CONCRETE: A benchmarking framework to CONtrol and Classify REpeatable Testbed Experiments

Authors: Stratos Keranidis, Wei Liu, Michael Mehari, Pieter Becue, Stefan Bouckaert, Ingrid Moerman, Thanasis Korakis, Iordanis Koutsopoulos and Leandros Tassiulas

Conference: FIRE Engineering Workshop 2012, Gent, Belgium, November 2012

The global research community has been recently directed towards implementation approaches and evaluation through experimentation in network- ing testbeds. A unique problem that arises during experimentation in such envi- ronments is that several di erent factors may impact the monitored performance of networks under consideration. More speci cally, the major factor that signif- icantly impacts performance is the e ect of interference that is generated either from testbed nodes that simultaneously transmit or from collocated commercial devices belonging to external networks. Moreover, as the level of interference signi cantly varies over time, the monitored performance may present high vari- ation among several executions of the same experiment. In order to ensure stable experimental conditions, speci c tools have to be developed that would be able to detect whether the gathered results have been a ected by external factors. In this work, we present a novel benchmarking framework, developed within the collaboration of CREW and OpenLab EU FP7 projects. The developed frame- work provides for proper evaluation of experimentally gathered results and thus enables the experimenter to arrive at solid conclusions regarding the validity of the conducted experiments.

Download paper: Keranidis_CONCRETE_FIRE_2012_paper.pdf