Exploration vs. Exploitation Challenge

Please could all participants that have downloaded the code below and are interested in submitting to the challenge email Zakria Hussain to indicate that you may wish to submit. This does not mean that you must submit but gives the organisers a number on possible participation. Thank you.

Java code download

Java-code1.2.zip (version 1.2 released 21st May 2006)

There is an API written in Java which is distributed as a .jar file (.class executables only) and some baseline Java code that implements some simple multi-armed bandit problem algorithms. You will need at least Java version 1.5 to run the Java code. You can compile the baseline code like so:

javac -classpath challenge1.2.jar applicationtest/*.java applicationtest/simplewindowprovider/*.java

After this you can run ApplicationTest like so:

java -classpath .;challenge1.2.jar -Xmx256m applicationtest/ApplicationTest

OR

java -classpath :challenge1.2.jar -Xmx256m applicationtest/ApplicationTest

The Java code contains provider classes and several (bandit) algorithms written using the provider classes. ApplicationTest calls these algorithms and runs them on all 6 tasks. All participants will need to write their algorithm as a provider class. Please look at the provider classes in the sample Java code to understand how to do this. The code contains very simplistic algorithms for bandit problems when arms do not vary over time. However, please note that the challenge is for the case where the rewards for an arm varies over time.

Version 1.1 contains:

1. Six new tasks
2. detailed output every 1000 requests to csv file. This contains:
1. the underlying response rates of the options in the task, and
2. the proportion of serves each option is receiving
This allows the user to construct clear graphical views of how an algorithm is performing over time - see for example UCBWindowProvider.JPG.
3. An efficiency metric has been added to getStatistics(). This ranges from 0 (equivalent to random serving) to 1 (equalling the maximum theoretical success rate).

Version 1.2 contains

1. UCBWindowProvider - a windowed version of UCB
2. Improved regret summary
3. improved commandline args:
1. -h help
2. -r number of requests
3. -s random seed set
4. -i produce results on interim seed set. The summary printed at the end of the test is sufficient for a submission
4. better javadoc of applicationtest and challenge1.2.jar

Here is the javadoc1.2 for the challenge1.2.jar package.

Matlab code download

Matlab-code1.2.zip (version 1.2 released 21st May 2006)

The Matlab code is a wrapper for the Java API but does not contain any baseline code like that available for the Java download. The Matlab wrapper calls in the challenge1.2.jar executables and all its functions can be used as in Java. Please read the documentation (preamble) in the matlab file in order to set it up to run correctly.