Results of the GECCO'2015 Track
Problem Summary
The GECCO 2015 track consisted of 1000 black box problems the characteristics of which are summarized in the following table.
| dimensions | problems per dimension | total number of problems | budget |
|---|---|---|---|
| 2, 4, 5, 8, 10, 16, 20, 32, 40, 64 | 100 | 1000 | 10 dim —100 dim |
"Raw" Result Data
On each problem participants were judged by the best (lowest) function value achieved within the given budget of function evaluations. There were 28 participants in the field. The best function value per problem and participant (1000 times 28 double precision numbers) is listed in this text file.
Participant Ranking
Participants were ranked based on aggregated problem-wise ranks (details here and here). The following results table lists participants with overall scores (higher is better) and the sum of ranks over all problems (lower is better) The table can be sorted w.r.t. these criteria.
| rank | participant | method | software | score  |
sum of ranks  |
|---|---|---|---|---|---|
| 1 | Sylvain Mouret (Artelys) | KNITRO | 1268.76 | 6900 | |
| 2 | Ling Chen Kelley | MVMO (proposed by István Erlich and José L. Rueda) | 1191.54 | 7021 | |
| 3 | Abdullah Shamil Hashim Al-Dujaili (mesop) | Naive Multi-Scale Optimization | 1099.04 | 8370 | |
| 4 | KMTM | [Faced Response Surface Design optimized with SCIP/YALMIP (dim 2-5) | CMA-ES (dim 8-16) | EGO - Kriging-Surrogate optimized with CMA-ES (dim 20..64)] Nelder-Mead (final refinement) | 1092.39 | 6872 | |
| 5 | rkar | Jumping Duplex | 879.125 | 8405 | |
| 6 | Al Jimenez | Curved Trajectories Algorithm (CTA) | 689.558 | 10103 | |
| 7 | Giacomo | RBFOpt | https://projects.coin-or.org/RBFOpt/wiki | 669.86 | 10328 |
| 8 | Tamer Moussa | Self-adaptive DE | 607.344 | 10368 | |
| 9 | radka | b6e6rl | 568.964 | 10411 | |
| 10 | JS | Algorithm that crossed differential evolution and CMAES | 537.304 | 10474 | |
| 11 | hdm | 449.83 | 15596 | ||
| 12 | bujok | MCRS | [paper] | 376.076 | 12218 |
| 13 | Grays | RD&R | 323.617 | 14058 | |
| 14 | anonymous | 293.915 | 18276 | ||
| 15 | Frank Hu | Learn to optimization | 278.599 | 14651 | |
| 16 | TRDF | Trust-region Derivative-free Algorithm for Constrained Optimization | 277.811 | 18352 | |
| 17 | Eric | apply machine learning algorithm to guide optimization | 218.893 | 14936 | |
| 18 | Makukhin Kirill | RBM-ES | 216.784 | 18020 | |
| 19 | Décio Lauro Soares | Memetic schema with selection features (first draft) | 204.664 | 18198 | |
| 20 | Poly Montreal | Mesh Adaptive Direct Search | 202.549 | 16379 | |
| 21 | decltypeme | CMA-ES flavors | 171.661 | 17881 | |
| 22 | Amin | DDS | 124.074 | 21910 | |
| 23 | Charlie Vanaret | DE | 115.44 | 16963 | |
| 24 | Glaedwine | Self-adaptive, Two-Phase Differential Evolution | 109.887 | 18529 | |
| 25 | Bugged | New custom method (with major bug) | 98.5314 | 20304 | |
| 26 | Pedro López-García, Enrique Onieva | GACE - Genetic Algorithm with Cross Entropy | [code] [paper] | 91.7493 | 17279 |
| 27 | Saber | BEA-GECCO2015 | 61.7187 | 19463 | |
| 28 | hsy | 41.3182 | 23807 |
The official ranking is based on the score. Hence the following participants receive prices:
| price | participant | affiliation |
|---|---|---|
| 1. 500 EUR | Sylvain Mouret | — |
| 2. 300 EUR | Ling Chen Kelley | Tsinghua University College of Humanities / Auburn University of Montgomery |
| 3. 200 EUR | Abdullah Shamil Hashim Al-Dujaili | Nanyang Technological University |
We would like to thank Microsoft Research - Inria joint centre for sponsoring the cash prizes of the competition, more specifically, Youssef Hamadi and Laurent Massoulie for their interest and support.
Visualization of Performance Data
The following figure shows an aggregated view on the performance data.
The following figures show the same data, but separately for each problem dimension.
