Results of the CEC'2015 Track
Problem Summary
The CEC 2015 track consisted of 1000 black box problems the characteristics of which are summarized in the following table.
| dimensions | problems per dimension | total number of problems | budget |
|---|---|---|---|
| 2, 4, 5, 8, 10, 16, 20, 32, 40, 64 | 100 | 1000 | 100 dim2 |
"Raw" Result Data
On each problem participants were judged by the best (lowest) function value achieved within the given budget of function evaluations. After removal of highly incomplete entries there were 25 participants left in the field. The best function value per problem and participant (1000 times 25 double precision numbers) is listed in this text file.
Participant Ranking
Participants were ranked based on aggregated problem-wise ranks (details here and here). The following results table lists participants with overall scores (higher is better), the sum of ranks over all problems (lower is better), and a ranking provided by the participants themselves (lower is better). The table can be sorted w.r.t. all three criteria.
| participant | method | software | score  |
sum of ranks  |
participant ranking |
|---|---|---|---|---|---|
| Mickey | MUOEAs | download | 1100.93 | 8125 | 2 |
| Simon Wessing | Two-stage algorithms | http://ls11-www.cs.uni-dortmund.de/staff/wessing/bbcomp | 960.623 | 6757 | 1.33 |
| Lepagnot | MDP | 864.597 | 8091 | 2.83 | |
| KMTM | [ RSM | CMA-ES ] NM | 855.67 | 8762 | 4.91 | |
| Charlie Vanaret | DE | 789.007 | 8512 | 4.16 | |
| rkar | Jumping Simplex | 759.892 | 10902 | 9 | |
| radka | b6e6rl | 654.521 | 9618 | 6.25 | |
| Poly Montreal | Mesh Adaptive Direct Search | 623.174 | 10156 | 8 | |
| jarabas | CMADE | 601.712 | 10255 | 7.16 | |
| Al Jimenez | Curved Trajectories Algorithm (CTA) | 526.609 | 11956 | 10.66 | |
| Artelys | KNITRO | 508.172 | 11391 | 9.83 | |
| bujok | SHADEcc | 493.541 | 12458 | 12.83 | |
| hdm | 447.931 | 13356 | 15.41 | ||
| anonymous | Dynamic Fireworks Algorithm with Orienting Mutation | 365.109 | 12966 | 15.41 | |
| Décio Lauro Soares | Memetic schema with selection features (first draft) | 340.808 | 15019 | 17.66 | |
| anonymous | IG-BSA | 315.634 | 13027 | 14.91 | |
| voglinio | 285.108 | 12702 | 14.83 | ||
| gmaxwell | 279.614 | 12359 | 14.08 | ||
| Ralf Sauermann | particle swarm optimization | 227.216 | 14163 | 18.66 | |
| Makukhin Kirill | RBM-ES | 103.561 | 16111 | 20 | |
| TRDF | Trust-region Derivative-free Algorithm for Constrained Optimization | 102.275 | 19194 | 22 | |
| anonymous | 82.372 | 16433 | 21.16 | ||
| Eric | apply machine learning algorithm to guide optimization | 52.7791 | 18324 | — | |
| 4fun | A modified simulated annealing algorithm | 8.29443 | 20662 | 22.83 | |
| Andrzej 'Yester' Fiedukowicz | DEArch | 1.45839 | 23127 | 24 |
The official ranking is based on the score. Hence the following participants received prices:
| price | participant | full name(s) of participant(s) | affiliation(s) |
|---|---|---|---|
| 1. 250 EUR | Mickey | Chao Yu | Peking University |
| 2. 150 EUR | Simon Wessing | Simon Wessing | Technische Universität Dortmund |
| 3. 100 EUR | Lepagnot | Julien Lepagnot, Lhassane Idoumghar | Université de Haute-Alsace |
Visualization of Performance Data
The following figure shows an aggregated view on the performance data.
The following figures show the same data, but separately for each problem dimension.
