Please visit my Google Scholar account. The list below might be incomplete and outdated.

Refereed Papers

• LM-CMA: an Alternative to L-BFGS for Large Scale Black-box Optimization
  Ilya Loshchilov
  to appear in Evolutionary Computation journal 2016, MIT press
              source code    
  Show abstract
  Hide abstract
  The limited memory BFGS method (L-BFGS) of Liu and Nocedal (1989) is often considered to be the method of choice for continuous optimization when first- and/or second- order information is available. However, the use of L-BFGS can be complicated in a black-box scenario where gradient information is not available and therefore should be numerically estimated. The accuracy of this estimation, obtained by finite difference methods, is often problem-dependent that may lead to premature convergence of the algorithm. In this paper, we demonstrate an alternative to L-BFGS, the limited memory Covariance Matrix Adaptation Evolution Strategy (LM-CMA) proposed by Loshchilov (2014). The LM-CMA is a stochastic derivative-free algorithm for numerical optimization of non-linear, non-convex optimization problems. Inspired by the L-BFGS, the LM-CMA samples candidate solutions according to a covariance matrix reproduced from $m$ direction vectors selected during the optimization process. The decomposition of the covariance matrix into Cholesky factors allows to reduce the memory complexity to $O(mn)$, where $n$ is the number of decision variables. The time complexity of sampling one candidate solution is also $O(mn)$, but scales as only about 25 scalar-vector multiplications in practice. The algorithm has an important property of invariance w.r.t. strictly increasing transformations of the objective function, such transformations do not compromise its ability to approach the optimum. The LM-CMA outperforms the original CMA-ES and its large scale versions on non-separable ill-conditioned problems with a factor increasing with problem dimension. Invariance properties of the algorithm do not prevent it from demonstrating a comparable performance to L-BFGS on non-trivial large scale smooth and nonsmooth optimization problems.

  Median (out of 11 runs) number of function evaluations required to find f(x) = 1e−10 for LM-CMA, L-BFGS with exact gradients, CL-BFGS with central differencing, active CMA-ES and VD-CMA. Dotted lines depict extrapolated results.

• Maximum Likelihood-based Online Adaptation of Hyper-parameters in CMA-ES
  Ilya Loshchilov, Marc Schoenauer, Michele Sebag and Nikolaus Hansen
  In: "Parallel Problem Solving from Nature (PPSN XIII)", ACM Press : p. 70-79. September 2014.
              source code    
  Show abstract
  Hide abstract
  The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is widely accepted as a robust derivative-free continuous optimization algorithm for non-linear and non-convex optimization problems. CMA-ES is well known to be almost parameterless, meaning that only one hyper-parameter, the population size, is proposed to be tuned by the user. In this paper, we propose a principled approach called self-CMA-ES to achieve the online adaptation of CMA-ES hyper-parameters in order to improve its overall performance. Experimental results show that for larger-than-default population size, the default settings of hyper-parameters of CMA-ES are far from being optimal, and that self-CMA-ES allows for dynamically approaching optimal settings.

  Evolution of learning rates c_1, c_{\mu}, c_c (lines with markers, left y-axis) and log10(objective function) (plain line, right y-axis) self-CMA-ES on 10- and 20-dimensional Sphere and Rosenbrock functions. The medians of 15 runs are shown.

  
• A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
  (nominated for the best paper at GECCO'14)
  Ilya Loshchilov
  In: "Genetic and Evolutionary Computation Conference (GECCO-2014)", ACM Press : p. 397-404. July 2014.
              source code    
  Show abstract
  Hide abstract
  We propose a computationally efficient limited memory Covariance Matrix Adaptation Evolution Strategy for large scale optimization, which we call the LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for numerical optimization of non-linear, non-convex optimization problems in continuous domain. Inspired by the limited memory BFGS method of Liu and Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a covariance matrix reproduced from m direction vectors selected during the optimization process. The decomposition of the covariance matrix into Cholesky factors allows to reduce the time and memory complexity of the sampling to O(mn), where n is the number of decision variables. When n is large (e.g., n > 1000), even relatively small values of m (e.g., m=20,30) are sufficient to efficiently solve fully non-separable problems and to reduce the overall run-time.

  Timing results of LM-CMA-ES on the separable Ellipsoid compared to sep-CMA-ES and Cholesky-CMA-ES. The results were computed using at most 10^5 function evaluations for sep-CMA-ES and LM-CMA-ES and using at most 10^4 for Cholesky-CMA-ES.

  The median of 11 runs on separable Ellipsoid function for different problem dimensions. The dotted lines correspond to extrapolated results by preserving the same scaling as between the last two actual estimations. Note that sep-CMA-ES is not rotation invariant and while it can solve the separable Ellipsoid, it cannot solve the non-separable one in a reasonable amount of function evaluations. However, LM-CMA-ES and Cholesky-CMA-ES are rotation invariant while the latter is limited to e.g. 10^3 ... 10^4 variables due to its quadratic time and space complexity per fuction evaluation.

  
• Intensive Surrogate Model Exploitation in Self-adaptive Surrogate-assisted CMA-ES (saACM-ES)
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2013)", ACM Press : p. 439-446. July 2013.
                 source code    
  Show abstract
  Hide abstract
  This paper presents a new mechanism for a better exploitation of surrogate models in the framework of Evolution Strategies (ESs). This mechanism is instantiated here on the self-adaptive surrogate-assisted Covariance Matrix Adaptation Evolution Strategy (ACM-ES), a recently proposed surrogate-assisted variant of CMA-ES. As well as in the original ACM-ES, the expensive function is optimized by exploiting the surrogate model, whose hyper-parameters are also optimized online. The main novelty concerns a more intensive exploitation of the surrogate model by using much larger population sizes for its optimization. The new variant of ACM-ES significantly improves the original ACM-ES and further increases the speed-up compared to the CMA-ES, especially on unimodal functions (e.g., on 20-dimensional Rotated Ellipsoid, ACM-ESis 6 times faster than aCMA-ES and almost by one order of magnitude faster than CMA-ES). The empirical validation on the BBOB-2013 noiseless testbed demonstrates the efficiency and the robustness of the proposed mechanism.

  Comparison of the proposed surrogate-assisted versions of IPOP-CMA-ES algorithms on 20-dimensional Rotated Ellipsoid function. The trajectories show the median of 15 runs.

  The proposed algorithms outperform BIPOP-aCMA-ES and demonstrate the best overall performance w.r.t. about 100 optimization algorithms tested during the BBOB-2009, 2010 and 2012. The performance of the proposed HCMA (a hybrid CMA-ES) is comparable to the portfolio 'best2009' algorithm (i.e., an artificial algorithm representing a combination of best results shown by a set of algorithms). See BBOB-2013 workshop paper for details.

  
• CMA-ES with Restarts for Solving CEC 2013 Benchmark Problems
  Ilya Loshchilov
  In: "Congress on Evolutionary Computation (CEC)", IEEE : p. 369-376. June 2013.
              original source code    source code with bound handling    a note on bound handling
  Show abstract
  Hide abstract
  This paper investigates the performance of 6 versions of Covariance Matrix Adaptation Evolution Strategy (CMAES) with restarts on a set of 28 noiseless optimization problems (including 23 multi-modal ones) designed for the special session on real-parameter optimization of CEC 2013. The experimental validation of the restart strategies shows that: i). the versions of CMA-ES with weighted active covariance matrix update outperform the original versions of CMA-ES, especially on illconditioned problems; ii). the original restart strategies with increasing population size (IPOP) are usually outperformed by the bi-population restart strategies where the initial mutation stepsize is also varied; iii). the recently proposed alternative restart strategies for CMA-ES demonstrate a competitive performance and are ranked first w.r.t. the proportion of function-target pairs solved after the full run on all 10-, 30- and 50-dimensional problems.

  Alternative restarts strategies of CMA-ES with active covariance matrix update (NBIPOP-aCMA-ES and NIPOP-aCMA-ES) demonstrate the overall best performance on 28 noiseless functions in dimensions 10, 30 and 50. The figure shows empirical cumulative distribution of all function-target pairs solved on all functions, dimensions and runs (in overall, 428400 pairs).

  
• KL-based Control of the Learning Schedule for Surrogate Black-Box Optimization
  Ilya Loshchilov
  In: "Conference sur l'Apprentissage Automatique (CAP)", ArXiv e-prints, 1308.2655. June 2013.
             
  Show abstract
  Hide abstract
  This paper investigates the control of an ML component within the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) devoted to black-box optimization. The known CMA-ES weakness is its sample complexity, the number of evaluations of the objective function needed to approximate the global optimum. This weakness is commonly addressed through surrogate optimization, learning an estimate of the objective function a.k.a. surrogate model, and replacing most evaluations of the true objective function with the (inexpensive) evaluation of the surrogate model. This paper presents a principled control of the learning schedule (when to relearn the surrogate model), based on the Kullback-Leibler divergence of the current search distribution and the training distribution of the former surrogate model. The experimental validation of the proposed approach shows significant performance gains on a comprehensive set of ill-conditioned benchmark problems, compared to the best state of the art including the quasi-Newton high-precision BFGS method.

  Ellipsoidal 95% confidence regions of the Gaussian distribution P_a (black thin line) and three other Gaussian distributions P_b (color marked lines) with same Kullback-Leibler divergence KL(P_a||P_b). In surrogate-assisted optimization an allowed KL-divergence from the original distribution when optimizing the surrogate can be linked with the estimated surrogate model error. The KL-divergence can be used to define some trust-region for local optimization.

  
• Alternative Restart Strategies for CMA-ES
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Parallel Problem Solving from Nature (PPSN XII)", Springer Verlag, LNCS 7491 : p. 296-305. September 2012.
              source code    
  Show abstract
  Hide abstract
  This paper focuses on the restart strategy of CMA-ES on multi-modal functions. A first alternative strategy proceeds by decreasing the initial step-size of the mutation while doubling the population size at each restart. A second strategy adaptively allocates the computational budget among the restart settings in the BIPOP scheme. Both restart strategies are validated on the BBOB benchmark; their generality is also demonstrated on an independent real-world problem suite related to spacecraft trajectory optimization.

  Restart performances of CMA-ES in the 2D hyper-parameter space (population size and initial mutation step size in log. coordinates). For each objective function (20 dimensional Rastrigin - top-left, Gallagher 21 peaks - top-right, Katsuuras - bottom-left and Lunacek bi-Rastrigin bottom-right), the median best function value out of 15 runs is indicated. Legends indicate that the optimum up to precision f(x) = 1e-10 is found always (+), sometimes (⊕) or never (◦). Black regions are better than white ones.

  
• Self-Adaptive Surrogate-Assisted Covariance Matrix Adaptation Evolution Strategy
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2012)", ACM Press : p. 321-328. July 2012.
                 source code     erratum
  Show abstract
  Hide abstract
  This paper presents a novel mechanism to adapt surrogate-assisted population-based algorithms. This mechanism is applied to ACM-ES, a recently proposed surrogate-assisted variant of CMA-ES. The resulting algorithm, saACM-ES, adjusts online the lifelength of the current surrogate model (the number of CMA-ES generations before learning a new surrogate) and the surrogate hyper-parameters. Both heuristics significantly improve the quality of the surrogate model, yielding a significant speed-up of saACM-ES compared to the ACM-ES and CMA-ES baselines. The empirical validation of saACM-ES on the BBOB-2012 noiseless testbed demonstrates the efficiency and the scalability w.r.t the problem dimension and the population size of the proposed approach, that reaches new best results on some of the benchmark problems.

  A comparison of the proposed saACM algorithms with 40+ classical (BFGS, GLOBAL, NEWOUA,...) and evolutionary optimization algorithms on 20-dimensional problems of the BBOB framework. The figure shows the proportion of functions which can be solved with a given precision (axis y) using a given number of function/model evaluations (axis x).

  
• Adaptive Coordinate Descent
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2011)", ACM Press : p. 885-892. July 2011.
                 source code
  Show abstract
  Hide abstract
  Independence from the coordinate system is one source of efficiency and robustness for the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The recently proposed Adaptive Encoding (AE) procedure generalizes CMA-ES adaptive mechanism, and can be used together with any optimization algorithm. Adaptive Encoding gradually builds a transformation of the coordinate system such that the new coordinates are as decorrelated as possible with respect to the objective function. But any optimization algorithm can then be used together with Adaptive Encoding, and this paper proposes to use one of the simplest of all, that uses a dichotomy procedure on each coordinate in turn. The resulting algorithm, termed Adaptive Coordinate Descent (ACiD), is analyzed on the Sphere function, and experimentally validated on BBOB testbench where it is shown to outperform the standard (1 + 1)-CMA-ES, and is found comparable to other state-of-the-art CMA-ES variants.

  CMA-like Adaptive Encoding Update (b) mostly based on Principal Component Analysis (a) is used to extend Coordinate Descent method (c) to the optimization of non-separable problems (d). Fast Adaptive Coordinate Descent has linear time complexity and is suitable for large-scale (D>>100) non-linear optimization.

  The adaptation of an appropriate coordinate system allows Adaptive Coordinate Descent to outperform Coordinate Descent on non-separable functions. The following figure illustrates the convergence of both algorithms on 2-dimensional Rosenbrock function up to a target function value 10^-10, starting from the initial point (-3,-4). The Adaptive Coordinate Descent reaches the target value after only 325 function evaluations (about 70 times faster than Coordinate Descent), that is comparable to gradient-based methods.

  
• Not all parents are equal for MO-CMA-ES
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Evolutionary Multi-Criterion Optimization 2011 (EMO 2011)", Springer Verlag, LNCS 6576 : p. 31-45. April 2011.
            
  Show abstract
  Hide abstract
  The Steady State variants of the Multi-Objective Covariance Matrix Adaptation Evolution Strategy (SS-MO-CMA-ES) generate one offspring from a uniformly selected parent. Some other parental selection operators for SS-MO-CMA-ES are investigated in this paper. These operators involve the definition of multi-objective rewards, estimating the expectation of the offspring survival and its Hypervolume contribution.Two selection modes, respectively using tournament, and inspired from the Multi-Armed Bandit framework, are used on top of these rewards. Extensive experimental validation comparatively demonstrates the merits of these new selection operators on unimodal MO problems.

  Multi-Armed Bandit (MAB) paradigm applied to parental selection in evolutionary multiobjective optimization allows to quickly catch the fruitful directions of search.

  Plots of all 10 populations found after 100,000 function evaluations by original (mu + 1)-MO-CMA (left), tournament-based (mu +1)-MOCMA (center) and MAB-based (mu + 1rank)-MO-CMA (right) in the x1-x2-x3 space on LZ09-2 and LZ09-3 30-dimensional multi-objective problems with complicated Pareto front in decision space (gray curve).

  
• Dominance-Based Pareto-Surrogate for Multi-Objective Optimization
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Simulated Evolution and Learning (SEAL 2010)", Springer Verlag, LNCS 6457 : p. 230-239. December 2010.
            
  Show abstract
  Hide abstract
  Mainstream surrogate approaches for multi-objective problems build one approximation for each objective. Mono-surrogate approaches instead aim at characterizing the Pareto front with a single model. Such an approach has been recently introduced using a mixture of regression Support Vector Machine (SVM) to clamp the current Pareto front to a single value, and one-class SVM to ensure that all dominated points will be mapped on one side of this value. A new mono-surrogate EMO approach is introduced here, relaxing the previous approach and modelling Pareto dominance within the rank-SVM framework. The resulting surrogate model is then used as a filter for offspring generation in standard Evolutionary Multi-Objective Algorithms, and is comparatively validated on a set of benchmark problems.

  In multi-criteria decision-making and multiobjective optimization it might be difficult to quantify the score of solutions according to given criteria/objectives. However, the comparison/preference relations is a sufficient source of information to build an efficient surrogate model of the problem.

  An approach to build comparison-based surrogate models of multi-objective problems using Ranking Support Vector Machine (Rank-SVM) is proposed. It learns the surrogate model using primary comparison constraints (<, =, >), then iteratively adds and learns the most violated secondary constraints. The model can take into account i). Pareto dominance relations ii). Quality indicators (Hypervolume) iii). Preference relations, defined by Decision-Maker.

  
• Comparison-Based Optimizers Need Comparison-Based Surrogates
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Parallel Problem Solving from Nature (PPSN XI)", Springer Verlag, LNCS 6238 : p. 364-373. September 2010.
                 source code     erratum
  Show abstract
  Hide abstract
  Taking inspiration from approximate ranking, this paper nvestigates the use of rank-based Support Vector Machine as surrogate model within CMA-ES, enforcing the invariance of the approach with respect to monotonous transformations of the fitness function. Whereas the choice of the SVM kernel is known to be a critical issue, the proposed approach uses the Covariance Matrix adapted by CMA-ES within a Gaussian kernel, ensuring the adaptation of the kernel to the currently explored region of the fitness landscape at almost no computational overhead. The empirical validation of the approach on standard benchmarks, comparatively to CMA-ES and recent surrogate-based CMA-ES, demonstrates the efficiency and scalability of the proposed approach.

  Surrogate models may predict very well f(x), but very poorly 1000*f(x) due to parametrization bias, while these two functions have exactly the same structure. We propose to use comparison-based surrogate models which are invariant to rank-preserving tranformation of the optimized function and also invariant to rotation of the search space, using adapted covariance matrix from the state-of-the-art continuous optimizer CMA-ES.

  
• A Mono Surrogate for Multiobjective Optimization
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2010)", ACM Press : p. 471-478. July 2010.
            
  Show abstract
  Hide abstract
  Most surrogate approaches to multi-objective optimization build a surrogate model for each objective. These surrogates can be used inside a classical Evolutionary Multiobjective Optimization Algorithm (EMOA) in lieu of the actual objectives, without modifying the underlying EMOA; or to filter out points that the models predict to be uninteresting. In contrast, the proposed approach aims at building a global surrogate model defined on the decision space and tightly characterizing the current Pareto set and the dominated region, in order to speed up the evolution progress toward the true Pareto set. This surrogate model is specified by combining a One-class Support Vector Machine (SVMs) to characterize the dominated points, and a Regression SVM to clamp the Pareto front on a single value. The resulting surrogate model is then used within state-of-the-art EMOAs to pre-screen the individuals generated by application of standard variation operators. Empirical validation on classical MOO benchmark problems shows a significant reduction of the number of evaluations of the actual objective functions.

  A mixture of One-Class Ranking Support Vector Machine (SVM) for dominated points and Regression SVM for non-dominated points can be used to build a mono surrogate for multiobjective optimization.

Papers in Workshop Proceedings

• BI-population CMA-ES Algorithms with Surrogate Models and Line Searches
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2013 Companion)", ACM Press : p. 1177-1184. July 2013.
  4th GECCO Workshop for Real-Parameter Optimization (Black-Box Optimization Benchmarking (BBOB), 2013
                 source code
  Show abstract
  Hide abstract
  In this paper, three extensions of the BI-population Covariance Matrix Adaptation Evolution Strategy with weighted active covariance matrix update (BIPOP-aCMA-ES) are investigated. First, to address expensive optimization, we benchmark a recently proposed extension of the self-adaptive surrogate-assisted CMA-ES which benefits from more intensive surrogate model exploitation (BIPOP-saACM-k). Second, to address separable optimization, we propose a hybrid of BIPOP-aCMA-ES and STEP algorithm with coordinatewise line search (BIPOP-aCMA-STEP). Third, we propose HCMA, a hybrid of BIPOP-saACM-k, STEP and NEWUOA to benefit both from surrogate models and line searches. All algorithms were tested on the noiseless BBOB testbed using restarts till a total number of function evaluations of 106n was reached, where n is the dimension of the function search space. The comparison shows that BIPOP-saACM-k outperforms its predecessor BIPOP-saACM up to a factor of 2 on illconditioned problems, while BIPOP-aCMA-STEP outperforms the original BIPOP-based algorithms on separable functions. The hybrid HCMA algorithm demonstrates the best overall performance compared to the best algorithms of the BBOB-2009, BBOB-2010 and BBOB-2012 when running for more than 100n function evaluations.

• Achieving optimization invariance w.r.t. monotonous transformations of the objective function and orthogonal transformations of the representation
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Probabilistic Numerics Workshop of NIPS 2012". December 2012.
      

• Black-box Optimization Benchmarking of NIPOP-aCMA-ES and NBIPOP-aCMA-ES on the BBOB-2012 Noiseless Testbed
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2012)", ACM Press : 269-276. July 2012.
  3rd GECCO Workshop for Real-Parameter Optimization (Black-Box Optimization Benchmarking (BBOB), 2012
                 source code
  Show abstract
  Hide abstract
  In this paper, we study the performance of NIPOP-aCMA- ES and NBIPOP-aCMA-ES, recently proposed alternative restart strategies for CMA-ES. Both algorithms were tested using restarts till a total number of function evaluations of 106D was reached, where D is the dimension of the function search space. We compared new strategies to CMA-ES with IPOP and BIPOP restart schemes, two algorithms with one of the best overall performance observed during the BBOB- 2009 and BBOB-2010. We also present the first benchmark- ing of BIPOP-CMA-ES with the weighted active covariance matrix update (BIPOP-aCMA-ES). The comparison shows that NIPOP-aCMA-ES usually out- performs IPOP-aCMA-ES and has similar performance with BIPOP-aCMA-ES, using only the regime of increasing the population size. The second strategy, NBIPOP-aCMA-ES, outperforms BIPOP-aCMA-ES in dimension 40 on weakly structured multi-modal functions thanks to the adaptive allocation of computation budgets between the regimes of restarts.

• Black-box optimization benchmarking of IPOP-saACM-ES and BIPOP-saACM-ES on the BBOB-2012 noiseless testbed
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2012)", ACM Press : 175-182. July 2012.
  3rd GECCO Workshop for Real-Parameter Optimization (Black-Box Optimization Benchmarking (BBOB), 2012
            
  Show abstract
  Hide abstract
  In this paper, we study the performance of IPOP-saACM-ES and BIPOP-saACM-ES, recently proposed self-adaptive surrogate-assisted Covariance Matrix Adaptation Evolution Strategies. Both algorithms were tested using restarts till a total number of function evaluations of 10{^6}D was reached, where D is the dimension of the function search space. We compared surrogate-assisted algorithms with their surrogate-less versions IPOP-saACM-ES and BIPOP-saACM-ES, two algorithms with one of the best overall performance observed during the BBOB-2009 and BBOB-2010. The comparison shows that the surrogate-assisted versions outperform the original CMA-ES algorithms by a factor from 2 to 4 on 8 out of 24 noiseless benchmark problems, showing the best results among all algorithms of the BBOB-2009 and BBOB-2010 on Ellipsoid, Discus, Bent Cigar, Sharp Ridge and Sum of different powers functions.

• Black-box optimization benchmarking of IPOP-saACM-ES on the BBOB-2012 noisy testbed
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2012)", ACM Press : 261-268. July 2012.
  3rd GECCO Workshop for Real-Parameter Optimization (Black-Box Optimization Benchmarking (BBOB), 2012
            
  Show abstract
  Hide abstract
  In this paper, we study the performance of IPOP-saACM-ES, recently proposed self-adaptive surrogate-assisted Covariance Matrix Adaptation Evolution Strategy. The algorithm was tested using restarts till a total number of function evaluations of 10{^6}D was reached, where D is the dimension of the function search space. The experiments show that the surrogate model control allows IPOP-saACM-ES to be as robust as the original IPOP-aCMA-ES and outperforms the latter by a factor from 2 to 3 on 6 benchmark problems with moderate noise. On 15 out of 30 benchmark problems in dimension 20, IPOP-saACM-ES exceeds the records observed during BBOB-2009 and BBOB-2010.

• A Pareto-Compliant Surrogate Approach for Multiobjective Optimization
  Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
  In: "Genetic and Evolutionary Computation Conference (GECCO-2010)", ACM Press : p. 1979-1982. July 2010.
  1st GECCO Workshop on Theoretical Aspects of Evolutionary Multiobjective Optimization, 2010
            
  Show abstract
  Hide abstract
  This paper discusses the idea of using a single Pareto-compliant surrogate model for multiobjective optimization. While most surrogate approaches to multi-objective optimization build a surrogate model for each objective, the recently proposed mono surrogate approach [3] aims at building a global surrogate model defined on the decision space and tightly characterizing the current Pareto set and the dominated region, in order to speed up the evolution progress toward the true Pareto set. This surrogate model is specified by combining a One-class Support Vector Machine (SVMs) to characterize the dominated points, and a Regression SVM to clamp the Pareto front on a single value. The aims of this paper are to identify issues of the proposed approach demanding further study and to raise the question of how to efficiently incorporate quality indicators, such as the hypervolume into the surrogate model.