The Genetic Aggregation response surface can be written as an ensemble using a weighted average of different metamodels:
where:
|
|
|
|
|
|
|
|
The weight factors satisfy:
To estimate the best weight factor values, DesignXplorer minimizes the Root Mean Square
Error (RMSE) of the Design of Experiments Points (or Design Point) on , and the RMSE of the same Design Points based on the cross-validation of
.
with:
where:
|
|
|
|
|
|
|
|
The cross-validation methods used in DesignXplorer are the "Leave-One-Out" and "K-Fold" methods.
Leave-One-Out Method:
Solves as many sub-metamodels as there are Design Points.
For a given response surface, DesignXplorer computes the
sub-metamodels, where each sub-response surface corresponds of the
response surface fitted to
Design Points.
The cross-validation error of the Design Point is the error at this point of the sub-response surface
built without the
Design Point
K-Fold Method:
Builds sub-metamodels of the
response surface, where each sub-response surface corresponds of the
response surface fitted to
Design Points.
The cross-validation error at the Design Point is the error at this point of the sub-response surface
built without the subset of
Design Points containing the
Design Point.
To improve the relevance of the k-fold strategy, the Design Points used as validation points of each fold are selected by
using the Maximum of Minimum Distance.

The Leave-One-Out method can be expensive, so DesignXplorer uses 10-fold cross-validation by default and switches to the Leave-One-Out method when the number of Design Points is too small to obtain a relevant 10-fold cross-validation. To reduce the computation cost, the cross-validation is done in parallel.
If you note , the cross-validation error of
response surface built without the
Design Point, then:
DesignXplorer computes analytically the weight factor values. This computation is based on a similar approach proposed by Viana et al. (2009).
Where is the identity matrix and
the matrix of the mean square errors:
To get positive weights, DesignXplorer uses only the diagonal elements of .
With:
and
are respectively the errors (combination of
,
, and the overfitting penalization ) of the
and
metamodels.
There are many types of metamodels, including Polynomial Regression, Kriging, Support Vector Regression, and Moving Least Squares. For each response surface, there are different settings. For example, on the Kriging response surface, you can control the type of kernel (Gaussian, exponential, and so on), the type of kernel variation (anisotropic or isotropic), and the type of polynomial regression (linear, quadratic, and so on).
To increase the chance of getting the most effective response surface, DesignXplorer generates a population of metamodels with different types and settings. This population corresponds to the first population of the genetic algorithm run by DesignXplorer. The next populations are obtained by cross-over and mutation of previous population.
There are two types of cross-over. The first one is the cross-over between two response surfaces of the same type (for example, two Kriging), and the second one is the cross-over between two response surfaces of different types (for example, a Kriging and a polynomial regression).
In the first case, DesignXplorer exchanges a part of settings from the first parent to the second parent (for example, the exchange of kernel type between two Kriging response surfaces).
In the second case, DesignXplorer creates a new response surface (an ensemble), which is a combination of the two parents (for example, the combination of a Kriging and a polynomial regression response surface).
DesignXplorer mutates one or several settings of the response surface (or of the response surfaces, in the case of a combination of response surfaces).
To keep a diversity of response surfaces, the genetic algorithm removes a part of the response surface type that is too present in the population, while it retains other response surfaces less present.
In the best case, the population contains similar metamodels in terms of prediction
accuracy () when the predicted values are different
: this increases the chance of error cancellation on the ensemble.
For external references, see Genetic Aggregation Response Surface References.
Gaussian Process regression methods, such as Kriging, provide a prediction distribution. The genetic aggregation can contain several surrogate models that do not afford a local prediction distribution.
Ben Salem et al. introduce a universal prediction distribution (UP distribution) for all surrogate models. This distribution relies on Cross-Validation sub-model predictions. Based on this empirical distribution, they define an adaptive sampling technique for global refinement (UP-SMART).
The Genetic Aggregation refinement is a hybrid variant of UP-SMART consisting in
adding at step a point:
With:
is a variant of the local UP variance at x point at step
and
denote a given distance on
is the sample set used to build the response surface
is the parameter space defined by the user
is a distance penalization
The Predicted Error exposed in DesignXplorer corresponds to .
References:
Malek Ben Salem, Olivier Roustant, Fabrice Gamboa, and Lionel Tomaso. Universal prediction distribution for surrogate models. arXiv preprint arXiv:1512.07560, 2015.