Let the input sample (as generated from a DOE method) be , where each
is an
-dimensional vector and represents an input variable. The objective is to
determine the equation of the form:
(2) |
Where is a weighting vector. In case of generic non-parametric case, the Equation 2 is rewritten as the following:
(3) |
where:
is the kernel map and the quantities
and
are Lagrange multipliers whose derivation are shown in later
sections.
To determine the Lagrange multipliers, you start with the assumption that the weight
vector must be minimized such that all (or most) of the sample points lie
within an error zone around the fitted surface. The following figure provides a simple
demonstration. It fits a regression line for a group of sample points with a tolerance of
ε, which is characterized by slack variables ξ* and ξ.
Thus, a primal optimization formulation is written as follows:
(4) |
Where, is an arbitrary
constant. To characterize the tolerance properly, you define a loss
function in the interval (-
,
), which de facto becomes the residual of the solution. In the present
implementation, you use the
-insensitive loss function, which is given by the equation:
(5) |
The primal problem in Equation 4 can be rewritten as the following using generalized loss functions:
(6) |
To solve this efficiently, a Lagrangian dual formulation is done on this to yield the following expression:
(7) |
After some simplification, the dual Lagrangian can be written as the following Equation 8:
(8) |
is a quadratic constrained optimization problem, and the design variables are the
vector . Once this is computed, the constant
in can be obtained by the application of Karush-Kuhn-Tucker (KKT)
conditions. Once the
-insensitive loss functions are used instead of the generic loss
functions l(
) in, this can be rewritten in a much simpler form as the
following:
(9) |
Which is to be solved by a QP optimizer to yield the Lagrange multipliers
and the constant
is obtained by applying the KKT conditions.