Non-Parametric Regression

Let the input sample (as generated from a DOE method) be , where each is an -dimensional vector and represents an input variable. The objective is to determine the equation of the form:

(2)

Where is a weighting vector. In case of generic non-parametric case, the Equation 2 is rewritten as the following:

(3)

where: is the kernel map and the quantities and are Lagrange multipliers whose derivation are shown in later sections.

To determine the Lagrange multipliers, you start with the assumption that the weight vector must be minimized such that all (or most) of the sample points lie within an error zone around the fitted surface. The following figure provides a simple demonstration. It fits a regression line for a group of sample points with a tolerance of ε, which is characterized by slack variables ξ* and ξ.

Thus, a primal optimization formulation is written as follows:

(4)

Where, is an arbitrary constant. To characterize the tolerance properly, you define a loss function in the interval (-, ), which de facto becomes the residual of the solution. In the present implementation, you use the -insensitive loss function, which is given by the equation:

(5)

The primal problem in Equation 4 can be rewritten as the following using generalized loss functions:

(6)

To solve this efficiently, a Lagrangian dual formulation is done on this to yield the following expression:

(7)

After some simplification, the dual Lagrangian can be written as the following Equation 8:

(8)

is a quadratic constrained optimization problem, and the design variables are the vector . Once this is computed, the constant in can be obtained by the application of Karush-Kuhn-Tucker (KKT) conditions. Once the -insensitive loss functions are used instead of the generic loss functions l() in, this can be rewritten in a much simpler form as the following:

(9)

Which is to be solved by a QP optimizer to yield the Lagrange multipliers and the constant is obtained by applying the KKT conditions.


Release 18.2 - © ANSYS, Inc. All rights reserved.