考虑一个最基本的凸优化问题:
最小二乘,也称为线性回归
设y=Ax+b,
x
∈
R
n
x \in R^n
x∈Rn, 最小二乘问题是求x使得y的平方和最小化。
首先,让我们举一个无约束的例子:
m=16;
n=8;
A=randn(m,n);
b=randn(m,1);
最小二乘解KaTeX parse error: Double superscript at position 7: x=(A^T^̲A)^-1^A^T^b很容易通过 backslash operator计算得到:
x_ls = A \ b;
这种简单问题没有必要使用CVX工具计算,但为了验证求解结果,我们求解如下(注意目标函数中的二范数最小化与y的平方和最小化是等价的):
cvx_begin
variable x(n)
minimize( norm(Ax-b) )
cvx_end
然后,考虑一个有约束的例子,这时就不能直接通过backslash operator计算得到最小二乘解。
例如x的下界为l,上界为u:
bnds = randn(n,2);
l = min( bnds, [], 2 );
u = max( bnds, [], 2 );
但可以使用matlab自带的QP solver求解,但需要转化为标准形式:
x_qp = quadprog( 2A’A, -2A’b, [], [], [], [], l, u );
转化过程详见quadprog函数说明。
此时我们使用CVX求解就显得方便多了:
cvx_begin
variable x(n)
minimize( norm(Ax-b) )
subject to
l <= x <= u
cvx_end
顺便说一句,CVX不会通过对目标函数求平方来将这个问题转换为QP求解, 而是将其转换为SOCP。可以验证,求解结果是相同的,并且转换是自动完成的。
总结:CVX工具就是尿不湿。
CVX工具安装使用链接
https://blog.csdn.net/Laynezhao/article/details/112118753
m=16;
n=8;
A=randn(m,n);
b=randn(m,1);
bnds = randn(n,2);
l = min( bnds, [], 2 );
u = max( bnds, [], 2 );
x_qp = quadprog( 2*A'*A, -2*A'*b, [], [], [], [], l, u )
cvx_begin
variable x(n)
minimize( norm(A*x-b) )
subject to
l <= x <= u
cvx_end
display(x,'x')
Minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the default value of the optimality tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.
<stopping criteria details>
x_qp =
0.5404
-0.0915
0.2436
-0.8536
1.2815
0.1250
-0.8840
-0.2598
Calling SDPT3 4.0: 33 variables, 9 equality constraints
For improved efficiency, SDPT3 is solving the dual problem.
------------------------------------------------------------
num. of constraints = 9
dim. of socp var = 17, num. of socp blk = 1
dim. of linear var = 16
*******************************************************************
SDPT3: Infeasible path-following algorithms
*******************************************************************
version predcorr gam expon scale_data
NT 1 0.000 1 0
it pstep dstep pinfeas dinfeas gap prim-obj dual-obj cputime
-------------------------------------------------------------------
0|0.000|0.000|1.6e+00|2.8e+00|1.7e+03| 7.161863e+01 0.000000e+00| 0:0:00| chol 1 1
1|0.927|0.966|1.1e-01|1.2e-01|1.3e+02| 5.081575e+01 -9.409293e+00| 0:0:00| chol 1 1
2|1.000|1.000|1.4e-06|2.7e-03|1.3e+01| 4.300286e+00 -8.185259e+00| 0:0:00| chol 1 1
3|0.867|0.771|2.6e-07|8.3e-04|2.4e+00|-5.141461e+00 -7.467459e+00| 0:0:00| chol 1 1
4|0.733|1.000|2.7e-07|2.7e-05|1.2e+00|-6.103270e+00 -7.252439e+00| 0:0:00| chol 1 1
5|1.000|0.807|1.5e-09|7.4e-06|2.2e-01|-6.889683e+00 -7.105866e+00| 0:0:00| chol 1 1
6|0.878|1.000|8.9e-10|2.7e-07|4.8e-02|-7.003738e+00 -7.051811e+00| 0:0:00| chol 1 1
7|1.000|0.902|6.9e-10|5.1e-08|4.3e-03|-7.038176e+00 -7.042472e+00| 0:0:00| chol 1 1
8|0.968|0.985|2.8e-10|3.6e-09|2.3e-04|-7.040772e+00 -7.041001e+00| 0:0:00| chol 1 1
9|0.984|0.988|1.4e-10|1.0e-10|3.5e-06|-7.040942e+00 -7.040945e+00| 0:0:00| chol 1 1
10|0.958|1.000|6.1e-12|2.8e-11|2.8e-07|-7.040944e+00 -7.040945e+00| 0:0:00| chol 1 1
11|1.000|1.000|1.9e-12|1.2e-12|1.7e-08|-7.040945e+00 -7.040945e+00| 0:0:00|
stop: max(relative gap, infeasibilities) < 1.49e-08
-------------------------------------------------------------------
number of iterations = 11
primal objective value = -7.04094466e+00
dual objective value = -7.04094468e+00
gap := trace(XZ) = 1.71e-08
relative gap = 1.14e-09
actual relative gap = 1.13e-09
rel. primal infeas (scaled problem) = 1.93e-12
rel. dual " " " = 1.23e-12
rel. primal infeas (unscaled problem) = 0.00e+00
rel. dual " " " = 0.00e+00
norm(X), norm(y), norm(Z) = 3.6e+00, 7.4e+00, 1.0e+01
norm(A), norm(b), norm(C) = 1.3e+01, 2.0e+00, 1.5e+01
Total CPU time (secs) = 0.37
CPU time per iteration = 0.03
termination code = 0
DIMACS: 1.9e-12 0.0e+00 2.0e-12 0.0e+00 1.1e-09 1.1e-09
-------------------------------------------------------------------
------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +7.04094
x =
0.5404
-0.0915
0.2436
-0.8536
1.2815
0.1250
-0.8840
-0.2598