【机器学习】ADMM 推导与总结

Posted by ShawnD on August 21, 2022

参考1

ADMM 一般用于求解如下带有等式约束的凸优化问题:

\[\min_{x, z} f(x) + g(z) \quad \text{s.t.} \quad Ax + Bz = c \tag{1}\]

$(1)$ 的增广拉格朗日函数为:

\[L_p(x, z, \lambda) = f(x) + g(z) + \lambda^\top (Ax + Bz - c) + \frac{\rho}{2} \|Ax + Bz - c \|^2 \tag{2}\]

其中$\lambda$ 是对偶变量, 常量 $\rho > 0$。

ADMM 具体迭代更新式如下:

\[x_{k+1} = \mathop{\text{argmin}}_x L_p(x, z_k, \lambda_k) \\ z_{k+1} = \mathop{\text{argmin}}_z L_p(x_{k+1}, z, \lambda_k) \\ \lambda_{k+1} = \lambda_k + \rho(Ax_{k+1} + Bz_{k+1} -c) \tag{3}\]

参考2

Reference

  1. 交替方向乘子法(ADMM)