A Symmetric Alternating Direction Method of Multipliers with Two Different Relaxation Factors for Solving Non-Separable Nonconvex Minimization Problems

Mei Lu, Zidan Wang^*

Key Laboratory of Optimization Theory and Applications, School of Mathematics and Information, China West Normal University, Nanchong, China.

DOI: 10.4236/jamp.2026.144081                

Abstract

This paper proposes a symmetric alternating direction method of multipliers with two different relaxation factors for solving nonconvex optimization problems with linear constraints and a non-separable structure. Although many studies have proposed variants of symmetric ADMM with a single relaxation factor, incorporating techniques such as Bregman distances, inertial terms, regularization terms, or linearization, the most basic form of symmetric ADMM with two different relaxation factors for solving non-separable problems has not yet been resolved. The introduction of two different relaxation factors in this method yields a broader range of parameters, making it applicable to more practical problems, and also provides a fundamental theoretical basis for accelerating the algorithm with other techniques. Based on the Kurdyka-Łojasiewicz property, we establish the convergence of the sequences generated by the proposed algorithm and analyze its convergence rate.

Keywords

Symmetric Alternating Direction Method of Multipliers, Non-Separable Structure, Nonconvex Optimization, Kurdyka-Łojasiewicz Inequality

1. Introduction

In many practical problems such as image processing, machine learning, and statistical modeling [1]-[3], the objective function often contains a coupling term $H\left(x,y\right)$ . That is, for the nonconvex and nonseparable problem considered in this paper, the iterative scheme is as follows:

$\begin{array}{l}\underset{x,y}{\text{min}}f\left(x\right)+g\left(y\right)+H\left(x,y\right),\\ \text{s}.\text{t}.Ax+y=b.\end{array}$ (1)

where $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function, possibly nonsmooth and nonconvex, both $g:{ℝ}^m\to ℝ$ and $H:{ℝ}^n\times {ℝ}^m\to ℝ$ are continuously differentiable and possibly nonconvex functions, the gradient $\nabla g$ is $L_g$ -Lipschitz continuous and the gradient $\nabla H$ is $L_h$ -Lipschitz continuous. $A\in {ℝ}^{m\times n}$ is a matrix and $b\in {ℝ}^m$ is a vector. In particular, when $H\left(x,y\right)=0$ , problem (1) reduces to a separable optimization problem of the following form:

$\begin{array}{l}\underset{x,y}{\text{min}}f\left(x\right)+g\left(y\right),\\ \text{s}.\text{t}.Ax+y=b.\end{array}$ (2)

The alternating direction method of multipliers (ADMM) is one of the most effective approaches for solving problem (2). The convergence of ADMM has been extensively studied [4]-[9], and the convergence rate analysis is relatively well-established for problem (2). However, when $H\left(x,y\right)\ne 0$ , for problem (1), the convergence results for ADMM are relatively limited. Gao et al. [10] proved the convergence of the proximal ADMM under the setting where $H$ is smooth, $f$ and $g$ are convex, combined with the assumption that $\nabla H$ is Lipschitz continuous. Chen et al. [11] proved the convergence of the extended ADMM when the coupling term $H$ is a quadratic function. In recent years, researchers have begun to focus on solving (1) by using ADMM in non-convex settings, and have employed tools such as the Kurdyka-Łojasiewicz (KL) inequality to prove the convergence of the algorithm. Guo et al. [12] [13] proved the convergence of the classic ADMM and extended it to the generalized ADMM (GADMM). Liu et al. [14] proposed a linearized ADMM and proved the convergence of the algorithm.

Regarding problem (2), it has been established in the literature that under convergence occurs, the symmetric ADMM often converges faster than the classical ADMM. Wu et al. [15] proposed a symmetric ADMM with one relaxation factor and verified its convergence. On this basis, some scholars have studied the symmetric ADMM with a relaxation factor and its variants for solving the nonseparable problem (1). For example, Dang et al. [16] proposed a linear proximal symmetric ADMM and verified that the sequence generated by the algorithm converges to a stationary point of the problem. The iterative scheme is as follows:

$\{\begin{array}{l}{x}^{k+1}\in \underset{x}{\arg \min }\{f\left(x\right)+\langle x-x^k,{\nabla }_{x}H\left(x^k,y^k\right)+\beta A^{\text{T}}\left(A{x}^k+B{y}^k-b\right)\rangle \underset{}{\overset{}{}}\\ -\langle {\lambda }^k,Ax\rangle +\frac{{\alpha }_x}{2}{\Vert x-x^k\Vert }^2\},\\ {\lambda }^{k+\frac{1}{2}}={\lambda }^k-\tau \beta \left(A{x}^{k+1}+B{y}^k-b\right),\\ y^{k+1}\in \underset{y}{\arg \min }\left\{g\left(y\right)+\langle y-y^k,{\nabla }_{y}H\left(x^{k+1},y^k\right)\rangle +\frac{\beta }{2}{\Vert A{x}^{k+1}+By-b-\frac{{\lambda }^{k+\frac{1}{2}}}{\beta }\Vert }^2\right\},\\ {\lambda }^{k+1}={\lambda }^{k+\frac{1}{2}}-\beta \left(A{x}^{k+1}+B{y}^{k+1}-b\right).\end{array}$

The parameters are selected as follows:

$\tau \in \left(-1,0\right),{\alpha }_y\ge l_h,$

${\alpha }_x\ge \frac{8{\mu }^2{l}_h^2}{\left(\tau +1\right)\beta }+\beta {\lambda }_{\max }\left(A^{\text{T}}A\right)+l_h,\beta \ge \frac{{\mu }^2\left(b-\sqrt{b^2-64\tau c}\right)}{4\tau }.$

Dang et al. [17] proposed an inertial Bregman symmetric ADMM algorithm and established its global convergence. The iterative scheme is as follows:

$\{\begin{array}{l}{x}^{k+1}\in \underset{x}{\arg \min }\left\{L_{\rho }\left(x,y^k,{\lambda }^k\right)+{\Delta }_{{\varphi }_1}\left(x,x^k\right)+{\theta }_{1k}\langle x,x^{k-1}-x^k\rangle +{\theta }_{2k}\langle x,x^{k-2}-x^{k-1}\rangle \right\},\\ {\lambda }^{k+\frac{1}{2}}={\lambda }^k-r\rho \left(A{x}^{k+1}+B{y}^k-b\right)+{\theta }_{1k}B\left(y^{k-1}-y^k\right)+{\theta }_{2k}B\left(y^{k-2}-y^{k-1}\right),\\ y^{k+1}\in \underset{y}{\arg \min }\{L_{\rho }\left(x^{k+1},y,{\lambda }^{k+\frac{1}{2}}\right)+{\Delta }_{{\varphi }_2}\left(y,y^k\right)+{\theta }_{1k}\langle By,B\left(y^{k-1}-y^k\right)\rangle \\ \begin{array}{c}\\ \end{array}+{\theta }_{2k}\langle By,B\left(y^{k-2}-y^{k-1}\right)\rangle \},\\ {\lambda }^{k+1}={\lambda }^{k+\frac{1}{2}}-\rho \left(A{x}^{k+1}+B{y}^{k+1}-b\right)+{\theta }_{1k}B\left(y^k-y^{k-1}\right)+{\theta }_{2k}B\left(y^{k-1}-y^{k-2}\right).\end{array}$

where ${\text{Δ}}_{{\varphi }_i}\left(s,t\right)$ , $i=1,2$ represents the Bregman distances, and $r\in \left(-1,1\right)$ is a relaxation factor. However, the convergence of the sequence generated by the algorithm depends on the strong convexity of the kernel function associated with the Bregman distance.

In recent studies on solving problem (2), we observe that Lu et al. [18] proposed a symmetric ADMM with two different relaxation factors, without introducing the Bregman distance, they proved its convergence with a broader range of parameters, numerical experiments verified that their algorithm converges faster than the algorithm proposed by [15]. Its iterative scheme is as follows:

$\{\begin{array}{l}{x}^{k+1}\in \underset{x}{\arg \min }\left\{{ℒ}_{\beta }^s\left(x,y^k,{\lambda }^k\right)\right\},\\ {\lambda }^{k+\frac{1}{2}}={\lambda }^k-\alpha \beta \left(A{x}^{k+1}+y^k-b\right),\\ y^{k+1}\in \underset{y}{\arg \min }\left\{{ℒ}_{\beta }^s\left(x^{k+1},y,{\lambda }^{k+\frac{1}{2}}\right)\right\},\\ {\lambda }^{k+1}={\lambda }^{k+\frac{1}{2}}-s\beta \left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$ (3)

Among them, ${ℒ}_{\beta }^s$ is the augmented Lagrangian function associated with problem (2), defined as follows:

${ℒ}_{\beta }^s\left(x,y,\lambda \right)=f\left(x\right)+g\left(y\right)-\langle \lambda ,Ax+y-b\rangle +\frac{s\beta }{2}{\Vert Ax+y-b\Vert }^2.$ (4)

where $\beta >0$ is the penalty parameter, $\alpha$ and s are two different relaxation factors.

The above research on symmetric ADMM algorithms for solving problem (1) is limited to those containing only one relaxation factor. However, inspired by the algorithm proposed by the work of Lu et al. [18], we consider introducing two relaxation factors to achieve wider applicability, faster convergence, and a more concise and unified algorithmic framework, this paper proposes using a symmetric ADMM with two different relaxation factors to solve the non-separable (1), the iterative scheme is as follows:

$\{\begin{array}{l}{x}^{k+1}\in \underset{x}{\arg \min }\left\{f\left(x\right)+H\left(x,y^k\right)-\langle {\lambda }^k,Ax\rangle +\frac{s\beta }{2}{\Vert Ax+y^k-b\Vert }^2\right\},\\ {\lambda }^{k+\frac{1}{2}}={\lambda }^k-\alpha \beta \left(A{x}^{k+1}+y^k-b\right),\\ y^{k+1}\in \underset{y}{\arg \min }\left\{g\left(y\right)+H\left(x^{k+1},y\right)-\langle {\lambda }^{k\frac{1}{2}},y\rangle +\frac{s\beta }{2}{\Vert A{x}^{k+1}+y-b\Vert }^2\right\},\\ {\lambda }^{k+1}={\lambda }^{k+\frac{1}{2}}-s\beta \left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$ (5)

Compared with the algorithm proposed by Dang et al. [16], our algorithm has a wider range of parameter values, allowing it to be applied to more practical scenarios through appropriate parameter tuning. Moreover, unlike the algorithm proposed by Dang et al. [17], our method does not require the introduction of the Bregman distance, thereby providing a unified iterative framework for solving non-separable problems via ADMM. The optimality conditions are as follows:

$\{\begin{array}{l}0\in \partial f\left(x^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^k\right)-A^{\text{T}}{\lambda }^k+s\beta A^{\text{T}}\left(A{x}^{k+1}+y^k-b\right),\\ {\lambda }^{k+\frac{1}{2}}={\lambda }^k-\alpha \beta \left(A{x}^{k+1}+y^k-b\right),\\ 0=\nabla g\left(y^{k+1}\right)+{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)-{\lambda }^{k+\frac{1}{2}}+s\beta \left(A{x}^{k+1}+y^{k+1}-b\right),\\ {\lambda }^{k+1}={\lambda }^{k+\frac{1}{2}}-s\beta \left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$ (6)

In the case where $H=0$ , when $a=0$ and $s=1$ , the proposed algorithm reduces to the classical ADMM. When $a\ne 0$ and $s=1$ , the algorithm reduces to the symmetric ADMM with a scaling factor. This demonstrates that our method provides an improved basic iterative framework for symmetric ADMM with relaxation factors.

2. Preliminaries

In this section, we provide the necessary definitions and properties required for the following study.

Notations:

• $ℝ$ , ${ℝ}^n$ , ${ℝ}^{m\times n}$ : real numbers, $n$ -dimensional real vectors, $m\times n$ -real matrices.

• $\langle \cdot ,\cdot \rangle$ and $\Vert \cdot \Vert$ : inner product and associated norm.

Set-valued mapping $F:{ℝ}^n\rightrightarrows {ℝ}^m$ :

• Domain: $\text{dom}F:=\left\{x\in {ℝ}^n|F\left(x\right)\ne \varnothing \right\}$ .

• Graph: $\text{Gra}F:=\left\{\left(x,y\right)\in {ℝ}^n\times {ℝ}^m:y\in F\left(x\right)\right\}$ .

Distance: For $S\subseteq {ℝ}^n$ and $x\in {ℝ}^n$ , $d\left(x,S\right):=\inf \left\{\Vert y-x\Vert |y\in S\right\}$ , with $d\left(x,S\right):=+\infty$ .

Definition 1. [19] The domain of function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is denoted by

$\text{dom}f=\left\{x\in R^n:f\left(x\right)<+\infty \right\},$

then $f\left(x\right)$ is called a proper function.

Definition 2. [19] A function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is said to be lower semicontinuous at $x\in {ℝ}^n$ if it satisfies

$f\left(x\right)\le \underset{k\to \infty }{\lim \inf }f\left(x_k\right).$

If this holds for every point in $\text{dom}f$ , then $f$ is said to be a lower semicontinuous function.

Definition 3. [20] Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function.

(i) The Fréchet subdifferential of $f$ at $x\in \text{dom}f$ , written $\widehat{\partial }f\left(x\right)$ , is the set of vectors $x^{\text{*}}\in {ℝ}^n$ that satisfy

$\underset{y\ne x}{\text{lim}}\underset{y\to x}{\text{inf}}\frac{f\left(y\right)-f\left(x\right)-\langle x^{\text{*}},y-x\rangle }{\Vert y-x\Vert }\ge 0.$

When $x\notin \text{dom}f$ , we set $\widehat{\partial }f\left(x\right)=\varnothing$ .

(ii) The limiting-subdifferential of $f$ at $x\in \text{dom}f$ , written $\partial f\left(x\right)$ , is defined as follows

$\partial f\left(x\right)=\left\{x^{\text{*}}\in {ℝ}^n,\exists x_n\to x,f\left(x_n\right)\to f\left(x\right),x_n^{\text{*}}\in \widehat{\partial }f\left(x_n\right),\text{with}{x}_n^{\text{*}}\to x^{\text{*}}\right\}.$

Lemma 1. [21] Let $g:R^m\to R$ be a continuously differentiable function and suppose that $\nabla g$ is $L$ -Lipschitz continuous. Then

$\Vert g\left(u\right)-g\left(v\right)-\langle u-v,\nabla g\left(v\right)\rangle \Vert \le \frac{L}{2}{\Vert u-v\Vert }^2,\forall u,v\in R^m.$

Definition 4. ([22], Kurdyka-Lojasiewicz inequality) Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function. For $-\infty <{\eta }_1<{\eta }_1\le +\infty$ , set

$\left[{\eta }_1<f<{\eta }_1\right]=\left\{x\in {ℝ}^n:{\eta }_1<f\left(x\right)<{\eta }_2\right\}.$

We say that function $f$ has the KL property at $x^{\text{*}}\in \text{dom}\left(\partial f\right)$ if there exist $\eta \in \left(0,+\infty \right]$ , a neighbourhood $U$ of $x^{\text{*}}$ , and a continuous concave function $\phi :\left[0,\eta \right)\to {ℝ}_{+}$ , such that

(i) $\phi \left(0\right)=0$ ;

(ii) $\phi$ is $C^1$ on $\left(0,\eta \right)$ and continuous at 0;

(iii) ${\phi }^{\prime }\left(x\right)>0,\forall x\in \left(0,\eta \right)$ ;

(iv) for all $x$ in $U\cap \left[f\left(x^{\text{*}}\right)<f<f\left(x^{\text{*}}\right)+\eta \right]$ , the Kurdyka-Lojasiewicz inequality holds

${\phi }^{\prime }\left(f\left(x\right)-f\left(x^{*}\right)\right)d\left(0,\partial f\left(x\right)\right)\ge 1,$

where $d\left(x,\partial f\left(x\right)\right)=\underset{y\in \partial f\left(x\right)}{\inf }\Vert y-x\Vert$ is the distance from $x$ to $\partial f\left(x\right)$ .

Lemma 2. ([23], Uniformized KL property) Let $\Omega$ be a compact set and $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper and lower semicontinuous function. Assume that $f$ is constant on $\Omega$ and satisfies the KL property at each point of $\Omega$ . Then, there exist $ϵ>0,\eta >0$ , and $\phi \in {\Phi }_{\eta }$ such that for all $\overline{x}\in \Omega$ and for all $x$ in the following intersection:

$\left\{x\in {ℝ}^n:d\left(x,\Omega \right)<ϵ\right\}\cap \left[f\left(\overline{x}\right)<f<f\left(\overline{x}\right)+\eta \right],$

one has

${\phi }^{\prime }\left(f\left(x\right)-f\left(\overline{x}\right)\right)d\left(0,\partial f\left(x\right)\right)\ge 1.$

Lemma 3. [24] Suppose that $F\left(x,y\right)=f\left(x\right)+g\left(x\right)$ , where $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ and $g:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ are proper lower semicontinuous functions. Then for all $\left(x,y\right)\in \text{dom}F=\text{dom}f\times \text{dom}g$ , we have

$\partial F\left(x,y\right)={\partial }_{x}F\left(x,y\right)\times {\partial }_{y}F\left(x,y\right).$

Definition 5. ([24], Kurdyka-Lojasiewicz function) If $f$ satisfies the KL property at each point of $\text{dom}\left(\partial f\right)$ , then $f$ is called a KL function.

Definition 6. $\left(x^{*},y^{*},{\lambda }^{*}\right)$ is a stationary point of the augmented Lagrangian function ${ℒ}_{\beta }^s(\cdot )$ for problem (1), if and only if

$\{\begin{array}{l}-{\nabla }_{x}H\left(x^{\text{*}},y^{\text{*}}\right)+A^{\text{T}}{\lambda }^{\text{*}}\in \partial f\left(x^{\text{*}}\right),\\ -{\nabla }_{y}H\left(x^{\text{*}},y^{\text{*}}\right)+{\lambda }^{\text{*}}=\nabla g\left(y^{\text{*}}\right),\\ A{x}^{\text{*}}+y^{\text{*}}=b.\end{array}$ (7)

Lemma 4. Let the iterative sequence generated by the algorithm (5) be denoted as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Then, the following holds

$\{\begin{array}{l}0\in \partial f\left(x^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^k\right)-A^{\text{T}}{\lambda }^{k+1}+\frac{\alpha }{\alpha +s}{A}^{\text{T}}\left({\lambda }^{k+1}-{\lambda }^k\right)\\ -\frac{s^2\beta }{\alpha +s}{A}^{\text{T}}\left(y^{k+1}-y^k\right),\\ \nabla g\left(y^{k+1}\right)={\lambda }^{k+1}-{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right),\\ {\lambda }^{k+1}={\lambda }^k-\beta \left[\left(\alpha +s\right)\left(A{x}^{k+1}+y^k-b\right)+s\left(y^{k+1}-y^k\right)\right].\end{array}$

Proof. Combining the second and fourth equations in (6) yields

${\lambda }^{k+1}={\lambda }^k-\alpha \beta \left(A{x}^{k+1}+y^k-b\right)-s\beta \left(x^{k+1}+y^{k+1}-b\right),$

${\lambda }^{k+1}={\lambda }^k-\left(\alpha +s\right)\beta \left(A{x}^{k+1}+y^k-b\right)-s\beta \left(y^{k+1}-y^k\right),$

subtracting the above two equations, we obtain

${\lambda }^{k+1}-{\lambda }^k=-\left(\alpha +s\right)\beta \left(A{x}^{k+1}+y^k-b\right)-s\beta \left(y^{k+1}-y^k\right),$ (8)

and thus

$A{x}^{k+1}+y^k-b=-\frac{1}{\left(\alpha +s\right)\beta }\left({\lambda }^{k+1}-{\lambda }^k\right)-\frac{s}{\alpha +s}\left(y^{k+1}-y^k\right),$ (9)

$A{x}^{k+1}+y^{k+1}-b=-\frac{1}{\left(\alpha +s\right)\beta }\left({\lambda }^{k+1}-{\lambda }^k\right)+\frac{\alpha }{\alpha +s}\left(y^{k+1}-y^k\right).$ (10)

According to the optimality condition (6) of the $x$ -subproblem, we have

$\begin{array}{c}0\in \partial f\left(x^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^k\right)-A^{\text{T}}{\lambda }^k+s\beta A^{\text{T}}\left(A{x}^{k+1}+y^k-b\right)\\ =\partial f\left(x^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^k\right)-A^{\text{T}}{\lambda }^{k+1}+A^{\text{T}}\left({\lambda }^{k+1}-{\lambda }^k\right)\\ +s\beta A^{\text{T}}\left(A{x}^{k+1}+y^k-b\right),\end{array}$ (11)

putting (9) into (11), we get

$\begin{array}{l}0\in \partial f\left(x^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^k\right)-A^{\text{T}}{\lambda }^{k+1}+\frac{\alpha }{\alpha +s}{A}^{\text{T}}\left({\lambda }^{k+1}-{\lambda }^k\right)\\ -\frac{s^2\beta }{\alpha +s}{A}^{\text{T}}\left(y^{k+1}-y^k\right).\end{array}$ (12)

According to the third and fourth equations in (6), we obtain

$\nabla g\left(y^{k+1}\right)={\lambda }^{k+1}-{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right).$ (13)

This completes the proof. □

3. Convergence Analysis

Assumption A. let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function, and let $g:{ℝ}^m\to ℝ$ be a continuously differentiable function with an $L_g$ -Lipschitz continuous gradient $\nabla g$ , and $H:{ℝ}^m\times {ℝ}^n\to ℝ$ be a continuously differentiable function with an $L_h$ -Lipschitz continuous gradient $\nabla H$ . Also assume that

• $\left\{\left(\alpha ,s\right)\in {ℝ}^2|\alpha <s<0\right\}$ ,

• $\left\{\beta \in ℝ|\beta >-\frac{1}{s}\left(L_g+L_h\right)\right\}$

Lemma 5. Let the iterative sequence generated by the algorithm (5) be denoted

as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Suppose that this sequence is bounded and that Assumption A holds. Then the following conclusion holds

${ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^k\right)\le -\eta \left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right),$ (14)

where $\eta >0$ (see Remark (0.1)).

Proof. The polarization identity implies

$\begin{array}{l}\frac{s\beta }{2}{\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2-\frac{s\beta }{2}{\Vert A{x}^{k+1}+y^k-b\Vert }^2\\ =-\frac{s\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2+s\beta \langle A{x}^{k+1}+y^{k+1}-b,y^{k+1}-y^k\rangle .\end{array}$ (15)

Due to $\nabla g$ is $L_g$ -Lipschitz continuous and $\nabla H$ is $L_h$ -Lipschitz continuous, we combining the lemma 1, then we have

$g\left(y^{k+1}\right)-g\left(y^k\right)\le \langle \nabla g\left(y^{k+1}\right),y^{k+1}-y^k\rangle +\frac{L}{2}{\Vert y^k-y^{k+1}\Vert }^2.$ (16)

$\begin{array}{l}H\left(x^{k+1},y^{k+1}\right)-H\left(x^{k+1},y^k\right)\\ \le \langle {\nabla }_{y}H\left(x^{k+1},y^{k+1}\right),y^{k+1}-y^k\rangle +\frac{L}{2}{\Vert y^k-y^{k+1}\Vert }^2.\end{array}$ (17)

From the fourth equation in the optimality condition (6) of the problem, we obtain

$\nabla g\left(y^{k+1}\right)={\lambda }^{k+\frac{1}{2}}-s\beta \left(A{x}^{k+1}+y^{k+1}-b\right).$ (18)

We obtain ${\lambda }^{k+1}=\nabla g\left(y^{k+1}\right)+{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)$ in (13) and combining the Lipschitz continuities of $\nabla g$ and $\nabla H$ , we have

$\begin{array}{c}{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2={\Vert \nabla g\left(y^{k+1}\right)-\nabla g\left(y^k\right)+{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)-{\nabla }_{y}H\left(x^k,y^k\right)\Vert }^2\\ \le 2{\Vert \nabla g\left(y^{k+1}\right)-\nabla g\left(y^k\right)\Vert }^2+2{\Vert {\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)-{\nabla }_{y}H\left(x^k,y^k\right)\Vert }^2\\ \le 2L_g^2{\Vert y^{k+1}-y^k\Vert }^2+2L_h^2{\Vert \left(x^{k+1},y^{k+1}\right)-\left(x^k,y^k\right)\Vert }^2\\ =2\left(L_g^2+L_h^2\right){\Vert y^{k+1}-y^k\Vert }^2+2L_h^2{\Vert x^{k+1}-x^k\Vert }^2.\end{array}$ (19)

Recall (9) and (10), we get

$\begin{array}{l}\alpha \beta {\Vert A{x}^{k+1}+y^k-b\Vert }^2+s\beta {\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2\\ =\frac{1}{\left(\alpha +s\right)\beta }{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }^2+\frac{\alpha s\beta }{\alpha +s}{\Vert y^k-y^{k+1}\Vert }^2.\end{array}$ (20)

From the definition of the augmented Lagrangian function ${ℒ}_{\beta }^s(\cdot )$ in (4), and in combination with (15), we have

$\begin{array}{l}{ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^{k+\frac{1}{2}}\right)\\ =g\left(y^{k+1}\right)-g\left(y^k\right)-\langle {\lambda }^{k+\frac{1}{2}},y^{k+1}-y^k\rangle +H\left(x^{k+1},y^{k+1}\right)-H\left(x^{k+1},y^k\right)\\ +\frac{s\beta }{2}{\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2-\frac{s\beta }{2}{\Vert A{x}^{k+1}+y^k-b\Vert }^2\\ =g\left(y^{k+1}\right)-g\left(y^k\right)-\langle {\lambda }^{k+\frac{1}{2}},y^{k+1}-y^k\rangle +H\left(x^{k+1},y^{k+1}\right)-H\left(x^{k+1},y^k\right)\\ -\frac{s\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2+s\beta \langle A{x}^{k+1}+y^{k+1}-b,y^{k+1}-y^k\rangle .\end{array}$ (21)

Then, combining it with (16) and (17) yields

$\begin{array}{l}{ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^{k+\frac{1}{2}}\right)\\ \le \langle \nabla g\left(y^{k+1}\right),y^{k+1}-y^k\rangle +\frac{L_g}{2}{\Vert y^k-y^{k+1}\Vert }^2-\langle {\lambda }^{k+\frac{1}{2}},y^{k+1}-y^k\rangle \\ +\langle {\nabla }_{y}H\left(x^{k+1},y^{k+1}\right),y^{k+1}-y^k\rangle +\frac{L_h}{2}{\Vert y^k-y^{k+1}\Vert }^2-\frac{s\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2\\ +s\beta \langle A{x}^{k+1}+y^{k+1}-b,y^{k+1}-y^k\rangle .\end{array}$ (22)

Substituting (13) into the above expression and simplifying, we obtain

${ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^{k+\frac{1}{2}}\right)\le \frac{L_g+L_h-s\beta }{2}{\Vert y^{k+1}-y^k\Vert }^2.$ (23)

Note that, by using (4), (6), (20) and (19), we have

$\begin{array}{l}{ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+1}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+\frac{1}{2}}\right)\\ +{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^k\right)\\ =\langle {\lambda }^{k+1}-{\lambda }^{k+\frac{1}{2}},A{x}^{k+1}+y^{k+1}-b\rangle +\langle {\lambda }^{k+\frac{1}{2}}-{\lambda }^k,A{x}^{k+1}+y^k-b\rangle \\ =s\beta {\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2+\alpha \beta {\Vert A{x}^{k+1}+y^k-b\Vert }^2\\ =\frac{1}{\left(\alpha +s\right)\beta }{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2+\frac{\alpha s\beta }{\alpha +s}{\Vert y^{k+1}-y^k\Vert }^2\\ \le \frac{2\left(L_g^2+L_h^2\right)+\alpha s{\beta }^2}{\left(\alpha +s\right)\beta }{\Vert y^{k+1}-y^k\Vert }^2+\frac{2L_h^2}{\left(\alpha +s\right)\beta }{\Vert x^{k+1}-x^k\Vert }^2.\end{array}$ (24)

Summing the inequalities (23), (24) and the first equation in Section (5) we obtain

$\begin{array}{l}{ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^k\right)\\ ={ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+1}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^s\left(x^{k+1},y^{k+1},{\lambda }^{k+\frac{1}{2}}\right)\\ -{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^k\right)\\ +{ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^k\right)-{ℒ}_{\beta }^s\left(x^k,y^k,{\lambda }^k\right)\\ \le \left[\frac{L_g+L_h-s\beta }{2}+\frac{2\left(L_g^2+L_h^2\right)+\alpha s{\beta }^2}{\left(\alpha +s\right)\beta }\right]{\Vert y^{k+1}-y^k\Vert }^2\\ +\frac{2L_h^2}{\left(\alpha +s\right)\beta }{\Vert x^{k+1}-x^k\Vert }^2.\end{array}$ (25)

Thus

${ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^k\right)\le -\eta \left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right),$ (26)

there $\eta :=\min \left\{\frac{s\beta -L_g-L_h}{2}-\frac{2\left(L_g^2+L_h^2\right)+\alpha s{\beta }^2}{\left(\alpha +s\right)\beta },-\frac{2L_h^2}{\left(\alpha +s\right)\beta }\right\}$ . This completes the proof. □

Remark 3.1. Thus, as long as $\eta >0$ , the sequence ${ℒ}_{\beta }^s\left(w_{k+1}\right)$ has sufficient descent properties, which means that ${ℒ}_{\beta }^s\left(w_{k+1}\right)$ is monotonically nonincreasing. Therefore, we can achieve $\eta >0$ by appropriately choosing the parameters $\left(\alpha ,s,\beta \right)$ . The constraints on these parameters are as follows:

• $\left\{\left(\alpha ,s\right)\in {ℝ}^2|\alpha <s<0\right\}$ ,

• $\left\{\beta \in ℝ|\beta >-\frac{1}{s}\left(L_g+L_h\right)\right\}$

Lemma 6. Let the iterative sequence generated by the algorithm (5) be denoted as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Suppose that this sequence is bounded, that Assumption A holds and $\eta >0$ . Then we have

${\displaystyle \sum }_{k=0}^{+\infty }{\Vert w^{k+1}-w^k\Vert }^2<+\infty .$

Proof. The boundedness of $\left\{w^k\right\}$ implies the existence of a subsequence $\left\{w^{k_j}\right\}$ converging to $w^{\text{*}}$ . Moreover, the lower semicontinuity of $f\left(x\right)$ together with the continuity of $g\left(y\right)$ ensures that ${ℒ}_{\beta }^s(\cdot )$ is lower semicontinuous. Hence,

${ℒ}_{\beta }^s\left(w^{*}\right)\le \underset{j\to +\infty }{\lim \inf }{ℒ}_{\beta }^s\left(w^{k_j}\right).$

Hence, $\left\{{ℒ}_{\beta }^s\left(w^{k_j}\right)\right\}$ is bounded below. The fact that it is also nonincreasing implies its convergence. Since $\left\{{ℒ}_{\beta }^s\left(w^k\right)\right\}$ is monotonic and contains a convergent subsequence, it follows that $\left\{{ℒ}_{\beta }^s\left(w^k\right)\right\}$ itself converges, satisfying ${ℒ}_{\beta }^s\left(w^k\right)\ge {ℒ}_{\beta }^s\left(w^{\text{*}}\right)$ . Finally, invoking Equation (14) yields

$\eta \left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right)\le {ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{k+1}\right),\forall k.$

By summing over $k=0,\cdots ,n$ , and observing that ${ℒ}_{\beta }^s\left(w^0\right)<+\infty$ , we arrive at

$\begin{array}{c}\eta {\displaystyle \sum }_{k=0}^n\left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right)\le {ℒ}_{\beta }^s\left(w^0\right)-{ℒ}_{\beta }^s\left(w^{n+1}\right)\\ \le {ℒ}_{\beta }^s\left(w^0\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)<+\infty .\end{array}$

The condition $\eta >0$ yields${\displaystyle \sum }_{k=0}^{\infty }{\Vert y^{k+1}-y^k\Vert }^2<+\infty$ and ${\displaystyle \sum }_{k=0}^{\infty }{\Vert x^{k+1}-x^k\Vert }^2<+\infty$ . Using (19), we further obtain ${\displaystyle \sum }_{k=0}^{\infty }{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2<+\infty$ . Hence, ${\displaystyle \sum }_{k=0}^{+\infty }{\Vert w^{k+1}-w^k\Vert }^2<+\infty$ . This completes the proof. □

Lemma 7. Let the iterative sequence generated by the algorithm (5) be denoted as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Suppose that this sequence is bounded and that Assumption A holds. We define

$\{\begin{array}{l}{\epsilon }_1^{k+1}=A^{\text{T}}\left({\lambda }^k-{\lambda }^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^{k+1}\right)-{\nabla }_{x}H\left(x^{k+1},y^k\right)+s\beta \left(y^{k+1}-y^k\right),\\ {\epsilon }_2^{k+1}=-\frac{s}{\alpha +s}\left({\lambda }^{k+1}-{\lambda }^k\right)+\frac{\alpha s\beta }{\alpha +s}\left(y^{k+1}-y^k\right),\\ {\epsilon }_3^{k+1}=\frac{1}{\left(\alpha +s\right)\beta }\left({\lambda }^{k+1}-{\lambda }^k\right)-\frac{\alpha }{\alpha +s}\left(y^{k+1}-y^k\right).\end{array}$ (27)

Hence, it holds that ${\epsilon }^{k+1}:=\left({\epsilon }_1^{k+1},{\epsilon }_2^{k+1},{\epsilon }_3^{k+1}\right)\in \partial {ℒ}_{\beta }^s\left(w^{k+1}\right)$ , and there exists a constant $\delta >0$ such that

$d\left(0,\partial {ℒ}_{\beta }^s\left(w^{k+1}\right)\right)\le \delta \left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right).$ (28)

Proof. From the definition of the function ${ℒ}_{\beta }^s(\cdot )$ in (4), the following system of equations holds

$\{\begin{array}{l}{\partial }_x{ℒ}_{\beta }^s\left(w^{k+1}\right)=\partial f\left(x^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^{k+1}\right)-A^{\text{T}}{\lambda }^{k+1}+s\beta A^{\text{T}}\left(A{x}^{k+1}+y^{k+1}-b\right),\\ {\partial }_y{ℒ}_{\beta }^s\left(w^{k+1}\right)=\nabla g\left(y^{k+1}\right)+{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)-{\lambda }^{k+1}+s\beta \left(A{x}^{k+1}+y^{k+1}-b\right),\\ {\partial }_{\lambda }{ℒ}_{\beta }^s\left(w^{k+1}\right)=-\left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$ (29)

From the optimality condition (6), after rearrangement, we have

$\{\begin{array}{l}{A}^{\text{T}}{\lambda }^k-{\nabla }_{x}H\left(x^{k+1},y^k\right)-s\beta A^{\text{T}}\left(A{x}^{k+1}+y^k-b\right)\in \partial f\left(x^{k+1}\right),\\ {\lambda }^{k+\frac{1}{2}}-{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)-s\beta A^{\text{T}}\left(A{x}^{k+1}+y^{k+1}-b\right)=\nabla g\left(y^{k+1}\right),\\ \frac{1}{\left(\alpha +s\right)\beta }\left({\lambda }^{k+1}-{\lambda }^k\right)-\frac{\alpha }{\alpha +s}\left(y^{k+1}-y^k\right)=-\left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$

Then, by substituting the above into (29), we obtain

$\{\begin{array}{l}{A}^{\text{T}}\left({\lambda }^k-{\lambda }^{k+1}\right)+{\nabla }_{x}H\left(x^{k+1},y^{k+1}\right)-{\nabla }_{x}H\left(x^{k+1},y^k\right)+s\beta \left(y^{k+1}-y^k\right)\in {\partial }_x{ℒ}_{\beta }^s\left(w^{k+1}\right),\\ -\frac{s}{\alpha +s}\left({\lambda }^{k+1}-{\lambda }^k\right)+\frac{\alpha s\beta }{\alpha +s}\left(y^{k+1}-y^k\right)\in {\partial }_y{ℒ}_{\beta }^s\left(w^{k+1}\right),\\ \frac{1}{\left(\alpha +s\right)\beta }\left({\lambda }^{k+1}-{\lambda }^k\right)-\frac{\alpha }{\alpha +s}\left(y^{k+1}-y^k\right)\in {\partial }_{\lambda }{ℒ}_{\beta }^s\left(w^{k+1}\right).\end{array}$

Consequently, applying Lemma 3 yields $\left({\epsilon }_1^{k+1},{\epsilon }_2^{k+1},{\epsilon }_3^{k+1}\right)\in \partial {ℒ}_{\beta }^s\left(w^{k+1}\right)$ . Based on the preceding relation, we can find ${\delta }_1,{\delta }_2$ such that

$\begin{array}{c}{\Vert {\epsilon }^{k+1}\Vert }^2={\Vert \left({\epsilon }_1^{k+1},{\epsilon }_2^{k+1},{\epsilon }_3^{k+1}\right)\Vert }^2\\ \le {\Vert {\epsilon }_1^{k+1}\Vert }^2+{\Vert {\epsilon }_2^{k+1}\Vert }^2+{\Vert {\epsilon }_3^{k+1}\Vert }^2\\ \le {\delta }_1^2{\Vert y^{k+1}-y^k\Vert }^2+{\delta }_2^2{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2.\end{array}$ (30)

Applying (19), there exists $\delta$ for which the following holds

$d^2\left(0,\partial {ℒ}_{\beta }^s\left(w^{k+1}\right)\right)\le {\Vert {\epsilon }^{k+1}\Vert }^2\le {\delta }^2\left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right),$

then

$d\left(0,\partial {ℒ}_{\beta }^s\left(w^{k+1}\right)\right)\le \delta \left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right).$ (31)

This completes the proof. □

Lemma 8. Let the iterative sequence generated by the algorithm (5) be denoted as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Suppose that this sequence is bounded and that Assumption A holds. Let $\Omega$ represent the set of all cluster points of the sequence $\left\{w^k\right\}$ . Then the following statement is true

(i) $\Omega$ is a nonempty compact set, and

$d\left(w^k,\Omega \right)\to 0,\text{as}k\to +\infty ;$

(ii) $\Omega \subset \text{crit}{ℒ}_{\beta }$ , where $\text{crit}{ℒ}_{\beta }$ denotes the set of all stationary points of ${ℒ}_{\beta }^s$ ;

(iii) ${ℒ}_{\beta }^s(\cdot )$ is finite and constant on $\Omega$ , which equals to

$\underset{k\in \mathbb{N}}{\text{inf}}{ℒ}_{\beta }^s\left(w^k\right)=\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^s\left(w^k\right).$

Proof. We now verify the above results one by one.

(i) This is immediate from the definition of limit points.

(ii) Suppose $w^{*}\in \Omega$ . Then there exists a subsequence $\left\{w^{k_j}\right\}$ of $\left\{w^k\right\}$ such that $w^{k_j}\to w^{\text{*}}$ . Applying Lemma 6 yields

$\underset{k\to +\infty }{\text{lim}}\Vert w^{k+1}-w^k\Vert =0,$ (32)

thus, $w^{k_j+1}\to w^{\text{*}}$ . Noting that $x^{k+1}$ minimize ${ℒ}_{\beta }^s\left(x,y^k,{\lambda }^k\right)$ with respect to $x$ , we get

${ℒ}_{\beta }^s\left(x^{k+1},y^k,{\lambda }^k\right)\le {ℒ}_{\beta }^s\left(x^{\text{*}},y^k,{\lambda }^k\right).$ (33)

With respect to the variables $y$ , $\lambda$ , and $\left(y^{k_j},{\lambda }^{k_j}\right)\to \left(y^{\text{*}},{\lambda }^{\text{*}}\right)$ . Hence, it follows that

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^s\left(x^{k_j+1},y^{k_j},{\lambda }^{k_j}\right)=\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^s\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right).$ (34)

Furthermore, applying (33) yields

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^s\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\le {ℒ}_{\beta }^s\left(x^{*},y^{*},{\lambda }^{*}\right).$ (35)

Since ${ℒ}_{\beta }^s(\cdot )$ is lower semicontinuous, we know that

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^s\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\ge {ℒ}_{\beta }^s\left(x^{*},y^{*},{\lambda }^{*}\right).$ (36)

From (34), (35) and (36), we obtain

$\underset{j\to +\infty }{\lim }f\left(x^{k_j+1}\right)=f\left(x^{*}\right).$

Taking the limit in (6) along the subsequence $\left\{\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\right\}$ using (32) again, we obtain

$\{\begin{array}{l}-{\nabla }_{x}H\left(x^{\text{*}},y^{\text{*}}\right)+A^{\text{T}}{\lambda }^{\text{*}}\in \partial f\left(x^{\text{*}}\right),\\ -{\nabla }_{y}H\left(x^{\text{*}},y^{\text{*}}\right)+{\lambda }^{\text{*}}=\nabla g\left(y^{\text{*}}\right),\\ A{x}^{\text{*}}+y^{\text{*}}-b=0.\end{array}$

Therefore, $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ satisfies the critical point condition of (4), which implies that $w^{\text{*}}\in \text{crit}{ℒ}_{\beta }^s$ . Thus, $\Omega \subset \text{crit}{ℒ}_{\beta }^s$ .

(iii) Take any $\left(x^{*},y^{*},{\lambda }^{*}\right)\in \Omega$ . Then there exists a subsequence $\left\{\left(x^{k_j},y^{k_j},{\lambda }^{k_j}\right)\right\}$ , such that $\left\{\left(x^{k_j},y^{k_j},{\lambda }^{k_j}\right)\right\}\to \left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ . Since ${ℒ}_{\beta }^s\left(w^k\right)$ is nonincreasing and has a convergent subsequence, the entire sequence ${ℒ}_{\beta }^s\left(w^k\right)$ converges, we have

$\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^s\left(x^k,y^k,{\lambda }^k\right)={ℒ}_{\beta }^s\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right).$

That is, ${ℒ}_{\beta }^s(\cdot )$ takes a constant value on $\Omega$ . Clearly,

$\underset{k\in \mathbb{N}}{\text{inf}}{ℒ}_{\beta }^s\left(w^k\right)=\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^s\left(w^k\right).$

This completes the proof. □

Theorem 9. Let the iterative sequence generated by the algorithm (5) be denoted as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Suppose that this sequence is bounded, that Assumption A holds and $\eta >0$ . When ${ℒ}_{\beta }^s(\cdot )$ is a KL function, then $\left\{w^k\right\}$ has finite length, that is

${\displaystyle \sum }_{k=0}^{+\infty }\Vert w^{k+1}-w^k\Vert <+\infty .$

Moreover, it converges to a critical point of ${ℒ}_{\beta }^s(\cdot )$ .

Proof. Lemma 8 implies that ${ℒ}_{\beta }^s\left(w^k\right)\to {ℒ}_{\beta }^s\left(w^{\text{*}}\right)$ for any $w^{*}\in \Omega$ . Next, we examine two cases.

(i) Suppose there exists $k_0$ with ${ℒ}_{\beta }^s\left(w^{k_0}\right)={ℒ}_{\beta }^s\left(w^{\text{*}}\right)$ , then using (14) and Remark 0.1, for every $k>k_0$ , we have

$\begin{array}{c}\eta \left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right)\le {ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{k+1}\right)\\ \le {ℒ}_{\beta }^s\left(w^{k_0}\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)=0.\end{array}$

Hence, $y^{k+1}=y^k$ and $x^{k+1}=x^k$ for any $k>k_0$ . Then, by (19), we further obtain ${\lambda }^{k+1}={\lambda }^k$ for any $k>k_0+1$ , which means $w^{k+1}=w^k$ .

(ii) If ${ℒ}_{\beta }^s\left(w^k\right)>{ℒ}_{\beta }^s\left(w^{\text{*}}\right)$ holds for all $k$ , then the following convergence properties hold:

• Since $d\left(w^k,\Omega \right)\to 0$ , for any ${\epsilon }_1>0$ there exists $k_1>0$ such that for all $k>k_1$ , it holds $d\left(w^k,\Omega \right)<{\epsilon }_1$ is true.

• Since ${ℒ}_{\beta }^s\left(w^k\right)\to {ℒ}_{\beta }^s\left(w^{\text{*}}\right)$ , for any ${\epsilon }_2>0$ there exists $k_2>0$ such that for all $k>k_2$ , it holds that${ℒ}_{\beta }^s\left(w^k\right)<{ℒ}_{\beta }^s\left(w^{\text{*}}\right)+{\epsilon }_2$ is true.

Now set $k>\tilde{k}=\text{max}\left\{k_1,k_2\right\}$ and any ${\epsilon }_1,{\epsilon }_2>0$ , we have

$d\left(w^k,\Omega \right)<{\epsilon }_1,{ℒ}_{\beta }^s\left(w^{*}\right)<{ℒ}_{\beta }^s\left(w^k\right)<{ℒ}_{\beta }^s\left(w^{*}\right)+{\epsilon }_2.$

By Lemma 8, we have established that $\Omega$ is a nonempty compact set and that ${ℒ}_{\beta }^s(\cdot )$ is constant $\Omega$ . Consequently, Lemma 2 implies that

${\phi }^{\prime }\left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)d\left(0,\partial {ℒ}_{\beta }^s\left(w^k\right)\right)\ge 1,\forall k>\tilde{k}.$(37)

Drawing on what has been clearly established, namely that

${ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{k+1}\right)={ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)-\left({ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right),$

and the concavity of $\phi (\cdot )$ , it follows that

$\begin{array}{l}\phi \left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right)\\ \ge {\phi }^{\prime }\left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right)\left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{k+1}\right)\right).\end{array}$

Now, taking the above inequality together with

$d\left(0,\partial {ℒ}_{\beta }^s\left(w^k\right)\right)\le \xi \Vert y^k-y^{k-1}\Vert ,{\phi }^{\prime }\left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)>0,$

and relation (37), it follows that

$\begin{array}{l}{ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{k+1}\right)\\ \le \frac{\phi \left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)-\phi \left({ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)}{{\phi }^{\prime }\left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)}\\ \le d\left(0,\partial {ℒ}_{\beta }^s\left(w^k\right)\right)\left[\phi \left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)-\phi \left({ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)\right]\\ \le \delta \left(\Vert x^k-x^{k-1}\Vert +\Vert y^k-y^{k-1}\Vert \right)\left[\phi \left({ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)-\phi \left({ℒ}_{\beta }^s\left(w^{k+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)\right].\end{array}$

For convenience, we define

${\text{Δ}}_{p,q}:=\phi \left({ℒ}_{\beta }^s\left(w^p\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^s\left(w^q\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right).$

Then the above equation is equivalent to

${ℒ}_{\beta }^s\left(w^k\right)-{ℒ}_{\beta }^s\left(w^{k+1}\right)\le \delta \left(\Vert x^k-x^{k-1}\Vert +\Vert y^k-y^{k-1}\Vert \right){\text{Δ}}_{k,k+1}.$ (38)

Together, Lemma 5 and (38), imply that

$\eta \left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right)\le \delta \left(\Vert x^k-x^{k-1}\Vert +\Vert y^k-y^{k-1}\Vert \right){\text{Δ}}_{k,k+1},k>\tilde{k},$

together with

$\frac{1}{2}{\left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right)}^2\le \left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2\right),$

and thereby

$2\left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right)\le 2\sqrt{\Vert x^k-x^{k-1}\Vert +\Vert y^k-y^{k-1}\Vert }\sqrt{\frac{2\delta }{\eta }{\text{Δ}}_{k,k+1}},k>\tilde{k}.$

Using the fact that $2\sqrt{\alpha \beta }\le \alpha +\beta$ , we have

$2\left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right)\le \Vert x^k-x^{k-1}\Vert +\Vert y^k-y^{k-1}\Vert +\frac{2\delta }{\eta }{\text{Δ}}_{k,k+1}.$ (39)

Taking the sum of (39) over $k=\tilde{k}+1,\cdots ,N$ gives

$2{\displaystyle \sum }_{k=\tilde{k}+1}^N\left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right)\le {\displaystyle \sum }_{k=\tilde{k}+1}^N\left(\Vert x^k-x^{k-1}\Vert +\Vert y^k-y^{k-1}\Vert \right)+\frac{2\delta }{\eta }{\text{Δ}}_{\tilde{k}+1,N+1}.$

From Definition, we have $\phi \left({ℒ}_{\beta }^s\left(w^{N+1}\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right)>0$ from Definition 4. Rearranging terms and taking $N\to +\infty$ gives

$\begin{array}{l}{\displaystyle \sum }_{k=\tilde{k}+1}^{+\infty }\left(\Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert \right)\\ \le \Vert x^{\tilde{k}+1}-x^{\tilde{k}}\Vert +\Vert y^{\tilde{k}+1}-y^{\tilde{k}}\Vert +\frac{2\delta }{\eta }\phi \left({ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^s\left(w^{\text{*}}\right)\right).\end{array}$(40)

Thus,

${\displaystyle \sum }_{k=0}^{+\infty }\Vert y^{k+1}-y^k\Vert <+\infty ,$ (41)

and

${\displaystyle \sum }_{k=0}^{+\infty }\Vert x^{k+1}-x^k\Vert <+\infty .$ (42)

Substituting into (19) yields

${\displaystyle \sum }_{k=0}^{+\infty }\Vert {\lambda }^{k+1}-{\lambda }^k\Vert <+\infty .$ (43)

Additionally, we note that

$\begin{array}{c}\Vert w^{k+1}-w^k\Vert ={\left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2+{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2\right)}^{1/2}\\ \le \Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert +\Vert {\lambda }^{k+1}-{\lambda }^k\Vert .\end{array}$

From (41), (42) and (43), we arrive at

${\displaystyle \sum }_{k=0}^{+\infty }\Vert w^{k+1}-w^k\Vert <+\infty .$

Finally, $\left\{w^k\right\}$ converges to a critical point of ${ℒ}_{\beta }^s(\cdot )$ by Lemma 8. This completes the proof. □

Lemma 10. Let $\left\{w^k:=\left(x^k,y^k,{\lambda }^k\right)\right\}$ be the sequence generated by the symmetric ADMM (5), If at least one of the following statements holds

(i) $\underset{\Vert x\Vert \to +\infty }{\lim \inf }f\left(x\right)=+\infty$ .

(ii) $\underset{x}{\text{inf}}f\left(x\right)>-\infty$ and $\underset{\Vert y\Vert \to +\infty }{\lim \inf }g\left(y\right)=+\infty$ .

Then we can conclude that the sequence $\left\{w^k:=\left(x^k,y^k,{\lambda }^k\right)\right\}$ is bounded.

Proof. In the first place, suppose that condition (i) holds. Based on Lemma 5, we arrive at

${ℒ}_{\beta }^s\left(x^k,y^k,{\lambda }^k\right)\le {ℒ}_{\beta }^s\left(x^1,y^1,{\lambda }^1\right).$

By combining (4) with $\nabla g\left(y^{k+1}\right)={\lambda }^{k+1}-{\nabla }_{y}H\left(x^{k+1},y^{k+1}\right)$ , we get

$\begin{array}{c}{ℒ}_{\beta }^s\left(x^1,y^1,{\lambda }^1\right)\ge f\left(x^k\right)+g\left(y^k\right)+H\left(x^k,y^k\right)-\langle {\lambda }^k,A{x}^k+y^k-b\rangle \\ +\frac{s\beta }{2}{\Vert A{x}^k+y^k-b\Vert }^2\\ =f\left(x^k\right)+g\left(y^k\right)+H\left(x^k,y^k\right)\\ -\langle \nabla g\left(y^k\right)+{\nabla }_{y}H\left(x^k,y^k\right),A{x}^k+y^k-b\rangle +\frac{s\beta }{2}{\Vert A{x}^k+y^k-b\Vert }^2\\ \ge f\left(x^k\right)+\left(\frac{L_g}{2}+\frac{L_h}{2}+\frac{s\beta }{2}\right){\Vert A{x}^k+y^k-b\Vert }^2.\end{array}$

Note that (i) implies that $\underset{x}{\text{inf}}f\left(x\right)>-\infty$ . Based on Assumption A, we can obtain $\frac{L_g}{2}+\frac{L_h}{2}+\frac{s\beta }{2}>0$ . Thus, we can deduce that $\left\{x^k\right\}$ and $\left\{y^k\right\}$ are bounded. Hence, $\left\{{\lambda }^k\right\}$ is also bounded, and thus $\left\{w^k\right\}$ is bounded. This completes the proof. □

Theorem 11. (Convergence rate) Let the iterative sequence generated by the algorithm (5) be denoted as $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}$ . Suppose that this sequence is bounded, that Assumption A holds and $\eta >0$ , and $\left\{w^k:\left(x^k,y^k,{\lambda }^k\right)\right\}\to \left\{w^{*}=\left(x^{*},y^{*},{\lambda }^{*}\right)\right\}$ . Assume that ${ℒ}_{\beta }^s(\cdot )$ possesses the KL property at $\left(x^{*},y^{*},{\lambda }^{*}\right)$ , and that the corresponding function is given by $\phi \left(s\right)=c{t}^{1-\theta }$ , with $\theta \in \left[0,1\right)$ , $c>0$ . Then the following three statements hold

(i) If $\theta =0$ , the sequence $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ converges in finitely many steps. This means we can find an index $k$ with $w^k=w^{*}$ .

(ii) If $\theta \in \left(0,\frac{1}{2}\right]$ , then there is a constant $c_1>0$ and $\tau \in \left[0,1\right)$ so that

$\Vert \left(x^k,y^k,{\lambda }^k\right)-\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\Vert \le c_1{\tau }^k.$

(iii) If $\theta \in \left(\frac{1}{2},1\right)$ , then there is a constant $c_2>0$ for which

$\Vert \left(x^k,y^k,{\lambda }^k\right)-\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\Vert \le c_2{k}^{\left(\theta -1\right)/\left(2\theta -1\right)}.$

Proof. For $\theta =0$ , we have $\phi \left(t\right)=ct$ and ${\phi }^{\prime }\left(t\right)=c$ . Suppose, contrary to the claim, that $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ does not terminate in finitely many steps. Then, for large enough $k$ , the KL property yields $cd\left(0,\partial {ℒ}_{\beta }^s\left(w^k\right)\right)\ge 1$ , which contradicts Lemma 7. Now let $\theta >0$ and set ${\text{Δ}}_k={\displaystyle \sum }_{i=k}^{+\infty }\left(\Vert x^{i+1}-x^i\Vert +\Vert y^{i+1}-y^i\Vert \right)$ for $k\ge 0$ . From (40) we deduce

${\Delta }_{\tilde{k}+1}\le {\Delta }_{\tilde{k}}-{\Delta }_{\tilde{k}+1}+\frac{2\delta }{\eta }\phi \left({ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right).$ (44)

Because ${ℒ}_{\beta }^s(\cdot )$ has the KL property at $w^{*}$ , we have

${\phi }^{\prime }\left({ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)d\left(0,\partial {ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)\right)\ge 1.$

This inequality can be rearranged as

${\left({ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)}^{\theta }\le c\cdot \left(1-\theta \right)d\left(0,\partial {ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)\right).$ (45)

Finally, applying Lemma 7 yields

$d\left(0,\partial {ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)\right)\le \delta \left(\Vert x^{\tilde{k}+1}-x^{\tilde{k}}\Vert +\Vert y^{\tilde{k}+1}-y^{\tilde{k}}\Vert \right)=\delta \left({\Delta }_{\tilde{k}}-{\Delta }_{\tilde{k}+1}\right).$ (46)

From (45) and (46), there exists $\gamma ={\left[c\left(1-\theta \right)\delta \right]}^{\frac{1-\theta }{\theta }}>0$ satisfying

$\phi \left({ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)=c\cdot {\left({ℒ}_{\beta }^s\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^s\left(w^{*}\right)\right)}^{1-\theta }\le \gamma {\left({\Delta }_{\tilde{k}}-{\Delta }_{\tilde{k}+1}\right)}^{\left(1-\theta \right)/\theta }.$

Inserting this into (44) gives

${\Delta }_{\tilde{k}+1}\le {\Delta }_{\tilde{k}}-{\Delta }_{\tilde{k}+1}+\frac{2\delta }{\eta }\gamma {\left({\Delta }_{\tilde{k}}-{\Delta }_{\tilde{k}+1}\right)}^{\left(1-\theta \right)/\theta }.$ (47)

Now, by (47) and the results of Attouch and Bolte [25], we obtain

• Case 1: $\theta \in \left(0,\frac{1}{2}\right]$ , then there exist $c_1>0$ and $\tau \in \left[0,1\right)$ such that

$\Vert x^k-x^{\text{*}}\Vert +\Vert y^k-y^{\text{*}}\Vert \le c_1{\tau }^k.$ (48)

• Case 2: $\theta \in \left(\frac{1}{2},1\right)$ , then there exists $c_2>0$ such that

$\Vert x^k-x^{\text{*}}\Vert +\Vert y^k-y^{\text{*}}\Vert \le c_2{k}^{\frac{\theta -1}{2\theta -1}}.$ (49)

Next, applying (19) yields

$\begin{array}{c}\Vert {\lambda }^k-{\lambda }^{\text{*}}\Vert \le \sqrt{2}{\left[L_g^2{\Vert y^k-y^{\text{*}}\Vert }^2+L_h^2{\Vert y^k-y^{\text{*}}\Vert }^2+L_h^2{\Vert x^k-x^{\text{*}}\Vert }^2\right]}^{\frac{1}{2}}\\ \le \sqrt{2}\left[L_g\Vert y^k-y^{\text{*}}\Vert +L_h^2\Vert y^k-y^{\text{*}}\Vert +L_h\Vert x^k-x^{\text{*}}\Vert \right].\end{array}$ (50)

Thus, (ii) and (iii) follow from (48)-(50).

4. Conclusion

In this paper, we propose a symmetric alternating direction method of multipliers with two different relaxation factors for minimizing the sum of two non-separable nonconvex functions. For problems where the objective function contains a coupling term, i.e., the case where $f$ and $g$ are non-separable, research remains limited in both convex and non-convex settings. We review the development of symmetric ADMM for solving non-separable problems and find that although many existing works have proposed symmetric ADMM variants incorporating techniques such as Bregman distances, inertial terms, regularization terms, or linearization, they often introduce only one relaxation factor. Inspired by this, we introduce two different relaxation factors and apply the algorithm to non-separable nonconvex problems, thereby refining the basic form of symmetric ADMM for solving non-separable problems. This makes the parameter range of the algorithm broader, allowing it to be adapted to more practical problems by adjusting the parameters. It also provides fundamental theoretical support for further integration with other techniques. Finally, based on the Kurdyka-Łojasiewicz (KL) property, we prove that the sequence generated by the algorithm converges to a stationary point of the problem and further analyze its finite-step convergence, linear convergence, and sublinear convergence.

Acknowledgements

Sincere thanks to the members of JAMP for their professional performance, and special thanks to managing editor Hellen XU for a rare attitude of high quality.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Shang, F., Xu, T., Liu, Y., Liu, H., Shen, L. and Gong, M. (2021) Differentially Private ADMM Algorithms for Machine Learning. IEEE Transactions on Information Forensics and Security, 16, 4733-4745.[CrossRef]
[2]	Chan, S.H. (2019) Performance Analysis of Plug-And-Play ADMM: A Graph Signal Processing Perspective. IEEE Transactions on Computational Imaging, 5, 274-286.[CrossRef]
[3]	Xu, H., Caramanis, C. and Mannor, S. (2013) Outlier-robust PCA: The High-Dimensional Case. IEEE Transactions on Information Theory, 59, 546-572.[CrossRef]
[4]	Glowinski, R. and Marroco, A. (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’automatique, informatique, recherche opérationnelle. Analyse numérique, 9, 41-76.[CrossRef]
[5]	Douglas, J. and Rachford, H.H. (1956) On the Numerical Solution of Heat Conduction Problems in Two and Three Space Variables. Transactions of the American Mathematical Society, 82, 421-439.[CrossRef]
[6]	He, B., Liu, H., Wang, Z. and Yuan, X. (2014) A Strictly Contractive Peaceman—Rachford Splitting Method for Convex Programming. SIAM Journal on Optimization, 24, 1011-1040.[CrossRef] [PubMed]
[7]	He, B., Ma, F. and Yuan, X. (2016) Convergence Study on the Symmetric Version of ADMM with Larger Step Sizes. SIAM Journal on Imaging Sciences, 9, 1467-1501.[CrossRef]
[8]	Jia, Z., Gao, X., Cai, X. and Han, D. (2021) The Convergence Rate Analysis of the Symmetric ADMM for the Nonconvex Separable Optimization Problems. Journal of Industrial & Management Optimization, 17, 1943-1971.[CrossRef]
[9]	Liu, P., Jian, J., He, B. and Jiang, X. (2022) Convergence of Bregman Peaceman-Rachford Splitting Method for Nonconvex Nonseparable Optimization. Journal of the Operations Research Society of China, 11, 707-733.[CrossRef]
[10]	Gao, X. and Zhang, S. (2016) First-Order Algorithms for Convex Optimization with Nonseparable Objective and Coupled Constraints. Journal of the Operations Research Society of China, 5, 131-159.[CrossRef]
[11]	Chen, C., Li, M., Liu, X. and Ye, Y. (2017) Extended ADMM and BCD for Nonseparable Convex Minimization Models with Quadratic Coupling Terms: Convergence Analysis and Insights. Mathematical Programming, 173, 37-77.[CrossRef]
[12]	Guo, K. and Han, D.R. and Wu, T.T. (2018) Convergence Analysis for Optimization Problems with Nonseparable Nonconvex Objective and Linear Constraints. Pacific Journal of Optimization, 14, 489-506. https://webofscience.clarivate.cn/wos/woscc/full-record/WOS:000461411400008
[13]	Guo, K. and Wang, X. (2018) Convergence of Generalized Alternating Direction Method of Multipliers for Nonseparable Nonconvex Objective with Linear Constraints. Journal of Mathematical Research with Applications, 38, 18. https://d.wanfangdata.com.cn/periodical/sxyjypl201805010
[14]	Liu, Q., Shen, X. and Gu, Y. (2019) Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis. IEEE Access, 7, 76131-76144.[CrossRef]
[15]	Wu, Z., Li, M., Wang, D.Z.W. and Han, D. (2017) A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems. Asia-Pacific Journal of Operational Research, 34, Article ID: 1750030.[CrossRef]
[16]	Dang, Y.Z. and Cui, T.T. (2023) Convergence Analysis of Linear Symmetric Proximal ADMM for Nonconvex Nonsmooth Nonseparable Optimization. Journal of Systems Science and Mathematical Sciences, 43, 2949-2969. (In Chinese)http://dx.chinadoi.cn/10.12341/jssms23295[CrossRef]
[17]	Dang, Y. and Liu, K. (2026) A Two-Step Inertial Bregman Symmetric Admm-Type Algorithm with Kl-Property for Nonconvex Nonsmooth Nonseparable Optimization Problems with Application. Journal of Computational and Applied Mathematics, 472, Article ID: 116815.[CrossRef]
[18]	Mei, L. and Ke, G. (2026) Convergence Analysis of the Symmetric Alternating Direction Method of Multipliers for Two-Block Separable Nonconvex Optimization Problems with Linear Constraints. Pacific Journal of Optimization.[CrossRef]
[19]	Beck, A. (2017) First-Order Methods in Optimization. Society for Industrial and Applied Mathematics.[CrossRef]
[20]	Guo, K., Han, D.R. and Wu, T.T. (2016) Convergence of Alternating Direction Method for Minimizing Sum of Two Nonconvex Functions with Linear Constraints. International Journal of Computer Mathematics, 94, 1653-1669.[CrossRef]
[21]	Nesterov, Y. (2013) Introductory Lectures on Convex Optimization: A Basic Course, Springer Science and Business Media.
[22]	Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010) Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality. Mathematics of Operations Research, 35, 438-457.[CrossRef]
[23]	Bolte, J., Sabach, S. and Teboulle, M. (2013) Proximal Alternating Linearized Minimization for Nonconvex and Nonsmooth Problems. Mathematical Programming, 146, 459-494.[CrossRef]
[24]	Attouch, H., Bolte, J. and Svaiter, B.F. (2011) Convergence of Descent Methods for Semi-Algebraic and Tame Problems: Proximal Algorithms, Forward-Backward Splitting, and Regularized Gauss-Seidel Methods. Mathematical Programming, 137, 91-129.[CrossRef]
[25]	Attouch, H. and Bolte, J. (2007) On the Convergence of the Proximal Algorithm for Nonsmooth Functions Involving Analytic Features. Mathematical Programming, 116, 5-16.[CrossRef]