^{1}

^{1}

We propose a generalized shift-splitting iteration method for saddle point problems with block three-by-three structure. As a new iteration method, the method converges to the unique solution of the saddle point problem unconditionally. When exploited as a preconditioner, the spectral distribu-tion of the preconditioned matrix is investigated. Numerical experiments show that the new variant is efficient in speeding up GMRES for solving the block three-by-three saddle point problem.

We consider the saddle point problems with the following block three-by-three structure

[ A B T 0 − B 0 − C T 0 C 0 ] [ x y z ] = [ f g h ] or A u = b , (1)

where A ∈ ℝ n × n is a symmetric positive definite matrix, B ∈ ℝ m × n and C ∈ ℝ s × m are matrices of full row rank, f ∈ ℝ n , g ∈ ℝ m and h ∈ ℝ s are given vectors. The linear system of equations (1) has many practical applications in scientific computing and engineering fields, including the constrained quadratic optimization problem, the constrained least squares problem and the computational fluid dynamics [

In practice, the coefficient matrix A in (1) is generally sparse and of large size. Therefore, iterative methods are usually preferred for their low memory requirement per iteration. Among these candidates, the Uzawa-type methods [

In [

A = M ( α ) − N ( α ) = 1 2 ( α I + A ) − 1 2 ( α I − A ) ,

which leads naturally to the shift-splitting iteration scheme

u k + 1 = M − 1 ( α ) N ( α ) u k + M − 1 ( α ) b , k = 0 , 1 , 2 , ⋯

Theoretical analysis shows that shift-splitting iteration is endowed with an unconditional convergence. By using this sound convergence property, Cao et al. come up with the shift-splitting to solve the saddle point problems and propose a shift-splitting preconditioner and a local shift-splitting variant [

By taking into account the special structure of (1), Cao [

The organization of this paper is as follows. In Section 2, we introduce the generalized shift-splitting iteration method for solving the block three-by-three saddle point problem and look into its convergence property. In Section 3, we obtain a GSS preconditioner derived from the GSS iteration and analyze the spectrum of the preconditioned matrix. In Section 4, some numerical experiments are carried out to illustrate the effectiveness of GSS by comparing it with other shift-splitting preconditioners given in [

The proposed generalized shift-splitting (GSS) iteration is based on the work of [

In its simplest form, the shift splitting (SS) of the coefficient matrix A in (1) goes as follows

A = 1 2 ( α I + A ) − 1 2 ( α I − A ) = 1 2 [ α I + A B T 0 − B α I − C T 0 C α I ] − 1 2 [ α I − A − B T 0 B α I C T 0 − C α I ] , (2)

where I is the identity matrix with conforming size. The parameter α in (2) is used to monitor the convergence of the SS iteration method and hence the spectrum of the preconditioned matrix. In other words, α plays the same role in adjusting the two block diagonal matrices in (2), i.e.,

[ A B T − B 0 ] ∈ ℝ ( m + n ) × ( m + n ) and 0 ∈ ℝ s × s . (3)

However, the reason for choosing a single parameter to control simultaneously these two block diagonal matrices above remains elusive. Instead, it seems more appealing to investigate the convergence if the two block diagonal matrices are affected by two different parameters separately. This is the main motivation of our work. To this end, we refine the splitting (2) by embedding an extra parameter β in the ( 3,3 ) -block, which yields the following generalized shift- splitting

A = M ( α , β ) − N ( α , β ) = 1 2 ( Ω + A ) − 1 2 ( Ω − A ) = 1 2 [ α I + A B T 0 − B α I − C T 0 C β I ] − 1 2 [ α I − A − B T 0 B α I C T 0 − C β I ] . (4)

Here α > 0 , β > 0 and Ω = [ α I α I β I ] . Then it is trivial to find that the generalized shift-splitting (4) reduces to the shift-splitting (2) if α = β .

Using the splitting (4), we obtain the following stationary iteration

u k + 1 = Γ G S S u k + c , (5)

where Γ G S S = M − 1 ( α , β ) N ( α , β ) , c = M − 1 ( α , β ) b and k = 0,1, ⋯ . To put it another way, the generalized shift-splitting method (GSS) for the block three-by- three saddle point problem (1) that can be stated as follows. Given an initial vector u 0 , compute

1 2 [ α I + A B T 0 − B α I − C T 0 C β I ] u k + 1 = 1 2 [ α I − A − B T 0 B α I C T 0 − C β I ] u k + [ f g h ] (6)

until u k converges, where k = 0,1, ⋯ .

It is known that the GSS iteration method (5) enjoys the unconditional convergence if the spectral radius of the iteration matrix Γ G S S is less than 1. The following results are adapted from [

Lemma 1. Suppose that A ∈ ℝ n × n is a symmetric positive definite matrix, B ∈ ℝ m × n and C ∈ ℝ s × m are of full row rank. Let λ be an eigenvalue of the saddle point matrix A , then

R e ( λ ) > 0.

Proof. The proof can be found in ( [

Lemma 2. Let W ∈ ℝ n × n be a non-Hermitian positive semidefinite matrix and E ( W , δ ) = ( δ I + W ) − 1 ( δ I − W ) be the extrapolated Cayley transform with δ > 0 . Then

1) the spectral radius of E ( W , δ ) is bounded by 1, i.e., ρ ( E ( W , δ ) ) ≤ 1 ;

2) ρ ( E ( W , δ ) ) = 1 if and only if the matrix W has a (reducing) eigenvalue of the form ι ξ with ξ ∈ ℝ and ι the imaginary unit.

Proof. See ( [

The following theorem on the convergence of GSS iteration method comes out as a result of Lemma 1 and Lemma 2.

Theorem 1. Suppose that A ∈ ℝ n × n is a symmetric positive definite matrix, B ∈ ℝ m × n and C ∈ ℝ s × m are matrices of full row rank. Let α > 0 and β > 0 . Then the spectral radius of Γ G S S defined in (5) satisfies

ρ ( Γ G S S ) < 1,

i.e., the GSS iteration method converges to the unique solution of (1) unconditionally.

Proof. By (4) and (5), we rewrite the iteration matrix Γ G S S as

Γ G S S = ( Ω + A ) − 1 ( Ω − A ) = Ω − 1 2 ( I + Ω − 1 2 A Ω − 1 2 ) − 1 ( I − Ω − 1 2 A Ω − 1 2 ) Ω 1 2 .

Let A ^ = Ω − 1 2 A Ω − 1 2 . It is easy to see that Γ G S S is similar to the matrix Γ ^ G S S = ( I + � ^ ) − 1 ( I − � ^ ) . Therefore, it suffices to prove ρ ( Γ ^ G S S ) < 1 . By the construction of A ^ = Ω − 1 2 A Ω − 1 2 , we have

A ^ = [ α I α I β I ] − 1 2 [ A B T 0 − B 0 − C T 0 C 0 ] [ α I α I β I ] − 1 2 = [ α − 1 A α − 1 B T 0 − α − 1 B 0 − ( α β ) − 1 2 C T 0 ( α β ) − 1 2 C 0 ] .

Since α and β are positive, then α − 1 A is a symmetric positive definite matrix, α − 1 B and ( α β ) − 1 2 C are matrices of full row rank. Thus A ^ is a

nonsymmetric positive semi-definite matrix. Moreover, it follows from Lemma 1 that R e ( λ ( � ^ ) ) > 0 . Therefore, it can be deduced from Lemma 2 that ρ ( Γ ^ G S S ) ≤ 1 . The equality ρ ( Γ ^ G S S ) = 1 holds if and only if A ^ has a (reducing) eigenvalue of the form ι ξ , which is impossible because R e ( λ ( A ^ ) ) > 0 . It turns out that ρ ( Γ ^ G S S ) < 1 . As a result, ρ ( Γ G S S ) < 1 which completes the proof.

Though the GSS iteration method is proved to converge unconditionally, its convergence can be rather slow when compared with the involved Krylov subspace methods [

In this section, we present the following generalized shift-splitting preconditioner (GSS)

P G S S = 1 2 [ α I + A B T 0 − B α I − C T 0 C β I ] . (7)

When applying a preconditioner, the convergence of the preconditioned linear system is closely related with the eigenvalue distribution. The following finding shows that the eigenvalues of GSS-preconditioned matrix A P G S S − 1 cluster in a circle centered at the point ( 1,0 ) with radius less than 1. We adopt the right preconditioning here such that the residual of the original saddle point problem is the same as that of its right-preconditioned counterpart.

Theorem 2. Suppose that A ∈ ℝ n × n is a symmetric positive definite matrix, B ∈ ℝ m × n and C ∈ ℝ s × m are matrices of full row rank. Let θ be an eigenvalue of A P G S S − 1 , then

| 1 − θ | < 1.

Proof. Since the preconditioned matrix A P G S S − 1 is similar to P G S S − 1 A , then it is only necessary to investigate the spectrum of P G S S − 1 A . Since P G S S = M ( α , β ) , then it follows from (4) that

P G S S − 1 A = P G S S − 1 ( M ( α , β ) − N ( α , β ) ) = M ( α , β ) − 1 ( M ( α , β ) − N ( α , β ) ) = I − Γ G S S .

From Theorem 1, we know that | 1 − θ | < 1 , where θ is an eigenvalue of P G S S − 1 A .

Before ending this section, we give some implementation details when applying the GSS preconditioner. In actual computations, a matrix-vector product P G S S − 1 r is often needed which can be solved from the linear system P G S S z = r . Note that the preconditioner P G S S can be decomposed as

P G S S = 1 2 [ I B T ( α I + 1 β C T C ) − 1 0 0 I − 1 β C T 0 0 I ] ⋅ [ α I + A + B T ( α I + 1 β C T C ) − 1 B 0 0 0 α I + 1 β C T C 0 0 0 β I ] ⋅ [ I 0 0 − ( α I + 1 β C T C ) − 1 B I 0 0 1 β C I ] . (8)

Let r = [ r 1 T , r 2 T , r 3 T ] T and z = [ z 1 T , z 2 T , z 3 T ] T , where r 1 , z 1 ∈ ℝ n , r 2 , z 2 ∈ ℝ m and r 3 , z 3 ∈ ℝ s . By using the decomposition (8), we obtain the following subroutine for solving z = P G S S − 1 r .

1) Solve w 1 from ( α I + 1 β C T C ) w 1 = 2 r 2 + 2 β C T r 3 ;

2) Solve z 1 from ( α I + A + B T ( α I + 1 β C T C ) − 1 B ) z 1 = 2 r 1 − B T w 1 ;

3) Solve w 2 from ( α I + 1 β C T C ) w 2 = B z 1 ;

4) Update z 2 = w 1 + w 2 ;

5) Update z 3 = 1 β ( 2 r 3 − C z 2 ) .

In computing P G S S − 1 r , the computational overhead by GSS is the same as that by SS [

In this section, we give some numerical experiments to illustrate the effectiveness of the new preconditioner GSS in terms of the number of iteration steps (iter), CPU time in seconds (time) and the relative residual norm (res). For completeness, we display both the number of restarts and the number of inner steps in the last restart; see, for instance,

The testing problem is adapted from [

A = [ I ⊗ T + T ⊗ I 0 0 I ⊗ T + T ⊗ I ] ∈ ℝ 2 p 2 × 2 p 2 , B = [ I ⊗ F F ⊗ I ] ∈ ℝ p 2 × 2 p 2 , C = E ⊗ F ∈ ℝ p 2 × p 2 ,

where T = 1 h 2 ⋅ tridiag ( − 1,2, − 1 ) ∈ ℝ p × p , F = 1 h ⋅ tridiag ( 0,1, − 1 ) ∈ ℝ p × p , E = diag ( 1, p + 1, ⋯ , p 2 − p + 1 ) with ⊗ being the Kronecker product and h = 1 p + 1 the grid size.

As mentioned in Section 2, the main motivation for embracing an extra parameter β in GSS is to strike a balance in controlling the sub-matrices in (3). A proper choice of α and β will make the GSS preconditioner more appealing. As shown in (7), the GSS preconditioner P G S S becomes closer to (half of) the saddle point matrix A as α and β decrease. This implies that the choice α = β = 0 seems to be the optimal one. However, we do not suggest using extremely small values of α and β for numerical concerns. Instead, we use the experimental (optimal) values of α and β in the following experiments.

To this end, we first tabulate the CPU time of the GSS preconditioner with varying α and β for p = 16 and p = 32 in ^{1} and CPU time of GSS with varying β when α is chosen to be 0.01, 0.1 and 1, respectively. Some remarks are in order. Loosely speaking, for a fixed value of α , the number of total iteration steps and CPU time increase as β goes up although there are some zigzags; see

Based on the above experiments, we use ( α , β ) = ( 0.01,0.001 ) as the experimental optimal choice for α and β in GSS. As for the optimal value of α

α time β | 0.001 | 0.01 | 0.1 | 1 | 5 | 10 | |||||
---|---|---|---|---|---|---|---|---|---|---|---|

0.001 | 0.0391 | 0.0306 | 0.0535 | 0.0704 | 0.0921 | 0.1283 | |||||

0.01 | 0.0336 | 0.0550 | 0.0512 | 0.0854 | 0.1321 | 0.1473 | |||||

0.1 | 0.0506 | 0.0515 | 0.0675 | 0.1122 | 0.1507 | 0.2250 | |||||

1 | 0.0739 | 0.0612 | 0.0886 | 0.1484 | 0.2838 | 0.4180 | |||||

5 | 0.1056 | 0.1041 | 0.1393 | 0.2813 | 0.7082 | 1.3993 | |||||

10 | 0.1443 | 0.1560 | 0.1744 | 0.4298 | 0.9397 | 2.1133 | |||||

in SS and RSS, we follow the choice α = 0.01 used in [

In this subsection, we compare the numerical performance of GSS with SS and

α time β | 0.001 | 0.01 | 0.1 | 1 | 5 | 10 |
---|---|---|---|---|---|---|

0.001 | 1.4348 | 1.3374 | 1.8127 | 2.5649 | 3.1475 | 4.3422 |

0.01 | 1.3786 | 2.1595 | 1.8248 | 2.7657 | 3.7720 | 4.5090 |

0.1 | 2.3747 | 2.0354 | 2.4876 | 3.1519 | 5.4760 | 7.0319 |

1 | 2.5129 | 2.5942 | 3.2291 | 5.2169 | 10.1217 | 15.6771 |

5 | 3.7049 | 3.8040 | 5.1990 | 9.5487 | 24.7657 | 47.8987 |

10 | 4.4818 | 4.4699 | 5.6668 | 13.9554 | 35.1132 | 74.5482 |

RSS regarding CPU time and iteration steps. It should be noted that all sub-linear equations involved in computing matrix-vector product P − 1 r are solved by direct methods.

It is known that the spectrum imposes a great effect on the convergence of the preconditioned linear system. To begin with the discussion, we plot respectively the spectral distributions of the original saddle point matrix A , the GSS-pre- conditioned matrix A P G S S − 1 , the SS-preconditioned matrix A P S S − 1 , and the RSS-preconditioned matrix A P R S S − 1 in

SS | RSS | GSS | |||||||
---|---|---|---|---|---|---|---|---|---|

size | iter | time | res | iter | time | res | iter | time | res |

p = 16 | 1(3) | 0.0744 | 9.3431e−09 | 1(3) | 0.0651 | 5.1792e−09 | 1(2) | 0.0415 | 4.5483e−07 |

p = 32 | 1(3) | 1.9894 | 8.5844e−09 | 1(3) | 1.8084 | 4.4832e−09 | 1(2) | 1.2381 | 3.4408e−07 |

p = 48 | 1(3) | 16.5567 | 7.8697e−09 | 1(3) | 16.2467 | 2.7591e−09 | 1(2) | 10.9178 | 3.1121e−07 |

p = 64 | 1(3) | 88.9495 | 7.6631e−09 | 1(3) | 84.8754 | 2.641e−09 | 1(2) | 57.3213 | 2.7896e−07 |

even if the problem size 4 p 2 soars. Fortunately, this desirable property has also been carried over to GSS. Furthermore, GSS outperforms SS and RSS in that it requires the shortest CPU time and the least number of iteration steps in all test problems, as observed in

The shift-splitting iteration method/preconditioner for solving the block three- by-three saddle point problem (1) uses a single parameter to control the convergence. Being aware of the different roles of the block diagonal matrices in the saddle point matrix, we introduce an additional parameter to monitor the ( 3,3 ) - block, which yields a generalized shift-splitting iteration method and a corresponding auxiliary preconditioner. By looking into the spectrum of the iteration matrix, we prove the unconditional convergence of the proposed method. We also apply the induced generalized shift-splitting preconditioner to speed up the convergence of GMRES. It is proved that the preconditioned matrix using the new preconditioner has a well-clustered spectrum which is verified by the numerical examples. Future works include finding the theoretically optimal choice of the parameters in the new scheme and develop other efficient shift-splitting preconditioners.

This work is supported by the National Natural Science Foundation (grant number: 11601323) and the Key Discipline Fund (2018) of College of Arts and Sciences in Shanghai Maritime University. The authors would like to thank the referees for their constructive suggestions.

The authors declare no conflicts of interest regarding the publication of this paper.

Wang, L.N. and Zhang, K. (2019) Generalized Shift-Split- ting Preconditioner for Saddle Point Pro- blems with Block Three-by-Three Structure. Open Access Library Journal, 6: e5968. https://doi.org/10.4236/oalib.1105968