Training Generative Adversarial Networks with Adaptive Composite Gradient

Gradient descent-based machine learning and deep learning methods have been widely used in various computer science tasks over the past several decades. Optimizing a single objective problem with gradient descent is easy to converge to a saddle point in some cases [1]. However, there is a growing set of multi-objective problems that need to be optimized in numerous fields, such as deep reinforcement learning [2, 3], Game Theory, Machine Learning and Deep Learning. Generative Adversarial Networks [4] is a kind of classical multi-objective problem in Deep Learning. GANs have a wide range of applications [5] because of their capability, which can learn to generate complex and high dimensional target distribution. The existing literature about GANs can be divided into four categories, including music generation [6, 7, 8], natural languages [9, 10, 11, 12], methods of training GANs [13, 14, 15, 16], images processing [17, 18, 19, 20]. GANs have obtained remarkable progress in image processing, such as video generation [21, 22], noise removal [23], deblur [24], image to image translation [25, 26], image super-resolution [17], medical image processing [27].

Generative adversarial networks’ framework consists of two deep neural networks: generator network and discriminator network correspondingly. The generator network is given a noise sample from a simple known distribution as input, and it can produce a fake sample as output. The generator learns to make such fake samples, not by directly using real data, just by adversarial training with a discriminator network. Bilinear games are two-player, non-cooperative zero-sum games with compact polytopal strategy sets. If the generator and discriminator have no information communication, then training GANs is a noncooperative zero-sum game. Therefore, GANs can be considered a bilinear game under suitable scenarios. The object function of GANs [4] is often formulated as a two-player min-max game with a Nash equilibrium at the saddle points:

Where x∼P_X (x) denotes an actual data sample and z∼P_Z (z) denotes a sample from a noise distribution (often using uniform distribution or Gaussian distribution). More different forms of GANs object function are mentioned in [28]. Though GANs have achieved remarkable applications, training stable and fast GANs [29, 30] still is a challenging task. Since it suffers from the strongly associated gradient vector field rotating around a Nash equilibrium (see Figure 1). Moreover, those gradient descent ascent-based methods used to optimize object function of GANs tend to lead the limit oscillatory behavior because of imaginary components in the Jacobian eigenvalues.

The main idea of this work is to reduce the computing cost of the Hessian matrix in consensus optimization and SGA. Motivated by [15, 16] and [31], we propose a novel Adaptive Composite Gradient method, which can be used to calibrate and accelerate the traditional methods such as SGD, RMSProp, Adam, consensus optimization, and SGA. The ACG method exploits three aspects of information in the iteration process, which includes gradients information of the past iteration steps, adaptive and predictive information for future iteration steps, and the projection information of the current iteration step mentioned in [16]. We fuse this information as the composite gradient to update the scheme in our algorithm, which can be deployed in deep networks and used to train GANS. The main contributions of this paper are as follows:

• We propose a novel adaptive composite gradient (ACG) method, which can alleviate cyclic behaviors in training GANs. Meanwhile, ACG can reduce the computational consumption of gradients and improve convergence speed.

• For purely adversarial bilinear game problems, we prove that the ACG method is linearly convergent under suitable conditions. In addition, we extend the ACG method to three-player game problems and verify its effect and efficiency with toy models.

• The comprehensive experiments are conducted to test the effect of training GANs and Deep Convolutional Generative Adversarial Networks. The proposed method can obtain competitive results over state-of-the-art (SoTA) methods.

There are several distinctive approaches to improve the training of GANs, but they show more or fewer limitations in some cases. Some of these are dependent closely on the previous assumptions, which leads to these methods not being valid. Moreover, some of these need to payoff the computing cost of the Hessian in the dynamics. Next, we will discuss some related researches in this section.

Symplectic Gradient Adjustment (SGA) [32]: Compared with the traditional games, do not constrain the players’ parameter sets or require the loss functions to be convex. The general games can be decomposed into a potential game, and a Hamiltonian game in [32]. To introduce our method, we firstly review the SGA method as follows.

Definition 2.1 A game is a set of players $[p]={1,2,…,n}$⁠, and the loss functions satisfy twice continuously differentiable ${li:Rd→R}i=1n$⁠. Players' parameters are $w=(w1,w2,…,wn)∈Rd$ with $wi∈Rdi$ where $∑indi=d$⁠. The i^th player can control w_i.

Using the g(w) notes the simultaneous gradient which is the gradient of the losses withe respect to players' parameters $g(w)=(∇w1l1,∇w2l2,…,∇wnln)$⁠. For a bilinear game, it requires the losses to satisfy $∑i=1nli=0$ such as follow:

This kind of games have a Nash equilibrium at (x,y) = (0,0). The simultaneous gradient g(x,y) = (Cy,-C^Tx) rotates around the Nash equilibrium shown in Fig. 6.

We can derive the Hessian of a n-player game with the simultaneous gradient g(w). The formula of Hessian is $H(w)=∇w·g(w)T=(∂gi(w)∂wj)i,j=1d$ where $H∈Rd×d$⁠. Further, the matrix formula of Hessian is as follows:

Applying the generalized Helmholtz decomposition [Lemma 1 in [32]] to the above mentioned Hessian of the game, we have H(w) = S(w)+A(w). David et al. (2018) [32] pointed that a game is a potential game if A(w)≡0. It is a Hamiltonian game if S(w)≡0. Potential games and Hamiltonian games are both well-studied, and they are easy to solve. Since the cyclic behavior around the Nash equilibrium is caused by simultaneous gradient, David et al. [32] proposed the Symplectic Gradient Adjustment method, which is as follows:

Where A is from the Helmholtz decomposition of Hessian. g_λ is used to replace the gradient among the iterates in the GAD-based methods, and using g_λ to train GANs can alleviate the cyclic behaviors. If we consider the players in a bilinear game as GANs, the SGA algorithm needs to pay an expensive computing cost of Hessian which can lower the algorithm efficiency.

Centripetal Acceleration [15]: The simultaneous gradient shows cyclic behaviors around the Nash. Hamiltonian games obey a conservation law in these gradient descent-based methods, so the cyclic behaviors can be considered as a uniform circular motion process. As well known, the direction of the centripetal acceleration in a consistent circular motion process points to the center of the circle. Using this characteristic modifies the direction of the simultaneous gradient vector field to alleviate the cyclic behaviors. Based on these observations, Peng et al. (2020) [15] propose the Centripetal acceleration methods, which are derived to two versions methods named Simultaneous Centripetal Acceleration (Grad-SCA) and Alternating Centripetal Acceleration (Grad-ACA), they are used to train GANs. In the next, we review the Centripetal Acceleration methods.

Given a bilinear game, the losses of this game are ℓ₁(θ,φ), ℓ₂(θ,φ) corresponding to player 1 and player 2. The parameter space is defined in θ × φ, where θ,φ ∈ ℝⁿ. Player 1 can control the parameter θ and tries to minimize payoff function ℓ₁, while player 2 can control parameter φ and tries to minimize payoff function under the non-cooperative situation. This game is a process of the two players adjusting their parameters to find a local Nash equilibrium that satisfies the following two requirements:

The centripetal acceleration methods require that the two-player game is differentiable. Then, the above two payoff functions can be combined into a joint payoff function because of the zero-sum property of the game:

The derivation of Eq. (1) leads a two-player game, which can be rewrite as V(θ,φ). The problem becomes to finds a local Nash equilibrium:

where

In order to introduce the Centripetal Acceleration methods, we fist review the simultaneous gradient descent method in [33] is

And the simultaneous gradient descent based alternating version is

where α is learning rate in the algorithms. While the Centripetal Acceleration methods directly utilize the item of centripetal acceleration to adjust the simultaneous gradient descent. Then gradient descent with simultaneous centripetal acceleration is introduced as:

We can also obtain the gradient descent with the alternating centripetal acceleration method:

where α₁, β₁, α₂, β₂ in above two versions of the centripetal acceleration methods are hyper parameters. The centripetal acceleration methods can calibrate other gradient-based methods. The intuitive understanding of the centripetal acceleration method is shown in Figure 2.

Predictive Projection Centripetal Acceleration (PPCA) [16]: For the centripetal acceleration methods, it uses the last iterative step information to update (φ_t+1 + φ_t+1). However, there are some methods which utilize the predictive step information to update (θ_t+1, φ_t+1), such as MPM, OMD, OGDA. MPM is introduced by Liang et al. (2019) [34] and its dynamics are as follows:

Motivated by MPM and centripetal acceleration methods, Li et al. (2020) [16] propose the predictive projection centripetal acceleration methods. They also consider approximately the cyclic behavior around a Nash as a uniform circular motion process. However, it is different from the Grad-SCA and Grad-ACA. They construct the item of centripetal acceleration to use the predictive step information replacing that of the last step. Meanwhile, they argue that the centripetal acceleration term points to the matched center approximately. To make the centripetal acceleration item point to the center precisely, they proposed the projection centripetal acceleration methods. PPCA can modify the gradient descent ascent and the alternating gradient descent ascent directly. We can understand PPCA intuitively from Figure 2. The dynamics of predictive projection centripetal acceleration are the following formula:

Where $∇V¯(θt,φt)=(-∇θV(θt,φt)∇φV(θt,φt))$ is the signed gradient vector at time t. The $∏∇v¯(θt,φt)[∇V¯(θt+12,φt+12)−∇V¯(θt,φt)]$ is the projection of the centripetal acceleration term $[∇V¯(θt+1'φt+12)-∇V¯(θt,φt)]$ onto the vector $∇V¯(θt,ϕt)$⁠.

Li et al. (2020) proposed two versions of the PPCA methods by constraining the coefficient matrix which must be full rank in bilinear games under the specified situation (Lemma 3.2 in [16]). The form of PPCA method for bilinear game is

And also, we can get the alternating PPCA formula is as follow:

where the all of γ, α, β are hyper parameters.

Although the mentioned methods have achieved significant work in training GANs, some of them require high computing costs and computer memory. The rest of them depend closely on the approximate circular motion process. If the practical numerical experiments do not satisfy this approximate theory, these methods will not be valid. In contrast to our approach, we propose the adaptive composite gradient method, which can reduce the computing cost and solve the limitation brought by the approximate circular motion process.

These Hessian-based methods, such as consensus optimization and SGA, optimize game problems, bringing high computing costs. The centripetal acceleration algorithm reduces expensive computational costs but depends on the approximately uniform circular motion process assumption, shown in Figure 3. PPCA is an improved version of the centripetal acceleration algorithm, shown in Figure 4(b). However, PPCA needs a full rank coefficient matrix, causing the projection of the PPCA method to be zero (Lemma 3.2 in [16]). The proposed ACG is motivated by two aspects. Firstly, we consider cyclic behavior as a generally circular motion process but not a uniform one. Therefore, similar to the centripetal acceleration method, we modify directions by adding the projection of the centripetal acceleration term at time t. Secondly, A³DMM provides an idea that can use past iterations to predict future iterations because the trajectory of the sequence Z_k is either straight linear or spiral. However, the cyclic behavior is also approximately a spiral. Motivated by the two aspects, we proposed a novel adaptive composite gradient (ACG) method to alleviate the cyclic behavior in training GANs. ACG can reduce computing costs and accelerate iteration by predicting the future step iterative information, which is called a Semi-Gradient-Free method.

Our idea is motivated by A³DMM [31]. Then, we review the A³DMM method. Give a optimisation problem

where the essential assumptions are proposed

• R ∈ Γ₀(ℝⁿ) and J ∈ Γ₀(ℝ^m) are proper convex and lower semi-continuous functions.

• A, B are injective linear operators.

• ri (dom (R) ∩ dom (J)) ≠ Ø and the set of minimizers is non-empty.

To derive the iteration scheme, considering the augmented Lagrangian and rewriting the optimisation problem, which reads

where γ >0 and Ψ is the Lagrangian multiplier, then we have iteration forms:

We can rewrite the above iteration into the following formula by introducing a new variable $Zk= def Ψk−1+γAxk,$

The trajectory of the sequence Z_k dependents closely on the value of γ, where k ∈ N. If selecting a proper γ, the eventual trajectory of Z_k is spiral as shown in Figure 4. since the trajectory of Z_k has the characteristic of spiral, which provides a way that using the previous q iterationd predicts the future s iterations. The Z_k of ADMM can be estimated by Z̄_k,s which is defined as follow:

for the choice of s = 1. Given a sequence Z_k-j, i = 1, 2, …, q+1, we can define v_i = Z_i − Z_i-1 and obtain the past $Vk−1,Vk−2,⋯,Vk−q$ which can be used the $Vk−1,Vk−2,⋯,Vk−q$ to estimate v_k. Let $Vk−1=[vk−1,Vk−2,⋯,Vk−q]∈Rn×q$ and $Ck=argminC∈Rq∥Vk−1C−Vk∥2=∥∑i=1qCiVk−i−Vk∥2$⁠. Then we can use the V_kC_k to approximate V_k+1 that is $VkCk≈Vk+1$⁠, we can compute $Z¯k+1=Zk+VkC≈Zk+1.$ By iterating s times, we can obtain $Z¯k,s≈Zk+s$⁠. This method is proposed by Clarice Poon et al. [31].

To more easily understand our method, we make some symbol conventions throughout the paper. Let $a→⊙b→$ is considered as the projection of $a→$ onto $b→$⁠, where $a→$ and $b→$ ∈ ℝⁿ and ⊙ denotes the projecting calculation between two vectors. w_tⁱ denotes the i^th player controlling parameter at time t and $Wt=(wt1,Wt2,?,Wtn)$⁠. we use ${ℓi:Rd?R}i=1n$⁠, to denote the losses corresponding to n players which is same as mentioned in Definition 2.1. Then we can obtain, which is the payoff vector of the n players at time t.

We consider a bilinear game problem with the following form

Desiderata. This two players game has a Nash equilibrium which must satisfy

D1. The two losses satisfy: $∑i=12ℓi(W1,W2)≡0;$

D2. For ℓ_i is differentiable over the parameter space Ω(w¹) ×Ω(w²), where $Ω(w1)×Ω(w2)⊆Rn×Rn$⁠.

The player 1 holds the parameter w¹ and tries to minimize the loss ℓ₁, while the player 2 holds the parameter w² and tries to minimize the loss ℓ₂. From the D1. we can get ℓ₁= -ℓ₂, the formula (22) can be rewritten as

The dynamics of gradient descent ascent based method is

The ACG method involves three parts which are consist of the composite gradients. Firstly, we introduce the predictive aspect. In this section, $Wt=(Wt1,Wt2)$ is the parameter vectors at time t. Similar to A³DMM, we utilize the W of the previous q iterations to predict the future s iterations which can be denoted by $W¯t,s$⁠. Then we can get the following formula of $W¯t,s$⁠:

for the value of s = 1. Define $vi=Wi−Wi−1$⁠, W_i is from sequence ${Wk−i}i=0q.$ We can use the past $Vk−1,Vk−2,⋯,Vk−q$ to approximate the last v_k. Then, $ck=argminc∈Rq∥Vk−1c−Vk∥2$⁠. Finally, we can obtain $W¯k,1=Wk+Vkck≈Wk+1.$ By looping of s times, we can get $W¯k,s≈Wk+s$⁠. The second part and third part of our ACG method are the $∇L((wt1,wt2))$ and the projection of centripetal acceleration term. The dynamics of proposed ACG method is

where the $∇wiL((wt1,wt2))$ denotes the partial derivatives of w_i corresponding to ℓ_i. at time t. The $a→i$ denotes $∇wiL((wt1,wt2))−∇wiL((wt−11,wt−12))$⁠. The $b→i$ denotes $a→i⊙∇wiL((wt−11,wt−12))$⁠, which is the projection of $a→i$ onto the vector $∇wiL((wt−11,wt−12))$⁠. The basic intuition of our proposed method is shown in Figure 5.

For clarity, we draft the scheme of proposed adaptive composite gradient method in Algorithm 1, which is applied in a two-player game problem. Note that the ACG method can calibrate any optimizer based on gradient descent ascent. Meanwhile, the ACG method can extend to a n-player game problem. Given a n-player game problem, let g(W_t) be the gradient of losses for all players at t. It is worth noting that all loss functions must be differentiable. We adopt the same way of Algorithm 1 to compute $W¯t$⁠. The dynamics of the ACG method for n-players reads

Where the $a→$ denotes $g(Wt)−g(Wt−1),b→$ denotes $a→⊙g(Wt−1)$ which is the projection of $a→$ onto g(W_t-1).

Remark 4.1 The value of k can be controlled both in Algorithm 1 and Algorithm 2. Let k = q + i where $i∈N+$⁠, we can set different acceleration ratio of the algorithms by adjusting the values of k, s.

In Algorithm 1 and Algorithm 2, there is no need to calculate the gradient of each iteration because of $W¯t+s$⁠. Therefore, we also name it a Semi-Gradient-Free method, whose merit is that it can reduce the computational cost and convergent fast. β₁, β₂ can be used to control the convergence speed in our algorithms.

In this subsection, we mainly discuss the convergence of Adaptive Composite Gradient Method in the bilinear game, which reads

For any local Nash equilibrium of the bilinear game has the point (θ∗, φ∗) satisfied the following conditions:

The local Nash equilibrium exists if and only if the ranks of A and A^T are the same as the dimension of B,C. By this way, without loss of generality, we can convert (θ, φ) to $(θ−θ∗,ϕ−ϕ∗)$⁠, which is used to rewrite bilinear game (29) as:

Before analyzing the convergence property of the Adaptive Composite Gradient Method in this situation, we introduce some essential theorems and propositions.

Theorem 5.1Suppose $F∈Rd×d$ is from the iterative system x_k+1 = Fx_k. If F is nonsingular and satisfies the spectral radius ρ (F) < 1, then the x_k of iterative system converges to 0 linearly.

Theorem 5.2 (OMD) Consider a bilinear game V(θ, φ) = θ^T Aφ, where$A∈Rd×d$⁠. Assume A is full rank. Then the following dynamics,

with the learning rate

obtain an ɛ -minimizer such that $(θT,ϕT)∈B2(ε),$ provided

under the assumption that$∥(θ0,ϕ0)∥,∥(θ1,ϕ1)∥≤δ.$

To discuss the convergence of the ACG method for the bilinear game, we divided the analysis process into three parts (two cases). Without loss of generality, let t represent the iterative step, and k is the previous steps. The convergence property of Algorithm 1 is as the following.

Case 1. mod(t,k)≠0 or mod(t,k) = 0&ρ (C_k)≥1. The ACG method adopts the dynamics:

Taking α = 2β = 2η in Case 1, the dynamics scheme reduces to OMD, which can be written as:

Theorem 5.2 has assigned the condition about learning rate of OMD and it is exponential Convergence. The convergence of OMD can be found in [34] (Theorem 3).

Case 2. mod(t,k) = 0&ρ (C_k)<1. From the Algorithm 1, firstly we shall compute the $W¯t+s$ by:

Using the fixed-point formulation of ADMM, (35) can be written as an unified $W¯t=ε(W¯t−1)$⁠, let $Vtct=σt$⁠, then $W¯t=ε(W¯t−1+σt)$⁠. We can obtain the convergence of (35), iff σ_t converges to 0. The convergence of (35) is based on the convergence of inexact Krasnosel'skil-Mann fixed-poin iteration in [31] (Proposition 5.34). The detail convergence analysis of $W¯t=ε(W¯t−1+σt)$ has been discussed in [31](Proposition 4.2).

Then we discuss the convergence of composite gradients update scheme, which is written as follow:

where the $Gw1,Gw2$ are defined as:

Proposition 5.3Any given two vector $a→,b→$⁠, the projection of vector $b→$ onto the vector $a→$ can be denoted as $γa→$⁠, where the$γ∈R$⁠.

According to proposition 5.3, our dynamics of the bilinear game, the composite gradients update scheme is reduced to be

We can obtain the iterative matrix as:

where τ is defined by (35).

According to the iterative matrix, it is easy to obtain that $[θt+1,ϕt+1,θt,ϕt,W¯θt+1,W¯ϕt+1]⊤=F[θt,ϕt,θt−1,ϕt−1,W¯θt,W¯ϕt]⊤$⁠, where the $(θt,ϕt)$ are generated by (38). With the assumption that A is square and nonsingular in Proposition 5.4, we use the well-known Theorem 5.1 to illustrate the linear convergence for the update scheme (38).

Proposition 5.4Suppose that A is square and nonsingular. Then, the eigenvalues of F are the roots of the sixth order polynomials:

where Sp(·) denotes the collection of all eigenvalues.

Proposition 5.5 Suppose that A is square and nonsingular. The$Δt:=∥θt∥2+∥ϕt∥2+∥θt+1∥2+∥ϕt+1∥2+∥w¯θt+1||2+∥w¯ϕt+1∥2$is linearly convergent to 0 for given γ with α and β₁, satisfy

where the$λmax (⋅),λmin (⋅)$denote the largest and the smallest eigenvalues of A^TA.

This subsection mainly discusses the convergence of the Adaptive Composite Gradient method in the general n-player game. The problem is described as in Definition 2.1, according to Algorithm 2, the convergence analysis process in general n players game is the same as that of the bilinear game with three parts and two cases. Before analyzing convergence property, we introduce some basic definitions.

Definition 5.6 Suppose that $f:Rn→R$ and it is convex, continuously. For $∀x,y∈Rn$⁠, the gradient of f is Lipschitz continuous with constant L such that:

we define that f belongs to the class $FL1,1$⁠. If f is strongly convex with modulus μ > 0 and such that:

we define that f belongs to $Fμ,L1,1$⁠.

Next, we suppose that the all $ℓi,i=1,2,⋯,n$ are belonging to $FL1,1$⁠. We can get the definition of fixed point which is also called Local Nash Equilibrium in game.

Definition 5.7W∗ is a Local Nash Equilibrium(fixed point) if W∗ satisfy g(W∗) = 0. We say that it is stable if g(W∗)≥0, unstable if g(W∗)≤0.

Theorem 5.8[Nesterov 1983] Let f be a convex and β-smooth function, and we can write the well-known Nesterov's Accelerated Gradient Descent update scheme as

Then Nesterov's Accelerated Gradient Descent satisfies

Nesterov (1983) proposed the accelerated gradient method which achieves the optimal $O(1t2)$ convergence rate.

The convergence of the ACG method for the general n player game is also divided into two cases. Let t represent the iterative step, and k is the previous steps. The convergence property of Algorithm 2 is as the following.

Case 1. mod(t,k)≠0 or mod(t,k) = 0&ρ (C_k)≥1. From Algorithm 2, if t and k satisfy the conditions of Case 1. Our proposed ACG method is the same as classical Gradient Descent and the update scheme is

where the α is a positive step-size parameter. According to Definition 5.7, let the Local Nash Equilibrium W∗ and $L∗=L(W∗)$⁠. Base the Definition 5.6, if $ℓi,i=1,2,⋯,n$ are belonging to $FL1,1$⁠, then $L(Wt)−L∗$ associated {W_t} converges at rate $O(1t)$⁠. For more detail convergence of averaged iterations with generalized gradient descent update scheme in convex-concave games is analyzed in [36, 37].

Case 2. mod(t,k) = 0&ρ (C_k)<1. According to Algorithm 2, firstly we should compute the $W¯t+s$ by:

The convergence of formula (44) is the same as that of Case 2 in previous section 5.1. For more detail about convergence of (44) has been discussed in [30] (Proposition 4.2).

In Case 2, we mainly analyze the convergence of composite gradient update scheme in formula (28). Before illustrating our proposed method convergence property, The formula (42) can be equivalently written as

where the α and β are step-size parameters. However, our proposed composite gradient method (28) can be transfer to the similar formula as (45) which is based on the Proposition 5.3. That is

where the $W¯t+s$ is equivalent to (W_t - W_t-1). Comparing (45) with (46) if the parameters are equivalently transformed, our proposed adaptive composite gradient method can be reduced to the Nesterov's Accelerated Gradient (NAG) method by this way. In 1983, Nesterov had given the convergence rate at $O(1t2)$for convex smooth optimization in [38]. And also there had given the convergence bounds for convex, non-convex and smooth optimization in [39] (Theorem 1 ~ Theorem 4).

Remark 5.1 In Case 2, our proposed Adaptive composite gradient method has the same convergence rate and the same convergence bounds as that of NAG method, since the assumption that all $ℓi,i=1,2,⋯,n$ are belonging to $FL1,1$⁠. We can naturally obtain the convergence rate of the ACG method, which is at $O(1t2)$ based on Theorem 5.8.

We tested our ACG methods in Algorithm 1 and Algorithm 2 corresponding with the bilinear game and general game with 3 players. We tested the ACG method in Algorithm 1 on the following bilinear game, which can be written as

It is obvious that the Nash Equilibrium (stationary point) is (0,0). We compared our ACG with some other methods whose results are presented in Figure 6 (a). From the behaviors of Figure 6, the Sim-GDA method diverges, and the Alt-GDA method is rotating around the stationary point. However, the other methods all converge to the Nash Equilibrium. We proposed the ACG method converges faster than other convergence methods.

In Figure 6 (b), we test our ACG method on the following general zero-sum game

The effects of the compared methods on this game show that all methods converge to the origin. Notably, the cyclic behavior of the Alt-GDA method has disappeared, and the Sim-GDA method converges. It is worth noting that the trajectory of our ACG method is the same as that of PPCA [16]. Both our ACG and PPCA [16] seem faster than others. We also compared ACG with other methods on the following general game

Its effects are presented in Figure 6 (c), which shows that Sim-GDA and Grad-SCA diverge, the rest methods converge. The APPCA [16] is faster than our ACG method in this game.

We used the last general zero-sum game (49) to test the robustness of the proposed ACG method in Figure 7. As the learning rate α increases through {0.01,0.05,0.1} and other parameters keeping same. ACG method converges faster when the learning rates setting with α = 0.01 and α = 0.05, although converge slower with learning rate α = 0.1, ACG method still converge to the origin rapidly.

The proposed Adaptive Composite Gradient method (ACG) is also suitable for the general game with n players. However, it is challenging to present the effect of the general n player game by toy function. To illustrate our proposed method of Algorithm 2 adaptive for n players game, we show the effects of Algorithm 2 by a general 3 players game. The payoff functions can be written as