第二类别算法通过不断计算策略期望总回报的梯度来更新参数得到最优策略，应

The second category of algorithms update the parameters to obtain the optimal strategy by continuously calculating the gradient of the expected total return of the strategy, which is more widely used. <br>The Deep Deterministic Policy Gradient (DDPG) algorithm proposed by Lillicrap et al. [9] is a typical deep policy gradient algorithm based on the Actor-Critic [10] architecture. <br>It draws on the idea of ​​DQN experience playback and separation of target networks, and successfully extends the discrete behavior space to the continuous behavior space. <br>In addition, in practical applications,

The second category algorithm obtains the optimal strategy by constantly calculating the gradient of the total return expected by the strategy, and is more widely used.<br>The Deep Deterministic Policy Gradient (DDPG) algorithm proposed by Lillicrap et al. is a typical deep policy gradient algorithm based on the Actor-Critic 10 architecture.<br>It draws on the idea of DQN empirical playback and target network separation, and successfully extends the discrete behavior space to the continuous behavior space.<br>In addition, in practical applications, need to be

In the second category, the optimal strategy is obtained by continuously calculating the gradient of the expected total return.<br>The deep deterministic policy gradient (ddpg) algorithm proposed by lillcrap et al. [9] is a typical deep deterministic policy gradient algorithm based on actor critical [10] architecture.<br>Based on the idea of dqn experience playback and target network separation, the discrete behavior space is successfully extended to the continuous behavior space.<br>In addition, in practical application, it is necessary to<br>