A method that increases the stability of a model’s convergence and helps it reach a better overall solution by preventing convergence to a local minima. To avoid drastic changes in the model’s weights during training, a copy of the current weights is created before updating the model’s weights. Then the model’s weights are updated to be the weighted average between the current weights and the post-optimization step weights.
