Exponential Moving Average

Share

A method that increases the stability of a model’s convergence and helps it reach a better overall solution by preventing convergence to a local minima. To avoid drastic changes in the model’s weights during training, a copy of the current weights is created before updating the model’s weights. Then the model’s weights are updated to be the weighted average between the current weights and the post-optimization step weights.

Filter terms by

Related resources

deci-updates-tensorboard-blog-featured-2
Open Source
pytorch-training-sg-new-features-featured-2
Open Source
new-hardware-support-featured
Engineering