A Blog Entry on Bayesian Computation by an Applied Mathematician
$$
$$
(Hinton et al., 2012), (Srivastava et al., 2014) による，ミニバッチごとに確率的に使わない結合を決定するという正則化の技法である．1
1 Dropout による正則化
1.1 Bayes からの説明
Dropout による正則化は，Gauss 過程による近似とも見れ，Bayes 手法の持つ正則化効果と相通ずることが指摘されている (Gal and Ghahramani, 2016)．
1.2 Monte Carlo Dropout
References
Gal, Y., and Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In M. F. Balcan and K. Q. Weinberger, editors, Proceedings of the 33rd international conference on machine learning,Vol. 48, pages 1050–1059. New York, New York, USA: PMLR.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems,Vol. 27, pages 1–9.
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors.
Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(56), 1929–1958.
Footnotes
GAN の数値実験 (Goodfellow et al., 2014, p. 6) にも，判別器を訓練するのに用いられている．↩︎