Effective neural network training with adaptive learning rate based on training loss.

Takase, Tomoumi; Oyama, Satoshi; Kurihara, Masahito

Takase, Tomoumi; Oyama, Satoshi; Kurihara, Masahito.

Afiliación

Takase T; Graduate School of Information Science and Technology, Hokkaido University, Kita 14 Nishi 9 Kita-ku, Sapporo, Japan. Electronic address: takase_t@complex.ist.hokudai.ac.jp.
Oyama S; Graduate School of Information Science and Technology, Hokkaido University, Kita 14 Nishi 9 Kita-ku, Sapporo, Japan. Electronic address: oyama@ist.hokudai.ac.jp.
Kurihara M; Graduate School of Information Science and Technology, Hokkaido University, Kita 14 Nishi 9 Kita-ku, Sapporo, Japan. Electronic address: kurihara@ist.hokudai.ac.jp.

Neural Netw ; 101: 68-78, 2018 May.

Article en En | MEDLINE | ID: mdl-29494873

RESUMEN

A method that uses an adaptive learning rate is presented for training neural networks. Unlike most conventional updating methods in which the learning rate gradually decreases during training, the proposed method increases or decreases the learning rate adaptively so that the training loss (the sum of cross-entropy losses for all training samples) decreases as much as possible. It thus provides a wider search range for solutions and thus a lower test error rate. The experiments with some well-known datasets to train a multilayer perceptron show that the proposed method is effective for obtaining a better test accuracy under certain conditions.

Asunto(s)

Aprendizaje Automático; Redes Neurales de la Computación; Entropía

Palabras clave

Beam search; Deep learning; Learning rate; Multilayer perceptron; Neural network training; Stochastic gradient descent

Texto completo

Añadir a Mi BVS

Imprimir

XML

PubMed Links

Buscar en Google

Texto completo: 1 Colección: 01-internacional Base de datos: MEDLINE Asunto principal: Redes Neurales de la Computación / Aprendizaje Automático Idioma: En Revista: Neural Netw Asunto de la revista: NEUROLOGIA Año: 2018 Tipo del documento: Article Pais de publicación: Estados Unidos

Texto completo

Añadir a Mi BVS

Imprimir

XML

PubMed Links

Buscar en Google