Contribution points mentioned by the author in the original representative work | We describe a new learning procedure, back-propagation, for networks of neurone-like units. |
---|---|
1. In MBGD, the learning rate is very important to the convergence speed and quality in training. Many different schemes, e.g., momentum [6], averaging [15], AdaGrad [16], RMSProp [17], Adam [18], etc., have been proposed to optimize the learning rate in neural network training. Adam may be the most popular one among them. [43.84%] | |
2. As for the extrapolation, a smooth activation function that only acts on the hidden layer(s) is recommended. Back-propagation is the second part of the algorithm [37]. This is the central mechanism that allows neural network methods to “learn.” [42.36%] | |
3. The feedforward multilayer perceptron is one of the most popular types of ANNs; it was developed by Rumelhart et al. [23], and it is presented in Supplementary 1. This network also consists of an input layer, one or more hidden layers, and one output layer. [10.95%] |
Highlights of Citing Sentence | Titles of Citing Paper | |
---|---|---|
1 | The steepest descent algorithm, also known as the error backpropagation (EBP) algorithm [8,9], dispersed the dark clouds on the field of artificial neural networks and could be regarded as one of the most significant breakthroughs for training neural networks. | Application of Neural Networks to Automatic Load Frequency Control |
2 | In addition to the development of new ANN algorithms that were more neural-inspired (e.g. Hopfield networks), another major breakthrough that helped lead to a resurgence in neural network research was the rediscovery of the backpropagation technique (LeCun, 1985, Rumelhart et al., 1986, Werbos, 1990). | A historical survey of algorithms and hardware architectures for neural-inspired and neuromorphic computing applications |
3 | The next major breakthrough happened in late 80s with the invention of back-propagation and a gradient-based optimization algorithm to train a neural network with one or two hidden layers with any desired number of nodes (Rumelhart et al., 1986). | Meta-analysis of deep neural networks in remote sensing: A comparative study of mono-temporal classification to support vector machines |
# | Author | Representative Papers | Citing Papers |
---|---|---|---|
1 | Hinton Geoffrey | Deep learning | 7,630 |
2 | Hinton Geoffrey | Reducing the dimensionality of data with neural networks | 4,509 |
3 | Hinton Geoffrey | A fast learning algorithm for deep belief nets | 4,538 |
4 | Hinton Geoffrey | Learning representations by back propagating errors | 6,598 |
5 | Hinton Geoffrey | Dropout: A Simple Way to Prevent Neural Networks from | 4,142 |
Overfitting |
Sentiment Category | Definition | Sentiment Score |
---|---|---|
Range (E) | ||
Positive | Holding commendatory, approving and admiring attitude | 1>E>0 |
Neutral | Brief statement or rephrasing, without obvious expression of sentiment | E=0 |
Negative | Describing defects, shortcomings or mistakes | 0>E>-1 |
Quantitative Analysis Tool | CiteOpinion | |
---|---|---|
Data object | Metadata of paper | Citing sentences |
Data granularity | Article | Sentence |
Analysis focus | Statistical indicators | Text content mining |
Result form | Relationship diagrams and data sheets | Evaluation evidence text, relationship diagrams and data sheets |