deep Sparse Rectifier Neural Networks: Difference between revisions
Jump to navigation
Jump to search
(→Method) |
|||
Line 7: | Line 7: | ||
== Sparsity == | == Sparsity == | ||
== | = Experiments = | ||
Networks with rectifier neurons were applied to the domains of image recognition (both black and white, colour and stereo images) and sentiment analysis. | |||
== Results == | == Results == |
Revision as of 21:15, 9 November 2015
Introduction
Two trends in Deep Learning can be seen in terms of architecture improvements. The first is increasing sparsity (for example, see convolutional neural nets) and increasing biological plausibility (biologically plausible sigmoid neurons performing better than tanh neurons). Rectified linear neurons are good for sparsity and for biological plausibility, thus should increase performance.
Biological Plausibility
Sparsity
Experiments
Networks with rectifier neurons were applied to the domains of image recognition (both black and white, colour and stereo images) and sentiment analysis.
Results
Criticism
Rectified linear neurons really aren't biologically plausible for a variety of reasons. However, they can be transformed into spiking neural networks.