deep Sparse Rectifier Neural Networks

Introduction

Rectified linear neurons are good for sparsity and for biological plausibility.

Criticism

Rectified linear neurons really aren't biologically plausible for a variety of reasons. However, they can be transformed into spiking neural networks.