Abstract: A smooth activation function with a parameter was constructed for the deep learning neural networks. The online correction formula for this parameter was established based on the error back propagation algorithm, which avoided the problems of gradients losing, nonsmooth and overfitting. Compared with some popular activation functions, the results show that the new activation function works well on many data sets.

Key words: activation function, convolutional neural network, machine learning