AI Dictionary
Activation Function
Definition
A mathematical function applied to a neural network node's output to introduce non-linearity.
Deep Dive
An activation function is a critical component within an artificial neural network, applied to the output of each neuron (or node) in the network. Its primary purpose is to introduce non-linearity into the model. Without activation functions, a neural network, no matter how many layers it has, would simply behave as a linear regression model, incapable of learning and representing complex, non-linear relationships within data. This non-linearity allows the network to approximate any arbitrary function and learn intricate patterns.
Examples & Use Cases
- 1Using a ReLU (Rectified Linear Unit) function in the hidden layers of a convolutional neural network to process image features
- 2Applying a Sigmoid function in the output layer of a binary classification model (e.g., spam detection) to produce a probability between 0 and 1
- 3Employing a Tanh (Hyperbolic Tangent) function in recurrent neural networks to help capture dependencies in sequential data
Related Terms
Neural NetworkNeuronBackpropagation