Neural Networks: Foundations to Applications

Artificial neuron, weights, and activation functions

An artificial neuron (node) is a basic unit in a neural network that receives real-valued inputs from other neurons (or data features), combines them using connection strengths called weights, and then produces an output by applying a nonlinear function. The neuron’s output is computed from the total input signal through an activation function, which introduces nonlinearity so networks can learn complex patterns rather than only linear relationships. Weights determine the strength and influence of each input connection and are adjusted during training to reduce prediction error. Neurons are organized into layers (input, hidden, and output), where each layer transforms its inputs and passes signals forward through the network. A deep neural network has multiple hidden layers and can learn hierarchical representations of data. Activation functions are crucial because they shape how input sums are converted into outputs and affect learning dynamics. Common activation functions mentioned include ReLU (rectified linear unit), which is widely used, and earlier historical activation choices such as Kolmogorov-Gabor polynomials in deep networks with multiplicative units.

0/2 modes complete

Network layers: input, hidden, and output

0/2 modes complete