Softmax function

An activation function that ensures all output values fall within the range $[0, 1]$ and their sum equal 1.

p_{i} = \frac{e^{y_{i}}}{\sum_{j} e^{y_{j}}}

Alternatively,

\begin{aligned} p_{i} & = s (y_{i}, y_{- i}) \\ p_{i} & = s (y_{i}, y_{- i}) = \frac{e^{y_{i}}}{\sum_{k = 0}^{K} e^{y_{k}}} \end{aligned}

The $y_{- i}$ notation consists of every element in vector $Y$ except $y_{i}$ :

y_{- i} = (y_{1}, . . ., y_{i - 1}, y_{i + 1}, . . ., y_{K})

This results in a transformation of the vector of values from

Y = (y_{0}, y_{1}, . . ., y_{k}) $ $ i n t o :

P = (p_0, p_1, ..., p_k)

where $\forall i$, $p_i \geq 0$, $\sum^{K}_{i=0}p_i = 1$ ```python def softmax(x): # Subtract the maximum value from each element of the input vector x # to avoid numerical instability (this is optional, but equivalent) x = x - np.max(x) # Compute the exponent of each element exp_x = np.exp(x) # Normalize the exponentiated values by their sum return exp_x / np.sum(exp_x) ``` We can take the highest value in the output vector of values when using it forMulti-class classification