Solution to Mathematics for Machine Learning Exercise 5.2

Compute the derivative $f’(x)$ of the logistic sigmoid $$f(x)=\frac{1}{1+\exp(-x)}.$$

Solution: By Chain rule (5.32), we have $$(1+\exp(-x))’=0+\exp(-x)(-x)’=-\exp(-x).$$ By the Quotient rule (5.30), we have\begin{align*}f’(x)=&\ \frac{(1)’(1+\exp(-x))-1(1+\exp(-x))’}{(1+\exp(-x))^2}\\=&\ \frac{0(1+\exp(-x))-(-\exp(-x))}{(1+\exp(-x))^2}\\=&\ \frac{\exp(-x)}{(1+\exp(-x))^2}.\end{align*}