Exercise 1 - Taylor Series

Compute the Taylor series expansion up to the second order term for the following multivariate functions around a given point:

1. \(f(x) = 5x^3\) around \(x_0=1\).
2. \(f(x,y) = x^2 \cdot y^3 + x^2\) around \(x_0=3\), \(y_0=2\).
3. \(f(\fx) = x_1^3 \cdot x_2 \cdot \log(x_2)\) around \(\fx_0=(2,1)^\top\).
4. \(f(\fx) = \sin(x_1) + \cos(x_2)\) around \(\fx_0=(-\pi,\pi)^\top\).

Exercise 2 - Eigenvalues, Eigenvectors

You are given the sets of eigevalues and eigenvectors. Compute the corresponding matrix.

1. \(\la_1 = 2\), \(\la_2 = 3\), \(\fv_1 = (1,0)^\top\), \(\fv_2 = (0,1)^\top\).
2. \(\la_1 = 2\), \(\la_2 = 3\), \(\fv_1 = (1,1)^\top\), \(\fv_2 = (1,-1)^\top\).

Exercise 3 - SGD with Momentum

Implement stochastic gradient descent with momentum and apply it to optimize some elementary functions in 1d and 2d.