Linear regression

Consider the following linear equation:

$y_i = \beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i} + \epsilon_i$

Assume you have access to data of the independent variables ($x_{1,i}$, $x_{2,i}$) and the dependent variable ($y_i$) for $N$ individuals, where $i$ indexes individuals. The variable $\epsilon_i$ is a mean-zero stochastic shock.

Assume the data generating process is given by:

The data you have access to is:

Question 1: Estimate the vector of coefficients $\mathbf{\beta} = (\beta_0,\beta_1,\beta_2)$ using ordinary least squares (OLS) implemented with matrix algebra by

$\hat{\mathbf{\beta}} = (\mathbf{X}^{\prime}\mathbf{X})^{-1}\mathbf{X}^{\prime}\mathbf{y}$

where $\mathbf{X}^{\prime}$ is the transpose of $\mathbf{X}$ and

Question 2: Construct a 3D plot, where the data is plotted as scattered points, and the prediction of the model is given by the plane

$\hat{y}_i = \hat{\beta}_0 + \hat{\beta}_1 x_{1,i} + \hat{\beta}_2 x_{2,i}$

Question 3: Esimtate the vector of coefficients $\mathbf{\beta} = (\beta_0,\beta_1,\beta_2)$ using a numerical solver to solve the ordinary least square problem, shown below, directly. Compare your results with the matrix algebra results.

$\min_{\mathbf{\beta}} \sum^N_{i=1} (y_i - (\beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i}) )^2$

Question 4: Estimate the vector of coefficients $\mathbf{\beta} = (\beta_0,\beta_1,\beta_2)$ using least absolute deviations (LAD) using a numerical solver to solve the following problem directly:

$\min_{\beta} \sum^N_{i=1} |y_i - (\beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i}) |$

where $|z|$ is the absolute value of $z$.

Question 5: Set $N = 50$. Repeat the estimation using the OLS and LAD methods $K=5000$ times, drawing a new random sample from the data generating process each time. Compare the estimates from each method using histograms. Which method do you prefer? Explain your choice.

Durable purchases

Consider a household living in two periods.

In the second period it gets utility from non-durable consumption, $c$, and durable consumption, $d+\chi x$:

where

• $m_2$ is cash-on-hand in the beginning of period 2
• $c$ is non-durable consumption
• $d$ is pre-commited durable consumption
• $x = m_2 - c$ is extra durable consumption
• $\rho > 1$ is the risk aversion coefficient
• $\alpha \in (0,1)$ is the utility weight on non-durable consumption
• $\chi \in (0,1)$ implies that extra durable consumption is less valuable than pre-comitted durable consumption
• the second constraint ensures the household cannot die in debt

The value function $v_2(m_2,d)$ measures the household's value of having $m_2$ at the beginning of period 2 with precomitted durable consumption of $d$. The optimal choice of non-durable consumption is denoted $c^{\ast}(m_2,d)$. The optimal extra durable consumption function is $x^{\ast}(m_2,d) = m_2-c^{\ast}(m_2,d)$.

Define the so-called end-of-period 1 value function as:

where

and

• $a$ is assets at the end of period 1
• $\beta > 0$ is the discount factor
• $\mathbb{E}_1$ is the expectation operator conditional on information in period 1
• $y$ is income in period 2
• $\Delta \in (0,1)$ is the level of income risk (mean-preserving)
• $r$ is the return on savings

In the first period, the household chooses it's pre-comitted level of durable consumption for the next-period,

where $m_1$ is cash-on-hand in period 1. The second constraint ensures the household cannot borrow. The value function $v_1(m_1)$ measures the household's value of having $m_1$ at the beginning of period 1. The optimal choice of pre-committed durable consumption is denoted $d^{\ast}(m_1)$.

The parameters and grids for $m_1$, $m_2$ and $d$ should be:

Question 1: Find and plot the functions $v_{2}(m_{2},d)$, $c^{\ast}(m_2,d)$, and $x^{\ast}(m_2,d)$. Comment.

Question 2: Find and plot the functions $v_{1}(m_{1})$ and $d^{\ast}(m_1)$. Comment.

Hint: For interpolation of $v_2(m_2,d)$ consider using interpolate.RegularGridInterpolator([GRID-VECTOR1,GRID-VECTOR2],VALUE-MATRIX,bounds_error=False,fill_value=None).

Next, consider an extension of the model, where there is also a period 0. In this period, the household makes a choice whether to stick with the level of durables it has, $z = 0$, or adjust its stock of durables, $z = 1$. If adjusting, the household loses a part of the value of its durable stock; more specificaly it incurs a proportional loss of $\Lambda \in (0,1)$.

Mathematically, the household problem in period 0 is:

The parameters and grids for $m_0$ and $d_0$ should be:

Question 3: For which values of $m_0$ and $d_0$ is the optimal choice not to adjust, i.e. $z = 0$? Show this in a plot. Give an interpretion of your results.

Gradient descent

Let $\boldsymbol{x} = \left[\begin{array}{c} x_1 \\ x_2\\ \end{array}\right]$ be a two-dimensional vector. Consider the following algorithm:

Algorithm: gradient_descent()

Goal: Minimize the function $f(\boldsymbol{x})$.

1. Choose a tolerance $\epsilon>0$, a scale factor $\Theta > 0$, and a small number $\Delta > 0$
2. Guess on $\boldsymbol{x}_0$ and set $n=1$

3. Compute a numerical approximation of the jacobian for $f$ by

   $\nabla f(\boldsymbol{x}_{n-1}) \approx \frac{1}{\Delta}\left[\begin{array}{c} f\left(\boldsymbol{x}_{n-1}+\left[\begin{array}{c} \Delta\\ 0 \end{array}\right]\right)-f(\boldsymbol{x}_{n-1})\\ f\left(\boldsymbol{x}_{n-1}+\left[\begin{array}{c} 0\\ \Delta \end{array}\right]\right)-f(\boldsymbol{x}_{n-1}) \end{array}\right]$

4. Stop if the maximum element in $|\nabla f(\boldsymbol{x}_{n-1})|$ is less than $\epsilon$

5. Set $\theta = \Theta$
6. Compute $f^{\theta}_{n} = f(\boldsymbol{x}_{n-1} - \theta \nabla f(\boldsymbol{x}_{n-1}))$
7. If $f^{\theta}_{n} < f(\boldsymbol{x}_{n-1})$ continue to step 9
8. Set $\theta = \frac{\theta}{2}$ and return to step 6
9. Set $x_{n} = x_{n-1} - \theta \nabla f(\boldsymbol{x}_{n-1})$
10. Set $n = n + 1$ and return to step 3

Question: Implement the algorithm above such that the code below can run.

Optimizer function:

Test case: