# Stochastic dynamic programming II.

Continuing my first post on dynamic programming, this post will introduce an example. Using the methods and results shown in the previous post on dynamic programming, we can solve this example to get an explicit equation that describes the motion of consumption, capital and consequently output in the economy.

Using these equations, we will be able to simulate economic growth and see whether the model’s predictions fit actual data.

Let us therefore construct an example. First, I will define the technology and utility functions. We will also assume some things about some of the constants included in the model. So let

$F(k_t, n_t) = k_t^{\alpha} n_t^{1 - \alpha},$

with 0 < alpha < 1. Furthemore let

$u(c_t) = \ln{c_t}, \delta = 1, E[\ln{z_t}] = \mu.$

Now let us postulate that the value function takes the form

$v(k_t, z_t) = A + B \ln{k_t} + D \ln{z_t}.$

We can try to plug this function into the Bellman equation we derived above and see if it’s possible to solve for A, B and D. If yes, then our guess regarding the form of the value function was right. Let us thus plug in our guess for the function v into the Bellman equation:

$A + B \ln{k_t} + D \ln{z_t} = max_{k_{t+1}} \ln{(z_t k_t^{\alpha} - k_{t+1})} + \beta E_t[A + B \ln{k_{t+1}} + D \ln{z_{t+1}}].$

To get this expression, we first plug in for the function v, and then solve the resource constraint for c as follows

$c_t = z_t f(k_t) + (1 - \delta) k_t - k_{t+1},$

where the middle term vanishes because delta = 1, and we plug in for f(k) = F(k, 1) while making use of the fact that n = 1 (see previous post). We plug the resulting value for c into the Bellman equation to get the expression above. We can evaluate the expectation operator

$A + B \ln{k_t} + D \ln{z_t} = max_{k_{t+1}} \ln{(z_t k_t^{\alpha} - k_{t+1})} + \beta A + \beta B \ln{k_{t+1}} + \beta D \mu.$

Here we make use of the fact that A, B and D are constants, that k_{t+1} is not stochastic and that the expected value of ln(z) is mu as specified above. We can solve this maximization problem by differentiating the right-hand side with respect to k_{t+1} and setting the derivative equal to zero. Doing this, we will get

$- \frac{1}{z_t k_t^{\alpha} - k_{t+1}} + \frac{\beta B}{k_{t+1}} = 0$

$\frac{k_{t+1}}{\beta B} = z_t k_t^{\alpha} - k_{t+1}$

$k_{t+1} + \beta B k_{t+1} = \beta B z_t k_t^{\alpha}$

$k_{t+1} (1 + \beta B) = \beta B z_t k_t^{\alpha}$

$k_{t+1} = \frac{\beta B}{1 + \beta B} z_t k_t^{\alpha}$

This is thus the level of capital that maximizes the Bellman equation. Therefore, if we plug this back into the Bellman equation we get the solution for the value function:

$A + B \ln{k_t} + D \ln{z_t} = \ln{(\frac{z_t k_t^{\alpha}}{1 + \beta B}}) + \beta A + \beta B \ln{(\frac{\beta B z_t k_t^{\alpha}}{1 + \beta B})} + \beta D \mu.$

Decompose this expression using the rules of logarithms into

$A + B \ln{k_t} + D \ln{z_t} = \ln{z_t} + \alpha \ln{k_t} - \ln{(1 + \beta B)} + \beta A + \beta B \ln{\beta B} + \beta B \ln{z_t} + \alpha \beta B \ln{k_t} - \beta B \ln{(1 + \beta B)} + \beta D \mu.$

This may seem like a daunting expression but we’re almost there. Notice that A is a constant on the left-hand side of the equation. Therefore, A must be a constant on the right-hand side as well. Collect all the constant terms on the right-hand side and you have A,

$A = \ln{\frac{1}{1 + \beta B}} + \beta A + \beta B \ln{\frac{\beta B}{1 + \beta B}} + \beta D \mu.$

Similarly, B and D multiply ln(k_t) and ln(z_t) terms on the left-hand side, respectively. So collect the coefficients of these terms on the right-hand side and you have B and D:

$B = \alpha + \alpha \beta B,$

$D = 1 + \beta B.$

Now solve for B:

$B = \frac{\alpha}{1 - \alpha \beta}.$

You can plug this into the expression for D and solve for D, then plug the expressions for B and D into the expression for A and solve for A. I will not do this because solving for B will already let us solve the expression for k_{t+1} that was derived above:

$k_{t+1} = \frac{\beta B}{1 + \beta B} z_t k_t^{\alpha}$

$k_{t+1} = \frac{\alpha \beta}{1 - \alpha \beta} \frac{1}{1 + \alpha \beta / (1 - \alpha \beta)} z_t k_t^{\alpha}$

$k_{t+1} = \frac{\alpha \beta}{1 - \alpha \beta} \frac{1}{1 / (1 - \alpha \beta)} z_t k_t^{\alpha}$

$k_{t+1} = \alpha \beta z_t k_t^{\alpha}$

Finally, since we know from above (from the resource constraint) that

$c_t = z_t k_t^{\alpha} - k_{t+1},$

we can also have an expression for consumption,

$c_t = (1 - \alpha \beta) z_t k_t^{\alpha}.$

Now these two equations for c_t and k_{t+1} describe our economy. Notice that they’re expressed in terms of the model’s constant parameters. The output can be found using

$y_t = z_t f(k_t) = z_t k_t^{\alpha},$

which was established in the previous post. The next step is to simulate the model and see if its properties fit actual data. I will do this in the third and last post in this series on dynamic programming.

Other posts in my series on dynamic programming: Stochastic dynamic programming I., Stochastic dynamic programming III..