Local level model to time series data on Stan

Overview

On the articles below, I tried local level modeling to time series data on Edward and am still struggling.

Time series analysis on TensorFlow and Edward: local level model

Deep learning and Machine learning methods blog

Time series analysis on TensorFlow and Edward: local level model:P.S. 1

On the article below, I tried to analyze time series data with local level model. On Stan, I could do it before without problem. But on Edward and TensorFlow, I have been struggling. Deep learning and Machine learning methods blog From the situation above, although it doesn't work well yet, I got some progress.

On this article, I’ll express by Stan what I wanted on Edward. In a nutshell, I’ll write local level model to time series data on Stan.

Data

I'll use the same data as the one which is used on the articles above. Simply, by following code, we can make the data.

import numpy as np

# make data
np.random.seed(59)
data = []
start_point = 10.0
data.append(start_point)
for i in range(100):
    if i == 0:
        temp = start_point
    else:
        next_temp = temp + np.random.normal(0, 1)
        data.append(next_temp)
        temp = next_temp

By visualizing, let's check the behavior of it.

import matplotlib.pyplot as plt
plt.plot(list(range(len(data))), data)
plt.show()

The purpose is to make model to trace this time series data.

Stan Modeling

Mathematically, the model can be expressed as below. This is simple local level model.
$u_t \sim Normal(u_{t-1}, \sigma_ξ^2)$
$y_t \sim Normal(u_t, \sigma_e^2)$
This time, the purpose is just to follow the data, meaning I want to get the points of $u_t$. So, I don’t use generated quantities block here. The model is like this.

data {
    int N;
    vector[N] X;
}

parameters {
    vector[N] u;
    real<lower=0> s_u;
    real<lower=0> s_x;
}

model {
    u[2:N] ~ normal(u[1:N-1], s_u);
    X ~ normal(u, s_x);
}

Save the code above to the file, “local_level_model.stan”.

Execute

On Python, we can execute.

import pystan

data_feed = {'X': data, 'N': len(data)}
fit = pystan.stan(file='local_level_model.stan', data=data_feed, iter=1000)

Check the outcome

We can visualize the sampled points by plot() method.

fit.plot()

But on this case, $u$ is vector. The plot doen't make it easy to understand the behaviors of $u$. So, we can extract the sampled points and check the behaviors of those.

samples = fit.extract(permuted=True)
u_mean = samples['u'].mean(axis=0)

The mean of the sampled points and the data can be plotted together.

plt.plot(list(range(len(data))), data)
plt.plot(list(range(len(data))), u_mean)
plt.show()

The $u$ looks valid. On Edward, I have been wanting to do same thing but now struggling.