Content

Trend only Estimation of trend only Elimination of trend by differencing Trend and seasonal component Estimation of trend and seasonal component Eliminating the trend and seasonal component <- Go Back

401-4623-00: Time Series Analysis

Section 2

Sample Estimation and Elimination of the Trend and Seasonal Component

Swiss Federal Institute of Technology Zurich

Eidgenössische Technische Hochschule Zürich

Last Edit Date: 12/16/2024

Disclaimer and Term of Use:

1. We do not guarantee the accuracy and completeness of the summary content. Some of the course material may not be included, and some of the content in the summary may not be correct. You should use this file properly and legally. We are not responsible for any results from using this file

2. This personal note is adapted from Professor Fadoua Balabdaoui and Professor Nicolai Meinshausen. Please contact us to delete this file if you think your rights have been violated.

3. This work is licensed under a Creative Commons Attribution 4.0 International License.

---

Let $\{X_t\}_t$ be a time series such that $X_t = m_t + S_t + Y_t$, $t \in \mathbb{Z}$ with $\{Y_t\}_t$ a 0-mean time series, $\{m_t\}_t$ the trend, and $\{S_t\}_t$ the seasonal component with period $d \in \{1, 2, \cdots\}$.

Trend only

Suppose we have $S_t = 0$ and $X_t = m_t + Y_t$ with $m_t = \mathbb{E}[X_t]$, where $t \in \mathbb{Z}$.

Estimation of trend only

2-sided moving average estimation

We assume that $\{m_t\}_t$ is approximately linear over $[t - q, t + q]$ for some integer $q \ge 1$. We also have $\exists a, b \in \mathbb{R}$ such that $m_{t'} = a + b t'$, where $\forall t' \in \{t - q, \cdots, t + q\}$.

Let us define the 2-sided moving average estimator at time point $t$ as

$$\hat{m}_t = \frac{1}{2 q + 1}\sum_{i = t - q}^{t + q} X_i = \frac{1}{2 q + 1}\sum_{j = - q}^{q} X_{t-j}.$$

Note that $\hat{m}_t$ uses the sample $X_1, X_2, \cdots, X_n \Rightarrow \{t-q, \cdots, t+q\} \subseteq \{1, \cdots, n\}$.

$$ \begin{align} \hat{m}_t &= \frac{1}{2 q + 1}\sum_{j = - q}^{q} \left( m_{t-j} + Y_{t-j} \right) \\ &= \frac{1}{2 q + 1}\sum_{j = - q}^{q} m_{t-j} + \frac{1}{2 q + 1}\sum_{j = - q}^{q} Y_{t-j} \\ \end{align} $$

As we have $m_{t'} = a + b t'$, we can get

$$ \begin{align} \frac{1}{2 q + 1}\sum_{j = - q}^{q} m_{t-j} &\approx \frac{1}{2 q + 1}\sum_{j = - q}^{q} \left( a + b (t - j) \right) \\ &= \frac{1}{2 q + 1}\sum_{j = - q}^{q} \left( a + bt - bj \right) \\ &= a + bt - \frac{b}{2 q + 1}\underbrace{\sum_{j = - q}^{q} j}_0 \\ &= a + bt \approx m_t \end{align} $$

The noise term is $\frac{1}{2 q + 1}\sum_{j = - q}^{q} Y_{t-j}$. Because of the law of large number, when $q$ is large enough, it will hold that

$$\frac{1}{2 q + 1}\sum_{j = - q}^{q} Y_{t-j} \approx 0$$

Mean squared error of $\hat{m}_t$

The mean squared error of $\hat{m}_t$ is given by

$$ \begin{align} \mathbb{E}[(\hat{m}_t - m_t)^2] &= \mathbb{E}\left[\left(\sum_{j = -q}^q X_{t - j} - m_t\right)^2\right] \\ &= \mathbb{E}\left[\left( \frac{1}{2q + 1} \sum_{j = -q}^q m_{t - j} - m_t + \frac{1}{2q + 1} \sum_{j = -q}^q Y_{t - j} \right)^2\right] \\ &= \left( \frac{1}{2q + 1} \sum_{j = -q}^q m_{t - j} - m_t \right)^2 + 2 \times \left( \frac{1}{2q + 1} \sum_{j = -q}^q m_{t - j} - m_t \right) \times 0 + \mathbb{E} \left[\left( \frac{1}{2q + 1} \sum_{j = -q}^q Y_{t - j} \right)^2\right] \\ &= \left( \frac{1}{2q + 1} \sum_{j = -q}^q m_{t - j} - m_t \right)^2 + \mathrm{Var}\left( \frac{1}{2q + 1} \sum_{j = -q}^q Y_{t - j} \right) \end{align} $$

If $\{Y_t\}_t \sim WN(0, \sigma^2)$, then $\mathrm{Var}\left( \frac{1}{2q + 1} \sum_{j = -q}^q Y_{t - j} \right) = \frac{2}{(2q + 1)^2} \sum_{j = -q}^q \sigma^2 = \frac{\sigma^2}{2q + 1}$, now we have

$$ \begin{align} \mathbb{E}[(\hat{m}_t - m_t)^2] &= \left( \frac{1}{2q + 1} \sum_{j = -q}^q m_{t - j} - m_t \right)^2 + \frac{\sigma^2}{2q + 1} \\ &= \underbrace{\ell^2(q)}_{\text{linear approximation gets worse with large q}} + \underbrace{\frac{\sigma^2}{2q + 1}}_{\text{almost equal to 0 as } q \rightarrow \infty} \end{align} $$

This is so called bias-variance trade off.

Polynomial fitting

We assume the trend $m_t = \alpha_0 + \alpha_1 t + \cdots + \alpha_k t^k$ for $\alpha_0, \cdots, \alpha_k \in \mathbb{R}$.

To estimate $\alpha_0, \cdots, \alpha_k$, we can look for

$$ (\hat{\alpha}_0, \hat{\alpha}_1, \cdots, \hat{\alpha}_k) = \underset{(\alpha_0, \cdots, \alpha_k)\in \mathbb{R}^{k + 1}}{\operatorname{argmin}} \sum_{t = 1}^n \left( X_t - \hat{\alpha}_0 + \hat{\alpha}_1 t + \cdots + \hat{\alpha}_k t^k \right)^2, $$

which is a least square estimation. Let us denote $C(\alpha_0, \alpha_1, \cdots, \alpha_l) = \sum_{t = 1}^n \left( X_t - \hat{\alpha}_0 + \hat{\alpha}_1 t + \cdots + \hat{\alpha}_k t^k \right)^2$. The estimator $(\hat{\alpha}_0, \hat{\alpha}_1, \cdots, \hat{\alpha}_k)$ is a stationary point of $C$.

$$ \frac{\partial C}{\partial \alpha_j} \Big|_{(\alpha_0, \alpha_1, \cdots, \alpha_k) = (\hat{\alpha}_0, \hat{\alpha}_1, \cdots, \hat{\alpha}_k)} = 0, \forall j \in \{0, 1, \cdots, k\} $$

$$ \Leftrightarrow -2 \sum_{t = 1}^n \left( X_t - \hat{\alpha}_0 - \hat{\alpha}_1 t - \cdots - \hat{\alpha}_k t^k \right) t^j = 0 $$

If $k = 1$, meaning that $j \in \{ 0, 1 \}$, then $\hat{\alpha}_0$ and $\hat{\alpha}_1$ solve the following linear system and get

$$ \begin{cases} \sum_{t = 1}^n \left( X_t - \hat{\alpha}_0 - \hat{\alpha}_1 t \right) = 0 & j = 0 \\ \sum_{t = 1}^n \left( X_t - \hat{\alpha}_0 - \hat{\alpha}_1 t \right)t = 0 & j = 1 \\ \end{cases} \Rightarrow \begin{cases} \hat{\alpha}_0 = \bar{X} - \frac{n + 1}{2} \hat{\alpha}_0 \\ \hat{\alpha}_1 = \frac{12}{n(n^2 - 1)} \sum_{t = 1}^n (X_t - \bar{X})t \end{cases} $$

where $\bar{X} = \frac{1}{n} \sum_{t = 1}^n X_t$.

Elimination of trend by differencing

Define $B$ as the backward shift operator. We can have

$$BX_t = X_{t - 1}, \forall t \in \mathbb{Z}$$

Moreover, define $\nabla$ as the lag-1 difference operator. We can have

$$\nabla X_t = X_t - X_{t - 1} = X_t - BX_t = (1-B)X_t$$

with $B^0 = 1$ is identity.

For any integer $j \ge 1$, we can define recursively

$$ B^j = B \circ B^{j - 1} \rightarrow B^j X_t = X_{t - j} $$

$$ \begin{align} \nabla^j = \nabla \circ \nabla^{j - 1} \rightarrow \nabla^j X_t &= \underbrace{(1 - B) \circ (1 - B) \circ \cdots \circ (1 - B)}_{j\text{ times}} X_t \\ &= (1 - B)^j X_t \\ &= \left( \sum_{k = 0}^j (-1)^l \binom{j}{k} B^k \right) X_t \\ &= \sum_{k = 0}^j (-1)^k \binom{j}{k} X_{t - k} \end{align} $$

If $j = 2$, then $\nabla^2 X_t = X_t - X_{t - 1} - (X_{t - 1} - X_{t - 2}) = X_t - 2X_{t - 1} + X_{t - 2}$.

• Assume $m_t = \alpha + \beta t$, then

  $$\nabla m_t = m_t - m_{t - 1} = \alpha + \beta t - a - \beta (t - 1) = \beta.$$

• Assume $m_t = \alpha_0 + \alpha_1 t + \cdots + \alpha_k t^k$, then

  $$\nabla^k m_t = k! \alpha_k.$$

  If $X_t = m_t + Y_t$ with $m_t = \sum_{j = 0}^k \alpha_j t^j$, then

  $$ \begin{align} \nabla^k X_t &= k! \alpha_k + \nabla^k Y_t \\ &= k! \alpha_k + Z_t \end{align} $$

  where $Z_t = \nabla^k Y_t = \sum_{i = 0}^k (-1)^i \binom{k}{i} Y_{t - i}$.

Trend and seasonal component

Suppose we have

$$X_t = \underbrace{m_t}_{\text{trend}} + \underbrace{S_t}_{\text{periodic function with period }d \ge 0} + \underbrace{Y_t}_{\text{noise (0-mean time series)}}, t \in \mathbb{Z}.$$

Additionally, we assume that

• $S_1 + S_2 + \cdots + S_d = 0$

• $X_t = m_t - a + S_t + a + Y_t$, note that $\underbrace{\sum_{t = 1}^d (S_t + a)}_{0} = \underbrace{\sum_{t = 1}^d S_t}_{0} + \underbrace{da}_{0} \Leftrightarrow a = 0$

Estimation of trend and seasonal component

2-sided moving average estimation

We assume the trend

$$m_{t'} \approx a + bt', \forall t' \in \{ t - q, \cdots, t + q \}$$

where $q = \begin{cases}\frac{d}{2} & d\text{ is even}\\ \frac{d-1}{2} & d\text{ is odd}\end{cases}$.

Estimation of $m_t$

The 2-sided moving average estimator of $m_t$ is given by

$$ \hat{m}_t = \begin{cases} \frac{1}{d} \{ \frac{1}{2} (X_{t - q} + X_{t + q}) + \sum_{j = -(q - 1)}^{q - 1} X_{t - j}\} & \text{if } d \text{ is even} \\ \frac{1}{d}\sum_{j = -q}^q X_{t - j} & \text{if } d \text{ is odd} \end{cases} $$

Let us focus on the case where $d$ is even (the other case can be treated similarly).

$$ \begin{align} \hat{m}_t &= \frac{1}{d} \Bigg\{ \frac{1}{2} (m_{t-q} + m_{t+q}) + \sum_{j=-(q-1)}^{q-1} m_{t-j} \\ &\quad + \frac{1}{2} (S_{t-q} + S_{t+q}) + \sum_{j=-(q-1)}^{q-1} S_{t-j} \\ &\quad + \frac{1}{2} (Y_{t-q} + Y_{t+q}) + \sum_{j=-(q-1)}^{q-1} Y_{t-j} \Bigg\} \\ &= \underbrace{\frac{1}{d} \Bigg\{ \frac{1}{2} (m_{t-q} + m_{t+q}) + \sum_{j=-(q-1)}^{q-1} m_{t-j} \Bigg\}}_{\text{term 1}} \\ &\quad + \underbrace{\frac{1}{d} \Bigg\{ \frac{1}{2} (S_{t-q} + S_{t+q}) + \sum_{j=-(q-1)}^{q-1} S_{t-j} \Bigg\}}_{\text{term 2}} \\ &\quad + \underbrace{\frac{1}{d} \Bigg\{ \frac{1}{2} (Y_{t-q} + Y_{t+q}) + \sum_{j=-(q-1)}^{q-1} Y_{t-j} \Bigg\}}_{\text{term 3}}. \end{align} $$

• For term 1, we have

  $$ \begin{align} \frac{1}{d} \left\{ \frac{1}{2} (m_{t-q} + m_{t+q}) + \sum_{j=-(q-1)}^{q-1} m_{t-j} \right\} &\approx \frac{1}{d} \left\{ \frac{1}{2} \left(a + b(t-q) + a + b(t+q)\right) + \sum_{j=-(q-1)}^{q-1} (a+b(t-j)) \right\} \\ &= \frac{1}{d} \left\{ a + bt + (2(q-1)+1) (a + bt) \right\} \\ &= \frac{1}{d} \left\{ 2q (a + bt) \right\} \\ &= \frac{1}{d} \left\{ 2 \left(\frac{d}{2}\right) (a + bt) \right\} \\ &= a + bt \approx m_t \end{align} $$

• For term 2, we have

  $$ \begin{align} \frac{1}{d} \left\{ \frac{1}{2} (S_{t-q} + S_{t+q}) + \sum_{j=-(q-1)}^{q-1} S_{t-j} \right\} &= \frac{1}{d} \left\{ \frac{1}{2} (S_{t-q} + S_{t-q+2q}) + \sum_{j=-(q-1)}^{q-1} S_{t-j} \right\} \\ &= \frac{1}{d} \left\{ \frac{1}{2} (S_{t-q} + S_{t-q+d}) + \sum_{j=-(q-1)}^{q-1} S_{t-j} \right\} \\ &= \frac{1}{d} \left\{ S_{t-q} + \sum_{j=-(q-1)}^{q-1} S_{t-j} \right\} \\ &= \frac{1}{d} \left\{ S_{t-q} + S_{t-q+1} + \cdots + S_{t+q-1} \right\} \\ \end{align} $$

  Note that $S_{t-q} = S_{t-q+d}$ because $d$ is even and the periodic property. Now let us denote $t' = t - q$.

  • If $t' = 1$, which is $t = q + 1$, then we have

    $$\frac{1}{d} \{S_1 + S_2 + \cdots + S_d\} = 0$$

  • If $t' > 1$, with periodic proerty, then we have

    $$\frac{1}{d} \left\{\underbrace{S_1 + \cdots + S_{t'-1}}_{0} + \underbrace{S_{t'} + \cdots + S_{t' + d - 1} - (S_1 + \cdots + S_{t' - 1})}_{0} \right\} = 0$$

  As a result, $\frac{1}{d} \left\{ \frac{1}{2} (S_{t-q} + S_{t+q}) + \sum_{j=-(q-1)}^{q-1} S_{t-j} \right\} = 0$.

• For term 3, we have

  $$\frac{1}{d} \left\{ \frac{1}{2} (Y_{t-q} + Y_{t+q}) + \sum_{j=-(q-1)}^{q-1} Y_{t-j} \right\} = 0$$

  if $q$ is large enough ($d$ is large enough). Law of large number.

Finally, we can have

$$\hat{m}_t \approx m_t.$$

Estimation of $S_t$

In order to estimate $\{S_t\}_t$, consider the following estomator

$$W_k = \frac{1}{N_k} \sum_{j: q+1 \le k+jd \le n-q} (X_{k+jd} - \hat{m}_{k+jd})$$

where $N_k = \text{number of }\{j: q+1 \le k+jd \le n-q\}$.

• For $t \in \{1,2, \cdots, d\}$

  $$\hat{S}_t = W_t - \frac{1}{d}\sum_{k=1}^d W_k$$

  Note that $\sum_{t = 1}^d \hat{S}_t = 0$.

• For $t > d$

  $$\hat{S}_t = \hat{S}_{t - d}$$

Eliminating the trend and seasonal component

Define $\nabla_d$ as the lag-d differencing operator, we have

$$ \begin{align} \nabla_d X_t &= X_t - X_{t - d}\\ &= (1 - B^d) X_t \end{align} $$

or we can also have

$$ \begin{align} \nabla_d X_t &= \nabla_d m_t + \nabla_d S_t + \nabla_d Y_t\\ &= m_t - m_{t-d} + Y_t - Y_{t-d}\\ &= m_t - m_{t-d} + Z_t \end{align} $$

note that we eliminated $\nabla_d S_t$ because of the periodic property of $S_t$, meaning $S_t = S_{t - d} \Rightarrow S_t - S_{t - d} = 0$.

We can eliminate the trend $m_t - m_{t-d}$ by applying the operator $\nabla^k$ if $m_t - m_{t-d}$ is a polynomial of degree $k$.

© Sample Academy