CS代考 ARMA Models Properties of AR(p) – cscodehelp代写
ARMA Models Properties of AR(p)
7.4 Properties of AR(p)
We will look at some properties of AR(p) processes
Xt =φ0 +φ1Xt−1 +φ2Xt−2 +···+φpXt−p +Wt.
In particular, Expectation
Autocovariance .
52/74
ARMA Models Properties of AR(p)
Expectation of AR(p)
We will sometimes work with the mean zero AR(p) by assuming φ0 = 0
Xt =φ1Xt−1 +φ2Xt−2 +···+φpXt−p +Wt. (1)
In the general case with φ0 not necessarily 0
Xt =φ0 +φ1Xt−1 +φ2Xt−2 +···+φpXt−p +Wt.
If 0 ̸= 1 − φ1 − · · · − φp , we set μX = φ0
1−φ1 −···−φp
and shift the process by Xt − μX . Re-arranging yields
Xt−μX = φ1(Xt−1−μX)+φ2(Xt−2−μX)+···+φp(Xt−p−μX)+Wt. Hence, E(Xt) = μx.
53/74
ARMA Models Properties of AR(p)
Backshift autoregression formulation
It is common to state the mean zero autoregression in (1) using the backshift or lag operator B Xt = Xt−1
Xt −φ1Xt−1 −φ2Xt−2 −···−φpXt−p =Wt (1−φ1B−φ2B2−···−φpBp)Xt =Wt
A particularly concise formulation is then
φ(B)Xt = Wt (2)
where φ(B) is called the autoregressive operator
φ(B) = (1 − φ1B − φ2B2 − · · · − φpBp). (3)
The AR(p) can then be seen as the solution: Xt= 1 Wt
φ(B )
Let us try to understand what 1 is the case of AR(1). φ(B )
54/74
ARMA Models Properties of AR(p)
Inverting AR(1)
ConsiderXt =φ1Xt−1+Wt.
We eliminate Xt−i from the l.h.s. to achieve Xt =
By recursive substitution we obtain:
1 Wt. φ(B )
Xt =
φ1Xt−1+Wt
= φ1(φ1Xt−2 + Wt−1) + Wt
= φ21Xt−2 + φ1Wt−1 + Wt
= .
= φkX +(φk−1W +···+φW +W)
1 t−k 1 t−k−1 1 t−1 t k−1
= φk1Xt−k +φj1Wt−j j=0
which can be written as:
k−1
φ k1 X t − k = X t − φ j1 W t − j
j=0
55/74
ARMA Models Properties of AR(p)
Inverting AR(1)
Let’s look at the limiting mean square difference:
k−1 2
lim E X −φjW = lim φ2kEX2 =0
k→∞ t 1 t−j k→∞ 1 t−k j=0
provided |φ1| < 1 and σW2 < ∞.
Hence, the limiting form of the zero mean AR(1) model with
|φ1|<1andσW2 <∞is
∞1
Xt =φj1Wt−j=φ(B)Wt.
j=0 So with B Wt = Wt−1
1∞
φ ( B ) = φ j1 B j
j=0
This is the Would decomposition of AR(1).
56/74
ARMA Models Properties of AR(p)
Now it’s easy to calculate the properties of the AR(1)
Expectation without φ0
μX = E(Xt) = φj1E(Wt−j) = 0
j=0
Expectation with φ0 using the formula for μX above
E(Xt)=μX= φ0 1−φ1
∞ IngeneralXt−μX =φj1Wt−j
j=0
SinceXt −μX isAR(1)withφX =0.
∞
57/74
ARMA Models Properties of AR(p)
Now it’s easy to calculate the properties of the AR(1)
Variance for
σX2 =
= E[(φj1Wt−j)2]
=
Var(Xt)=E[(Xt −μX)2] ∞
j=0 ∞
= φ2jE[W2 ] 1 t−j
j=0
∞
= σ2 φ2j W1
j=0
σW2
1 − φ 21
58/74
ARMA Models Properties of AR(p)
Now it’s easy to calculate the properties of the AR(1)
Autocovariance γX (h) =
Cov[(Xt −μX),(Xt+|h| −μX)] ∞∞
1 = Cov φj Wt−j,φk1Wt+|h|−k
j=0 k=0 ∞∞
= CovφjW ,φkW 1 t−j 1
j=0 k=0 ∞
t+|h|−k
= Cov φj W j=0
∞ σ 2 φ | h |
= φj φ|h|+jσ2 = W 1
11 W 1−φ21
j=0
= σ2 φ|h| X1
,φ|h|+jW 1t−j1 t−j
59/74
ARMA Models Properties of AR(p)
Now it’s easy to calculate the properties of the AR(1)
Autocorrelation
ρX(h) = γX(h) = φ|h| σX2 1
Hence, if |φ1| < 1 and σW2 < ∞, AR(1) is stationary.
60/74
ARMA Models Properties of AR(p)
Example: AR(1) autocovariance and autocorrelation
AR(1): Xt = 0.5Xt−1 + Wt time series
0 100
Estimated autocovariance
200
300
400 500
Estimated autocorrelation
Time, t
●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
sample acf ● true acf
●
●●
●●●●●●●●●●●●●●●●●●●●●
0 5 10 15 20 25
0 5 10 15 20 25
lag, h
lag, h
●
●
61/74
0.0
0.2
0.4 0.6 0.8 1.0
1.2
0.0
0.2
0.4 0.6 0.8
1.0
Autocovariance, gamma(h)
Autocorrelation, rho(h)
Xt
−2 0 2 4
ARMA Models Properties of AR(p)
Example: AR(1) autocovariance and autocorrelation
AR(1): Xt = 0.9Xt−1 + Wt time series
0 100
Estimated autocovariance
200
300
400 500
Estimated autocorrelation
Time, t
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
sample acf ● true acf
●●
●●●
0 5 10 15 20 25
0 5 10 15 20 25
lag, h
lag, h
●
●
62/74
Autocovariance, gamma(h)
123456
Autocorrelation, rho(h)
0.0 0.2 0.4 0.6 0.8 1.0
Xt
−4 −2 0 2 4 6 8
ARMA Models Properties of AR(p)
Example: AR(1) autocovariance and autocorrelation
AR(1): Xt = −0.9Xt−1 + Wt time series
0 100 200
Estimated autocovariance
Time, t
300 400 500
Estimated autocorrelation
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
sample acf ● true acf
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
0 5 10 15 20 25
0 5 10 15 20 25
lag, h
lag, h
63/74
Autocovariance, gamma(h)
−4 −2 0 2 4 6
Autocorrelation, rho(h)
−0.5 0.0 0.5 1.0
Xt
−6 −4 −2 0 2 4 6
ARMA Models Properties of AR(p)
What if |φ1| ≥ 1?
We know if |φ1| < 1, AR(1) is stationary.
If φ1 = 1, AR(1) is the random walk.
Xt =Xt−1+Wt =X0+Wi
σ2
The formula Var (Xt ) = W 2 is not applicable.
t i=0
1−φ1
If we assume that X0 is fixed, say X0 = 0, then
t Var(Xt)=Var Wi =tσW2
i=0
Hence, the random walk is not stationary. The variance
depends on t and increases over time.
This is even worse for |φ1| > 1. These AR(1) processes will literally explode.
64/74
ARMA Models Properties of AR(p)
Example: random walk
The random walk Xt = Xt−1 + Wt is not stationary.
65/74
ARMA Models Properties of AR(p)
Example: explosive AR(1)
Xt = 1.02Xt−1 + Wt
0 50 100 150 200
Time, t
The AR(1) process Xt = 1.02Xt−1 + Wt is not stationary and goes rapidly to ∞ or −∞. It is called an explosive process.
66/74
Xt
0 50 100 150 200 250
ARMA Models Properties of AR(p)
The general case AR(p) with p > 1
Iterating backwards to explore stationarity is easy enough for
an AR(1).
But it is cumbersome for AR(p) with p > 1.
A common approach to the general case is to use the backshift representation of AR(p) models that we mentioned before.
67/74
ARMA Models Properties of AR(p)
Back to . . . backshift of AR(p) model
The backshift operator representation of an AR(p) model is
Xt −φ1Xt−1 −φ2Xt−2 −···−φpXt−p =Wt (1−φ1B−φ2B2−···−φpBp)Xt =Wt.
where B Xt = Xt−1. This gives the concise notation φ(B)Xt =Wt.
We call φ(B) the autoregressive operator for this model φ(B) = 1 − φ1B − φ2B2 − · · · − φpBp.
AR(p) can then be seen as the solution to Xt= 1 Wt=ψ(B)Wt
φ(B )
where ψ is the inverse of the polynomial φ.
68/74
ARMA Models Properties of AR(p)
Example: backshift for AR(1) model
For an AR(1) model φ(B) = 1 − φ1B. Let’s take a look at the limiting form again
∞
X t = φ j1 W t − j .
j=0
Now rewrite in the form of the inverse operator
∞
Xt =ψjWt−j =ψ(B)Wt
j=0
where ψ(B) = ∞j=0 ψjBj and ψj = φj1.
But how can we find the inverse coefficients ψj without going through the iterative substitution above?
69/74
ARMA Models Properties of AR(p)
Inverse polynomial method 1: coefficient matching
Since ψ(B) is the inverse of φ(B), φ(B)ψ(B) = 1
which gives in the case of AR(1)
(1 − φ1B) (1 + ψ1B + ψ2B2 + . . . ) = 1
Rearranging gives:
1 + (ψ1 − φ1)B + (ψ2 − ψ1φ1)B2 + · · · = 1.
Coefficient matching between the lhs and rhs gives Clearly, ψ1 = φ1, ψ2 = φ21, …
In general, ψj = φj1 as before.
This method works as well for AR(p) with p > 1.
70/74
ARMA Models Properties of AR(p)
Inverse polynomial method 2: polynomial inversion
It does not matter if we think of φ(B) and ψ(B) as backshift operations or as (infinite) polynomials in complex numbers.
Let z ∈ C be a complex variable. Then still φ(z)ψ(z) = 1 and ψ(z) = φ−1(z).
Working with the autoregressiv operator is just like working with polynomials.
φ(z) = 1 − φ1z − φ2z2 − · · · − φpzp
is called the characteristic polynomial of AR(p).
In the AR(1) case φ(z) = 1 − φ1 z. If |φ1| < 1,
φ−1(z) = 1 = 1+φ1z +φ21z2 +φ31z3 +... for |z| ≤ 1. 1−φ1z
71/74
ARMA Models Properties of AR(p)
Obervation for AR(1)
The root of the characteristic polynomial φ(z) = 1 − φ1 z is z=1.
φ1
So |z| > 1 if and only if |φ1| < 1.
Hence, the process is stationary if and only if |z| > 1.
The root of the backshift polynomial dictates whether the AR(1) is stationary.
This observation is also true for general AR(p) models.
72/74
ARMA Models Properties of AR(p)
Stationarity condition for AR(p) Model Theorem
A (unique) stationary solution to φ(B)Xt = Wt for an AR(p) model exists if and only if
φ ( z ) = 1 − φ 1 z − φ 2 z 2 − · · · − φ p z p = 0 =⇒ | z | > 1 . A complex polynomial of degree p has p roots with
multiplicity.
All roots of the characteristic polynomial must be outside of
the unit circle in C for the AR(p) process to be stationary. We will only use AR(p) with this property in ARMA models.
If a root is exactly on the circle |z| = 1, the process a called a unit root process. It is not stationary.
The random walk a a unit root process.
73/74
ARMA Models Properties of AR(p)
Example: AR(2) with complex roots
AR(2): Xt = Xt−1 − 0.9Xt−2 + Wt .
φ(z) = 1−z +0.9z2
Roots: z1 ≈ 0.556 + i0.896 and z2 ≈ 0.556 − i0.896 |z1| = |z2| ≈ 1.054
The process is stationary.
74/74
