Regression with ARMA Errors

This lecture will cover:

• Regression with ARMA Errors

The regression model with ARMA errors is given by:

\[ Y_t = \beta_{0t} + \beta_{1t} X_{1t} + \ldots + \beta_{pt} X_{pt} + \varepsilon_t \]

where \(\varepsilon_t\) is an ARMA(p,q) time series model:

\[ \phi(B)\varepsilon_t = \theta(B) Z_t \]

\(Z_t\) is a white noise.

US Change Data

Daily air quality measurements in New York, May to September 1973.

library(itsmr)
library(astsa)
library(forecast)
library(fpp3)

autoplot(us_change,Consumption) +
  xlab("Year") + ylab("") + 
  ggtitle("Quarterly changes in US consumption")

autoplot(us_change,Income) +
  xlab("Year") + ylab("") + 
  ggtitle("Quarterly changes in US Personal Income")

fit = auto.arima(us_change$Consumption, 
                 xreg=us_change$Income)
fit

Series: us_change$Consumption 
Regression with ARIMA(1,0,2) errors 

Coefficients:
         ar1      ma1     ma2  intercept    xreg
      0.7070  -0.6172  0.2066     0.5949  0.1976
s.e.  0.1068   0.1218  0.0741     0.0850  0.0462

sigma^2 = 0.3113:  log likelihood = -163.04
AIC=338.07   AICc=338.51   BIC=357.8

The equation of the model:

\[ Y_t = 0.5949 + 0.1976 Income_t + \varepsilon_t \] and \[ \varepsilon_t = 0.7\varepsilon_{t-1} -0.62 Z_{t-1} + 0.21 Z_{t-2} + Z_t \]

Overall,

\[ Y_t = 0.5949 + 0.1976 Income_t + 0.7\varepsilon_{t-1} -0.62 Z_{t-1} + 0.21 Z_{t-2} + Z_t \]

Type of Residuals

## regression residuals
test(residuals(fit,type="regression"))

Null hypothesis: Residuals are iid noise.
Test                        Distribution Statistic   p-value
Ljung-Box Q                Q ~ chisq(20)     52.62     1e-04 *
McLeod-Li Q                Q ~ chisq(20)     13.06     0.875
Turning points T  (T-130.7)/5.9 ~ N(0,1)       139    0.1582
Diff signs S       (S-98.5)/4.1 ~ N(0,1)       100    0.7126
Rank P         (P-9751.5)/466.1 ~ N(0,1)      8656    0.0188 *

## ARMA residuals -innovation-
test(residuals(fit,type="innovation"))

Null hypothesis: Residuals are iid noise.
Test                        Distribution Statistic   p-value
Ljung-Box Q                Q ~ chisq(20)     13.03    0.8763
McLeod-Li Q                Q ~ chisq(20)     26.53    0.1489
Turning points T  (T-130.7)/5.9 ~ N(0,1)       137    0.2835
Diff signs S       (S-98.5)/4.1 ~ N(0,1)       104    0.1768
Rank P         (P-9751.5)/466.1 ~ N(0,1)      8952    0.0863

Air Quality data

summary(airquality)

     Ozone           Solar.R           Wind             Temp      
 Min.   :  1.00   Min.   :  7.0   Min.   : 1.700   Min.   :56.00  
 1st Qu.: 18.00   1st Qu.:115.8   1st Qu.: 7.400   1st Qu.:72.00  
 Median : 31.50   Median :205.0   Median : 9.700   Median :79.00  
 Mean   : 42.13   Mean   :185.9   Mean   : 9.958   Mean   :77.88  
 3rd Qu.: 63.25   3rd Qu.:258.8   3rd Qu.:11.500   3rd Qu.:85.00  
 Max.   :168.00   Max.   :334.0   Max.   :20.700   Max.   :97.00  
 NA's   :37       NA's   :7                                       
     Month            Day      
 Min.   :5.000   Min.   : 1.0  
 1st Qu.:6.000   1st Qu.: 8.0  
 Median :7.000   Median :16.0  
 Mean   :6.993   Mean   :15.8  
 3rd Qu.:8.000   3rd Qu.:23.0  
 Max.   :9.000   Max.   :31.0  
                               

fit.temp = auto.arima(y = airquality$Temp, 
                      xreg = cbind(airquality$Wind,airquality$Ozone))
fit.temp

Series: airquality$Temp 
Regression with ARIMA(2,1,0) errors 

Coefficients:
          ar1      ar2  drift    xreg1   xreg2
      -0.5881  -0.2861  0.023  -0.1363  0.0719
s.e.   0.1040   0.1063  0.212   0.1417  0.0182

sigma^2 = 18.61:  log likelihood = -349.91
AIC=711.82   AICc=712.4   BIC=729.96

The equation

\[ y_t = -0.13 Wind + 0.072 Ozone + \varepsilon_t \]

where

\[ (1+0.58B+0.28B^2)(1-B)\varepsilon_t = 0.023 + Z_t \]

Lake Data

plot(lake,type="o")

t = 1:98
fit.lake = auto.arima(lake, xreg = t)
fit.lake

Series: lake 
Regression with ARIMA(1,0,1) errors 

Coefficients:
         ar1     ma1  intercept     xreg
      0.6526  0.3566    10.0822  -0.0211
s.e.  0.0944  0.1149     0.5104   0.0089

sigma^2 = 0.476:  log likelihood = -101.2
AIC=212.4   AICc=213.05   BIC=225.32

Equation:

\[ y_t = 10.08 -0.02t + 0.65\varepsilon_{t-1} + 0.35 Z_{t-1} + Z_t \]

test(residuals(fit.lake,"innovation"))

Null hypothesis: Residuals are iid noise.
Test                        Distribution Statistic   p-value
Ljung-Box Q                Q ~ chisq(20)      9.42    0.9775
McLeod-Li Q                Q ~ chisq(20)      20.6     0.421
Turning points T     (T-64)/4.1 ~ N(0,1)        67    0.4682
Diff signs S       (S-48.5)/2.9 ~ N(0,1)        48    0.8618
Rank P         (P-2376.5)/162.9 ~ N(0,1)      2332    0.7847