revision.Rmd

---
title: "ETC3550/ETC5550 Applied forecasting"
author: "Revision"
date: "OTexts.org/fpp3/"
classoption: aspectratio=169
titlepage: title16x9.png
titlecolor: burntorange
toc: true
output:
  binb::monash:
    colortheme: monashwhite
    fig_width: 7.5
    fig_height: 3.5
    keep_tex: no
    includes:
      in_header: header.tex
---

```{r setup, include=FALSE}
source("setup.R")
melsyd <- tsibbledata::ansett %>%
  filter(Airports == "MEL-SYD")
```

# Assignment 1

## Assignment 1

**Stock price forecasting** (Q1 and Q5)

* Hard to beat naive forecast
* Random walk model says forecast variance = $h\sigma^2$.

\pause

**Maximum temperature at Melbourne airport on 12 April 2021.** (Q2)

* Weather is relatively stationary over similar time of year and recent years.
* So take mean and var of max temp in April over last 10 years.

## Assignment 1

**Difference in points in AFL match** (Q3)

* Teams vary in strength from year to year.
* Could look at distribution of for-against points from 2020 across all games for each team. Assume distributions independent.

\pause

**Seasonally adjusted estimate of total employment** (Q4)

* Probably locally trended.
* Perhaps use drift method based on average monthly change in last 2 years.

# Some case studies

## CASE STUDY 1: Paperware company

\fontsize{11.5}{13}\sf

\begin{textblock}{9.2}(0.2,1.5)
\textbf{Problem:} Want forecasts of each of hundreds of
items. Series can be stationary, trended or seasonal. They currently
have a large forecasting program written in-house but it doesn't seem
to produce sensible forecasts. They want me to fix it.

\textbf{Additional information}\vspace*{-0.2cm}\fontsize{11.5}{13}\sf
\begin{itemize}\itemsep=0cm\parskip=0cm
\item  Program  written in COBOL making numerical calculations limited. It is not possible to do any optimisation.
\item Their programmer has little experience in numerical computing.
\item They employ no statisticians and want the program to produce forecasts automatically.
\end{itemize}
\end{textblock}

\placefig{10.2}{1.4}{width=5.8cm}{tableware2}

## CASE STUDY 1: Paperware company

### Methods currently used

A
: 12 month average

C
: 6 month average

E
: straight line regression over last 12 months

G
: straight line regression over last 6 months

H
: average slope between last year's and this year's values.
  (Equivalent to differencing at lag 12 and taking mean.)

I
: Same as H except over 6 months.

K
: I couldn't understand the explanation.

## CASE STUDY 2: PBS

\fullwidth{pills}

## CASE STUDY 2: PBS

### The Pharmaceutical Benefits Scheme (PBS) is the Australian government drugs subsidy scheme.

  * Many drugs bought from pharmacies are subsidised to allow more equitable access to modern drugs.
  * The cost to government is determined by the number and types of drugs purchased. Currently nearly 1\% of GDP.
  * The total cost is budgeted based on forecasts of drug usage.

## CASE STUDY 2: PBS

\fullheight{pbs2}

## CASE STUDY 2: PBS

  * In 2001: \$4.5 billion budget, under-forecasted by \$800 million.
  * Thousands of products. Seasonal demand.
  * Subject to covert marketing, volatile products, uncontrollable expenditure.
  * Although monthly data available for 10 years, data are aggregated to annual values, and only the first three years are used in estimating the forecasts.
  * All forecasts being done with the \texttt{FORECAST} function in MS-Excel!

## CASE STUDY 3: Car fleet company

**Client:** One of Australia's largest car fleet companies

**Problem:** how to forecast resale value of vehicles? How
should this affect leasing and sales policies?

\pause

### Additional information
 - They can provide a large amount of data on previous vehicles and their eventual resale values.
 - The resale values are currently estimated by a group of specialists. They see me as a threat and do not cooperate.

## CASE STUDY 4: Airline

\fullheight{ansettlogo}

## CASE STUDY 4: Airline

```{r, echo=FALSE}
melsyd %>%
  filter(Class == "Economy") %>%
  autoplot(Passengers) +
  labs(
    title = "Economy class passengers",
    subtitle = "Melbourne-Sydney",
    x = "Year",
    y = "Thousands"
  )
```

## CASE STUDY 4: Airline

**Problem:** how to forecast passenger traffic on major routes?

### Additional information

  * They can provide a large amount of data on previous routes.
  * Traffic is affected by school holidays, special events such as
the Grand Prix, advertising campaigns, competition behaviour, etc.
  * They have a highly capable team of people who are able to do
most of the computing.

# Review of topics covered

## 1. Introduction to forecasting and R

  - Time series data and `tsibble` objects.
  - What makes things hard/easy to forecast. \pause

## 2. Time series graphics

  - Time plots
  - Seasonal plots
  - Seasonal subseries plots
  - Lag plots
  - ACF
  - White noise

## 3: Time series decomposition

  - Describing a time series: seasonality, trend, cycles, changing variance, unusual features.
  - Transformations (and adjustments)
  - Difference between seasonality and cyclicity.
  - Interpreting a decomposition.
  - Seasonal adjustment.
  - Forecasting and decomposition.

## 5. The forecasters' toolbox

  - Four benchmark methods: naïve, seasonal naïve, drift, mean.
  - Forecasting involves distributions of future observations.
  - Residual diagnostics: white noise, ACF, LB test.
  - Problem of over-fitting.
  - Out-of-sample accuracy. Training/test sets.
  - Measures of forecast accuracy: MAE, MSE, RMSE, MAPE, MASE.
  - Time series cross-validation.
  - One-step prediction intervals based on RMSE from residuals.

## 8: Exponential smoothing

  - Simple exponential smoothing.
  - Holt's local trend method.
  - Damped trend methods.
  - Holt-Winters seasonal method (additive and multiplicative versions).
  - ETS state space formulation.
  - Interpretation of output in R.
  - Computing forecasts by setting future $\varepsilon_t$ to 0.
  - Assumptions for prediction intervals.
  - You have access to formula in the textbook.

## 9: ARIMA models

  - Stationarity.
  - Transformations
  - Differencing (first- and seasonal-differences). What to use when.
  - White noise, random walk, random walk with drift, AR($p$), MA($q$), ARMA($p$,$q$), ARIMA($p,d,q$), ARIMA($p,d,q$)($P,D,Q$)$_m$.
  - ACF, PACF.  Model identification.
  - ARIMA models, Seasonal ARIMA models
  - Order selection and goodness of fit (AICc)
  - Interpretation of output in R.

## 9: ARIMA models (cont'd)

  - Backshift operator notation.
  - Expanding out an ARIMA model for forecasting.
  - Finding point forecasts for given ARIMA process.
  - Assumptions for prediction intervals.
  - One-step prediction intervals based on RMSE.
  - Effect of differencing on forecasts.
  - Effect of a constant on forecasts.
  - ARIMA vs ETS.

## 7: Multiple regression

  - Interpretation of coefficients and R output and residual diagnostics.
  - Dummy variables, seasonal dummies, piecewise linear trends, interventions.
  - Harmonic regression.
  - Variable selection.
  - AIC, AICc, BIC, $R^2$, adjusted $R^2$.
  - Ex ante vs ex post forecasts.
  - Scenario forecasting.
  - (Matrix formulation.)

## 10: Dynamic regression models

  - Problems with OLS and autocorrelated errors.
  - Regression with ARIMA errors.
  - Difference between regression residuals and ARIMA (innovation) residuals.
  - Dynamic harmonic regression (and other specifications). Review the last lecture examples.
  - Stochastic vs deterministic trends.
  - Using lagged predictors
  - Forecasting for dynamic regression models with ARIMA errors

# Exam

## Exam: 9.00am (AEST) 15 June
\fontsize{13}{15}\sf

Five Sections, all to be attempted.

A
: Short answers/explanations. Write about 1/4 page on four topics (out of six possible topics). Nuanced answers required. \pause

B
: Describing a time series, decomposition, choosing a forecasting method. \pause

C, D, E
: Benchmarks, ETS models, ARIMA models, Dynamic regression models, forecast evaluation.\pause

Sections \alert{B}, \alert{C}, \alert{D} and \alert{E} require interpretation of R output, but no coding. \pause

###
  $\bullet$ [\alert{E-Assessment}](https://www.monash.edu/students/admin/exams/electronic-exams)\qquad\qquad\qquad $\bullet$ Open access\qquad\qquad\qquad $\bullet$ Invigilated\newline
  $\bullet$ 2 hours 10 minutes + (30 minutes to upload images).

## Preparing for the exam

  - Exams from 2019--2021 on Moodle already.
  - Solutions to follow soon.
  - Exercises. Make sure you have done them all (especially the last two topics - revise the lecture examples)!
  - Identify your weak points and practice them.
  - Write your own summary of the material.
  - Practice explaining the material to a class-mate.

## Help available

  * See us during the consultation times (for details refer to the moodle page.
  * Discuss on the moodle forum.

## Useful resources for forecasters
\fontsize{14}{15}\sf

\alert{Organization:}

 * International Institute of Forecasters.

\alert{Annual Conference:}

 * International Symposium on Forecasting

    *   Oxford, UK, July 4-7, 2022.
    *   Free to our student members ($25).

\alert{Journals:}

 * International Journal of Forecasting
 * Foresight (the practitioner's journal)

###
Links to all of the above at **www.forecasters.org**

## IIF Best Student Award
\fontsize{14}{16}\sf

  - https://forecasters.org/programs/research-awards/students/
  - US$100
  - A certificate of achievement from the IIF
  - One year free membership of the Institute with all attendant benefits. Subscriptions to:

    -   the International Journal of Forecasting
    -   the practitioner journal: Foresight
    -   The Oracle newsletter

Discounts on conference and workshop fees, and links to a worldwide community of forecasters in many disciplines.

## Happy forecasting

\begin{block}{}
Good forecasters are not smarter than everyone else, they merely have their ignorance better organised.
\end{block}

\vspace*{2cm}\pause

\begin{alertblock}{}\centering
Please fill in your SETU
\end{alertblock}