knitr::opts_chunk$set( collapse = TRUE, comment = "#>" )  # Data library(PointFore) library(ggplot2) ggplot(GDP)+ geom_line(aes(x=date,y=observation))+ geom_point(aes(x=date,y=forecast), color = 'red', size = 2, shape=4)  Next to this forecast, which was generated by taking the forecast issued closest to the midpoint of the quarter before the considered quarter, we also consider the forecast issued latest in the respective quarter. ggplot(GDP)+ geom_line(aes(x=date,y=forecast-forecast_late), color = 'red',alpha=.5)+ geom_point(aes(x=date,y=forecast-forecast_late), color = 'red', size = .5, shape=4)+ ylim(-10,10)  We see there is considerable adjustement. Starting 1980, there are normally 2 instead of 3 forecasts issued each quarter. The publication dates vary across time. # Analysis Now, let us analyse the forecasts. First, we consider the forecast issued closest to the midpoint of the respective quarter. Then, we analyze the latest issued forecast with potentially more information. We begin by estimating and testing constant quantile forecasts. First, for the main forecast: res <- estimate.functional(model = constant, state = GDP$forecast, Y=GDP$observation,X=GDP$forecast)
summary(res)


Optimality is rejected with a p-value of r round(summary(res$gmm)$stest$test,digits=2). If we consider less restrictive information sets, the same holds true. res <- estimate.functional(model = constant, instruments = c("lag(lag(Y))","X"), state = GDP$forecast, Y=GDP$observation,X=GDP$forecast)
summary(res)


However, if we consider the forecast issued later in the quarter, the tests cannot reject optimality.

res <- estimate.functional(model = constant,
state = GDP$forecast_late, Y=GDP$observation,X=GDP$forecast_late) summary(res) car::linearHypothesis(res$gmm,"Theta=.5")


Noteworthy, the median forecast is still rejected, which is indicative of an optimal but asymmetric forecast. In the following we try to understand under which conditions the main forecast may be optimal.

## Replication of the results in "Interpretation of Point Forecasts with Unknown Directive" by Schmidt, Katzfuss and Gneiting

First, we replicate the main result from the original paper.

res <- estimate.functional(model = probit_linear,
state = GDP$forecast, Y=GDP$observation,X=GDP$forecast) summary(res) plot(res,hline = TRUE)  The evidence can be summarized as follows. The forecast indicates overshooting in the presence of extreme expected growth. There is strong evidence of overshooting in times of high growth (current forecast > 3) and weak evidence in times of low growth (current forecast < 0, recession expected). Further, there is no evidence that the forecast is sub-optimal as the optimality test has a p-value of r round(summary(res$gmm)$stest$test,digits=2). Hence, the assumption that the forecast is generate as state-dependent quantile $$X_t = q_{m(X_{t},\theta)}(Y_t|\mathcal{F}_t).$$ seems valid.

Further, there is evidence that the forecasted level actually depends on the current forecast as the Wald-test for $\theta_2=0$ has a p-value of r round(summary(res$gmm)$coefficients[2,4],digits=3). That is the assumption that the forecast is generate as optimal constant quantile forecast does not seem to hold:

$$X_t = q_{\tau}(Y_t|\mathcal{F}_t).$$

## Different information sets (instrumental vectors)

In the following we apply the same analysis with different instrumental vectors. The standard in the estimate.functional function is c("lag(Y)","X") which signifies that the instruments are given by $$w_t=(1,y_{t-1},x_t)'.$$

As first alternative, we consider only lagged values as instruments. That is the instruments are given by $$w_t=(1,y_{t-1},y_{t-2})'.$$

r, Y_{t-2})$.'} res <- estimate.functional(model = probit_linear, instruments = c("lag(Y)","lag(lag(Y))"), state = GDP$forecast, Y=GDP$observation,X=GDP$forecast) summary(res) plot(res,hline = TRUE)

As argued in the main paper, this choice often lags power against identifying misspecified models. And indeed, we observe large confidence bands and a high p-value indicating that the model fits well. Arguably this is because the instruments generate a rather limited information set, such that forecast optimality with respect to this limited information set is a weak requirement.

As second alternative, we consider that the lagged outcome $Y_{t-1}$ may not be in the information set of the forecast. As instruments we use just the forecasts $$W_t = \left( 1, X_t, X_{t-1} \right)' '.$$

r)$.'} res <- estimate.functional(model = probit_linear, instruments = c( "X", "lag(X)"), state = GDP$forecast, Y=GDP$observation,X=GDP$forecast)
summary(res)
plot(res,hline = TRUE)


We observe similar results to the main specification in the paper: The test of overidentifying restrictions does not reject optimality, strong evidence of overshooting in times of high expected growth and weak evidence of undershootin in times of low expected growth.

As third alternative, we consider the instrument $$W_t = \left( 1, X_t, Y_{t-2} \right).$$

r)$.'} res <- estimate.functional(model = probit_linear, instruments = c( "X", "lag(lag(Y))"), state = GDP$forecast, Y=GDP$observation,X=GDP$forecast) summary(res) plot(res,hline = TRUE)

We observe similar results to the main specification in the paper: The test of overidentifying restrictions does not reject optimality, strong evidence of overshooting in times of high expected growth and weak evidence of undershootin in times of low expected growth.

Finally, we consider the larger set of instruments used in the other simulation study in the main paper. That is the instruments are given by $$W_t = \left( 1, X_t, X_{t-1} - Y_{t-1}, (X_{t-1} - Y_{t-1})^2, X_{t-1}, X_{t-2} - Y_{t-2}, (X_{t-2} - Y_{t-2})^2 \right)' '.$$

r - Y_{t-1}, (X_{t-1} - Y_{t-1})^2, X_{t-1}, X_{t-2} - Y_{t-2}, (X_{t-2} - Y_{t-2})^2$.'} res <- estimate.functional(model = probit_linear, instruments = c( "X", "lag(X-Y)", "lag(X-Y)^2", "lag(X)", "lag(lag(X-Y))", "lag(lag(X-Y))^2"), state = GDP$forecast, Y=GDP$observation,X=GDP$forecast)
summary(res)
plot(res,hline = TRUE)


Again, the results are robust.

## Different specification models

In this section, we want to allow for other specification models as the linear model with probit link.

First, we consider splines in the state variable. We limit our analysis to quadratic and cubic splines. As the spline models have more parameters, we use again the high dimensional instrumental vector from the simulation setting.

res_quadratic <- estimate.functional(model = probit_spline2,
instruments = c( "X", "lag(X-Y)", "lag(X-Y)^2",
"lag(X)", "lag(lag(X-Y))", "lag(lag(X-Y))^2"),
state = GDP$forecast, Y=GDP$observation,X=GDP$forecast) summary(res_quadratic) res_cubic <- estimate.functional(model = probit_spline3, instruments = c( "X", "lag(X-Y)", "lag(X-Y)^2", "lag(X)", "lag(lag(X-Y))", "lag(lag(X-Y))^2"), state = GDP$forecast, Y=GDP$observation,X=GDP$forecast)
summary(res_cubic)

plot(res_quadratic,hline = TRUE)
plot(res_cubic,hline = TRUE)


The specification models include the linear model as a special case. Hence, it is without surprise that the optimality tests are passed in this case, too. Again we observe that the results are largely in line with the main analysis in the paper in that there is strong evidence for overshooting in times of high expected growth and weaker evidence in times of low growth (or recession). The large confidence bands for more extreme forecasts indicate that the linear model arguably is overly confident for those areas for its strong parametric assumption.

## Different state variables

In this section, we use another typical state variable, namely the lagged value of the observation $Y_{t-1}$.

res <- estimate.functional(model = probit_linear,
state = lag(GDP$observation), Y=GDP$observation,X=GDP$forecast) summary(res) #plot(res,hline = TRUE)  In this case, the optimality of the forecast is rejected with a p-value of r round(summary(res$gmm)$stest$test,digits=2). That is the assumption that the forecast is generate as state-dependent quantile does not seem to hold:

$$X_t = q_{m(Y_{t-1},\theta)}(Y_t|\mathcal{F}_t).$$ In particular, this also rejects the subcase that the forecast is simply a optimal quantile forecast for any level.

## Try the PointFore package in your browser

Any scripts or data that you put into this service are public.

PointFore documentation built on May 2, 2019, 9:42 a.m.