knitr::opts_chunk$set( collapse = TRUE, comment = "#>" )
This vignette provides a "jump-start" for users that want to quickly understand how to use entropy-pooling (EP) to construct views on the market distribution. The methodology fits well in a variety of scenarios: stress-testing, macro factors, non-linear payoffs, and more.
To demonstrate entropy-pooling's firepower, the current vignette uses some of the cases Meucci (2008)[^1] leaves for the reader in the appendix. In particular, the usage of the following functions are covered in depth:
view_on_mean()
view_on_volatility()
view_on_correlation()
view_on_rank()
view_on_marginal_distribution()
view_on_copula()
[^1]: Meucci, Attilio, Fully Flexible Views: Theory and Practice (August 8, 2008). Fully Flexible Views: Theory and Practice, Risk, Vol. 21, No. 10, pp. 97-102, October 2008, Available at SSRN: https://www.ssrn.com/abstract=1213325
Along with the examples, the EuStockMarkets
dataset - that comes with the installation of R
- is used as a proxy for "the market":
library(ffp) library(ggplot2) set.seed(321) # invariance / stationarity x <- diff(log(EuStockMarkets)) dim(x) head(x)
Assume an investor believes the FTSE
index return will be $20\%$ above average in the near future. In order to process this view, the investor needs to minimize the relative entropy:
$$ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) $$
Subject to the restriction:
$$ \ x_j V_{j, k} = \mu_{k} $$
In which $x_{j}$ is a yet to be discovered posterior probability; $V_{j,k}$ is a $1859 \times 1$ vector with the FTSE
returns; and $\mu_{k}$ is a $1 \times 1$ scalar with the investor projected return for the FTSE
index. In this case, the $k$ subscript represents the fourth column in x
[^2].
[^2]: The commands x[ , 4]
or x[ , "FTSE"]
yield the same results.
Views on expected returns can be constructed with view_on_mean
:
# Returns 20% higher than average mu <- mean(x[ , "FTSE"]) * 1.2 # ffp views constructor views <- view_on_mean(x = as.matrix(x[ , "FTSE"]), mean = mu) views
The views
object is a list
with two components, Aeq
and beq
, that are equivalent to the elements $V_{j,k}$ transposed and $\mu_{k}$, respectively.
The investor also needs to formulate a vector of prior probabilities, $p_j$, which is usually - but not necessary - a equal-weight vector:
# Prior probabilities p_j <- rep(1 / nrow(x), nrow(x))
Once the prior and the views are established, the optimization can take place with entropy_pooling
:
# Solve Minimum Relative Entropy (MRE) ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nlminb" ) ep
What is the interpretation for ep
? Among all the possible probability vectors, ep
is the one that can satisfy the views in the "least" evasive way (in which the term "least evasive" is direct reference to the prior). Therefore, ep
is the best candidate for a posterior distribution.
In real world, the investor can have many views. Extending the current example, say the investor has a new belief: the CAC
index returns will be $10\%$ bellow average[^3]:
[^3]: An ARIMA, GARCH or VAR model could be used as well. The focus is not on the best forecasting method, but how to input the view once the forecast has already been made.
mu <- c( mean(x[ , "FTSE"]) * 1.2, # Return 20% higher than average mean(x[ , "CAC"]) * 0.9 # Return 10% bellow average ) # ffp views constructor views <- view_on_mean(x = as.matrix(x[ , c("FTSE", "CAC")]), mean = mu) views
In which the elements Aeq
and beq
now have $2$ rows each, one for every view.
The minimum relative entropy problem is solved in the exact same way:
# Solve MRE ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nlminb" ) ep
It's easier to analyse the output of ep
visually.
class(ep)
To that end, the autoplot
method is available for objects of the ffp
class:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "Views on Expected Returns", x = NULL, y = NULL)
The plot reinforces the intuition of $x_j$ - the posterior probability - as a distribution that distorts the prior in order to accommodate the views.
It's easy to double-check the investor views by computing the ratio of the conditional vs. unconditional returns:
conditional <- ffp_moments(x, ep)$mu unconditional <- colMeans(x) conditional / unconditional - 1
According to expectations, CAC
outlook is now $10\%$ bellow average and FTSE
expected return is $20\%$ above average. ViolĂ !
Say the investor is confident that markets will be followed by a period of lull and bonanza. Without additional insights about expected returns, the investor anticipates that market volatility for the CAC
index will drop by $10\%$.
To impose views on volatility, the investor needs to minimize the expression:
$$ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) $$ Subject to the constraint:
$$ \sum_{j=1}^{J} x_j V_{j, k}^2 = \mu_{k}^2 + \sigma_{k}^2 $$
In which, $x_j$ is a vector of yet to be defined posterior probabilities; $V_{j,k}^2$ is a $1859 \times 1$ vector with CAC
squared returns; $\mu_{k}^2$ and $\sigma_{k}^2$ are the CAC
sample mean and variance; and $k$ is the column position of CAC
in the panel $V$.
The view_on_volatility
function can be used for this purpose:
# opinion vol_cond <- sd(x[ , "CAC"]) * 0.9 # views views <- view_on_volatility(x = as.matrix(x[ , "CAC"]), vol = vol_cond) views
The views
object holds a list with two components - Aeq
and beq
- that are equivalent to the elements $V_{j,k}^2$ transposed and $\mu_{k}^2 + \sigma_{k}^2$, respectively.
To solve the relative entropy use the entropy_pooling
function:
ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nlminb" ) ep
Once again, the ep
vector is what we call posterior: the probability vector that causes the "least" amount of distortion in the prior, but still obeys the constraints (views).
The posterior probabilities can be observed with the autoplot
method:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "View on Volatility", x = NULL, y = NULL)
To check if the posterior probabilities are valid as a way to display the investor's view, compare the conditional vs. unconditional volatilities:
unconditional <- apply(x, 2, sd) conditional <- sqrt(diag(ffp_moments(x, ep)$sigma)) conditional / unconditional - 1
The posterior volatility for the CAC
index is reduced by $10\%$, in line with the investor's subjective judgment. However, by emphasizing tranquil periods more often, other assets are also affected, but in smaller magnitudes.
Assume the investor believes the correlation between FTSE
and DAX
will increase by $30\%$, from $0.64$ to $0.83$. To construct views on the correlation matrix, the general expression has to be minimized:
$$ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) $$ Subject to the constraints:
$$ \sum_{j=1}^{J} x_j V_{j, k} V_{j, l} = \mu_{k} \mu_{l} + \sigma_{k} \sigma_{l} C_{k,l} $$
In which, the term $V_{j, k} V_{j, l}$ on the left hand-side of the restriction consists of cross-correlations among assets; the terms $\mu_{k} \mu_{l} + \sigma_{k} \sigma_{l} C_{k,l}$ carry the investor target correlation structure; and $x_j$ is a yet to be defined probability vector.
To formulate this view, first compute the unconditional correlation matrix. Second, add a "perturbation" in the corresponding element that is consistent with the perceived view:
cor_view <- cor(x) # unconditional correlation matrix cor_view["FTSE", "DAX"] <- 0.83 # perturbation (investor belief) cor_view["DAX", "FTSE"] <- 0.83 # perturbation (investor belief)
Finally, pass the adjusted correlation matrix into the view_on_correlation
function:
views <- view_on_correlation(x = x, cor = cor_view) views
In which Aeq
is equal to $V_{j, k} V_{j, l}$ transposed and beq
is a $10 \times 1$ vector with $\mu_{k} \mu_{l} + \sigma_{k} \sigma_{l} C_{k,l}$.
Notice that even though the investor has a view in just one parameter[^4] the constraint vector has $10$ rows, because every element of the lower/upper diagonal of the correlation matrix has to match.
[^4]: cor(ftse, dax) = 0.83
.
Once again, the minimum entropy is solved by entropy_pooling
:
ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nlminb" ) ep
And the autoplot
method is available:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "View on Correlation", x = NULL, y = NULL)
To fact-check the investor view, compute the posterior correlation matrix:
cov2cor(ffp_moments(x, p = ep)$sigma)
And notice that the linear association between FTSE
and DAX
is now $0.83$!
Assume the investor believes the DAX
index will outperform the SMI
by some amount, but he doesn't know by how much. If the investor has only a mild view on the performance of assets, he may want to minimize the following expression:
$$ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) $$ Subject to the constraint:
$$ \sum_{j=1}^{J} x_j (V_{j, k} - V_{j, l}) \ge 0 $$
In which, the $x_j$ is a yet to be determined probability vector; $V_{j, k}$ is a $1859 \times 1$ column vector with DAX
returns and $V_{j, l}$ is a $1859 \times 1$ column vector with the SMI
returns. In this case, the $k$ and $l$ subscripts refers to the column positions of DAX
and SMI
in the object x
, respectively.
Views on relative performance can be imposed with view_on_rank
:
views <- view_on_rank(x = x, rank = c(2, 1)) views
Assets that are expected to outperform enter in the rank
argument with their column positions to the right: assets in the left underperform and assets in the right outperform. Because the investor believes that DAX
$\geq$ SMI
(the asset in the first column will outperform the asset in the second column), we fill rank = c(2, 1)
[^5].
[^5]: In the x
object, DAX
it's in the first column and the SMI
it's in the second column.
The optimization is, once again, guided by the entropy_pooling
function:
ep <- entropy_pooling( p = p_j, A = views$A, b = views$b, solver = "nloptr" ) ep
Two important observations:
nlminb
solver. Use nloptr
or solnl
, instead;A
and b
to be fulfilled rather than Aeq
and beq
.The posterior probabilities are presented bellow:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "View on Ranking/Momentum", x = NULL, y = NULL)
To fact-check the ongoing view, compare the conditional vs. unconditional returns:
conditional <- ffp_moments(x, ep)$mu unconditional <- colMeans(x) conditional / unconditional - 1
Indeed, returns for DAX
are adjusted upwards by $17\%$, while returns for SMI
are revised downwards by $6.5\%$. Notice that returns for CAC
and FTSE
were also affected. This behavior could be tamed by combining the ranking condition with views on expected returns. See the function bind_views()
for an example.
Assume the investor has a view on the entire marginal distribution. One way to add views on marginal distributions is by matching the first moments exactly, up to a given order. Mathematically, the investor minimizes the relative entropy:
$$ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) $$ With, say, two equality constraints (one for $\mu$ and one for $\sigma^2$):
$$ \sum_{j=1}^J x_j V_{j,k} = \sum_{j=z}^Z p_z \hat{V}{z,k} $$ $$ \sum{j=1}^J x_j (V_{j,k})^2 = \sum_{z=1}^Z p_z (\hat{V}_{z,k})^2 $$
In which $x_j$ is a yet to be defined probability vector; $V_{j,k}$ is a matrix with the empirical marginal distributions; $p_z$ is vector of prior probabilities; and $\hat{V}{z,k}$ an exogenous panel with simulations that are consistent with the investor's _views.
When $j = z$, the panels $V_{j,k}$ and $\hat{V}{z,k}$ have the same number of rows and the dimensions in both sides of the restrictions match. However, it's possible to set $z \ge j$ to simulate a larger panel for $\hat{V}{z, k}$. Keep in mind though that, if $z \ne j$, two prior probabilities will have to be specified: one for $p_j$ (the objective function) and one for $p_z$ (the views).
Continuing on the example, consider the margins of x
can be approximated by a symmetric multivariate t-distribution. If this is the case, the estimation can be conducted by the amazing ghyp
package, that covers the entire family of generalized hyperbolic distributions:
library(ghyp) # multivariate t distribution dist_fit <- fit.tmv(data = x, symmetric = TRUE, silent = TRUE) dist_fit
The investor then, can construct a large panel with statistical properties similar to the margins he envisions:
set.seed(123) # random numbers from `dist_fit` simul <- rghyp(n = 100000, object = dist_fit)
Where the $100.000 \times 4$ matrix is used to match the views with view_on_marginal_distribution
:
p_z <- rep(1 / 100000, 100000) views <- view_on_marginal_distribution( x = x, simul = simul, p = p_z ) views
The objects simul
and p_z
corresponds to the terms $\hat{V}_{z,k}$ transposed and $p_z$, respectively. Note that $8$ restrictions are created, four for each moment of the marginal distribution (there are four assets in "the market").
With the prior and the views at hand, the optimization can be quickly implemented:
ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nloptr" ) ep
As the visual inspection of the vector ep
:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "View on Marginal Distribution", x = NULL, y = NULL)
It's easy to extend the current example. For instance, the investor could "tweak" some of the fitted parameters for stress-testing (i.e. change the degrees of freedom, increase/decrease expected returns, etc) to verify the immediate impact on the P&L (VaR, CVaR, etc).
Acknowledge that when the restrictions are to harsh, the scenarios will be heavily distorted to satisfy the views. In this case, one may need to compute the effective number of scenarios (ENS). This is very easy to do with ens()
:
ens(ep)
Rest on the investor to calibrate this statistic in order to get a desired confidence level[^6].
[^6]: Meucci, Attilio, Effective Number of Scenarios in Fully Flexible Probabilities (January 30, 2012). GARP Risk Professional, pp. 32-35, February 2012, Available at SSRN: https://www.ssrn.com/abstract=1971808.
Assume the investor would like to simulate a different dependence structure for the market. Views that change the interdependence of assets can be implemented by minimizing the relative entropy:
$$ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) $$ Subject to the following restrictions:
$$ \sum_{j=1}^J x_i U_{j,k} = 0.5$$ $$ \sum_{j=1}^J x_j U_{j,k}U_{j,l} = \sum_{z=1}^Z p_z \hat{U}{z,k}\hat{U}{z,l} $$
$$ \sum_{j=1}^J x_j U_{j,k}U_{j,l}U_{j,i} = \sum_{j=1}^J p_j \hat{U}{j,k}\hat{U}{j,l}\hat{U}_{j,i} $$
In which, the first restriction matches the first moment of the uniform distribution; the second and third restrictions pair the cross-moments of the empirical copula, $U$, with the simulated copula, $\hat{U}$; $x_j$ is a yet to be discovered posterior distribution; $p_z$ is a prior probability; When $j = z$, the dimensions of $p_j$ and $p_z$ match.
Among many of the available copulas, say the investor wants to model the dependence of the market as a clayton copula to ensure the lower tail dependency does not go unnoticed. The estimation is simple to implement with the package copula
:
library(copula) # copula (pseudo-observation) u <- pobs(x) # copula estimation cop <- fitCopula( copula = claytonCopula(dim = ncol(x)), data = u, method = "ml" ) # simulation r_cop <- rCopula( n = 100000, copula = claytonCopula(param = cop@estimate, dim = 4) )
First, the copulas are computed component-wise by applying the marginal empirical distribution in every $k$ column of the dataset. Second, the pseudo-observations are used to fit a Clayton copula by the Maximum-Likelihood (ML) method. Finally, a large panel with dimension $100.000 \times 4$ is constructed to match a Clayton copula with $\hat \alpha = 1.06$, as in the object cop
.
The views on the market is constructed with the view_on_copula
:
views <- view_on_copula(x = u, simul = r_cop, p = p_z) views
Once again, the solution of the minimum relative entropy can be found in one line of code:
ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nloptr" ) ep
And the autoplot
method is available for the objects of the ffp
class:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "View on Copulas", x = NULL, y = NULL)
To stretch the current example, assume the investor stresses the parameter $\hat \alpha = 1.06$^9. Higher $\alpha$ values are linked to extreme occurrences on the left tails of the distribution. Whence, consider the case where $\hat \alpha = 5$:
cop@estimate <- 5
Rerun the previous steps (simulation, views and optimization):
# simulation r_cop <- rCopula( n = 100000, copula = claytonCopula(param = cop@estimate, dim = 4) ) # #views views <- view_on_copula(x = u, simul = r_cop, p = p_z) # relative entropy ep <- entropy_pooling( p = p_j, Aeq = views$Aeq, beq = views$beq, solver = "nloptr" )
To find the new posterior probability vector the satisfy the stress-test condition:
autoplot(ep) + scale_color_viridis_c(option = "C", end = 0.75) + labs(title = "Posterior Probability Distribution", subtitle = "View on Copulas", x = NULL, y = NULL)
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.