knitr::opts_chunk$set( message = FALSE, warning = FALSE, collapse = TRUE, comment = "#>", out.width = "100%" )
In recent years, an irrational phenomenon in financial markets is grabbing the attention of behavioral economists: the disposition effect. Firstly discovered by H. Shefrin and M. Statman (1985), the disposition effect consists in the realization that investors are more likely to sell an asset when it is gaining value compared to when it is losing value. A phenomenon which is closely related to sunk costs’ bias, diminishing sensitivity, and loss aversion.
From 1985 until now, the disposition effect has been documented in US retail stock investors as well as in foreign retail investors and even among professionals and institutions. By the time, it is a well-established fact that the disposition effect is a real behavioral anomaly that strongly influences the final profits (or losses) of investors. Furthermore, being able to correctly capture these irrational behaviors timely is even more important in periods of high financial volatility as nowadays.
The dispositionEffect
package allows to quickly evaluate the presence of
disposition effect’s behaviors of an investor based solely on his transactions
and the market prices of the traded assets.
library(dispositionEffect) library(dplyr) library(tidyr) library(lubridate) library(skimr) library(ggplot2) library(ggridges)
The dataset DEanalysis
is provided within the package allowing to
reproduce a full analysis on the disposition effect.
help("DEanalysis")
The disposition effect analysis is performed on two fundamental types of data frames:
portfolio transactions, that is all the financial transactions an investor did during a specific period of time. A single transaction is made up of 6 features: the investor id, the asset id, the type of the transaction (it can be a buy or a sell), the traded quantity, the traded price, and the datetime.
market prices, that is the prices found on the stock markets for each traded asset and each transaction datetimes.
First of all, we need to extract and understand the structure of the transaction dataset.
trx <- DEanalysis$transactions # transactions head(trx)
skimr::skim(trx)
The portfolio transaction dataset is made up of the six
fundamental variables described above.
This real sample dataset contains transactions on 10 investors
on 337 traded assets, from January 2010 until December 2018.
One important feature is the type
variable. It states if
a transaction is a "Buy" (B) or a "Sell" (S), and only this
two values are allowed.
unique(trx$type)
Moreover, as expected, not all the investors are active on the whole period of analysis.
# number of transactions of each investor over years trx %>% dplyr::mutate(year = lubridate::year(datetime)) %>% dplyr::count(investor, year) %>% dplyr::arrange(year) %>% tidyr::pivot_wider(names_from = year, values_from = n) %>% dplyr::left_join(dplyr::count(trx, investor), by = "investor")
Clearly, they have similar number of transactions as a whole, but they traded on different years.
The market prices dataset needs only to have three variables: asset, datetime and price.
mkt <- DEanalysis$marketprices # market prices head(mkt)
skimr::skim(mkt)
Again, not all the assets prices are available in every year because we only need the market prices for those assets for the traded datetime (i.e. the datetime of the transactions dataset).
mkt %>% dplyr::mutate(year = lubridate::year(datetime)) %>% dplyr::count(asset, year) %>% dplyr::arrange(year) %>% tidyr::pivot_wider(names_from = year, values_from = n) %>% head(10)
Now, first proceed to analyze the behavior of a single investor.
# Investor QZ621 trx_QZ621 <- dplyr::filter(trx, investor == "QZ621") # transactions mkt_QZ621 <- dplyr::filter(mkt, asset %in% unique(trx_QZ621$asset)) # market prices
Based solely on this two data frames it is possible to compute the so-called realized gains (RG), realized losses (RL), paper gains (PG), and paper losses (PL), as defined by L. Mazzucchelli et al. (2021).
To sum up the main concepts are the followings:
Realized Gain / Loss => whenever an investor closes a position in its portfolio in gain / loss
Paper Gain / Loss => all the open positions at the moment of the transaction, and all the partially closed positions.
The portfolio_compute
is the core interface of the package and
it is used to perform all the gains and losses computations.
p_res <- portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621) head(p_res)[, -5] skimr::skim(p_res)
Hence, the result is a data frame containing:
Once that gains and losses have been computed, it is finally possible to evaluate both the disposition effect of the investor and of each traded assets, where the disposition effect is defined as:
$$DE = \bigg(\frac{RG}{RG + PG}\bigg) - \bigg(\frac{RL}{RL + PL}\bigg)$$
The DE varies between -1 and 1. Positive DE values show the presence of disposition effect irrational behaviors, while negative values show the presence of opposite disposition effect behaviors. A value of zero show that no disposition effect exists.
You almost never want to compute the disposition effect directly
via the disposition_effect
function, but you will mostly rely
on the quicker and easier disposition_compute
interface, since
it designed to handle many situations.
de <- disposition_compute(gainslosses = p_res) head(de)
As can be seen, disposition_compute
calculates a value of disposition
effect for each asset. In order to obtain the value of disposition
effect of the investor, one can simply compute an aggregate statistic,
such as the mean or the median, on the assets' values.
To do this we can simply use once again disposition_compute
specifying
the desired aggregate_fun
.
disposition_compute(gainslosses = p_res, aggregate_fun = mean, na.rm = TRUE)
Moreover, by means of the disposition_summary
function
it is also easy to summarize the disposition effect behavior
of the investor, obtaining common summary statistics.
de_stat <- disposition_summary(gainslosses = p_res) de_stat
Until now, we limited our analysis to the default parameters of
portfolio_compute
. However, this function has many different
arguments that can be used both to fine tune the analysis and to
perform more advanced calculations, such as the so-called
portfolio driven disposition effect and the
time series disposition effect.
Hence, we focus here on the usage of five fundamental different arguments.
Let's start by the method
argument. It is probably the most relevant
parameter the user can control since it allows to perform five different
types of analysis.
If set to "none", no gains and losses are computed but the investor's portfolio is updated at every transaction, resulting in the actual portfolio of the investors at time T (the end of the period).
portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, method = "none") %>% head()
If set to one of "count", "total", "value", or "duration", gains and losses are computed for the corresponding method.
portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, method = "count") %>% head() portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, method = "total") %>% head() portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, method = "value") %>% head() portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, method = "duration") %>% head()
In particular:
count
computes gains and losses as simple counts total
calculates gains and losses as the sum of the quantity value
measures the expected percentage gains and losses based on prices duration
calculates the time in hours of held gains and losses. Instead, when method is set to "all", then all the four measures are computed.
p_res_all <- portfolio_compute(portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, method = "all") skimr::skim(p_res_all)
It is important to notice that the disposition effect is
only meaningful for methods coount
and total
. In all
the other cases the disposition difference is used instead.
disposition_compute(gainslosses = p_res_all, aggregate_fun = mean, na.rm = TRUE)
The allow_short
argument, instead, allows for short selling
transactions. If set to FALSE, short selling will not be allowed
and no gains or losses will be computed when this happens.
The time_threshold
argument is a fundamental fine tuning
parameter. It essentially controls the minimum time distance
necessary to compute gains and losses.
By default it is set to "0 mins", implying that gains and losses
are always computed.
However, this may not be desirable since investor's behaviors
are not expected to change with very high frequencies.
Hence, for instance, setting it to "60 mins" states that gains and
losses are calculated only if 60 minutes are passed from the last
transaction.
portfolio_compute( portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, time_threshold = "60 mins" )
This parameter is very important also because it allows to somewhat filters human operations from machines operations, without actually removing them from the analysis.
Different units may be specified.
The argument exact_market_prices
is set to TRUE by default,
since it is expected that the user provides market prices of
each traded asset for each transaction datetime.
However, when this is not the case, one may want to set it to
FALSE to allow for non exact market prices. It essentially
means that the nearest price in time is used.
portfolio_compute( portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, exact_market_prices = FALSE )
Note, however, that with exact_market_prices
set to FALSE,
unreliable results may be obtained when transactions occur
with low frequency, since the market prices used as reference
for the calculation may be outdated.
The verbose
and progress
arguments may be useful for
interactive use and for very long calculations on large
portfolios of transactions.
portfolio_compute( portfolio_transactions = trx_QZ621, market_prices = mkt_QZ621, verbose = c(1, 1), progress = TRUE )
See Portfolio Driven Disposition Effect and Time Series Disposition Effect for a guide on the usage of other, more advanced, arguments.
Although the analysis of disposition effect can be performed simply on a single investor, the real advantages of this analysis derive from the capacity to study and understand the behaviors of many different investors that actively operate on the financial markets.
Hence, to fully grasp the power of dispositionEffect
package, we can
proceed to jointly analyze all the 10 investors' transactions that are
available into the DEanalysis
dataset.
Furthermore, if you are interested in computing the disposition effect on large datasets, please see Disposition Effect in Parallel to understand how the benefits of parallel computing can be exploited within this framework.
# list of transactions separated by investor trx_list <- trx %>% dplyr::group_by(investor) %>% dplyr::group_split()
This time to calculate gains and losses for each investor's
portfolio we can simply map
portfolio_compute
on the list
of transactions, specifying all the other necessary arguments
as usual.
p_res_full <- purrr::map(trx_list, portfolio_compute, market_prices = mkt)
load("figures/p_res_full.RData")
The same procedure can be used to quickly compute the disposition effect on each resulting portfolio.
de <- purrr::map(p_res_full, disposition_compute) %>% dplyr::bind_rows() skimr::skim(de)
As it is shown, de
is a data frame containing disposition effect
results (variable DE_count
) on the 10 investors for all their 337
traded assets.
Also the average disposition effects of the investors can be easily obtained,
de_mean <- purrr::map(p_res_full, disposition_compute, aggregate_fun = mean, na.rm = TRUE) %>% dplyr::bind_rows() %>% dplyr::arrange(dplyr::desc(DE_count)) de_mean
and the disposition effect summary statistics.
de_stat <- purrr::map(p_res_full, disposition_summary) %>% dplyr::bind_rows() head(de_stat, 7)
It is clearer now that some investors display irrational behaviors while other don't.
The graphical inspection of disposition effect results allows to easily understand what is going on and to spot possible interesting behaviors.
One may want to investigate the overall distribution, or the distributions of every statistics obtained.
ggplot(de, aes(x = DE_count)) + geom_histogram(aes(y = ..density..), color = "darkblue", fill = "yellow") + geom_density(aes(y = ..density..), color = "darkblue", fill = "yellow", alpha = 0.4) + scale_x_continuous(limits = c(-1, 1)) + theme( panel.background = element_rect(fill = "grey92"), plot.background = element_rect(fill = "grey85", colour = NA), plot.title = element_text(size = 20), legend.position = "none" ) + labs(title = "Disposition Effect Distribution", x = "Disposition Effect", y = "Frequency")
de_stat <- de_stat %>% dplyr::filter(stat != "StDev") %>% dplyr::mutate(stat = factor(stat, levels = c("Min", "Q1", "Median", "Mean", "Q3", "Max")))
ggplot(de_stat, aes(x = DE_count, y = stat, fill = stat)) + geom_density_ridges() + scale_fill_viridis_d() + scale_x_continuous(limits = c(-1, 1)) + theme( panel.background = element_rect(fill = "grey92"), plot.background = element_rect(fill = "grey85", colour = NA), plot.title = element_text(size = 20), legend.position = "none" ) + labs( title = "Disposition Effect Statistics' Distributions", x = "Disposition Effect", y = "" )
Or deeper, one can also analyze investors' behaviors on some specific assets to understand whether there exists on the market assets that are more subject to irrationality.
top5_assets <- trx %>% dplyr::count(asset) %>% dplyr::arrange(dplyr::desc(n)) %>% dplyr::slice(1:6) %>% dplyr::pull(asset) dplyr::filter(de, asset %in% top5_assets) %>% ggplot(aes(x = asset, y = DE_count, fill = asset)) + # geom_half_boxplot(center = TRUE, width = 0.8, nudge = 0.02) + # geom_half_violin(side = "r", nudge = 0.02, alpha = 0.8) + geom_boxplot() + geom_jitter(color = "grey40") + scale_fill_viridis_d() + theme( panel.background = element_rect(fill = "grey92"), plot.background = element_rect(fill = "grey85", colour = NA), plot.title = element_text(size = 20), legend.position = "none" ) + labs(title = "Volatility & Disposition Effect", x = "", y = "Disposition Effect")
For more tutorials on disposition effect visit dispositionEffect.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.