txnsim | R Documentation |
Running simulations with similar properties as the backtest or production portfolio may allow the analyst to evaluate the distribution of returns possible with similar trading approaches and evaluate skill versus luck or overfitting.
txnsim(
Portfolio,
n = 10,
replacement = TRUE,
tradeDef = c("increased.to.reduced", "flat.to.flat", "flat.to.reduced"),
...,
CI = 0.95
)
Portfolio |
string identifying a portfolio |
n |
number of simulations, default = 100 |
replacement |
sample with or without replacement, default TRUE |
tradeDef |
string to determine which definition of 'trade' to use. See |
... |
any other passthrough parameters |
CI |
numeric specifying desired Confidence Interval used in hist.txnsim(), default 0.95 |
Statisticians talk about the 'stylized facts' of a data set. If you consider
the stylized facts of a series of transactions that are the output of a
discretionary or systematic trading strategy, it should be clear that there
is a lot of information available to work with. Initial analysis such as
tradeStats
and perTradeStats
can describe the
results of the series of transactions which resulted from the trading
strategy. What else can we learn from these transactions regarding trading
style or the skill of the trader? txnsim
seeks to conduct a simulation
over the properties of sampled round turn trades to help evaluate this.
With tradeDef='flat.to.flat'
, the samples are simply rearranging
quantity and duration of round turns. This may be enough for a strategy that
only puts on a single level per round turn.
With tradeDef='increased.to.reduced'
, typically used for more complex
strategies, the simulation is also significantly more complicated, especially
with replacement=TRUE
. In this latter case, the simulation must try
to retain stylized facts of the observed strategy, specifically:
percent time in market
percent time flat
ratio of long to short position taking
number of levels or layered trades observed
In order to do this, samples are taken and randomized for flat periods, short periods, and long periods, and then these samples are interleaved and layered to construct the random strategy. The overall goal is to construct a random strategy that preserves as many of the stylized facts (or style) of the observed strategy as possible, while demonstrating no skill. The round turn trades of the random replicate strategies, while outwardly resembling the original strategy in summary time series statistics, are the result of random combinations of observed features taking place at random times in the tested time period.
It should be noted that the first opened trade of the observed series and the replicates will take place at the same time. Quantity and duration may differ, but the trade will start at the same time, unless the first sampled period is a flat one. We may choose to relax this in the future and add or subtract a short amount of duration to the replicates to randomize the first entry more fully as well. Inclusion of flat periods should provide a fair amount of variation, so this may not be an issue.
The user may wish to pass Interval
in dots to mark the portfolio at a
different frequency than the market data, especially for intraday market
data. Note that market data must be available to call
updatePortf
on.
We are including p-values for some sample statistics in the output as well.
Some notes are in order on how this is calculatated, and how it may be
interpreted. With small n
, these p-values are meaningless. With large
n
, they should be fairly stable. Per North et. al. (2002) who use
Davison & Hinkley (1997) as their source, the correct unbiased p-value for a
simulation sample statistic is:
\frac{rank_bt+1}{n_samples+1}
where the rank of the observed statistic is compared against statistics calculated on the simulation. Interpretation of this result takes some care. The skeptical analyst would prefer to see a low p-value (e.g. the customary 0.05). The same analyst should be concerned about overfitting if an extraordinarily low p-value (e.g. 0.0001) is observed, or conclude that there may be room to improve the strategy is available if the p-value is low but not compelling (e.g. 0.15). Issues of multiple testing bias should also be considered. Interpretation of the p-value of the mean is most easily fit into the customary p<0.05 target. Appropriate critical values for other statistics may be lower or higher.
A list object of class txnsim
. The list contains the following elements:
replicates
: A list by symbol containing all the resampled start, quantity, and duration time series replicates.
transactions
: A list by symbol for each replicate of the Txn object passed to addTxns
.
backtest.trades
: A list by symbol containing trade start, quantity, and duration from the original backtest.
cumpl
: An xts
object containing the cumulative P&L of each replicate portfolio.
initEq
: A numeric variable containing the initEq
of the portfolio, for starting portfolio value.
seed
: The value of .Random.seed
for replication, if required.
call
: An object of type call
that contains the txnsim
call.
Portfolio
: A string identifying a portfolio.
n
: The number of simulations, default = 100.
replacement
: Whether to sample with or without replacement, default TRUE
.
samplestats
: A numeric data.frame
of various statistics for each replicate series.
original
: A numeric data.frame
of the statistics for the original series.
ranks
: A numeric data.frame
containing the ranking of the statistics.
pvalues
: A numeric data.frame
containing the p-values for the observed backtest compared to the sampled ranks.
stderror
: A numeric data.frame
of the standard error of the statistics for the replicates.
CI
: Numeric specifying the desired Confidence Interval used in hist.txnsim()
, default 0.95
.
CIdf
: A numeric data.frame
of the Confidence Intervals of the statistics for the bootstrapped replicates.
Note that this object and its slots may change in the future.
Slots replicates
,transactions
, and call
are likely
to exist in all future versions of this function, but other slots may be added
and removed as S3method
's are developed.
The backtest.trades
object contains the stylized facts of the observed
series, and consists of a list with one slot per instrument in the input
portfolio. Each slot in that list contains a data.frame
of
Start:
time stamp of the start of the round turn, discarded later
duration:
duration (difference from beginning or end) of the observed round turn trade
quantity:
quantity of the round turn trade, or 0 for flat periods
with additional attributes for the observed stylized facts:
calendar.duration:
total length/duration of the observed series
trade.duration:
total length/duration used by round turn trades
flat.duration:
aggregate length/duration of periods when observed series was flat
flat.stddev:
standard deviation of the duration of individual flat periods
first.start:
timestamp of the start of the first trade, to avoid starting simulations during a training period
period:
periodicity of the observed series
Jasen Mackie, Brian G. Peterson
Burns, Patrick. 2006. Random Portfolios for Evaluating Trading Strategies. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=881735
North, B.V., D. Curtis, and P.C. Sham. Aug 2002. A Note on the Calculation of Empirical P Values from Monte Carlo Procedures. Am J Hum Genet. 2002 Aug; 71(2): 439-441. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC379178/
Davison & Hinkley. 1997. Bootstrap methods and their application.
mcsim
, updatePortf
, perTradeStats
, hist.txnsim
, quantile.txnsim
## Not run:
n <- 10
ex.txnsim <- function(Portfolio
,n=10
,replacement=FALSE
, tradeDef='increased.to.reduced'
, chart=FALSE
)
{
out <- txnsim(Portfolio,n,replacement, tradeDef = tradeDef)
if(isTRUE(chart)) {
portnames <- blotter:::txnsim.portnames(Portfolio, replacement, n)
for (i in 1:n){
p<- portnames[i]
symbols<-names(getPortfolio(p)$symbols)
for(symbol in symbols) {
dev.new()
chart.Posn(p,symbol)
}
}
}
invisible(out)
} # end ex.txnsim
demo('longtrend',ask=FALSE)
lt.nr <- ex.txnsim('longtrend',n, replacement = FALSE)
lt.wr <- ex.txnsim('longtrend',n, replacement = TRUE, chart = TRUE)
plot(lt.wr)
hist(lt.wr)
require('quantstrat') #sorry for the circular dependency
demo('bbands',ask=FALSE)
bb.nr <- ex.txnsim('bbands',n, replacement = FALSE)
bb.wr <- ex.txnsim('bbands',n, replacement = TRUE, chart = TRUE)
plot(rsi.wr)
hist(bb.wr)
## End(Not run) #end dontrun
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.