samplecomp: Performance assessment of sampling algorithms against a known...

In BiDAG: Bayesian Inference for Directed Acyclic Graphs

Performance assessment of sampling algorithms against a known Bayesian network

Description

This function compute 8 different metrics of structure fit of an object of classes orderMCMC and partitionMCMC to the ground truth DAG (or CPDAG). First posterior probabilities of single edges are calculated based on a sample stores in the object of class orderMCMC or partitionMCMC. This function computes structure fit of each of the consensus graphs to the ground truth one based on a defined range of posterior thresholds. Computed metrics include: TP, FP, TPR, FPR, FPRn, FDR, SHD. See metrics description in see also compareDAGs.

Usage

samplecomp(
  MCMCchain,
  truedag,
  p = c(0.99, 0.95, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2),
  pdag = TRUE,
  burnin = 0.2,
  trans = TRUE
)

## S3 method for class 'samplecomp'
plot(x, ..., vars = c("FP", "TP"), type = "b", col = "blue", showp = NULL)

## S3 method for class 'samplecomp'
print(x, ...)

## S3 method for class 'samplecomp'
summary(object, ...)

Arguments

MCMCchain	an object of class partitionMCMC or orderMCMC, representing the output of structure sampling function partitionMCMC or orderMCMC (the latter when parameter chainout=TRUE;
truedag	ground truth DAG which generated the data used in the search procedure; represented by an object of class graphNEL
p	a vector of numeric values between 0 and 1, defining posterior probabilities according to which the edges of assessed structures are drawn, please note very low barriers can lead to very dense structures; by default p=c(0.99, 0.95, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2)
pdag	logical, if TRUE (default) all DAGs in the MCMCchain are first converted to equivalence class (CPDAG) before the averaging
burnin	number between 0 and 1, indicates the percentage of the samples which will be the discarded as ‘burn-in’ of the MCMC chain; the rest of the samples will be used to calculate the posterior probabilities; 0.2 by default
trans	logical, for DBNs indicates if model comparions are performed for transition structure; when trans equals FALSE the comparison is performed for initial structures of estimated models and the ground truth DBN; for usual BNs the parameter is disregarded
x	object of class 'samplecomp'
...	ignored
vars	a tuple of variables which will be used for 'x' and 'y' axes; possible values: "SHD", "TP", "FP", "TPR", "FPR", "FPRn", "FDR"
type	type of line in the plot; "b" by default
col	colour of line in the plotl; "blue" by default
showp	logical, defines if points are labelled with the posterior threshold corresponding to the assessed model
object	object of class 'samplecomp'

Value

an object if class samplesim, a matrix with the number of rows equal to the number of elements in 'p', and 8 columns reporting for the consensus graphss (corresponfing to each of the values in 'p') the number of true positive edges ('TP'), the number of false positive edges ('FP'), the number of false negative edges ('FN'), the true positive rate ('TPR'), the structural Hamming distance ('SHD'), false positive rate ('FPR'), false discovery rate ('FDR') and false positive rate normalized by TP+FN ('FPRn').

Author(s)

Polina Suter

Examples

gsim.score<-scoreparameters("bge", gsim)
## Not run: 
MAPestimate<-learnBN(gsim.score,"orderIter",scoreout=TRUE)
ordersample<-sampleBN(gsim.score, "order", scoretable=getSpace(MAPestimate))
samplecomp(ordersample, gsimmat)

## End(Not run)

BiDAG documentation built on May 31, 2023, 6:46 p.m.