Simulation_Based_Calibration_histogram: Draw a histogram of the rank statistics

Description Usage Arguments Value Examples

View source: R/Simulation_Based_Calibration.R

Description

To validate that the MCMC procedure is correct or not, we show the histogram of rank statistics. If the resulting histogram is uniformly distributed, then we can conclude that the MCMC sampling is correct. If the histogram is far from uniformity, then the MCMC sampling or specification of priors is not correct or not appropriate.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Simulation_Based_Calibration_histogram(
  N = 3,
  sd = 5,
  C = 5,
  initial.seed.for.drawing.a.rank.statistics = 1234567,
  fun = stats::var,
  NI = 259,
  NL = 259,
  initial.seed.for.drawing.a.data = 1234,
  ModifiedPoisson = FALSE,
  ite = 1111,
  DrawCurve = FALSE
)

Arguments

N

samples size of the rank statistics.

sd

Standard Deviation of priors

C

No. of Confidence levels

initial.seed.for.drawing.a.rank.statistics

seed

fun

An one dimensional real valued function defined on the parameter space. This is used in the definition of the rank statistics. Generally speaking, the element of the parameter space is a vector, so the function should be defined on vectors. In my model parameter is mean, standard deviation, C thresholds of the latent Gaussian, so this function should be defined on the C+2 dimensional Euclidean space.

NI

No. of images

NL

No. of Lesions

initial.seed.for.drawing.a.data

seed

ModifiedPoisson

Logical, that is TRUE or FALSE.

If ModifiedPoisson = TRUE, then Poisson rate of false alarm is calculated per lesion, and model is fitted so that the FROC curve is an expected curve of points consisting of the pairs of TPF per lesion and FPF per lesion.

Similarly,

If ModifiedPoisson = TRUE, then Poisson rate of false alarm is calculated per image, and model is fitted so that the FROC curve is an expected curve of points consisting of the pair of TPF per lesion and FPF per image.

For more details, see the author's paper in which I explained per image and per lesion. (for details of models, see vignettes , now, it is omiited from this package, because the size of vignettes are large.)

If ModifiedPoisson = TRUE, then the False Positive Fraction (FPF) is defined as follows (F_c denotes the number of false alarms with confidence level c )

\frac{F_1+F_2+F_3+F_4+F_5}{N_L},

\frac{F_2+F_3+F_4+F_5}{N_L},

\frac{F_3+F_4+F_5}{N_L},

\frac{F_4+F_5}{N_L},

\frac{F_5}{N_L},

where N_L is a number of lesions (signal). To emphasize its denominator N_L, we also call it the False Positive Fraction (FPF) per lesion.

On the other hand,

if ModifiedPoisson = FALSE (Default), then False Positive Fraction (FPF) is given by

\frac{F_1+F_2+F_3+F_4+F_5}{N_I},

\frac{F_2+F_3+F_4+F_5}{N_I},

\frac{F_3+F_4+F_5}{N_I},

\frac{F_4+F_5}{N_I},

\frac{F_5}{N_I},

where N_I is the number of images (trial). To emphasize its denominator N_I, we also call it the False Positive Fraction (FPF) per image.

The model is fitted so that the estimated FROC curve can be ragraded as the expected pairs of FPF per image and TPF per lesion (ModifiedPoisson = FALSE )

or as the expected pairs of FPF per image and TPF per lesion (ModifiedPoisson = TRUE)

If ModifiedPoisson = TRUE, then FROC curve means the expected pair of FPF per lesion and TPF.

On the other hand, if ModifiedPoisson = FALSE, then FROC curve means the expected pair of FPF per image and TPF.

So,data of FPF and TPF are changed thus, a fitted model is also changed whether ModifiedPoisson = TRUE or FALSE. In traditional FROC analysis, it uses only per images (trial). Since we can divide one image into two images or more images, number of trial is not important. And more important is per signal. So, the author also developed FROC theory to consider FROC analysis under per signal. One can see that the FROC curve is rigid with respect to change of a number of images, so, it does not matter whether ModifiedPoisson = TRUE or FALSE. This rigidity of curves means that the number of images is redundant parameter for the FROC trial and thus the author try to exclude it.

Revised 2019 Dec 8 Revised 2019 Nov 25 Revised 2019 August 28

ite

A variable to be passed to the function rstan::sampling() of rstan in which it is named iter. A positive integer representing the number of samples synthesized by Hamiltonian Monte Carlo method, and, Default = 10000.

DrawCurve

Logical: TRUE of FALSE. Whether the curve is to be drawn. TRUE or FALSE. If you want to draw the FROC and AFROC curves, then you set DrawCurve =TRUE, if not then DrawCurve =FALSE. The reason why the author make this variable DrawCurve is that it takes long time in MRMC case to draw curves, and thus Default value is FALSE in the case of MRMC data.

Value

samples of rank statistics

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
## Not run: 
  g <-Simulation_Based_Calibration_histogram(N=2,ite = 2222)

  graphics::hist(g$rank.statistics)




  g <- Simulation_Based_Calibration_histogram(
  NI=1111111,
  NL=1111111,
  # N =100 would be better  more than N =10
  # But this is only example, we take very small N
  N=10,
  ite=3333,
  sd=1,
  initial.seed.for.drawing.a.rank.statistics = 123456789,
  DrawCurve = TRUE
  )



   g <- Simulation_Based_Calibration_histogram(
   NI=1111111,
   NL=1111111,
  # N =100 would be better  more than N =10
  # But this is only example, we take very small N
   N=10,
   ite=3333,
   sd=1,initial.seed.for.drawing.a.rank.statistics = 123456789,
   DrawCurve = TRUE,
   C=11)
#=======      The Second Example:       =================================================

# If you want to see the replicated data, then the following code is available.
# In the following, I extract the dataset which is very small rank statistics, e.g.
# less than 10. And draw the CFP and CTP for observation of dataset.




gggg <- Simulation_Based_Calibration_histogram(
NI=1111111,
NL=1111111,
N=22,
ite=2222)


 a <- gggg$rank.statistics<10

 aa <- the_row_number_of_logical_vector(a)


  draw.CFP.CTP.from.dataList(gggg$fit.list[[  aa[1]  ]]@dataList)



## End(Not run)#\dontrun

BayesianFROC documentation built on Jan. 13, 2021, 5:22 a.m.