CPTtools-package: Tools for Creating Conditional Probability Tables

In ralmond/CPTtools: Tools for Creating Conditional Probability Tables

Tools for Creating Conditional Probability Tables

Description

Provides support parameterized tables for Bayesian networks, particularly the IRT-like DiBello tables. Also, provides some tools for visualing the networks.

Details

The DESCRIPTION file: This package was not yet installed at build time.

CPTtools is a collection of various bits of R code useful for processing Bayes net output. Some were designed to work with ETS's proprietary StatShop code, and some with RNetica. The code collected in this package is all free from explicit dependencies on the specific Bayes net package and will hopefully be useful with other systems as well.

The majority of the code are related to building conditional probability tables (CPTs) for Bayesian networks. The package has two output representations for a CPT. The first is a data.frame object where the first several columns are factor variables corresponding the the parent variables, and the remaining columns are numeric variables corresponding to the state of the child variables. The rows represent possible configurations of the parent variables. An example is shown below.

       S1       S2       Full    Partial       None
1    High     High 0.81940043 0.15821522 0.02238436
2  Medium     High 0.46696668 0.46696668 0.06606664
3     Low     High 0.14468106 0.74930671 0.10601223
4    High   Medium 0.76603829 0.14791170 0.08605000
5  Medium   Medium 0.38733177 0.38733177 0.22533647
6     Low   Medium 0.10879020 0.56342707 0.32778273
7    High      Low 0.65574465 0.12661548 0.21763987
8  Medium      Low 0.26889642 0.26889642 0.46220715
9     Low      Low 0.06630741 0.34340770 0.59028489
10   High LowerYet 0.39095414 0.07548799 0.53355787
11 Medium LowerYet 0.11027649 0.11027649 0.77944702
12    Low LowerYet 0.02337270 0.12104775 0.85557955

The second representation is a table (matrix) with just the numeric part. Two approaches to building these tables from parameters are described below. The more flexible discrete partial credit model is used for the basis of the parameterized networks in the Peanut package.

In addition to the code for building partial credit networks, this package contains some code for building Bayesian network structures from (inverse) correlation matrixes, and graphical displays for Bayes net output. The latter includes some diagnostic plots and additional diagnostic tests.

Discrete Partial Credit Framework

The original parameterization for creating conditional probability tables based on Almond et al (2001) proved to be insufficiently flexible. Almond (2015) describes a newer parameterization based on three steps:

1. Translate the parent variables onto a numeric effective theta scale (effectiveThetas).

2. Combine the parent effective thetas into a single effective theta using a combination rule (Compensatory, OffsetConjunctive).

3. Convert the effective theta for each row of the table into conditional probabilities using a link function (gradedResponse, partialCredit, normalLink).

The partialCredit link function is particularly flexible as it allows different parameterizations and different combination rules for each state of the child variable. This functionality is best captured by the two high level functions:

calcDPCTable

Creates the probability table for the discrete partial credit model given the parameters.

mapDPC

Finds an MAP estimate for the parameters given an observed table of counts.

This parameterization serves as basis for the model used in the Peanut package.

Other parametric CPT models

The first two steps of the discrete partial credit framework outlined above are due to a suggestion by Lou DiBello (Almond et al, 2001). This lead to an older framework, in which the link function was hard coded into the conditional probability table formation. The models were called DiBello-XX, where XX is the name of the link function. Almond et al. (2015) describes several additional examples.

calcDDTable

Calculates DiBello-Dirichlet model probability and parameter tables.

calcDNTable

Creates the probability table for DiBello-Normal distribution. This is equivalent to using the normalLink in the DPC framework. This also uses a link scale parameter.

calcDSTable

Creates the probability table for DiBello-Samejima distribution. This is equivalent to using the gradedResponse in the DPC framework.

calcDSllike

Calculates the log-likelihood for data from a DiBello-Samejima (Normal) distribution.

Diez (1993) and Srinivas (1993) describe an older parametric framework for Bayes nets based on the noisy-or or noisy-max function. These are also available.

calcNoisyAndTable

Calculate the conditional probability table for a Noisy-And or Noisy-Min distribution.

calcNoisyOrTable

Calculate the conditional probability table for a Noisy-Or distribution.

Building Bayes nets from (inverse) correlation matrixes

Almond (2010) noted that in many cases the best information about the relationship among variables came from a procedure that produces a correlation matrix (e.g., a factor analysis). Applying a trick from Whittaker (1990), connecting pairs of nodes corresponding to nonzero entries in an inverse correlation matrix produces an undirected graphical model. Ordering in the nodes in a perfect ordering allows the undirected model to be converted into a directed model (Bayesian network). The conditional probability tables can then be created through a series of regressions.

The following functions implement this protocol:

structMatrix

Finds graphical structure from a covariance matrix.

mcSearch

Orders variables using Maximum Cardinality search.

buildParentList

Builds a list of parents of nodes in a graph.

buildRegressions

Creates a series of regressions from a covariance matrix.

buildRegressionTables

Builds conditional probability tables from regressions.

Other model construction tools

These functions are a grab bag of lower level utilities useful for building CPTs:

areaProbs

Translates between normal and categorical probabilities.

numericPart

Splits a mixed data frame into a numeric matrix and a factor part..

dataTable

Constructs a table of counts from a setof discrete observations..

eThetaFrame

Constructs a data frame showing the effective thetas for each parent combination..

effectiveThetas

Assigns effective theta levels for categorical variable.

getTableStates

Gets meta data about a conditional probability table..

rescaleTable

Rescales the numeric part of the table.

scaleMatrix

Scales a matrix to have a unit diagonal.

scaleTable

Scales a table according to the Sum and Scale column.

Bayes net output displays and tests

Almond et al. (2009) suggested using hanging barplots for displaying Bayes net output and gives several examples. The function stackedBars produces the simple version of this plot and the function compareBars compares two distributions (e.g., prior and posterior). The function buildFactorTab is useful for building the data and the function colorspread is useful for building color gradients.

Madigan, Mosurski and Almond (1997) describe a graphical weight of evidence balance sheet (see also Almond et al, 2015, Chapter 7; Almond et al, 2013). The function woeHist calculates the weights of evidence for a series of observations and the function woeBal produces a graphical display.

Sinharay and Almond (2006) propose a graphical fit test for conditional probability tables (see also, Almond et al, 2015, Chapter 10). The function OCP implements this test, and the function betaci creates the beta credibility intervals around which the function is built.

The key to Bayesian network models are the assumptions of conditional independence which underlie the model. The function localDepTest tests these assumptions based on observed (or imputed) data tables.

The function mutualInformation calculates the mutual information of a two-way table, a measure of the strength of association. This is similar to the measure used in many Bayes net packages (e.g., MutualInfo).

Data sets

Two data sets are provided with this package:

ACED

Data from ACED field trial (Shute, Hansen, and Almond, 2008). This example is based on a field trial of a Bayesian network based Assessment for Learning system, and contains both item-level response and high-level network summaries. A complete description of the Bayes net can be found at http://ecd.ralmond.net/ecdwiki/ACED/ACED.

MathGrades

Grades on 5 mathematics tests from Mardia, Kent and Bibby (from Whittaker, 1990).

Index

Complete index of all functions.

Index: This package was not yet installed at build time.

Acknowledgements

We are grateful to support from the following projects for supporting the work in the development and maintenance of this package.

• Bill \& Melinda Gates Foundation grant "Games as Learning/Assessment: Stealth Assessment" (\#0PP1035331, Val Shute, PI)

• National Science Foundation grant "DIP: Game-based Assessment and Support of STEM-related Competencies" (\#1628937, Val Shute, PI).

• National Scient Foundation grant "Mathematical Learning via Architectual Design and Modeling Using E-Rebuild." (\#1720533, Fengfeng Ke, PI).

Author(s)

Russell Almond

Maintainer: Russell Almond <[e-mail]>

References

Almond, R.G. (2015). An IRT-based Parameterization for Conditional Probability Tables. Paper submitted to the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence conference.

Almond, R.G., Mislevy, R.J., Steinberg, L.S., Williamson, D.M. and Yan, D. (2015) Bayesian Networks in Educational Assessment. Springer.

Almond, R. G. (2010). ‘I can name that Bayesian network in two matrixes.’ International Journal of Approximate Reasoning. 51, 167-178.

Almond, R. G., Shute, V. J., Underwood, J. S., and Zapata-Rivera, J.-D (2009). Bayesian Networks: A Teacher's View. International Journal of Approximate Reasoning. 50, 450-460.

Almond, R.G., DiBello, L., Jenkins, F., Mislevy, R.J., Senturk, D., Steinberg, L.S. and Yan, D. (2001) Models for Conditional Probability Tables in Educational Assessment. Artificial Intelligence and Statistics 2001 Jaakkola and Richardson (eds)., Morgan Kaufmann, 137–143.

Diez, F. J. (1993) Parameter adjustment in Bayes networks. The generalized noisy OR-gate. In Heckerman and Mamdani (eds) Uncertainty in Artificial Intelligence 93. Morgan Kaufmann. 99–105.

Muraki, E. (1992). A Generalized Partial Credit Model: Application of an EM Algorithm. Applied Psychological Measurement, 16, 159-176. DOI: 10.1177/014662169201600206

Samejima, F. (1969) Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph No. 17, 34, (No. 4, Part 2).

Shute, V. J., Hansen, E. G., & Almond, R. G. (2008). You can't fatten a hog by weighing it—Or can you? Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence and Education, 18(4), 289-316.

Sinharay, S. and Almond, R.G. (2006). Assessing Fit of Cognitively Diagnostic Models: A case study. Educational and Psychological Measurement. 67(2), 239–257.

Srinivas, S. (1993) A generalization of the Noisy-Or model, the generalized noisy OR-gate. In Heckerman and Mamdani (eds) Uncertainty in Artificial Intelligence 93. Morgan Kaufmann. 208–215.

Whittaker, J. (1990). Graphical Models in Applied Multivariate Statistics. Wiley.

Madigan, D., Mosurski, K. and Almond, R. (1997) Graphical explanation in belief networks. Journal of Computational Graphics and Statistics, 6, 160-181.

Almond, R. G., Kim, Y. J., Shute, V. J. and Ventura, M. (2013). Debugging the Evidence Chain. In Almond, R. G. and Mengshoel, O. (Eds.) Proceedings of the 2013 UAI Application Workshops: Big Data meet Complex Models and Models for Spatial, Temporal and Network Data (UAI2013AW), 1-10. http://ceur-ws.org/Vol-1024/paper-01.pdf

See Also

RNetica ~~ Peanut ~~

Examples

## Set up variables
skill1l <- c("High","Medium","Low") 
skill2l <- c("High","Medium","Low","LowerYet") 
correctL <- c("Correct","Incorrect") 
pcreditL <- c("Full","Partial","None")
gradeL <- c("A","B","C","D","E") 

## New Discrete Partial Credit framework:

## Complex model, different rules for different levels
cptPC2 <- calcDPCFrame(list(S1=skill1l,S2=skill2l),pcreditL,
                          list(full=log(1),partial=log(c(S1=1,S2=.75))),
                          betas=list(full=c(0,999),partial=1.0),
                          rule=list("OffsetDisjunctive","Compensatory"))

## Graded Response using the older DiBello-Samejima framework.
cptGraded <- calcDSTable(list(S1=skill1l),gradeL, 0.0, 0.0, dinc=c(.3,.4,.3))

## Building a Bayes net from a correlation matrix.
data(MathGrades)
pl <- buildParentList(structMatrix(MathGrades$var),"Algebra")
rt <- buildRegressions(MathGrades$var,MathGrades$means,pl)
tabs <- buildRegressionTables(rt, MathGrades$pvecs, MathGrades$means,
                              sqrt(diag(MathGrades$var)))

## Stacked Barplots:
margins.prior <- data.frame (
 Trouble=c(Novice=.19,Semester1=.24,Semester2=.28,Semseter3=.20,Semester4=.09),
 NDK=c(Novice=.01,Semester1=.09,Semester2=.35,Semseter3=.41,Semester4=.14),
 Model=c(Novice=.19,Semester1=.28,Semester2=.31,Semseter3=.18,Semester4=.04)
)

margins.post <- data.frame(
 Trouble=c(Novice=.03,Semester1=.15,Semester2=.39,Semseter3=.32,Semester4=.11),
 NDK=c(Novice=.00,Semester1=.03,Semester2=.28,Semseter3=.52,Semester4=.17),
 Model=c(Novice=.10,Semester1=.25,Semester2=.37,Semseter3=.23,Semester4=.05))

stackedBars(margins.post,3,
            main="Marginal Distributions for NetPASS skills",
            sub="Baseline at 3rd Semester level.",
            cex.names=.75, col=hsv(223/360,.2,0.10*(5:1)+.5))

compareBars(margins.prior,margins.post,3,c("Prior","Post"),
            main="Margins before/after Medium Trouble Shooting Task",
            sub="Observables:  cfgCor=Medium, logCor=High, logEff=Medium",
            legend.loc = "topright",
            cex.names=.75, col1=hsv(h=.1,s=.2*1:5-.1,alpha=1),
            col2=hsv(h=.6,s=.2*1:5-.1,alpha=1))

## Weight of evidence balance sheets
sampleSequence <- read.csv(system.file("testFiles","SampleStudent.csv",
                                       package="CPTtools"),
                           header=TRUE,row.names=1)

woeBal(sampleSequence[,c("H","M","L")],c("H"),c("M","L"),lcex=1.25)


### Observable Characteristic Plot
pi <- c("+"=.15,"-"=.85)
nnn <- c("(0,0,0)"=20,"(0,0,1)"=10,
         "(0,1,0)"=10,"(0,1,0)"=5,
         "(1,0,0)"=10,"(1,0,1)"=10,
         "(1,1,1)"=10,"(1,1,1)"=25)
xx1 <- c("(0,0,0)"=2,"(0,0,1)"=5,
       "(0,1,0)"=1,"(0,1,1)"=3,
       "(1,0,0)"=0,"(1,0,1)"=2,
       "(1,1,0)"=5,"(1,1,1)"=24)
grouplabs <- c(rep("-",3),"+")
grouplabs1 <- rep(grouplabs,each=2)
OCP2 (xx1,nnn,grouplabs1,pi,c("-","+"),ylim=c(0,1), reflty=c(2,4),
      setlabs=c("Low Skill3","High Skill3"),setat=-.8,
      main="Data for which Skill 3 is relevant")

ralmond/CPTtools documentation built on Dec. 27, 2024, 7:15 a.m.