pmine: PST based pattern mining

In PST: Probabilistic Suffix Trees and Variable Length Markov Chains

Description

Mine for (sub)sequences satisfying user defined criteria in a state sequence object

Usage

## S4 method for signature 'PSTf,stslist'
pmine(object, data, l, pmin=0, pmax=1, prefix, lag, average=FALSE,
output="sequences", with.prefix=TRUE, sorted=TRUE, decreasing=TRUE, score.norm=FALSE)

Arguments

object	A fitted PST, that is an object of class PSTf as returned by the pstree or prune method.
data	A sequence object of class 'stslist' as defined with the seqdef function of the TraMineR library.
l	integer. Length of the subsequence to search for.
pmin	numeric. (Sub)-sequences having average or per state probability greater or equal than pmin are selected. Default to 1, meaning no lower threshold for the probability.
pmax	numeric. (Sub)-sequences having average or per state probability less or equal than pmax are selected. Default to 1, meaning no upper threshold for the probability.
prefix	character. Subsequences are searched in sequences starting with 'prefix', where 'prefix' is a string representing a subsequence with states separated by '-'. This option can be used to search for -most- likely patterns in sequences starting with 'prefix'.
lag	integer. The lag first states in the sequence are omitted. If prefix is
average	logical. If TRUE, the pmin or pmax probability is supposed to be the average state probability in the (sub)sequence. If FALSE (sub)sequences having every state probability less than pmax or greater than pmin are selected.
output	character. If output='sequences' the whole sequence(s) where the user defined criteria is satisfied are returned. If output='patterns' only the (sub)sequences satisfying the user defined criteria are returned.
with.prefix	logical. If 'output=patterns', should the patterns in the output be preceeded by their prefix, that is by the whole sub-sequence preceding the pattern.
sorted	logical. If 'sorted=TRUE', selected patterns or sequences are sorted according to their score, i.e., their average probability.
decreasing	logical. If 'sorted=TRUE', should sort order be decreasing or increasing ?
score.norm	logical. If TRUE, the score attached to each selected pattern or (sub)-sequence (the weights in the returned sequence object) is the average per state probability, and is thus normalized by the length of the pattern. If FALSE, the score is the whole (sub)-sequence probability.

Details

The likelihood P^{S}(x) of a whole sequence x is computed from the state probabilities at each position in the sequence. However, the likelihood of the first states is usually lower than at higher position due to a reduced memory available for prediction. A sequence may not appear as very likely if its first state has a low relative frequency, even if the model predicts high probabilities for the states at higher positions.

The pmine function allows for advanced pattern mining with user defined parameters. It is controlled by the lag and pmin arguments. For example, by setting lag=2 and pmin=0.40 (example 1), we select all sequences with average (the geometric mean is used) state probability from position lag+1, …, \ell above pmin. Instead of considering the average state probability at positions lag+1, …, \ell, it is also possible to select frequent patterns that do not contain any state with probability below the threshold. This prevents from selecting sequences having many states with high probability but one ore several states with a low probability.

It is also possible to mine the sequence data for frequent patterns of length \ell_{j} < \ell, regardless of the position in the sequence where they occur. By using the output="patterns" argument, the pmine function returns the patterns (as a sequence object) instead of the whole set of distinct sequences containing the patterns. Since the probability of a pattern can be different depending on the context (previous states) the returned subsequences also contain the context preceding the pattern. For more details, see Gabadinho 2016.

Value

A state sequence object, that is an object of class stslist, where weights are the probability score of (sub)sequences.

Author(s)

Alexis Gabadinho

References

Gabadinho, A. & Ritschard, G. (2016). Analyzing State Sequences with Probabilistic Suffix Trees: The PST R Package. Journal of Statistical Software, 72(3), pp. 1-39.

See Also

cmine for context mining

Examples

## activity calendar for year 2000
## from the Swiss Household Panel
## see ?actcal
data(actcal)

## selecting individuals aged 20 to 59
actcal <- actcal[actcal$age00>=20 & actcal$age00 <60,]

## defining a sequence object
actcal.lab <- c("> 37 hours", "19-36 hours", "1-18 hours", "no work")
actcal.seq <- seqdef(actcal,13:24,labels=actcal.lab)

## building a PST
actcal.pst <- pstree(actcal.seq, nmin=2, ymin=0.001)

## pruning
## Cut-offs for 5% and 1% (see ?prune)
C99 <- qchisq(0.99,4-1)/2
actcal.pst.C99 <- prune(actcal.pst, gain="G2", C=C99)

## example 1
pmine(actcal.pst.C99, actcal.seq, pmin=0.4, lag=2)

## example 2: patterns of length 6 having p>=0.6
pmine(actcal.pst.C99, actcal.seq, pmin=0.6, l=6)

Example output

Loading required package: TraMineR

TraMineR stable version 2.0-11.1 (Built: 2019-05-12)
Website: http://traminer.unige.ch
Please type 'citation("TraMineR")' for citation information.

Loading required package: RColorBrewer

PST version 0.94 (Built: "Sun,)
Website: http://r-forge.r-project.org/projects/pst
 [>] 4 distinct states appear in the data: 
     1 = A
     2 = B
     3 = C
     4 = D
 [>] state coding:
       [alphabet]  [label]  [long label] 
     1  A           A        > 37 hours
     2  B           B        19-36 hours
     3  C           C        1-18 hours
     4  D           D        no work
 [>] 1472 sequences in the data set
 [>] min/max sequence length: 12/12
 [>] 1472 sequence(s) - min/max length: 12/12
 [>] max. depth L=11, nmin=2, ymin=0.001
     [L]  [nodes]
       0        1
       1        4
       2       16
       3       40
       4       68
       5       79
       6       86
       7       87
       8       79
       9       69
      10       55
      11       39
 [>] computing sequence(s) likelihood ... (0.222 secs)
 [>] total time: 0.973 secs
 [>] pruning results: 
     [L]  [nodes]  [pruned]
      11       39        39
      10       55        55
       9       69        69
       8       79        79
       7       87        87
       6       86        86
       5       79        77
       4       68        65
       3       40        34
       2       16         9
       1        4         0
 [>] computing sequence(s) likelihood ... (0.245 secs)
 [>] 152 sequence(s) - min/max length: 12/12
 [>] max. context length: L=5
 [>] found 23 distinct context(s)
 [>] total time: 0.022 secs
 [>] 7 sequence(s) selected
     Sequence               
6946 A-A-A-A-A-A-A-A-A-A-A-A
2848 B-B-B-B-B-B-B-B-B-B-B-B
1872 D-B-B-B-B-B-B-B-B-B-B-B
4516 D-D-D-D-D-D-D-D-D-D-D-D
4977 C-C-C-C-C-C-C-C-C-C-C-C
5665 B-D-D-D-D-D-D-D-D-D-D-D
4867 C-D-D-D-D-D-D-D-D-D-D-D
 [>] 152 sequence(s) - min/max length: 12/12
 [>] max. context length: L=5
 [>] found 23 distinct context(s)
 [>] total time: 0.02 secs
 [>] 94 sequence(s) selected
     Sequence               
6946 A-A-A-A-A-A-A-A-A-A-A-A
3092 A-A-A-A-A-A-A-A-A-A-D-D
731  A-A-A-A-A-A-A-A-A-A-B-B
132  A-A-A-A-A-A-A-A-A-A-A-B
4762 A-A-A-A-A-A-A-A-A-B-B-B
6096 B-B-B-A-A-A-A-A-A-A-A-A
3395 A-A-A-A-A-A-A-A-A-D-D-D
2885 A-A-A-A-A-A-A-A-A-C-C-C
5864 A-A-A-A-A-A-A-A-A-A-C-C
3340 A-A-A-A-A-A-A-A-A-A-A-D
639  A-A-A-A-A-A-A-A-A-A-A-C
3607 A-A-A-A-A-A-A-A-D-D-B-B
2710 A-A-A-A-A-A-A-A-C-C-C-C
3233 A-A-A-A-A-A-A-A-B-B-B-B
4066 A-A-A-A-A-A-A-A-B-B-A-A
896  A-A-A-A-A-A-A-A-D-D-D-D
950  A-A-A-A-A-A-A-B-D-D-D-D
5975 A-A-A-A-A-A-A-B-B-B-B-B
877  A-A-A-A-A-A-A-D-D-D-D-D
3268 A-A-A-A-A-A-A-D-D-D-A-A
2848 B-B-B-B-B-B-B-B-B-B-B-B
1872 D-B-B-B-B-B-B-B-B-B-B-B
5238 B-B-B-B-B-B-B-B-B-B-B-C
1279 B-B-B-B-B-B-B-B-B-C-C-C
4267 B-B-B-B-B-B-B-B-B-B-B-A
1647 B-B-B-B-B-B-B-B-B-A-A-A
2106 B-B-B-B-B-B-B-B-B-B-A-A
3953 D-D-D-D-B-B-B-B-B-B-B-B
4279 B-B-B-B-B-B-B-B-B-B-C-C
5482 D-B-B-B-B-B-B-B-B-C-C-C
3694 B-B-B-B-B-B-B-B-B-D-D-D
1018 B-B-B-B-B-B-B-B-B-B-D-D
6922 D-C-C-C-C-B-B-B-B-B-B-B
3293 A-A-B-B-B-B-B-B-B-B-B-B
647  B-B-B-B-B-B-B-B-A-A-A-A
131  B-B-B-B-B-B-B-D-D-D-D-D
1206 B-B-B-B-B-B-B-A-A-A-A-A
1959 B-B-B-B-B-B-B-B-B-C-A-A
5169 B-B-B-B-B-B-B-B-D-C-C-C
3708 B-B-B-B-B-B-B-B-D-D-B-B
5644 B-B-B-B-B-B-B-B-A-A-B-B
3580 B-B-B-B-B-B-B-B-B-D-B-B
1423 B-B-B-B-B-B-B-B-C-C-C-C
4516 D-D-D-D-D-D-D-D-D-D-D-D
2833 D-D-D-D-D-D-D-D-D-D-D-A
3157 D-D-D-D-D-D-D-D-D-C-C-C
1055 D-D-D-D-D-D-D-D-D-D-D-C
4402 D-D-D-D-D-D-D-D-D-D-B-B
4971 D-D-D-D-D-D-D-D-D-D-A-A
3093 D-D-D-D-D-D-D-D-D-D-C-C
4932 D-D-D-D-D-D-D-D-D-B-B-B
1180 D-D-D-D-D-D-D-D-D-D-D-B
2626 D-D-D-D-D-D-D-D-D-C-C-B
5665 B-D-D-D-D-D-D-D-D-D-D-D
4867 C-D-D-D-D-D-D-D-D-D-D-D
1202 D-D-D-D-D-D-D-D-B-B-B-B
41   D-D-D-D-D-D-D-D-C-C-C-C
26   D-D-D-D-D-D-D-D-A-A-A-A
2620 D-D-D-D-D-D-D-D-C-B-B-B
2068 D-D-D-D-D-D-D-A-D-D-D-D
2715 D-D-D-D-D-D-D-C-C-C-D-D
708  D-D-D-D-D-D-D-C-C-C-C-C
1893 D-D-D-D-D-D-D-A-A-A-A-A
3047 D-D-D-D-D-D-D-B-B-B-B-B
318  D-D-D-D-D-D-D-B-B-B-D-D
4164 D-D-D-D-D-D-D-A-A-A-B-B
4344 D-D-A-A-A-A-A-A-A-A-A-A
654  C-C-C-C-C-C-C-C-C-B-B-B
4977 C-C-C-C-C-C-C-C-C-C-C-C
1929 C-C-C-C-C-C-C-B-B-B-B-B
1152 C-C-C-C-C-C-C-C-C-C-C-B
529  C-C-C-C-C-C-C-D-D-D-D-D
408  C-C-C-C-C-C-C-C-B-B-B-B
752  C-C-C-C-C-C-C-C-C-D-D-D
3366 C-C-C-C-C-C-C-C-C-C-D-D
2581 C-C-C-C-C-C-C-C-C-C-B-B
1133 C-C-C-C-C-C-C-C-C-C-C-A
3360 C-C-C-C-C-C-C-C-D-D-D-D
4059 C-C-C-C-C-C-C-C-C-C-C-D
2444 C-C-C-C-C-C-C-C-D-D-D-C
475  B-D-C-C-C-C-C-C-C-C-C-C
5272 D-D-C-C-C-C-C-C-C-C-C-C
1907 C-D-D-C-C-C-C-C-C-C-C-C
4959 D-D-D-D-C-C-C-C-C-C-C-C
700  A-A-A-D-D-D-D-D-D-D-D-B
5088 D-D-D-D-D-C-C-C-C-C-C-C
1496 A-A-A-D-A-A-A-A-A-A-A-A
1230 D-D-D-D-A-A-A-A-A-A-A-D
1374 A-A-A-D-D-A-A-A-A-A-A-A
4522 C-C-D-D-D-D-D-D-D-D-D-C
5555 D-C-C-D-D-D-D-D-D-D-D-D
899  C-D-D-D-D-D-D-D-D-D-B-B
2261 C-D-D-D-D-D-D-D-B-B-B-B
6838 C-C-C-C-C-D-D-D-D-D-D-D

PST documentation built on Nov. 25, 2020, 3 p.m.