crf_evaluation: Basic classification evaluation metrics for multi-class...

Description Usage Arguments Value Examples

Description

The accuracy, precision, recall, specificity, F1 measure and support metrics are provided for each label in a one-versus the rest setting.

Usage

1
2
crf_evaluation(pred, obs, labels = na.exclude(unique(c(as.character(pred),
  as.character(obs)))), labels_overall = setdiff(labels, "O"))

Arguments

pred

a factor with predictions

obs

a factor with gold labels

labels

a character vector of possible values that pred and obs can take. Defaults to the values in the data

labels_overall

a character vector of either labels which is either the same as labels or a subset of labels in order to compute a weighted average of the by-label statistics

Value

a list with 2 elements:

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
pred <- sample(LETTERS, 1000, replace = TRUE)
gold <- sample(LETTERS, 1000, replace = TRUE)
crf_evaluation(pred = pred, obs = gold, labels = LETTERS) 

## Not run: 
x <- ner_download_modeldata("conll2002-nl")
x <- crf_cbind_attributes(x, terms = c("token", "pos"), 
                          by = c("doc_id", "sentence_id"))
crf_train <- subset(x, data == "ned.train")
crf_test <- subset(x, data == "testa")
attributes <- grep("token|pos", colnames(x), value=TRUE)
model <- crf(y = crf_train$label, 
             x = crf_train[, attributes], 
             group = crf_train$doc_id, 
             method = "lbfgs") 
             
## Use the model to score on existing tokenised data
scores <- predict(model, 
                  newdata = crf_test[, attributes], 
                  group = crf_test$doc_id)
crf_evaluation(pred = scores$label, obs = crf_test$label)
crf_evaluation(pred = scores$label, obs = crf_test$label, 
  labels = c("O", 
             "B-ORG", "I-ORG", "B-PER", "I-PER", 
             "B-LOC", "I-LOC", "B-MISC", "I-MISC"))
             
             
library(udpipe)
pred <- txt_recode(scores$label, 
                   from = c("B-ORG", "I-ORG", "B-PER", "I-PER", 
                            "B-LOC", "I-LOC", "B-MISC", "I-MISC"),
                   to = c("ORG", "ORG", "PER", "PER", 
                          "LOC", "LOC", "MISC", "MISC"))
obs <- txt_recode(crf_test$label, 
                  from = c("B-ORG", "I-ORG", "B-PER", "I-PER", 
                           "B-LOC", "I-LOC", "B-MISC", "I-MISC"),
                  to = c("ORG", "ORG", "PER", "PER", 
                         "LOC", "LOC", "MISC", "MISC"))
crf_evaluation(pred = pred, obs = obs, 
               labels = c("ORG", "LOC", "PER", "MISC", "O"))

## End(Not run)

crfsuite documentation built on May 31, 2019, 5:04 p.m.