In katilingban/ennet: Utilities to Extract and Analyse Text Data from the Emergency Nutrition Network Forum

knitr::opts_chunk$set(
  message = FALSE,
  warning = FALSE,
  error = FALSE,
  collapse = TRUE,
  comment = "#>"
)

library(ennet)

ennet package provides a set of functions that extracts information from the en-net online forum. This set of functions was built on top of the rvest package which provides robust and performant web scraping functions and the dplyr package which provides a full suite of data manipulation functions. The ennet package was designed to be able to interact with how the en-net online forum has been structured.

en-net website structure

The en-net online forum website has a very clear and clean structure. The opening page is a list of thematic areas which are linked to each of their respective webpages. In each of these thematic area webpages is another list, this time a list of topics raised within the thematic area. These topics are the text that an online user provides as the title for the question she/he is going to ask. Each of the topics are then again linked to their respective webpages that show the actual full question raised and the ensuing responses and discussion stemming from that question.

The en-net online forum structure can be summarised graphically as follows:

knitr::include_graphics("../man/figures/ennet_structure.png")

Getting list of thematic areas

To get a list of thematic areas along with the link to the webpage of each thematic area, we use the get_themes() function as follows:

## Load ennet package
library(ennet)

## Get all thematic areas from en-net
get_themes()

which results in

library(ennet)
get_themes()

The resulting table has two columns - the first is named themes which contains the various thematic areas on the en-net online forum, and the second is named links which contains the corresponding URL for the webpages for each of the thematic areas.

This will be useful when choosing which themes to focus on when extracting information. This function outputs an object of the appropriate class and structure as the required input for the get_themes_topics() function which would lend to piped operations between the two functions (see below).

Getting list of topics from thematic area/s

To get a list of topics for a particular theme, we use the get_theme_topics() function as follows:

## Load dplyr to facilitate data manipulation
library(dplyr)

## Extract data from "Assessment and Surveillance" theme
get_themes() %>%
  filter(themes == "Assessment and Surveillance") %>%
  select(links) %>%
  as.character() %>%
  get_theme_topics()

which results in

## Load dplyr to facilitate data manipulation
library(dplyr)

## Extract data from "Assessment and Surveillance" theme
get_themes() %>%
  filter(themes == "Assessment and Surveillance") %>%
  select(links) %>%
  as.character() %>%
  get_theme_topics()

The resulting table contains information on all the topics within the Assessment and Surveillance thematic area including URL links to the corresponding webpages for each topic

To get a list of topics for multiple themes, we use the get_themes_topics() function as follows:

## Extract data from "Assessment and Surveillance" theme and "Food assistance" theme
get_themes() %>%
  filter(themes %in% c("Assessment and Surveillance", "Food assistance")) %>%
  get_themes_topics()

which results in

## Extract data from "Assessment and Surveillance" theme and "Food assistance" theme
get_themes() %>%
  filter(themes %in% c("Assessment and Surveillance", "Food assistance")) %>%
  get_themes_topics()

The resulting table contains information on all the topics within the Assessment and Surveillance and Food assistance thematic area including URL links to the corresponding webpages for each topic.

Getting discussions from topic/s

To get a list of discussions for a particular topic, we use the get_topic_discussions() function as follows:

get_themes() %>%
  filter(themes == "Assessment and Surveillance") %>%
  get_themes_topics() %>%
  filter(Topic == "Resources for coverage assessment") %>%
  select(Link) %>%
  as.character() %>%
  get_topic_discussions()

which results in

get_themes() %>%
  filter(themes == "Assessment and Surveillance") %>%
  get_themes_topics() %>%
  filter(Topic == "Resources for coverage assessment") %>%
  select(Link) %>%
  as.character() %>%
  get_topic_discussions()

The resulting table contains information on all the discussions within the topic on Resources for coverage assessment within the thematic area of Assessment and Surveillance including the text data on the question and the ensuing reply/replies to the question.

To get a list of discussions for a set of topics, we use the get_topics_discussions() function as follows:

get_themes() %>%
  filter(themes %in% c("Assessment and Surveillance", "Food assistance")) %>%
  get_themes_topics() %>%
  get_topics_discussions()

which results in

get_themes() %>%
  filter(themes %in% c("Assessment and Surveillance", "Food assistance")) %>%
  get_themes_topics() %>%
  get_topics_discussions()

The resulting table contains information on all the discussions within all the topics within the thematic areas of Assessment and Surveillance and Food assistance including the text data on the question and the ensuing reply/replies to the question.

katilingban/ennet documentation built on Nov. 3, 2022, 4:39 p.m.