README.md
In bryanwhiting/generalconference: General Conference text corpus and web scrapers
generalconference
General
Conference
is a semi-annual event where members of The Church of Jesus Christ of
Latter-day Saints gather to listen to church prophets, apostles, and
other leaders.
This package both scrapes General Conference talks and provides all
talks in a data package for analysis in R.
Install the package
# install.packages('devtools')
devtools::install_github("bryanwhiting/generalconference")
Load the package:
library(generalconference)
Load the General Conference corpus, which is a tibble with nested data
for each conference, session, talk, and paragraph.
data("genconf")
head(genconf)
#> # A tibble: 6 × 4
#> year month date sessions
#> <dbl> <dbl> <date> <list>
#> 1 2021 4 2021-04-01 <tibble [5 × 4]>
#> 2 2020 10 2020-10-01 <tibble [5 × 4]>
#> 3 2020 4 2020-04-01 <tibble [5 × 4]>
#> 4 2019 10 2019-10-01 <tibble [5 × 4]>
#> 5 2019 4 2019-04-01 <tibble [5 × 4]>
#> 6 2018 10 2018-10-01 <tibble [5 × 4]>
Getting Started
Unnest it to analyze individual talks, which can be unnested further to
the paragraph level.
library(dplyr)
library(tidyr)
genconf %>%
tidyr::unnest(sessions) %>%
tidyr::unnest(talks) %>%
head()
#> # A tibble: 6 × 14
#> year month date session_name session_id session_url talk_urls
#> <dbl> <dbl> <date> <chr> <int> <chr> <chr>
#> 1 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 2 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 3 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 4 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 5 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 6 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> # … with 7 more variables: talk_session_id <int>, url <chr>, title1 <chr>,
#> # author1 <chr>, author2 <chr>, kicker1 <chr>, paragraphs <list>
Analyze individual paragraphs that contain the word “faith”:
library(gt)
genconf %>%
# unpack/unnest the dataframe, which is a tibble of lists
tidyr::unnest(sessions) %>%
tidyr::unnest(talks) %>%
tidyr::unnest(paragraphs) %>%
# extract just the date, title, author and paragraph
# date, title, and author will be repeated fields, with paragraph unique
select(date, title1, author1, paragraph) %>%
# Filter to just the paragraphs that mention the word "faith"
filter(stringr::str_detect(paragraph, "faith")) %>%
# take top 5 records
head(5) %>%
# convert into a gt() table with row groups for date/title/author
# (use row groups since these data are replicated by paragraph)
group_by(date, title1, author1) %>%
gt() %>%
tab_options(
row_group.background.color = 'lightgray'
) %>%
tab_header(
title='Paragraphs on Faith',
subtitle='Grouped by talk'
)
html {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}
#kjgmesdonl .gt_table {
display: table;
border-collapse: collapse;
margin-left: auto;
margin-right: auto;
color: #333333;
font-size: 16px;
font-weight: normal;
font-style: normal;
background-color: #FFFFFF;
width: auto;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #A8A8A8;
border-right-style: none;
border-right-width: 2px;
border-right-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #A8A8A8;
border-left-style: none;
border-left-width: 2px;
border-left-color: #D3D3D3;
}
#kjgmesdonl .gt_heading {
background-color: #FFFFFF;
text-align: center;
border-bottom-color: #FFFFFF;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_title {
color: #333333;
font-size: 125%;
font-weight: initial;
padding-top: 4px;
padding-bottom: 4px;
border-bottom-color: #FFFFFF;
border-bottom-width: 0;
}
#kjgmesdonl .gt_subtitle {
color: #333333;
font-size: 85%;
font-weight: initial;
padding-top: 0;
padding-bottom: 6px;
border-top-color: #FFFFFF;
border-top-width: 0;
}
#kjgmesdonl .gt_bottom_border {
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
}
#kjgmesdonl .gt_col_headings {
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_col_heading {
color: #333333;
background-color: #FFFFFF;
font-size: 100%;
font-weight: normal;
text-transform: inherit;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
vertical-align: bottom;
padding-top: 5px;
padding-bottom: 6px;
padding-left: 5px;
padding-right: 5px;
overflow-x: hidden;
}
#kjgmesdonl .gt_column_spanner_outer {
color: #333333;
background-color: #FFFFFF;
font-size: 100%;
font-weight: normal;
text-transform: inherit;
padding-top: 0;
padding-bottom: 0;
padding-left: 4px;
padding-right: 4px;
}
#kjgmesdonl .gt_column_spanner_outer:first-child {
padding-left: 0;
}
#kjgmesdonl .gt_column_spanner_outer:last-child {
padding-right: 0;
}
#kjgmesdonl .gt_column_spanner {
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
vertical-align: bottom;
padding-top: 5px;
padding-bottom: 5px;
overflow-x: hidden;
display: inline-block;
width: 100%;
}
#kjgmesdonl .gt_group_heading {
padding: 8px;
color: #333333;
background-color: #D3D3D3;
font-size: 100%;
font-weight: initial;
text-transform: inherit;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
vertical-align: middle;
}
#kjgmesdonl .gt_empty_group_heading {
padding: 0.5px;
color: #333333;
background-color: #D3D3D3;
font-size: 100%;
font-weight: initial;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
vertical-align: middle;
}
#kjgmesdonl .gt_from_md > :first-child {
margin-top: 0;
}
#kjgmesdonl .gt_from_md > :last-child {
margin-bottom: 0;
}
#kjgmesdonl .gt_row {
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
margin: 10px;
border-top-style: solid;
border-top-width: 1px;
border-top-color: #D3D3D3;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
vertical-align: middle;
overflow-x: hidden;
}
#kjgmesdonl .gt_stub {
color: #333333;
background-color: #FFFFFF;
font-size: 100%;
font-weight: initial;
text-transform: inherit;
border-right-style: solid;
border-right-width: 2px;
border-right-color: #D3D3D3;
padding-left: 12px;
}
#kjgmesdonl .gt_summary_row {
color: #333333;
background-color: #FFFFFF;
text-transform: inherit;
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
}
#kjgmesdonl .gt_first_summary_row {
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
}
#kjgmesdonl .gt_grand_summary_row {
color: #333333;
background-color: #FFFFFF;
text-transform: inherit;
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
}
#kjgmesdonl .gt_first_grand_summary_row {
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
border-top-style: double;
border-top-width: 6px;
border-top-color: #D3D3D3;
}
#kjgmesdonl .gt_striped {
background-color: rgba(128, 128, 128, 0.05);
}
#kjgmesdonl .gt_table_body {
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
}
#kjgmesdonl .gt_footnotes {
color: #333333;
background-color: #FFFFFF;
border-bottom-style: none;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 2px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 2px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_footnote {
margin: 0px;
font-size: 90%;
padding: 4px;
}
#kjgmesdonl .gt_sourcenotes {
color: #333333;
background-color: #FFFFFF;
border-bottom-style: none;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 2px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 2px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_sourcenote {
font-size: 90%;
padding: 4px;
}
#kjgmesdonl .gt_left {
text-align: left;
}
#kjgmesdonl .gt_center {
text-align: center;
}
#kjgmesdonl .gt_right {
text-align: right;
font-variant-numeric: tabular-nums;
}
#kjgmesdonl .gt_font_normal {
font-weight: normal;
}
#kjgmesdonl .gt_font_bold {
font-weight: bold;
}
#kjgmesdonl .gt_font_italic {
font-style: italic;
}
#kjgmesdonl .gt_super {
font-size: 65%;
}
#kjgmesdonl .gt_footnote_marks {
font-style: italic;
font-weight: normal;
font-size: 65%;
}
Paragraphs on Faith
Grouped by talk
paragraph
2021-04-01 - God among Us - By Elder Dieter F. Uchtdorf
The doubting He would infuse with faith and courage to believe.
He would recognize and honor honesty, humility, integrity, faithfulness, compassion, and charity.
To you, my dear brothers and sisters, my dear friends, and to all who are searching for answers, truth, and happiness, I do offer the same counsel: keep searching with faith and patience.
2021-04-01 - Essential Conversations - By Joy D. Jones
Have faith in Christ, the Son of the living God.
Elder D. Todd Christofferson said, “Certainly the adversary is pleased when parents neglect to teach and train their children to have faith in Christ and be spiritually born again.”
- See Example Analysis for more
examples on how to analyze the data.
- See genconf for a schema of the data.
See documentation for scrapers, if you need them. But you shouldn’t need
them since all the data is available.
bryanwhiting/generalconference documentation built on Dec. 19, 2021, 11:51 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
generalconference
General Conference is a semi-annual event where members of The Church of Jesus Christ of Latter-day Saints gather to listen to church prophets, apostles, and other leaders.
This package both scrapes General Conference talks and provides all talks in a data package for analysis in R.
Install the package
# install.packages('devtools')
devtools::install_github("bryanwhiting/generalconference")
Load the package:
library(generalconference)
Load the General Conference corpus, which is a tibble with nested data for each conference, session, talk, and paragraph.
data("genconf")
head(genconf)
#> # A tibble: 6 × 4
#> year month date sessions
#> <dbl> <dbl> <date> <list>
#> 1 2021 4 2021-04-01 <tibble [5 × 4]>
#> 2 2020 10 2020-10-01 <tibble [5 × 4]>
#> 3 2020 4 2020-04-01 <tibble [5 × 4]>
#> 4 2019 10 2019-10-01 <tibble [5 × 4]>
#> 5 2019 4 2019-04-01 <tibble [5 × 4]>
#> 6 2018 10 2018-10-01 <tibble [5 × 4]>
Getting Started
Unnest it to analyze individual talks, which can be unnested further to the paragraph level.
library(dplyr)
library(tidyr)
genconf %>%
tidyr::unnest(sessions) %>%
tidyr::unnest(talks) %>%
head()
#> # A tibble: 6 × 14
#> year month date session_name session_id session_url talk_urls
#> <dbl> <dbl> <date> <chr> <int> <chr> <chr>
#> 1 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 2 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 3 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 4 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 5 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> 6 2021 4 2021-04-01 Saturday Morn… 1 /study/general-… /study/gene…
#> # … with 7 more variables: talk_session_id <int>, url <chr>, title1 <chr>,
#> # author1 <chr>, author2 <chr>, kicker1 <chr>, paragraphs <list>
Analyze individual paragraphs that contain the word “faith”:
library(gt)
genconf %>%
# unpack/unnest the dataframe, which is a tibble of lists
tidyr::unnest(sessions) %>%
tidyr::unnest(talks) %>%
tidyr::unnest(paragraphs) %>%
# extract just the date, title, author and paragraph
# date, title, and author will be repeated fields, with paragraph unique
select(date, title1, author1, paragraph) %>%
# Filter to just the paragraphs that mention the word "faith"
filter(stringr::str_detect(paragraph, "faith")) %>%
# take top 5 records
head(5) %>%
# convert into a gt() table with row groups for date/title/author
# (use row groups since these data are replicated by paragraph)
group_by(date, title1, author1) %>%
gt() %>%
tab_options(
row_group.background.color = 'lightgray'
) %>%
tab_header(
title='Paragraphs on Faith',
subtitle='Grouped by talk'
)
html {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}
#kjgmesdonl .gt_table {
display: table;
border-collapse: collapse;
margin-left: auto;
margin-right: auto;
color: #333333;
font-size: 16px;
font-weight: normal;
font-style: normal;
background-color: #FFFFFF;
width: auto;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #A8A8A8;
border-right-style: none;
border-right-width: 2px;
border-right-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #A8A8A8;
border-left-style: none;
border-left-width: 2px;
border-left-color: #D3D3D3;
}
#kjgmesdonl .gt_heading {
background-color: #FFFFFF;
text-align: center;
border-bottom-color: #FFFFFF;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_title {
color: #333333;
font-size: 125%;
font-weight: initial;
padding-top: 4px;
padding-bottom: 4px;
border-bottom-color: #FFFFFF;
border-bottom-width: 0;
}
#kjgmesdonl .gt_subtitle {
color: #333333;
font-size: 85%;
font-weight: initial;
padding-top: 0;
padding-bottom: 6px;
border-top-color: #FFFFFF;
border-top-width: 0;
}
#kjgmesdonl .gt_bottom_border {
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
}
#kjgmesdonl .gt_col_headings {
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_col_heading {
color: #333333;
background-color: #FFFFFF;
font-size: 100%;
font-weight: normal;
text-transform: inherit;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
vertical-align: bottom;
padding-top: 5px;
padding-bottom: 6px;
padding-left: 5px;
padding-right: 5px;
overflow-x: hidden;
}
#kjgmesdonl .gt_column_spanner_outer {
color: #333333;
background-color: #FFFFFF;
font-size: 100%;
font-weight: normal;
text-transform: inherit;
padding-top: 0;
padding-bottom: 0;
padding-left: 4px;
padding-right: 4px;
}
#kjgmesdonl .gt_column_spanner_outer:first-child {
padding-left: 0;
}
#kjgmesdonl .gt_column_spanner_outer:last-child {
padding-right: 0;
}
#kjgmesdonl .gt_column_spanner {
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
vertical-align: bottom;
padding-top: 5px;
padding-bottom: 5px;
overflow-x: hidden;
display: inline-block;
width: 100%;
}
#kjgmesdonl .gt_group_heading {
padding: 8px;
color: #333333;
background-color: #D3D3D3;
font-size: 100%;
font-weight: initial;
text-transform: inherit;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
vertical-align: middle;
}
#kjgmesdonl .gt_empty_group_heading {
padding: 0.5px;
color: #333333;
background-color: #D3D3D3;
font-size: 100%;
font-weight: initial;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
vertical-align: middle;
}
#kjgmesdonl .gt_from_md > :first-child {
margin-top: 0;
}
#kjgmesdonl .gt_from_md > :last-child {
margin-bottom: 0;
}
#kjgmesdonl .gt_row {
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
margin: 10px;
border-top-style: solid;
border-top-width: 1px;
border-top-color: #D3D3D3;
border-left-style: none;
border-left-width: 1px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 1px;
border-right-color: #D3D3D3;
vertical-align: middle;
overflow-x: hidden;
}
#kjgmesdonl .gt_stub {
color: #333333;
background-color: #FFFFFF;
font-size: 100%;
font-weight: initial;
text-transform: inherit;
border-right-style: solid;
border-right-width: 2px;
border-right-color: #D3D3D3;
padding-left: 12px;
}
#kjgmesdonl .gt_summary_row {
color: #333333;
background-color: #FFFFFF;
text-transform: inherit;
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
}
#kjgmesdonl .gt_first_summary_row {
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
}
#kjgmesdonl .gt_grand_summary_row {
color: #333333;
background-color: #FFFFFF;
text-transform: inherit;
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
}
#kjgmesdonl .gt_first_grand_summary_row {
padding-top: 8px;
padding-bottom: 8px;
padding-left: 5px;
padding-right: 5px;
border-top-style: double;
border-top-width: 6px;
border-top-color: #D3D3D3;
}
#kjgmesdonl .gt_striped {
background-color: rgba(128, 128, 128, 0.05);
}
#kjgmesdonl .gt_table_body {
border-top-style: solid;
border-top-width: 2px;
border-top-color: #D3D3D3;
border-bottom-style: solid;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
}
#kjgmesdonl .gt_footnotes {
color: #333333;
background-color: #FFFFFF;
border-bottom-style: none;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 2px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 2px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_footnote {
margin: 0px;
font-size: 90%;
padding: 4px;
}
#kjgmesdonl .gt_sourcenotes {
color: #333333;
background-color: #FFFFFF;
border-bottom-style: none;
border-bottom-width: 2px;
border-bottom-color: #D3D3D3;
border-left-style: none;
border-left-width: 2px;
border-left-color: #D3D3D3;
border-right-style: none;
border-right-width: 2px;
border-right-color: #D3D3D3;
}
#kjgmesdonl .gt_sourcenote {
font-size: 90%;
padding: 4px;
}
#kjgmesdonl .gt_left {
text-align: left;
}
#kjgmesdonl .gt_center {
text-align: center;
}
#kjgmesdonl .gt_right {
text-align: right;
font-variant-numeric: tabular-nums;
}
#kjgmesdonl .gt_font_normal {
font-weight: normal;
}
#kjgmesdonl .gt_font_bold {
font-weight: bold;
}
#kjgmesdonl .gt_font_italic {
font-style: italic;
}
#kjgmesdonl .gt_super {
font-size: 65%;
}
#kjgmesdonl .gt_footnote_marks {
font-style: italic;
font-weight: normal;
font-size: 65%;
}
Paragraphs on Faith
Grouped by talk
paragraph
2021-04-01 - God among Us - By Elder Dieter F. Uchtdorf
The doubting He would infuse with faith and courage to believe.
He would recognize and honor honesty, humility, integrity, faithfulness, compassion, and charity.
To you, my dear brothers and sisters, my dear friends, and to all who are searching for answers, truth, and happiness, I do offer the same counsel: keep searching with faith and patience.
2021-04-01 - Essential Conversations - By Joy D. Jones
Have faith in Christ, the Son of the living God.
Elder D. Todd Christofferson said, “Certainly the adversary is pleased when parents neglect to teach and train their children to have faith in Christ and be spiritually born again.”
- See Example Analysis for more examples on how to analyze the data.
- See genconf for a schema of the data.
See documentation for scrapers, if you need them. But you shouldn’t need them since all the data is available.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.