clean_text: Clean Corpus with BBC News article

In unimi-dse/1ed5ff0d: BBC News article text analysis

Description

Clean documents in Corpus which include BBC News article text

Usage

clean_text(url_end)

Arguments

url_end

character string, an ending part of BBC News particular atricle URL (everything after https://www.bbc.com/news/). For example, article URL is "https://www.bbc.com/news/world-us-canada-51381625". Only "world-us-canada-51381625" should be pasted

Details

Transform the symbols like "/", "\" into space. Stem text, remove numbers, remove english stopwords, extra spaces and punctuation, convert text to lower case

Value

art_c - Corpus with cleaned and transformed text documents. Corpus represents a collection of text documents with an article text (each article paragraph as a single document in Corpus)

Note

Please, check that URL (url_end) exists before running the function, otherwise you will get an "Error in open.connection(x, "rb") : HTTP error 404". Please, insert URLs of articles in English only. Only for BBC News, not BBC Sports , Travel, Worklife, etc.

Examples

clean_text("world-us-canada-51381625")
clean_text("entertainment-arts-35232060")

unimi-dse/1ed5ff0d documentation built on Feb. 10, 2020, 12:21 a.m.