PDE_extr_data_from_pdfs: Extracting data from PDF (Portable Document Format) files
In erikstricker/PDE: Extract Sentences and Tables from PDFs with User Interface

Description Usage Arguments Value See Also Examples

PDE_extr_data_from_pdfs extracts sentences or tables from a single PDF file and writes output in the corresponding folder.

PDE_extr_data_from_pdfs(
  pdfs,
  whattoextr,
  out = ".",
  filter.words = "",
  ignore.case.fw = FALSE,
  filter.word.times = 20,
  table.heading.words = "",
  ignore.case.th = FALSE,
  search.words,
  ignore.case.sw = FALSE,
  eval.abbrevs = TRUE,
  out.table.format = ".csv (WINDOWS-1252)",
  dev = 20,
  context = 0,
  write.table.locations = FALSE,
  exp.nondetc.tabs = FALSE,
  write.tab.doc.file = FALSE,
  write.txt.doc.file = FALSE,
  delete = TRUE,
  verbose = TRUE
)

`pdfs`	String. A list of paths to the PDF files to be analyzed.
`whattoextr`	String. Either txt, tab, or tabandtxt for PDFS2TXT (extract sentences from a PDF file) or PDFS2TABLE (table of a PDF file to a Microsoft Excel file) extraction. tab allows the extraction of tables with and without search words while txt and tabandtxt require search words.
`out`	String. Directory chosen to save analysis results in. Default: `"."`.
`filter.words`	List of strings. The list of filter words. If not `NA` or `""` a hit will be counted every time a word from the list is detected in the article. Regex rules apply (see also https://rstudio.com/wp-content/uploads/2016/09/RegExCheatsheet.pdf). Default: `""`.
`ignore.case.fw`	Logical. Are the filter words case-sensitive (does capitalization matter)? Default: `FALSE`.
`filter.word.times`	Numeric. The minimum number of hits described for `filter.words` for a paper to be further analyzed. Default: `20`.
`table.heading.words`	List of strings. Different than standard (TABLE, TAB or table plus number) headings to be detected. Regex rules apply (see also https://rstudio.com/wp-content/uploads/2016/09/RegExCheatsheet.pdf). Default = `""`.
`ignore.case.th`	Logical. Are the additional table headings (see `table.heading.words`) case-sensitive (does capitalization matter)? Default = `FALSE`.
`search.words`	List of strings. List of search words. To extract all tables from the PDF files leave `search.words = ""`. Regex rules apply (see also https://rstudio.com/wp-content/uploads/2016/09/RegExCheatsheet.pdf).
`ignore.case.sw`	Logical. Are the search words case-sensitive (does capitalization matter)? Default: `FALSE`.
`eval.abbrevs`	Logical. Should abbreviations for the search words be automatically detected and then replaced with the search word + "$*"? Default: `TRUE`.
`out.table.format`	String. Output file format. Either comma separated file `.csv` or tab separated file `.tsv`. The encoding indicated in parantheses should be selected according to the operational system exported tables are opened in, i.e., Windows: `"(WINDOWS-1252)"`; Mac: `(macintosh)`; Linux: `(UTF-8)`. Default: `".csv"` and encoding depending on the operational system.
`dev`	Numeric. For a table the size of indention which would be considered the same column. Default: `20`.
`context`	Numeric. Number of sentences extracted before and after the sentence with the detected search word. If `0` only the sentence with the search word is extracted. Default: `0`.
`write.table.locations`	Logical. If `TRUE`, a separate file with the headings of all tables, their relative location in the generated html and txt files, as well as information if search words were found will be generated. Default: `FALSE`.
`exp.nondetc.tabs`	Logical. If `TRUE`, if a table was detected in a PDF file but is an image or cannot be read, the page with the table with be exported as a png. Default: `FALSE`.
`write.tab.doc.file`	Logical. If `TRUE`, if search words are used for table detection and no search words were found in the tables of a PDF file, a no.table.w.search.words. Default: `FALSE`.
`write.txt.doc.file`	Logical. If `TRUE`, if no search words were found in the sentences of a PDF file, a file will be created with the PDF filename followed by no.txt.w.search.words. If the PDF file is empty, a file will be created with the PDF filename followed by no.content.detected. If the filter word threshold is not met, a file will be created with the PDF filename followed by no.txt.w.filter.words. Default: `FALSE`.
`delete`	Logical. If `TRUE`, the intermediate txt, keeplayouttxt and html copies of the PDF files will be deleted. Default: `TRUE`.
`verbose`	Logical. Indicates whether messages will be printed in the console. Default: `TRUE`.

If tables were extracted from the PDF file the function returns a list of following tables/items: 1) htmltablelines, 2) txttablelines, 3) keeplayouttxttablelines, 4) id, 5) out_msg. The tablelines are tables that provide the heading and position of the detected tables. The id provide the name of the PDF file. The out_msg includes all messages printed to the console or the suppressed messages if verbose=FALSE.

PDE_pdfs2table,PDE_pdfs2table_searchandfilter,PDE_pdfs2txt_searchandfilter

## Running a simple analysis with filter and search words to extract sentences and tables
if(PDE_check_Xpdf_install() == TRUE){
 outputtables <- PDE_extr_data_from_pdfs(pdfs = c(paste0(PDE_path(),
                                                 "examples/Methotrexate/29973177_!.pdf"),
                                                 paste0(PDE_path(),
                                                 "examples/Methotrexate/31083238_!.pdf")),
 whattoextr = "tabandtxt",
 out = paste0(PDE_path(),"examples/MTX_all_files+-0_test/"),
 filter.words = strsplit("cohort;case-control;group;study population;study participants", ";")[[1]],
 ignore.case.fw = TRUE,
 search.words = strsplit("(M|m)ethotrexate;(T|t)rexal;(R|r)heumatrex;(O|o)trexup", ";")[[1]],
 ignore.case.sw = FALSE)
}

## Running an advanced analysis with filter and search words to
## extract sentences and tables and obtain documentation files
if(PDE_check_Xpdf_install() == TRUE){
 outputtables <- PDE_extr_data_from_pdfs(pdfs = c(paste0(PDE_path(),
                                                 "examples/Methotrexate/29973177_!.pdf"),
                                                  paste0(PDE_path(),
                                                 "examples/Methotrexate/31083238_!.pdf")),
 whattoextr = "tabandtxt",
 out = paste0(PDE_path(),"examples/MTX_all_files+-1_test/"),
 context = 1,
 dev = 20,
 filter.words = strsplit("cohort;case-control;group;study population;study participants",";")[[1]],
 ignore.case.fw = TRUE,
 filter.word.times = 20,
 table.heading.words = "",
 ignore.case.th = FALSE,
 search.words = strsplit("(M|m)ethotrexate;(T|t)rexal;(R|r)heumatrex;(O|o)trexup", ";")[[1]],
 ignore.case.sw = FALSE,
 eval.abbrevs = TRUE,
 out.table.format = ".csv (WINDOWS-1252)",
 write.table.locations = TRUE,
 write.tab.doc.file = TRUE,
 write.txt.doc.file = TRUE,
 exp.nondetc.tabs = TRUE,
 delete = TRUE)
}