rt_get_fields_worker: Extract robotstxt fields

View source: R/rt_get_fields_worker.R

rt_get_fields_workerR Documentation

Extract robotstxt fields

Description

Extract robotstxt fields

Usage

rt_get_fields_worker(txt, type = "all", regex = NULL, invert = FALSE)

Arguments

txt

content of the robots.txt file

type

name or names of the fields to be returned, defaults to all fields

regex

subsetting field names via regular expressions

invert

field selection


ropenscilabs/robotstxt documentation built on Sept. 4, 2024, 6 p.m.