is_suspect_robotstxt: is_suspect_robotstxt

Description Usage Arguments

View source: R/is_suspect_robotstxt.R

Description

function that checks if file is valid / parsable robots.txt file

Usage

1

Arguments

text

content of a robots.txt file provides as character vector


robotstxt documentation built on Sept. 4, 2020, 1:08 a.m.