is_suspect_robotstxt: Check if file is valid / parsable robots.txt file

View source: R/is_suspect_robotstxt.R

is_suspect_robotstxtR Documentation

Check if file is valid / parsable robots.txt file

Description

Function that checks if file is valid / parsable robots.txt file

Usage

is_suspect_robotstxt(text)

Arguments

text

content of a robots.txt file provides as character vector


ropenscilabs/robotstxt documentation built on Sept. 4, 2024, 6 p.m.