is_valid_robotstxt: function that checks if file is valid / parsable robots.txt...

Description Usage Arguments

View source: R/is_valid_robotstxt.R

Description

function that checks if file is valid / parsable robots.txt file

Usage

1

Arguments

text

content of a robots.txt file provides as character vector


petermeissner/robotstxt documentation built on May 25, 2019, 1:56 a.m.