rt_get_rtxt: Load robots.txt files saved along with the package
In petermeissner/robotstxt: A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker
rt_get_rtxt R Documentation
Load robots.txt files saved along with the package
Description
load robots.txt files saved along with the package:
these functions are very handy for testing (not used otherwise)
Usage
rt_get_rtxt(name = sample(rt_list_rtxt(), 1))
Arguments
name
name of the robots.txt files, defaults to a random drawn file ;-)
petermeissner/robotstxt documentation built on Nov. 17, 2024, 9:50 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| rt_get_rtxt | R Documentation |
Load robots.txt files saved along with the package
Description
load robots.txt files saved along with the package: these functions are very handy for testing (not used otherwise)
Usage
rt_get_rtxt(name = sample(rt_list_rtxt(), 1))
Arguments
name |
name of the robots.txt files, defaults to a random drawn file ;-) |
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.