rt_get_comments: extracting comments from robots.txt
In robotstxt: A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker
View source: R/rt_get_comments.R
Description
extracting comments from robots.txt
Usage
1
rt_get_comments(txt)
Arguments
txt
content of the robots.txt file
robotstxt documentation built on Sept. 4, 2020, 1:08 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
View source: R/rt_get_comments.R
Description
extracting comments from robots.txt
Usage
1 | rt_get_comments(txt)
|
Arguments
txt |
content of the robots.txt file |
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.