parse_url: parse_url

In robotstxt: A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker

Description

parse_url

Usage

parse_url(url)

Arguments

url	url to parse into its components

Value

data.frame with columns protocol, domain, path

Examples

## Not run: 
url <-
c(
  "google.com",
  "google.com/",
  "www.google.com",
  "http://google.com",
  "https://google.com",
  "sub.domain.whatever.de"
  "s-u-b.dom-ain.what-ever.de"
)

parse_url(url)

## End(Not run)

robotstxt documentation built on Sept. 4, 2020, 1:08 a.m.