R/fix_url.R

In robotstxt: A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker

Defines functions fix_url

Documented in fix_url

#' fix_url
#'
#'
#' @param url a character string containing a single URL
#'
fix_url <-
  function(url){
    parsed_url <- httr::parse_url(url)
    if ( is.null(parsed_url$scheme) ){
      url <- paste0("http://", url)
    }
    url
  }

Try the robotstxt package in your browser