In hrbrmstr/urlscan: Analyze Websites and Resources They Request

urlscan

Analyze Websites and Resources They Request

Description

The service provides an 'API' enabling analysis of websites and the resources they request. Much like the 'Inspector' of your browser, will let you take a look at the individual resources that are requested when a site is loaded. Tools are provided to search public scan submissions/results and submit URLs for scanning.

What's Inside The Tin

The following functions are implemented:

• urlscan_search: Perform a urlscan.io query
• urlscan_result: Retrieve detailed results for a given scan ID
• urlscan_submit: Submit a URL for scanning

Installation

devtools::install_git("https://git.sr.ht/~hrbrmstr/urlscan")
# or
devtools::install_gitlab("hrbrmstr/urlscan")
# or
devtools::install_github("hrbrmstr/urlscan")

options(width=120)

Usage

library(urlscan)
library(tidyverse) # for demos

# current verison
packageVersion("urlscan")

x <- urlscan_search("domain:r-project.org")

as_tibble(x$results$task) %>% 
  bind_cols(as_tibble(x$results$page)) %>% 
  mutate(
    time = anytime::anytime(time),
    id = x$results$`_id`
  ) %>%
  arrange(desc(time)) %>% 
  select(url, country, server, ip, id) -> xdf

ures <- urlscan_result(xdf$id[2], include_dom = TRUE, include_shot = TRUE)

ures

magick::image_write(ures$screenshot, "img/shot.png")

urlscan Metrics

cloc::cloc_pkg_md()

Code of Conduct

Please note that the 'urlscan' project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

hrbrmstr/urlscan documentation built on May 15, 2019, 3:30 p.m.