queue: wanted HTML file (*absolute* url) queue

In dongzhuoer/rget: fetch contents of a website

Description

wanted HTML file (absolute url) queue

Usage

.queue()

Format

A list of three character

1. current files to be downloaded

2. succeed files successfully downloaded

3. failed files failed to be downloaded

Details

the url can contain http:// or not.

But it must be HTML files, and actually linked by other files (? and # removed). That is to say,

• if some file links to www.foo.com/bar, you shouldn't add www.foo.com/bar/index.html to the queue

• if some file links to /bar.html, you shouldn't add www.foo.com/bar.html to the queue

Thus ensure setdiff(queue$current, queue$succeed) makes sense

dongzhuoer/rget documentation built on July 29, 2020, 1:45 p.m.