Man pages for digital-geopolitics/dgblogs
Scrape text from blogs
| default_get_article_urls | Extract article links from index page |
| default_get_index_urls | Get urls for index pages |
| download_article | Download article |
| download_articles | Download all articles |
| download_blog | Download blog articles |
| get_node_text | Extract text from html/xml node |
| get_url | HTTPrequest |
| load_config | Load blog configuration from file |
| load_index_parsers | Import html parsing functions from file |
| merge_xp | Merge xpath expressions |
| parse_all_articles | Parse all articles |
| parse_article | Parse blog article |
| parse_blog_articles | Parse blog |
| parse_date | Parse date from string |
| pipe | Pipe operator |
| print_article_content | Print article content |
| scrape_article_urls | Find new article urls on index pages and insert into db |
| scrape_blog | Scrape article urls from blog |
| was_redirected | Check if download was redirected |
