JohnCoene/chirp
'Twitter' Networks Analyser

Package index
Search the JohnCoene/chirp package
Vignettes
Functions
33
Source code
8
Man pages
4
Home
/
GitHub
/
JohnCoene/chirp
/
website/node_modules/parse5/docs/build/index.md

website/node_modules/parse5/docs/build/index.md
In JohnCoene/chirp: 'Twitter' Networks Analyser

parse5

NPM Version

WHATWG HTML5 specification-compliant, fast and ready for production HTML parsing/serialization toolset for Node.

parse5 contains nearly everything what you will need to deal with the HTML. It's the fastest spec-compliant HTML parser for Node to the date and will parse HTML the way the latest version of your browser does. It's stable and used by such projects as jsdom, Angular2, Polymer and many more.

Table of contents

Install

$ npm install parse5

Usage

var parse5 = require('parse5');

var document     = parse5.parse('<!DOCTYPE html><html><body>Hi there!</body></html>');
var documentHtml = parse5.serialize(document);


var fragment     = parse5.parseFragment('<td>Yo!</td>');
var fragmentHtml = parse5.serialize(fragment);

For more advanced examples see API reference and FAQ.

API Reference

Objects

parse5 : object

Typedefs

ElementLocationInfo : Object LocationInfo : Object ParserOptions : Object SAXParserOptions : Object SerializerOptions : Object TreeAdapter : Object

parse5 : object

Kind: global namespace

parse5.ParserStream ⇐ stream.Writable

Kind: instance class of parse5 Extends: stream.Writable

new ParserStream(options)

Streaming HTML parser with the scripting support. Writable stream.

| Param | Type | Description | | --- | --- | --- | | options | ParserOptions | Parsing options. |

Example

var parse5 = require('parse5');
var http = require('http');

// Fetch google.com content and obtain it's <body> node
http.get('http://google.com', function(res) {
 var parser = new parse5.ParserStream();

 parser.on('finish', function() {
     var body = parser.document.childNodes[0].childNodes[1];
 });

 res.pipe(parser);
});

parserStream.document : ASTNode.<document>

Resulting document node.

Kind: instance property of ParserStream

"script" (scriptElement, documentWrite(html), resume)

Raised then parser encounters <script> element. If event has listeners then parsing will be suspended on event emission. So, if <script> has src attribute you can fetch it, execute and then resume parser like browsers do.

Kind: event emitted by ParserStream

| Param | Type | Description | | --- | --- | --- | | scriptElement | ASTNode | Script element that caused the event. | | documentWrite(html) | function | Write additional html at the current parsing position. Suitable for the DOM document.write and document.writeln methods implementation. | | resume | function | Resumes the parser. |

Example

var parse = require('parse5');
var http = require('http');

var parser = new parse5.ParserStream();

parser.on('script', function(scriptElement, documentWrite, resume) {
  var src = parse5.treeAdapters.default.getAttrList(scriptElement)[0].value;

  http.get(src, function(res) {
     // Fetch script content, execute it with DOM built around `parser.document` and
     // `document.write` implemented using `documentWrite`
     ...
     // Then resume the parser
     resume();
  });
});

parser.end('<script src="example.com/script.js"></script>');

parse5.SAXParser ⇐ stream.Transform

Kind: instance class of parse5 Extends: stream.Transform

new SAXParser(options)

Streaming SAX-style HTML parser. Transform stream (which means you can pipe through it, see example).

| Param | Type | Description | | --- | --- | --- | | options | SAXParserOptions | Parsing options. |

Example

var parse5 = require('parse5');
var http = require('http');
var fs = require('fs');

var file = fs.createWriteStream('/home/google.com.html');
var parser = new SAXParser();

parser.on('text', function(text) {
 // Handle page text content
 ...
});

http.get('http://google.com', function(res) {
 // SAXParser is the Transform stream, which means you can pipe
 // through it. So you can analyze page content and e.g. save it
 // to the file at the same time:
 res.pipe(parser).pipe(file);
});

saxParser.stop()

Stops parsing. Useful if you want parser to stop consume CPU time once you've obtained desired info from input stream. Doesn't prevents piping, so data will flow through parser as usual.

Kind: instance method of SAXParser Example

var parse5 = require('parse5');
var http = require('http');
var fs = require('fs');

var file = fs.createWriteStream('/home/google.com.html');
var parser = new parse5.SAXParser();

parser.on('doctype', function(name, publicId, systemId) {
 // Process doctype info ans stop parsing
 ...
 parser.stop();
});

http.get('http://google.com', function(res) {
 // Despite the fact that parser.stop() was called whole
 // content of the page will be written to the file
 res.pipe(parser).pipe(file);
});

"startTag" (name, attributes, selfClosing, [location])

Raised then parser encounters start tag.

Kind: event emitted by SAXParser

| Param | Type | Description | | --- | --- | --- | | name | String | Tag name. | | attributes | String | List of attributes in { key: String, value: String } form. | | selfClosing | Boolean | Indicates if tag is self-closing. | | [location] | LocationInfo | Start tag source code location info. Available if location info is enabled in SAXParserOptions. |

"endTag" (name, [location])

Raised then parser encounters end tag.

Kind: event emitted by SAXParser

| Param | Type | Description | | --- | --- | --- | | name | String | Tag name. | | [location] | LocationInfo | End tag source code location info. Available if location info is enabled in SAXParserOptions. |

"comment" (text, [location])

Raised then parser encounters comment.

Kind: event emitted by SAXParser

| Param | Type | Description | | --- | --- | --- | | text | String | Comment text. | | [location] | LocationInfo | Comment source code location info. Available if location info is enabled in SAXParserOptions. |

"doctype" (name, publicId, systemId, [location])

Raised then parser encounters document type declaration.

Kind: event emitted by SAXParser

| Param | Type | Description | | --- | --- | --- | | name | String | Document type name. | | publicId | String | Document type public identifier. | | systemId | String | Document type system identifier. | | [location] | LocationInfo | Document type declaration source code location info. Available if location info is enabled in SAXParserOptions. |

"text" (text, [location])

Raised then parser encounters text content.

Kind: event emitted by SAXParser

| Param | Type | Description | | --- | --- | --- | | text | String | Text content. | | [location] | LocationInfo | Text content code location info. Available if location info is enabled in SAXParserOptions. |

parse5.SerializerStream ⇐ stream.Readable

Kind: instance class of parse5 Extends: stream.Readable

new SerializerStream(node, [options])

Streaming AST node to HTML serializer. Readable stream.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node to serialize. | | [options] | SerializerOptions | Serialization options. |

Example

var parse5 = require('parse5');
var fs = require('fs');

var file = fs.createWriteStream('/home/index.html');

// Serialize parsed document to the HTML and write it to file
var document = parse5.parse('<body>Who is John Galt?</body>');
var serializer = new parse5.SerializerStream(document);

serializer.pipe(file);

parse5.treeAdapters

Provides built-in tree adapters which can be used for parsing and serialization.

Kind: instance property of parse5 Properties

| Name | Type | Description | | --- | --- | --- | | default | TreeAdapter | Default tree format for parse5. | | htmlparser2 | TreeAdapter | Quite popular htmlparser2 tree format (e.g. used by cheerio and jsdom). |

Example

var parse5 = require('parse5');

// Use default tree adapter for parsing
var document = parse5.parse('<div></div>', { treeAdapter: parse5.treeAdapters.default });

// Use htmlparser2 tree adapter with SerializerStream
var serializer = new parse5.SerializerStream(node, { treeAdapter: parse5.treeAdapters.htmlparser2 });

parse5.parse(html, [options]) ⇒ ASTNode.<Document>

Parses HTML string.

Kind: instance method of parse5 Returns: ASTNode.<Document> - document

| Param | Type | Description | | --- | --- | --- | | html | string | Input HTML string. | | [options] | ParserOptions | Parsing options. |

Example

var parse5 = require('parse5');

var document = parse5.parse('<!DOCTYPE html><html><head></head><body>Hi there!</body></html>');

parse5.parseFragment([fragmentContext], html, [options]) ⇒ ASTNode.<DocumentFragment>

Parses HTML fragment.

Kind: instance method of parse5 Returns: ASTNode.<DocumentFragment> - documentFragment

| Param | Type | Description | | --- | --- | --- | | [fragmentContext] | ASTNode | Parsing context element. If specified, given fragment will be parsed as if it was set to the context element's innerHTML property. | | html | string | Input HTML fragment string. | | [options] | ParserOptions | Parsing options. |

Example

var parse5 = require('parse5');

var documentFragment = parse5.parseFragment('<table></table>');

//Parse html fragment in context of the parsed <table> element
var trFragment = parser.parseFragment(documentFragment.childNodes[0], '<tr><td>Shake it, baby</td></tr>');

parse5.serialize(node, [options]) ⇒ String

Serializes AST node to HTML string.

Kind: instance method of parse5 Returns: String - html

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node to serialize. | | [options] | SerializerOptions | Serialization options. |

Example

var parse5 = require('parse5');

var document = parse5.parse('<!DOCTYPE html><html><head></head><body>Hi there!</body></html>');

//Serialize document
var html = parse5.serialize(document);

//Serialize <body> element content
var bodyInnerHtml = parse5.serialize(document.childNodes[0].childNodes[1]);

ElementLocationInfo : Object

Kind: global typedef Extends: LocationInfo Properties

| Name | Type | Description | | --- | --- | --- | | startTag | LocationInfo | Element's start tag LocationInfo. | | endTag | LocationInfo | Element's end tag LocationInfo. |

LocationInfo : Object

Kind: global typedef Properties

| Name | Type | Description | | --- | --- | --- | | line | Number | One-based line index | | col | Number | One-based column index | | startOffset | Number | Zero-based first character index | | endOffset | Number | Zero-based last character index |

ParserOptions : Object

Kind: global typedef Properties

| Name | Type | Default | Description | | --- | --- | --- | --- | | locationInfo | Boolean | false | Enables source code location information for the nodes. When enabled, each node (except root node) has __location property. In case the node is not an empty element, __location will be ElementLocationInfo object, otherwise it's LocationInfo. If element was implicitly created by the parser it's __location property will be null. | | treeAdapter | TreeAdapter | parse5.treeAdapters.default | Specifies resulting tree format. |

SAXParserOptions : Object

Kind: global typedef Properties

| Name | Type | Default | Description | | --- | --- | --- | --- | | locationInfo | Boolean | false | Enables source code location information for the tokens. When enabled, each token event handler will receive LocationInfo object as the last argument. |

SerializerOptions : Object

Kind: global typedef Properties

| Name | Type | Default | Description | | --- | --- | --- | --- | | treeAdapter | TreeAdapter | parse5.treeAdapters.default | Specifies input tree format. |

TreeAdapter : Object

Kind: global typedef

TreeAdapter.createDocument() ⇒ ASTNode.<Document>

Creates document node

Kind: static method of TreeAdapter Returns: ASTNode.<Document> - document See: default implementation.

TreeAdapter.createDocumentFragment() ⇒ ASTNode.<DocumentFragment>

Creates document fragment node

Kind: static method of TreeAdapter Returns: ASTNode.<DocumentFragment> - fragment See: default implementation.

TreeAdapter.createElement(tagName, namespaceURI, attrs) ⇒ ASTNode.<Element>

Creates element node

Kind: static method of TreeAdapter Returns: ASTNode.<Element> - element See: default implementation.

| Param | Type | Description | | --- | --- | --- | | tagName | String | Tag name of the element. | | namespaceURI | String | Namespace of the element. | | attrs | Array | Attribute name-value pair array. Foreign attributes may contain namespace and prefix fields as well. |

TreeAdapter.createElement(data) ⇒ ASTNode.<CommentNode>

Creates comment node

Kind: static method of TreeAdapter Returns: ASTNode.<CommentNode> - comment See: default implementation.

| Param | Type | Description | | --- | --- | --- | | data | String | Comment text. |

TreeAdapter.setDocumentType(document, name, publicId, systemId)

Sets document type. If document already have document type node in it then name, publicId and systemId properties of the node will be updated with the provided values. Otherwise, creates new document type node with the given properties and inserts it into document.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | document | ASTNode.<Document> | Document node. | | name | String | Document type name. | | publicId | String | Document type public identifier. | | systemId | String | Document type system identifier. |

TreeAdapter.setQuirksMode(document)

Sets document quirks mode flag.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | document | ASTNode.<Document> | Document node. |

TreeAdapter.setQuirksMode(document) ⇒ Boolean

Determines if document quirks mode flag is set.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | document | ASTNode.<Document> | Document node. |

TreeAdapter.detachNode(node)

Removes node from it's parent.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.insertText(parentNode, text)

Inserts text into node. If the last child of the node is the text node then provided text will be appended to the text node content. Otherwise, inserts new text node with the given text.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | parentNode | ASTNode | Node to insert text into. | | text | String | Text to insert. |

TreeAdapter.insertTextBefore(parentNode, text, referenceNode)

Inserts text into node before the referenced child node. If node before the referenced child node is the text node then provided text will be appended to the text node content. Otherwise, inserts new text node with the given text before the referenced child node.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | parentNode | ASTNode | Node to insert text into. | | text | String | Text to insert. | | referenceNode | ASTNode | Node to insert text before. |

TreeAdapter.adoptAttributes(recipientNode, attrs)

Copies attributes to the given node. Only those nodes which are not yet present in the node are copied.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | recipientNode | ASTNode | Node to copy attributes into. | | attrs | Array | Attributes to copy. |

TreeAdapter.getFirstChild(node) ⇒ ASTNode

Returns first child of the given node.

Kind: static method of TreeAdapter Returns: ASTNode - firstChild See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.getChildNodes(node) ⇒ Array

Returns array of the given node's children.

Kind: static method of TreeAdapter Returns: Array - children See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.getParentNode(node) ⇒ ASTNode

Returns given node's parent.

Kind: static method of TreeAdapter Returns: ASTNode - parent See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.getAttrList(node) ⇒ Array

Returns array of the given node's attributes in form of the name-value pair. Foreign attributes may contain namespace and prefix fields as well.

Kind: static method of TreeAdapter Returns: Array - attributes See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.getTagName(element) ⇒ String

Returns given element's tag name.

Kind: static method of TreeAdapter Returns: String - tagName See: default implementation.

| Param | Type | Description | | --- | --- | --- | | element | ASTNode.<Element> | Element. |

TreeAdapter.getNamespaceURI(element) ⇒ String

Returns given element's namespace.

Kind: static method of TreeAdapter Returns: String - namespaceURI See: default implementation.

| Param | Type | Description | | --- | --- | --- | | element | ASTNode.<Element> | Element. |

TreeAdapter.getTextNodeContent(textNode) ⇒ String

Returns given text node's content.

Kind: static method of TreeAdapter Returns: String - text See: default implementation.

| Param | Type | Description | | --- | --- | --- | | textNode | ASTNode.<Text> | Text node. |

TreeAdapter.getTextNodeContent(commentNode) ⇒ String

Returns given comment node's content.

Kind: static method of TreeAdapter Returns: String - commentText See: default implementation.

| Param | Type | Description | | --- | --- | --- | | commentNode | ASTNode.<Comment> | Comment node. |

TreeAdapter.getDocumentTypeNodeName(doctypeNode) ⇒ String

Returns given document type node's name.

Kind: static method of TreeAdapter Returns: String - name See: default implementation.

| Param | Type | Description | | --- | --- | --- | | doctypeNode | ASTNode.<DocumentType> | Document type node. |

TreeAdapter.getDocumentTypeNodePublicId(doctypeNode) ⇒ String

Returns given document type node's public identifier.

Kind: static method of TreeAdapter Returns: String - publicId See: default implementation.

| Param | Type | Description | | --- | --- | --- | | doctypeNode | ASTNode.<DocumentType> | Document type node. |

TreeAdapter.getDocumentTypeNodeSystemId(doctypeNode) ⇒ String

Returns given document type node's system identifier.

Kind: static method of TreeAdapter Returns: String - systemId See: default implementation.

| Param | Type | Description | | --- | --- | --- | | doctypeNode | ASTNode.<DocumentType> | Document type node. |

TreeAdapter.isTextNode(node) ⇒ Boolean

Determines if given node is a text node.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.isCommentNode(node) ⇒ Boolean

Determines if given node is a comment node.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.isDocumentTypeNode(node) ⇒ Boolean

Determines if given node is a document type node.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

TreeAdapter.isElementNode(node) ⇒ Boolean

Determines if given node is an element.

Kind: static method of TreeAdapter See: default implementation.

| Param | Type | Description | | --- | --- | --- | | node | ASTNode | Node. |

FAQ

Q: I want to work with my own document tree format. How can I achieve this?

You can create a custom tree adapter so parse5 can work with your own DOM-tree implementation. Then just pass it to the parser or serializer via option:

var parse5 = require('parse5');

var myTreeAdapter = {
   //Adapter methods...
};

var document = parse5.parse('<div></div>', { treeAdapter: myTreeAdapter });

var html = parse5.serialize(document, { treeAdapter: myTreeAdapter });

You can find description of the methods which should be exposed by tree adapter and links to their default implementation in the API reference.

Q: How can I use parse5 in the browser?

Just compile it with browserify and you're set.

Q: I'm parsing <img src="foo"> with the SAXParser and I expect selfClosing flag to be true for the <img> tag. But it's not. Is there something wrong with parser?

No. Self-closing tag is the tag that has / before the closing bracket. E.g: <br/>, <meta/>. In the provided example tag just doesn't have end tag. Self-closing tags and tags without end tags are differently treated by the parser: in case of self-closing tag parser will not lookup for the appropriate closing tag and expects element to not have any content. But if start tag is not self-closing parser will treat everything after it (with the few exceptions) as the element content. However, if the start tag is in the list of void elements parser expects corresponding element to not have content and behaves the same way as the if element is self-closing. So, semantically if element is the void element self-closing tags and tags without closing tags are equivalent, but it's not true for all other tags.

TL;DR: selfClosing is the part of the lexical information and will be set only if the tag in source code has / before the closing bracket.

Q: I have some weird output from the parser, seems like it's a bug.

More likely, it's not. There are a lot of weird edge cases in HTML5 parsing algorithm, e.g.:

<b>1<p>2</b>3</p>

will be parsed as

<b>1</b><p><b>2</b>3</p>

Just try it in the latest version of your browser before submitting the issue.

Version history

2.0.0

1.5.0

1.4.2

1.4.1

1.4.0

1.3.2

1.3.1

1.3.0

1.2.0

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.0.0

0.8.3

0.8.1

0.6.1

0.6.0

License

Copyright (c) 2013-2015 Ivan Nikulin (ifaaan@gmail.com, https://github.com/inikulin)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.



JohnCoene/chirp documentation built on May 25, 2021, 6:33 p.m.

R Package Documentation

rdrr.io home R language documentation Run R code online

Browse R Packages

CRAN packages Bioconductor packages R-Forge packages GitHub packages

We want your feedback!

Note that we can't provide technical support on individual packages. You should contact the package authors for that.
GitHub issue tracker
ian@mutexlabs.com
Personal blog

 
Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.

Close