Class Tokenizer

Constructors

constructor

Properties

active consumedAfterSnapshot currentAttr currentCharacterToken currentLocation currentToken entityDecoder entityStartPos handler inForeignNode inLoop lastStartTagName options paused preprocessor returnState state

Methods

_advanceBy _appendCharToCurrentCharacterToken _callState _consume _consumeSequenceIfMatch _createAttr _createCharacterToken _createCommentToken _createDoctypeToken _createEndTagToken _createStartTagToken _emitChars _emitCodePoint _emitCurrentCharacterToken _emitEOFToken _ensureHibernation _err _flushCodePointConsumedAsCharacterReference _isCharacterReferenceInAttribute _leaveAttrName _leaveAttrValue _runParsingLoop _startCharacterReference _stateAfterAttributeName _stateAfterAttributeValueQuoted _stateAfterDoctypeName _stateAfterDoctypePublicIdentifier _stateAfterDoctypePublicKeyword _stateAfterDoctypeSystemIdentifier _stateAfterDoctypeSystemKeyword _stateAmbiguousAmpersand _stateAttributeName _stateAttributeValueDoubleQuoted _stateAttributeValueSingleQuoted _stateAttributeValueUnquoted _stateBeforeAttributeName _stateBeforeAttributeValue _stateBeforeDoctypeName _stateBeforeDoctypePublicIdentifier _stateBeforeDoctypeSystemIdentifier _stateBetweenDoctypePublicAndSystemIdentifiers _stateBogusComment _stateBogusDoctype _stateCdataSection _stateCdataSectionBracket _stateCdataSectionEnd _stateCharacterReference _stateComment _stateCommentEnd _stateCommentEndBang _stateCommentEndDash _stateCommentLessThanSign _stateCommentLessThanSignBang _stateCommentLessThanSignBangDash _stateCommentLessThanSignBangDashDash _stateCommentStart _stateCommentStartDash _stateData _stateDoctype _stateDoctypeName _stateDoctypePublicIdentifierDoubleQuoted _stateDoctypePublicIdentifierSingleQuoted _stateDoctypeSystemIdentifierDoubleQuoted _stateDoctypeSystemIdentifierSingleQuoted _stateEndTagOpen _stateMarkupDeclarationOpen _statePlaintext _stateRawtext _stateRawtextEndTagName _stateRawtextEndTagOpen _stateRawtextLessThanSign _stateRcdata _stateRcdataEndTagName _stateRcdataEndTagOpen _stateRcdataLessThanSign _stateScriptData _stateScriptDataDoubleEscapeEnd _stateScriptDataDoubleEscapeStart _stateScriptDataDoubleEscaped _stateScriptDataDoubleEscapedDash _stateScriptDataDoubleEscapedDashDash _stateScriptDataDoubleEscapedLessThanSign _stateScriptDataEndTagName _stateScriptDataEndTagOpen _stateScriptDataEscapeStart _stateScriptDataEscapeStartDash _stateScriptDataEscaped _stateScriptDataEscapedDash _stateScriptDataEscapedDashDash _stateScriptDataEscapedEndTagName _stateScriptDataEscapedEndTagOpen _stateScriptDataEscapedLessThanSign _stateScriptDataLessThanSign _stateSelfClosingStartTag _stateTagName _stateTagOpen emitCurrentComment emitCurrentDoctype emitCurrentTagToken getCurrentLocation handleSpecialEndTag insertHtmlAtCurrentPos pause prepareToken resume write

Constructors

constructor

Properties

active

active: boolean = false

Protected consumedAfterSnapshot

consumedAfterSnapshot: number = -1

Protected currentAttr

currentAttr: Attribute = ...

Protected currentCharacterToken

currentCharacterToken: null | CharacterToken = null

Protected currentLocation

currentLocation: null | Location

Protected currentToken

currentToken: null | Token = null

Protected entityDecoder

entityDecoder: EntityDecoder

We use entities' EntityDecoder to parse character references.

All of the following states are handled by the EntityDecoder:

  • Named character reference state
  • Numeric character reference state
  • Hexademical character reference start state
  • Hexademical character reference state
  • Decimal character reference state
  • Numeric character reference end state

Protected entityStartPos

entityStartPos: number = 0

Protected handler

handler: TokenHandler

inForeignNode

inForeignNode: boolean = false

Indicates that the current adjusted node exists, is not an element in the HTML namespace, and that it is not an integration point for either MathML or HTML.

See

https://html.spec.whatwg.org/multipage/parsing.html#tree-construction

Protected inLoop

inLoop: boolean = false

Ensures that the parsing loop isn't run multiple times at once.

lastStartTagName

lastStartTagName: string = ''

Protected options

options: TokenizerOptions

Protected paused

paused: boolean = false

preprocessor

preprocessor: Preprocessor

Protected returnState

returnState: State = State.DATA

state

state: State = State.DATA

Methods

Protected _advanceBy

Protected _appendCharToCurrentCharacterToken

Protected _callState

Protected _consume

Protected _consumeSequenceIfMatch

  • _consumeSequenceIfMatch(pattern, caseSensitive): boolean

  • Parameters

    • pattern: string
    • caseSensitive: boolean

    Returns boolean

Protected _createAttr

  • _createAttr(attrNameFirstCh): void

  • Parameters

    • attrNameFirstCh: string

    Returns void

Protected _createCharacterToken

Protected _createCommentToken

Protected _createDoctypeToken

  • _createDoctypeToken(initialName): void

  • Parameters

    • initialName: null | string

    Returns void

Protected _createEndTagToken

Protected _createStartTagToken

Protected _emitChars

Protected _emitCodePoint

Protected _emitCurrentCharacterToken

Protected _emitEOFToken

Protected _ensureHibernation

Protected _err

Protected _flushCodePointConsumedAsCharacterReference

  • _flushCodePointConsumedAsCharacterReference(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _isCharacterReferenceInAttribute

Protected _leaveAttrName

Protected _leaveAttrValue

Protected _runParsingLoop

Protected _startCharacterReference

Protected _stateAfterAttributeName

Protected _stateAfterAttributeValueQuoted

Protected _stateAfterDoctypeName

Protected _stateAfterDoctypePublicIdentifier

Protected _stateAfterDoctypePublicKeyword

Protected _stateAfterDoctypeSystemIdentifier

Protected _stateAfterDoctypeSystemKeyword

Protected _stateAmbiguousAmpersand

Protected _stateAttributeName

Protected _stateAttributeValueDoubleQuoted

Protected _stateAttributeValueSingleQuoted

Protected _stateAttributeValueUnquoted

Protected _stateBeforeAttributeName

Protected _stateBeforeAttributeValue

Protected _stateBeforeDoctypeName

Protected _stateBeforeDoctypePublicIdentifier

Protected _stateBeforeDoctypeSystemIdentifier

Protected _stateBetweenDoctypePublicAndSystemIdentifiers

  • _stateBetweenDoctypePublicAndSystemIdentifiers(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _stateBogusComment

Protected _stateBogusDoctype

Protected _stateCdataSection

Protected _stateCdataSectionBracket

Protected _stateCdataSectionEnd

Protected _stateCharacterReference

Protected _stateComment

Protected _stateCommentEnd

Protected _stateCommentEndBang

Protected _stateCommentEndDash

Protected _stateCommentLessThanSign

Protected _stateCommentLessThanSignBang

Protected _stateCommentLessThanSignBangDash

Protected _stateCommentLessThanSignBangDashDash

Protected _stateCommentStart

Protected _stateCommentStartDash

Protected _stateData

Protected _stateDoctype

Protected _stateDoctypeName

Protected _stateDoctypePublicIdentifierDoubleQuoted

  • _stateDoctypePublicIdentifierDoubleQuoted(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _stateDoctypePublicIdentifierSingleQuoted

  • _stateDoctypePublicIdentifierSingleQuoted(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _stateDoctypeSystemIdentifierDoubleQuoted

  • _stateDoctypeSystemIdentifierDoubleQuoted(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _stateDoctypeSystemIdentifierSingleQuoted

  • _stateDoctypeSystemIdentifierSingleQuoted(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _stateEndTagOpen

Protected _stateMarkupDeclarationOpen

Protected _statePlaintext

Protected _stateRawtext

Protected _stateRawtextEndTagName

Protected _stateRawtextEndTagOpen

Protected _stateRawtextLessThanSign

Protected _stateRcdata

Protected _stateRcdataEndTagName

Protected _stateRcdataEndTagOpen

Protected _stateRcdataLessThanSign

Protected _stateScriptData

Protected _stateScriptDataDoubleEscapeEnd

Protected _stateScriptDataDoubleEscapeStart

Protected _stateScriptDataDoubleEscaped

Protected _stateScriptDataDoubleEscapedDash

Protected _stateScriptDataDoubleEscapedDashDash

Protected _stateScriptDataDoubleEscapedLessThanSign

  • _stateScriptDataDoubleEscapedLessThanSign(cp): void

  • Parameters

    • cp: number

    Returns void

Protected _stateScriptDataEndTagName

Protected _stateScriptDataEndTagOpen

Protected _stateScriptDataEscapeStart

Protected _stateScriptDataEscapeStartDash

Protected _stateScriptDataEscaped

Protected _stateScriptDataEscapedDash

Protected _stateScriptDataEscapedDashDash

Protected _stateScriptDataEscapedEndTagName

Protected _stateScriptDataEscapedEndTagOpen

Protected _stateScriptDataEscapedLessThanSign

Protected _stateScriptDataLessThanSign

Protected _stateSelfClosingStartTag

Protected _stateTagName

Protected _stateTagOpen

Protected emitCurrentComment

Protected emitCurrentDoctype

Protected emitCurrentTagToken

Protected getCurrentLocation

Protected handleSpecialEndTag

insertHtmlAtCurrentPos

pause

Protected prepareToken

resume

  • resume(writeCallback?): void

  • Parameters

    • Optional writeCallback: (() => void)
        • (): void

        • Returns void

    Returns void

write

  • write(chunk, isLastChunk, writeCallback?): void

  • Parameters

    • chunk: string
    • isLastChunk: boolean
    • Optional writeCallback: (() => void)
        • (): void

        • Returns void

    Returns void

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc