Lexer

public protocol Lexer : GrammarComponent

A Lexer manages the custom parsing of terminals.

Given a position in some input, and a given terminal with given associated input parameter, the lexer returns a set of tokens.

Note

Terminals can not only be parsed via the lexer, but also via rules, therefore enabling scannerless parsing. In those cases the returned set of tokens will typically be empty, but does not have to be, thus making it possible to mix scannerless and scannerful parsing of terminals. Nevertheless, even in a fully scannerless parser, there needs to be one terminal parsed via the lexer, representing a general character.

Seealso

Rule

  • The type of character inputs processed by the lexer have.

    Declaration

    Swift

    associatedtype Char

  • Given a position in input, and a given terminal with given associated input parameter, the lexer returns a set of parsed tokens.

    Declaration

    Swift

    func parse(input: Input<Char>, position: Int, key: TerminalKey<Param>) -> Set<Token<Param, Result>>

    Parameters

    input

    The input providing the characters to parse.
    position

    The position in the input from where to start parsing.
    key

    The terminal key distinguishing the terminal with associated input parameter that is being parsed.

    Return Value

    A set of tokens, each token representing a successful parse. An empty set is returned in case of a failed parse.