Lexer
The result of compiling a lexer definition.
This abstract class represents a compiled lexer that can tokenize input. It is generated by the lexer macro and provides methods to access tokens and perform tokenization.
Type parameters
- Ctx
-
the global context type
Attributes
- Source
- Lexer.scala
- Graph
-
- Supertypes
-
trait Selectableclass Objecttrait Matchableclass Any
Members list
Type members
Types
The lexer's tokens by name, as a named tuple: what MyLexer.NAME selects.
The lexer's tokens by name, as a named tuple: what MyLexer.NAME selects.
Attributes
- Source
- Lexer.scala
A lexeme of this lexer, with its context fields: def show(lexeme: MyLexer.Lexeme).
A lexeme of this lexer, with its context fields: def show(lexeme: MyLexer.Lexeme).
Attributes
- Source
- Lexer.scala
The context fields every lexeme and error of this lexer carries, as a named tuple.
The context fields every lexeme and error of this lexer carries, as a named tuple.
Attributes
- Source
- Lexer.scala
An error of this lexer, with its context fields: def report(error: MyLexer.LexerError).
An error of this lexer, with its context fields: def report(error: MyLexer.LexerError).
Attributes
- Source
- Lexer.scala
An error of a parser run on this lexer's lexemes, with their context fields: def report(error: MyLexer.ParserError).
An error of a parser run on this lexer's lexemes, with their context fields: def report(error: MyLexer.ParserError).
Attributes
- Source
- Lexer.scala
Value members
Abstract methods
Backs token selection by name (myLexer.PLUS); not meant to be called directly.
Backs token selection by name (myLexer.PLUS); not meant to be called directly.
Attributes
- Source
- Lexer.scala
Concrete methods
Tokenizes the input character sequence.
Tokenizes the input character sequence.
Processes the input from start to finish, matching tokens and building a list of lexemes. Input that matches no token does not throw: it is reported as a LexerError, and the context's ErrorHandling decides whether tokenizing stops there or skips it and goes on. An exception thrown by a rule body, such as a failed require, propagates out of tokenize.
Value parameters
- input
-
the input to tokenize
Attributes
- Returns
-
the lexemes and the final lexer context, or the errors if any input did not match; with a recovering strategy the lexemes collected despite the errors are the failure's
recoveredvalue - Source
- Lexer.scala