alpaca.internal.lexer
Members list
Type members
Experimental classlikes
Trait representing error handling strategies for a lexer context.
Trait representing error handling strategies for a lexer context.
This trait allows the specification of error handling strategies for various lexer contexts by transforming a given lexer context to an instance of ErrorHandling.Strategy. The strategies determine how the lexer should proceed when it encounters an error, such as invalid tokens or characters during parsing.
Error handling is essential for customizing lexer behavior in response to specific scenarios, including ignoring or stopping on errors, or throwing specific exceptions.
Users must define an implicit instance of this trait to provide the error handling behavior for a custom lexer context.
Type parameters
- Ctx
-
The lexer context that this error handling strategy applies to. Must be a subtype of
LexerCtx.
Attributes
- Companion
- object
- Experimental
- true
- Source
- ErrorHandling.scala
- Supertypes
Provides mechanisms for defining error-handling strategies when processing input. This object contains a sealed enumeration Strategy that specifies the different ways errors can be addressed during tokenization or parsing workflows.
Provides mechanisms for defining error-handling strategies when processing input. This object contains a sealed enumeration Strategy that specifies the different ways errors can be addressed during tokenization or parsing workflows.
Attributes
- Companion
- trait
- Experimental
- true
- Source
- ErrorHandling.scala
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
ErrorHandling.type
A lazy character sequence that reads from a Reader on demand.
A lazy character sequence that reads from a Reader on demand.
This class is useful for tokenizing large files without loading them entirely into memory. It buffers characters as they are accessed and can efficiently skip over processed characters.
Value parameters
- reader
-
the underlying Reader to read from
- size
-
the total size of the input (if known)
Attributes
- Companion
- object
- Experimental
- true
- Source
- LazyReader.scala
- Supertypes
-
trait Closeabletrait AutoCloseabletrait CharSequenceclass Objecttrait Matchableclass AnyShow all
Factory methods for creating LazyReader instances.
Factory methods for creating LazyReader instances.
Attributes
- Companion
- class
- Experimental
- true
- Source
- LazyReader.scala
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
LazyReader.type
A trait for contexts that track line numbers.
A trait for contexts that track line numbers.
This trait adds line number tracking to a lexer context. The line number is incremented each time a newline character is matched.
Attributes
- Companion
- object
- Experimental
- true
- Source
- LineTracking.scala
- Supertypes
- Known subtypes
-
class Default
Attributes
- Companion
- trait
- Experimental
- true
- Source
- LineTracking.scala
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
LineTracking.type
Attributes
- Companion
- trait
- Experimental
- true
- Source
- OnTokenMatch.scala
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
OnTokenMatch.type
A hook for updating context between lexing stages.
A hook for updating context between lexing stages.
This trait defines a function that is called after each token match to update the global context. It can be used to track line numbers, column positions, or other custom state.
Type parameters
- Ctx
-
the global context type
Attributes
- Companion
- object
- Experimental
- true
- Source
- OnTokenMatch.scala
- Supertypes
A trait for contexts that track character positions.
A trait for contexts that track character positions.
This trait adds character position tracking to a lexer context. The position tracks the column number within the current line and is reset to 1 when a newline is encountered.
Attributes
- Companion
- object
- Experimental
- true
- Source
- PositionTracking.scala
- Supertypes
- Known subtypes
-
class Default
Attributes
- Companion
- trait
- Experimental
- true
- Source
- PositionTracking.scala
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
PositionTracking.type
Base trait for all token types.
Base trait for all token types.
A token represents a lexical unit matched by the lexer. It contains information about the token's name, pattern, and how to manipulate the lexer context when matched.
Type parameters
- Ctx
-
the global context type
- Name
-
the token name type
- Value
-
the value type extracted from the matched text
Attributes
- Experimental
- true
- Source
- Token.scala
- Supertypes
-
class Objecttrait Matchableclass Any
The result of compiling a lexer definition.
The result of compiling a lexer definition.
This abstract class represents a compiled lexer that can tokenize input. It is generated by the lexer macro and provides methods to access tokens and perform tokenization.
Type parameters
- Ctx
-
the global context type
Attributes
- Experimental
- true
- Source
- Tokenization.scala
- Supertypes
-
trait Selectableclass Objecttrait Matchableclass Any
Value members
Experimental methods
Attributes
- Experimental
- true
- Source
- Lexer.scala