SnuggleTeX 1.3.0 · uk.ac.ed.ph.snuggletex.internal.LaTeXTokeniser

Modifier and Type	Class and Description
pack-priv static class	LaTeXTokeniser.ArgumentDefinitionResult Trivial result Object for `LaTeXTokeniser#advanceOverUserDefinedCommandOrEnvironmentArgumentDefinition(String, ArgumentDefinitionResult)`
pack-priv static class	LaTeXTokeniser.BuiltinCommandOrEnvironmentArgumentSearchResult Trivial "struct" Object to hold the results of searching for command and/or environment arguments.
public static class	LaTeXTokeniser.ModeState Trivial "struct" encapsulating information about the state of a tokenisation mode.
public static class	LaTeXTokeniser.PatternTerminator This implementation of `Terminator` represents terminators specified as regular expressions.
public static class	LaTeXTokeniser.StringTerminator This implementation of `Terminator` is used to handle fixed `String`s as terminators.
public static interface	LaTeXTokeniser.Terminator Represents the "terminator" characters that signify the end of a parsing mode.
public static enum	LaTeXTokeniser.TokenisationMode Enumeration of the various "tokenisation modes" we will be running in. This is only really used for documentation during debugging!
pack-priv static class	LaTeXTokeniser.UserDefinedCommandOrEnvironmentArgumentSearchResult Trivial "struct" Object to hold the results of searching for user-defined command and/or environment arguments, which are initially treated as unparsed but balanced `WorkingDocument`s.

Modifier and Type	Field and Description
private LaTeXTokeniser.ModeState	currentModeState Current parsing "mode"
private final ArrayListStack<LaTeXTokeniser.ModeState>	modeStack Stack of parsing modes, with the last entry being equal to `currentModeState`
private final NumberMatcher	numberMatcher `NumberMatcher` being used
private final ArrayListStack<String>	openEnvironmentStack Stack of open environments
private int	position Current position within `workingDocument`.
public static final Set<String>	reservedCommands Set of reserved commands.
private final SessionContext	sessionContext Provides access to the current `SessionContext`.
private int	startTokenIndex Index of the start of the current token being parsed.
private static final String	UDE_POST_BEGIN Name of internal command that gets temporarily appended after the begin clauses of a user-defined environment has been substituted in order to make sure the `openEnvironmentStack` is kept in the correct order.
private WorkingDocument	workingDocument Current "working document", taking into account command/environment substitutions.

Access	Constructor and Description
public	LaTeXTokeniser(final SessionContext sessionContext)

Method Summary

Modifier and Type	Method and Description
private String	advanceOverBracesAndEnvironmentName() Reads text of the form '{envName}' from the current position onwards, skipping any leading comments/whitespace and advancing the current position to just after the final '}'.
private ErrorToken	Returns: ErrorToken if parsing failed, null otherwise. advanceOverBuiltinCommandOrEnvironmentArguments(final CommandOrEnvironment commandOrEnvironment, final LaTeXTokeniser.BuiltinCommandOrEnvironmentArgumentSearchResult blank result Object that will be filled in by this method. result) This helper reads in the optional and required arguments for a Command or Environment, starting at the current position.
private ErrorToken	advanceOverUserDefinedCommandOrEnvironmentArgumentDefinition(final String commandOrEnvironmentName, final LaTeXTokeniser.ArgumentDefinitionResult result) Reads in the argument specification for a new command or environment.
private ErrorToken	Returns: ErrorToken if parsing failed, null otherwise. advanceOverUserDefinedCommandOrEnvironmentArguments(final CommandOrEnvironment commandOrEnvironment, LaTeXTokeniser.UserDefinedCommandOrEnvironmentArgumentSearchResult blank result Object that will be filled in by this method. result) This helper reads in the optional and required arguments for a Command or Environment, starting at the current position.
private ErrorToken	checkDefinitionArguments(final FrozenSlice definitionSlice, final String commandOrEnvironmentName, final LaTeXTokeniser.ArgumentDefinitionResult argumentDefinitionResult, final ErrorCode errorCode) This helper checks the definition of a user-defined command or environment to make sure that any argument references are in line with what is defined.
private ErrorToken	createError(final ErrorCode errorCode, final int errorStartIndex, final int errorEndIndex, final Object... arguments)
private int	findEndCurlyBrackets(final int openBraceIndex) Returns the index of the next balanced '}', or -1 if no balance was found.
private int	findEndSquareBrackets(final int openSquareBracketIndex) Returns the index of the next ']', handling any balanced braces appropriately.
private FlowToken	finishBeginBuiltinEnvironment(final BuiltinEnvironment environment being read in environment) Finishes off reading a built-in environment.
private FlowToken	finishBeginEnvironment() Handles `\\begin...`
private FlowToken	finishBeginUserDefinedEnvironment(final UserDefinedEnvironment environment) Finishes the handling of the start of a user-defined environment.
private FlowToken	finishBuiltinCommand(final BuiltinCommand command) Finishes reading in a `BuiltinCommand`, catering for the different types of those commands.
private FlowToken	finishCombiningCommand(final BuiltinCommand command) Deals with pulling in the next token after something like `\not` PRE-CONDITION: position will point to the character immediately after `\commandName`.
private FlowToken	finishCommand(final String name of the command being read in commandName) Finishes reading a Command, which will either be a `BuiltinCommand` or a `UserDefinedCommand`.
private FlowToken	finishCommandDefinition(final BuiltinCommand definitionCommand) Finishes reading the definition of a user-defined command specified using `\\newcommand` or similar.
private FlowToken	finishComplexCommand(final BuiltinCommand command) Finishes the process or reading in a "complex" command, by searching for any required and optional arguments.
private FlowToken	finishEndEnvironment() Handles `\\end...`
private FlowToken	finishEndUserDefinedEnvironment(final UserDefinedEnvironment environment) Finishes the handling of the end of a user-defined environment.
private FlowToken	finishEnvironmentDefinition(final BuiltinCommand definitionCommand) Finishes reading the definition of a user-defined environment specified using `\\newenvironment` or similar.
private FlowToken	finishSimpleCommand(final BuiltinCommand command) Finishes the reading of a simple command.
private FlowToken	finishUserDefinedCommand(final UserDefinedCommand command) Finishes the process of reading in and evaluating a `UserDefinedCommand`.
private FlowToken	finishVerbToken(BuiltinCommand verbCommand) This is called once it has become clear that the next token is `\verb` or `\verb*`.
private FlowToken	handleUserDefinedEnvironmentControl() Handles the special internal "begin user-defined environment finished" token that is substituted into the input when processing the start of a user-defined environment.
private ErrorToken	makeSubstitutionAndRewind(final int startIndex, final int endIndex, final CharSequence replacement) Makes a substitution within the working document, replacing the position of the `WorkingDocument` from startIndex to endIndex with the given replacement `CharSequence`.
private BraceContainerToken	readBraceRegion() Reads the content of a region explicitly delimited inside `{....}`.
private String	Returns: command name String, which will contain at least one character, or null if there were no further characters in the Slice. readCommandOrEnvironmentName(final int index to start reading from. startCommandNameIndex) Helper to read in the command or environment name starting at the given index, obeying the esoteric rules of naming.
private FlowToken	readCommandOrEnvironmentOrVerb() Reads in a command, `\\verb(*)` or environment control token.
private FlowToken	readDollarMath() Reads in a Math environment opened with `$` or `$$`.
private FlowToken	readNextMathNumberOrSymbol()
private SimpleToken	readNextSimpleTextParaMode() Reads either the next text lump or "new paragraph" tokens, until the start of something else.
private FlowToken	Returns: next token read in, or null if the input ended or we hit the terminator for the current `ModeState`. (You can test whether we hit a terminator by checking `ModeState#foundTerminator`.) readNextToken() Reads the next token before the current terminator, returning null on end of input or if the terminator is discovered.
private FlowToken	readNextTokenMathMode()
private FlowToken	readNextTokenTextMode()
private FlowToken	readNextTokenVerbatimMode() This reads the next token in `LaTeXMode#VERBATIM` Mode, which is "simply" a case of pulling in everything until the next match of the required terminator.
private FlowToken	readSlashToken() Reads in a token starting with a `\\`
public void	reset() Resets the parsing state of this tokeniser.
private void	skipOverComment() Advances the current position past a comment, indicated by a '%' symbol.
private void	skipOverComments() Advances the current position past any comments and/or whitespace, including newlines.
private void	skipOverCommentsAndWhitespace() Advances the current position past any comments and/or whitespace, including newlines.
private void	skipOverTrailingWhitespace() Advances the current position past any whitespace, excluding newlines.
private String	substituteArguments(final FrozenSlice slice, final UserDefinedCommandOrEnvironment commandOrEnvironment, final LaTeXTokeniser.UserDefinedCommandOrEnvironmentArgumentSearchResult argumentSearchResult) This helper substitutes the arguments provided for a user-defined command/environment into the content of the given `FrozenSlice`, handling correctly escaped arguments such as `\#1`.
public RootToken	tokenise(final SnuggleInputReader reader) Tokenises the input specified by the given `SnuggleInputReader`, returning an `ArgumentContainerToken` containing the roots of the parsed token tree.
private LaTeXTokeniser.ModeState	tokeniseInNewState(final LaTeXTokeniser.TokenisationMode tokenisationMode, final LaTeXTokeniser.Terminator terminator, final LaTeXMode latexMode) Tokenises the `workingDocument` from the current `position` in the given `LaTeXMode` using the newly provided `TokenisationMode` and terminator.
private SimpleToken	Returns: SimpleToken representing the number, or null if input wasn't a number. tryReadMathNumber() Attempts to read in a (positive) number at the current position, returning null if the input doesn't look like a number.

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

public final Class LaTeXTokeniser

Nested and Inner Type Summary

Field Summary

Constructor Summary

Method Summary