Top Description Inners Fields Constructors Methods
org.antlr.v4.tool

public Class GrammarParserInterpreter

extends ParserInterpreter
Class Inheritance
Imports
org.antlr.v4.runtime.BailErrorStrategy, .DefaultErrorStrategy, .InputMismatchException, .InterpreterRuleContext, .Parser, .ParserInterpreter, .ParserRuleContext, .RecognitionException, .Token, .TokenStream, .Vocabulary, org.antlr.v4.runtime.atn.ATN, .ATNDeserializer, .ATNSerializer, .ATNState, .DecisionState, .PredictionMode, .RuleStartState, .StarLoopEntryState, org.antlr.v4.runtime.misc.IntegerList, .Interval, org.antlr.v4.runtime.tree.Trees, java.lang.reflect.Constructor, java.util.ArrayList, .Arrays, .BitSet, .Collection, .List

A heavier weight ParserInterpreter that creates parse trees that track alternative numbers for subtree roots.
Since
4.5.1

Nested and Inner Type Summary

Modifier and TypeClass and Description
public static class
GrammarParserInterpreter.BailButConsumeErrorStrategy

We want to stop and track the first error but we cannot bail out like BailErrorStrategy as consume() constructs trees.

Field Summary

Modifier and TypeField and Description
protected BitSet
protected final Grammar
g

The grammar associated with this interpreter.

protected int[][]
Inherited from org.antlr.v4.runtime.ParserInterpreter:
_parentContextStackatndecisionToDFAgrammarFileNameoverrideDecisionoverrideDecisionAltoverrideDecisionInputIndexoverrideDecisionReachedoverrideDecisionRootrootContextruleNamessharedContextCachetokenNames

Constructor Summary

AccessConstructor and Description
public
GrammarParserInterpreter(Grammar g, String grammarFileName, Vocabulary vocabulary, Collection<String> ruleNames, ATN atn, TokenStream input)

public

Method Summary

Modifier and TypeMethod and Description
protected InterpreterRuleContext
createInterpreterRuleContext(ParserRuleContext parent, int invokingStateNumber, int ruleIndex)

Overrides org.antlr.v4.runtime.ParserInterpreter.createInterpreterRuleContext.

Provide simple "factory" for InterpreterRuleContext's.
public static ParserInterpreter
deriveTempParserInterpreter(Grammar g, Parser originalParser, TokenStream tokens)

Derive a new parser from an old one that has knowledge of the grammar.

public BitSet
findOuterMostDecisionStates()

identify the ATN states where we need to set the outer alt number.

public static List<ParserRuleContext>

Returns:

The list of all possible interpretations of the input for the decision in ambiguityInfo. The actual interpretation chosen by the parser is always given first because this method retests the input in alternative order and ANTLR always resolves ambiguities by choosing the first alternative that matches the input. The subtree returned
getAllPossibleParseTrees
(Grammar
From which grammar should we drive alternative numbers and alternative labels.
g
,
Parser
The parser used to create ambiguityInfo; it is not modified by this routine and can be either a generated or interpreted parser. It's token stream *is* reset/seek()'d.
originalParser
,
TokenStream
A stream of tokens to use with the temporary parser. This will often be just the token stream within the original parser but here it is for flexibility.
tokens
,
int
Which decision to try different alternatives for.
decision
,
BitSet
The set of alternatives to try while re-parsing.
alts
,
int
The index of the first token of the ambiguous input or other input of interest.
startIndex
,
int
The index of the last token of the ambiguous input. The start and stop indexes are used primarily to identify how much of the resulting parse tree to return.
stopIndex
,
int
The start rule for the entire grammar, not the ambiguous decision. We re-parse the entire input and so we need the original start rule.
startRuleIndex
)

Given an ambiguous parse information, return the list of ambiguous parse trees.

public static List<ParserRuleContext>
getLookaheadParseTrees(Grammar g, ParserInterpreter originalParser, TokenStream tokens, int startRuleIndex, int decision, int startIndex, int stopIndex)

Return a list of parse trees, one for each alternative in a decision given the same input.

public void
reset()

Overrides org.antlr.v4.runtime.ParserInterpreter.reset.

reset the parser's state
protected int
visitDecisionState(DecisionState p)

Overrides org.antlr.v4.runtime.ParserInterpreter.visitDecisionState.

Override this method so that we can record which alternative was taken at each decision point.
Inherited from org.antlr.v4.runtime.ParserInterpreter:
addDecisionOverrideenterRecursionRulegetATNgetATNStategetGrammarFileNamegetOverrideDecisionRootgetRootContextgetRuleNamesgetTokenNamesgetVocabularyparserecoverrecoverInlinevisitRuleStopStatevisitState

Field Detail

decisionStatesThatSetOuterAltNumInContextback to summary
protected BitSet decisionStatesThatSetOuterAltNumInContext
gback to summary
protected final Grammar g

The grammar associated with this interpreter. Unlike the ParserInterpreter from the standard distribution, this can reference Grammar, which is in the tools area not purely runtime.

stateToAltsMapback to summary
protected int[][] stateToAltsMap

Cache LeftRecursiveRule#getPrimaryAlts() and LeftRecursiveRule#getRecursiveOpAlts() for states in decisionStatesThatSetOuterAltNumInContext. It only caches decisions in left-recursive rules.

Constructor Detail

GrammarParserInterpreterback to summary
public GrammarParserInterpreter(Grammar g, String grammarFileName, Vocabulary vocabulary, Collection<String> ruleNames, ATN atn, TokenStream input)
GrammarParserInterpreterback to summary
public GrammarParserInterpreter(Grammar g, ATN atn, TokenStream input)

Method Detail

createInterpreterRuleContextback to summary
protected InterpreterRuleContext createInterpreterRuleContext(ParserRuleContext parent, int invokingStateNumber, int ruleIndex)

Overrides org.antlr.v4.runtime.ParserInterpreter.createInterpreterRuleContext.

Doc from org.antlr.v4.runtime.ParserInterpreter.createInterpreterRuleContext.

Provide simple "factory" for InterpreterRuleContext's.

Annotations
@Override
deriveTempParserInterpreterback to summary
public static ParserInterpreter deriveTempParserInterpreter(Grammar g, Parser originalParser, TokenStream tokens)

Derive a new parser from an old one that has knowledge of the grammar. The Grammar object is used to correctly compute outer alternative numbers for parse tree nodes. A parser of the same type is created for subclasses of ParserInterpreter.

findOuterMostDecisionStatesback to summary
public BitSet findOuterMostDecisionStates()

identify the ATN states where we need to set the outer alt number. For regular rules, that's the block at the target to rule start state. For left-recursive rules, we track the primary block, which looks just like a regular rule's outer block, and the star loop block (always there even if 1 alt).

getAllPossibleParseTreesback to summary
public static List<ParserRuleContext> getAllPossibleParseTrees(Grammar g, Parser originalParser, TokenStream tokens, int decision, BitSet alts, int startIndex, int stopIndex, int startRuleIndex) throws RecognitionException

Given an ambiguous parse information, return the list of ambiguous parse trees. An ambiguity occurs when a specific token sequence can be recognized in more than one way by the grammar. These ambiguities are detected only at decision points. The list of trees includes the actual interpretation (that for the minimum alternative number) and all ambiguous alternatives. The actual interpretation is always first. This method reuses the same physical input token stream used to detect the ambiguity by the original parser in the first place. This method resets/seeks within but does not alter originalParser. The trees are rooted at the node whose start..stop token indices include the start and stop indices of this ambiguity event. That is, the trees returned will always include the complete ambiguous subphrase identified by the ambiguity event. The subtrees returned will also always contain the node associated with the overridden decision. Be aware that this method does NOT notify error or parse listeners as it would trigger duplicate or otherwise unwanted events. This uses a temporary ParserATNSimulator and a ParserInterpreter so we don't mess up any statistics, event lists, etc... The parse tree constructed while identifying/making ambiguityInfo is not affected by this method as it creates a new parser interp to get the ambiguous interpretations. Nodes in the returned ambig trees are independent of the original parse tree (constructed while identifying/creating ambiguityInfo).

Parameters
g:Grammar

From which grammar should we drive alternative numbers and alternative labels.

originalParser:Parser

The parser used to create ambiguityInfo; it is not modified by this routine and can be either a generated or interpreted parser. It's token stream *is* reset/seek()'d.

tokens:TokenStream

A stream of tokens to use with the temporary parser. This will often be just the token stream within the original parser but here it is for flexibility.

decision:int

Which decision to try different alternatives for.

alts:BitSet

The set of alternatives to try while re-parsing.

startIndex:int

The index of the first token of the ambiguous input or other input of interest.

stopIndex:int

The index of the last token of the ambiguous input. The start and stop indexes are used primarily to identify how much of the resulting parse tree to return.

startRuleIndex:int

The start rule for the entire grammar, not the ambiguous decision. We re-parse the entire input and so we need the original start rule.

Returns:List<ParserRuleContext>

The list of all possible interpretations of the input for the decision in ambiguityInfo. The actual interpretation chosen by the parser is always given first because this method retests the input in alternative order and ANTLR always resolves ambiguities by choosing the first alternative that matches the input. The subtree returned

Exceptions
RecognitionException:
Throws upon syntax error while matching ambig input.
Since
4.5.1
getLookaheadParseTreesback to summary
public static List<ParserRuleContext> getLookaheadParseTrees(Grammar g, ParserInterpreter originalParser, TokenStream tokens, int startRuleIndex, int decision, int startIndex, int stopIndex)

Return a list of parse trees, one for each alternative in a decision given the same input. Very similar to getAllPossibleParseTrees except that it re-parses the input for every alternative in a decision, not just the ambiguous ones (there is no alts parameter here). This method also tries to reduce the size of the parse trees by stripping away children of the tree that are completely out of range of startIndex..stopIndex. Also, because errors are expected, we use a specialized error handler that more or less bails out but that also consumes the first erroneous token at least. This ensures that an error node will be in the parse tree for display.

Notes

// we must parse the entire input now with decision overrides // we cannot parse a subset because it could be that a decision // above our decision of interest needs to read way past // lookaheadInfo.stopIndex. It seems like there is no escaping // the use of a full and complete token stream if we are // resetting to token index 0 and re-parsing from the start symbol. // It's not easy to restart parsing somewhere in the middle like a // continuation because our call stack does not match the // tree stack because of left recursive rule rewriting. grrrr!

Since
4.5.1
resetback to summary
public void reset()

Overrides org.antlr.v4.runtime.ParserInterpreter.reset.

Doc from org.antlr.v4.runtime.Parser.reset.

reset the parser's state

Annotations
@Override
visitDecisionStateback to summary
protected int visitDecisionState(DecisionState p)

Overrides org.antlr.v4.runtime.ParserInterpreter.visitDecisionState.

Override this method so that we can record which alternative was taken at each decision point. For non-left recursive rules, it's simple. Set decisionStatesThatSetOuterAltNumInContext indicates which decision states should set the outer alternative number.

Left recursive rules are much more complicated to deal with: there is typically a decision for the primary alternatives and a decision to choose between the recursive operator alternatives. For example, the following left recursive rule has two primary and 2 recursive alternatives.

e : e '*' e | '-' INT | e '+' e | ID ;

ANTLR rewrites that rule to be

e[int precedence] : ('-' INT | ID) ( {...}? '*' e[5] | {...}? '+' e[3] )* ;

So, there are two decisions associated with picking the outermost alt. This complicates our tracking significantly. The outermost alternative number is a function of the decision (ATN state) within a left recursive rule and the predicted alternative coming back from adaptivePredict(). We use stateToAltsMap as a cache to avoid expensive calls to getRecursiveOpAlts().

Annotations
@Override
org.antlr.v4.tool back to summary

public Class GrammarParserInterpreter.BailButConsumeErrorStrategy

extends DefaultErrorStrategy
Class Inheritance

We want to stop and track the first error but we cannot bail out like BailErrorStrategy as consume() constructs trees. We make sure to create an error node during recovery with this strategy. We consume() 1 token during the "bail out of rule" mechanism in recover() and let it fall out of the rule to finish constructing trees. For recovery in line, we throw InputMismatchException to engage recover().

Field Summary

Modifier and TypeField and Description
public int
Inherited from org.antlr.v4.runtime.DefaultErrorStrategy:
errorRecoveryModelastErrorIndexlastErrorStatesnextTokensContextnextTokensState

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
public void
recover(Parser
the parser instance
recognizer
,
RecognitionException
the recognition exception to recover from
e
)

Overrides org.antlr.v4.runtime.DefaultErrorStrategy.recover.

Implements org.antlr.v4.runtime.ANTLRErrorStrategy.recover.

This method is called to recover from exception e.
public Token
recoverInline(Parser
the parser instance
recognizer
)

Overrides org.antlr.v4.runtime.DefaultErrorStrategy.recoverInline.

Implements org.antlr.v4.runtime.ANTLRErrorStrategy.recoverInline.

This method is called when an unexpected symbol is encountered during an inline match operation, such as Parser#match.
public void
sync(Parser
the parser instance
recognizer
)

Overrides org.antlr.v4.runtime.DefaultErrorStrategy.sync.

Implements org.antlr.v4.runtime.ANTLRErrorStrategy.sync.

The default implementation of ANTLRErrorStrategy#sync makes sure that the current lookahead symbol is consistent with what were expecting at this point in the ATN.
Inherited from org.antlr.v4.runtime.DefaultErrorStrategy:
beginErrorConditionconsumeUntilendErrorConditionescapeWSAndQuotegetErrorRecoverySetgetExpectedTokensgetMissingSymbolgetSymbolTextgetSymbolTypegetTokenErrorDisplayinErrorRecoveryModereportErrorreportFailedPredicatereportInputMismatchreportMatchreportMissingTokenreportNoViableAlternativereportUnwantedTokenresetsingleTokenDeletionsingleTokenInsertion

Field Detail

firstErrorTokenIndexback to summary
public int firstErrorTokenIndex

Constructor Detail

BailButConsumeErrorStrategyback to summary
public BailButConsumeErrorStrategy()

Method Detail

recoverback to summary
public void recover(Parser recognizer, RecognitionException e)

Overrides org.antlr.v4.runtime.DefaultErrorStrategy.recover.

Implements org.antlr.v4.runtime.ANTLRErrorStrategy.recover.

Doc from org.antlr.v4.runtime.DefaultErrorStrategy.recover.

This method is called to recover from exception e. This method is called after reportError by the default exception handler generated for a rule method.

The default implementation resynchronizes the parser by consuming tokens until we find one in the resynchronization set--loosely the set of tokens that can follow the current rule.

Parameters
recognizer:Parser

Doc from org.antlr.v4.runtime.ANTLRErrorStrategy.recover.

the parser instance

e:RecognitionException

Doc from org.antlr.v4.runtime.ANTLRErrorStrategy.recover.

the recognition exception to recover from

Annotations
@Override
recoverInlineback to summary
public Token recoverInline(Parser recognizer) throws RecognitionException

Overrides org.antlr.v4.runtime.DefaultErrorStrategy.recoverInline.

Implements org.antlr.v4.runtime.ANTLRErrorStrategy.recoverInline.

Doc from org.antlr.v4.runtime.DefaultErrorStrategy.recoverInline.

This method is called when an unexpected symbol is encountered during an inline match operation, such as Parser#match. If the error strategy successfully recovers from the match failure, this method returns the Token instance which should be treated as the successful result of the match.

This method handles the consumption of any tokens - the caller should not call Parser#consume after a successful recovery.

Note that the calling code will not report an error if this method returns successfully. The error strategy implementation is responsible for calling Parser#notifyErrorListeners as appropriate.

The default implementation attempts to recover from the mismatched input by using single token insertion and deletion as described below. If the recovery attempt fails, this method throws an InputMismatchException.

EXTRA TOKEN (single token deletion)

LA(1) is not what we are looking for. If LA(2) has the right token, however, then assume LA(1) is some extra spurious token and delete it. Then consume and return the next token (which was the LA(2) token) as the successful result of the match operation.

This recovery strategy is implemented by singleTokenDeletion.

MISSING TOKEN (single token insertion)

If current token (at LA(1)) is consistent with what could come after the expected LA(1) token, then assume the token is missing and use the parser's TokenFactory to create it on the fly. The "insertion" is performed by returning the created token as the successful result of the match operation.

This recovery strategy is implemented by singleTokenInsertion.

EXAMPLE

For example, Input i=(3; is clearly missing the ')'. When the parser returns from the nested call to expr, it will have call chain:

stat → expr → atom
and it will be trying to match the ')' at this point in the derivation:
=> ID '=' '(' INT ')' ('+' atom)* ';'
                   ^
The attempt to match ')' will fail when it sees ';' and call recoverInline. To recover, it sees that LA(1)==';' is in the set of tokens that can follow the ')' token reference in rule atom. It can assume that you forgot the ')'.
Parameters
recognizer:Parser

Doc from org.antlr.v4.runtime.ANTLRErrorStrategy.recoverInline.

the parser instance

Annotations
@Override
Exceptions
RecognitionException:

Doc from org.antlr.v4.runtime.ANTLRErrorStrategy.recoverInline.

if the error strategy was not able to recover from the unexpected input symbol

syncback to summary
public void sync(Parser recognizer)

Overrides org.antlr.v4.runtime.DefaultErrorStrategy.sync.

Implements org.antlr.v4.runtime.ANTLRErrorStrategy.sync.

Doc from org.antlr.v4.runtime.DefaultErrorStrategy.sync.

The default implementation of ANTLRErrorStrategy#sync makes sure that the current lookahead symbol is consistent with what were expecting at this point in the ATN. You can call this anytime but ANTLR only generates code to check before subrules/loops and each iteration.

Implements Jim Idle's magic sync mechanism in closures and optional subrules. E.g.,

a : sync ( stuff sync )* ;
sync : {consume to what can follow sync} ;
At the start of a sub rule upon error, sync performs single token deletion, if possible. If it can't do that, it bails on the current rule and uses the default error recovery, which consumes until the resynchronization set of the current rule.

If the sub rule is optional ((...)?, (...)*, or block with an empty alternative), then the expected set includes what follows the subrule.

During loop iteration, it consumes until it sees a token that can start a sub rule or what follows loop. Yes, that is pretty aggressive. We opt to stay in the loop as long as possible.

ORIGINS

Previous versions of ANTLR did a poor job of their recovery within loops. A single mismatch token or missing token would force the parser to bail out of the entire rules surrounding the loop. So, for rule

classDef : 'class' ID '{' member* '}'
input with an extra token between members would force the parser to consume until it found the next class definition rather than the next member definition of the current class.

This functionality cost a little bit of effort because the parser has to compare token set at the start of the loop and at each iteration. If for some reason speed is suffering for you, you can turn off this functionality by simply overriding this method as a blank { }.

Parameters
recognizer:Parser

Doc from org.antlr.v4.runtime.ANTLRErrorStrategy.sync.

the parser instance

Annotations
@Override