ParserInterpreter that creates parse trees
that track alternative numbers for subtree roots.
| Modifier and Type | Class and Description |
|---|---|
| public static class | GrammarParserInterpreter.
We want to stop and track the first error but we cannot bail out like
|
| Modifier and Type | Field and Description |
|---|---|
| protected BitSet | |
| protected final Grammar | g
The grammar associated with this interpreter. |
| protected int[][] |
| Access | Constructor and Description |
|---|---|
| public | GrammarParserInterpreter(Grammar g, String grammarFileName, Vocabulary vocabulary, Collection<String> ruleNames, ATN atn, TokenStream input)
|
| public |
| Modifier and Type | Method and Description |
|---|---|
| protected InterpreterRuleContext | createInterpreterRuleContext(ParserRuleContext parent, int invokingStateNumber, int ruleIndex)
Overrides org. |
| public static ParserInterpreter | deriveTempParserInterpreter(Grammar g, Parser originalParser, TokenStream tokens)
Derive a new parser from an old one that has knowledge of the grammar. |
| public BitSet | |
| public static List | Returns: The list of all possible interpretations of the input for the decision in ambiguityInfo. The actual interpretation chosen by the parser is always given first because this method retests the input in alternative order and ANTLR always resolves ambiguities by choosing the first alternative that matches the input. The subtree returnedFrom which grammar should we drive alternative
numbers and alternative labels. g, Parser The parser used to create ambiguityInfo; it
is not modified by this routine and can be either
a generated or interpreted parser. It's token
stream *is* reset/seek()'d. originalParser, TokenStream A stream of tokens to use with the temporary parser.
This will often be just the token stream within the
original parser but here it is for flexibility. tokens, int Which decision to try different alternatives for. decision, BitSet The set of alternatives to try while re-parsing. alts, int The index of the first token of the ambiguous
input or other input of interest. startIndex, int The index of the last token of the ambiguous input.
The start and stop indexes are used primarily to
identify how much of the resulting parse tree
to return. stopIndex, int The start rule for the entire grammar, not
the ambiguous decision. We re-parse the entire input
and so we need the original start rule. startRuleIndex)Given an ambiguous parse information, return the list of ambiguous parse trees. |
| public static List | getLookaheadParseTrees(Grammar g, ParserInterpreter originalParser, TokenStream tokens, int startRuleIndex, int decision, int startIndex, int stopIndex)
Return a list of parse trees, one for each alternative in a decision given the same input. |
| public void | |
| protected int | visitDecisionState(DecisionState p)
Overrides org. |
| decisionStatesThatSetOuterAltNumInContext | back to summary |
|---|---|
| protected BitSet decisionStatesThatSetOuterAltNumInContext | |
| g | back to summary |
|---|---|
| protected final Grammar g The grammar associated with this interpreter. Unlike the
| |
| stateToAltsMap | back to summary |
|---|---|
| protected int[][] stateToAltsMap Cache | |
| GrammarParserInterpreter | back to summary |
|---|---|
| public GrammarParserInterpreter(Grammar g, String grammarFileName, Vocabulary vocabulary, Collection<String> ruleNames, ATN atn, TokenStream input) | |
| GrammarParserInterpreter | back to summary |
|---|---|
| public GrammarParserInterpreter(Grammar g, ATN atn, TokenStream input) | |
| createInterpreterRuleContext | back to summary |
|---|---|
| protected InterpreterRuleContext createInterpreterRuleContext(ParserRuleContext parent, int invokingStateNumber, int ruleIndex) Overrides org. Doc from org. Provide simple "factory" for InterpreterRuleContext's.
| |
| deriveTempParserInterpreter | back to summary |
|---|---|
| public static ParserInterpreter deriveTempParserInterpreter(Grammar g, Parser originalParser, TokenStream tokens) Derive a new parser from an old one that has knowledge of the grammar.
The Grammar object is used to correctly compute outer alternative
numbers for parse tree nodes. A parser of the same type is created
for subclasses of | |
| findOuterMostDecisionStates | back to summary |
|---|---|
| public BitSet findOuterMostDecisionStates() identify the ATN states where we need to set the outer alt number. For regular rules, that's the block at the target to rule start state. For left-recursive rules, we track the primary block, which looks just like a regular rule's outer block, and the star loop block (always there even if 1 alt). | |
| getAllPossibleParseTrees | back to summary |
|---|---|
| public static List Given an ambiguous parse information, return the list of ambiguous parse trees. An ambiguity occurs when a specific token sequence can be recognized in more than one way by the grammar. These ambiguities are detected only at decision points. The list of trees includes the actual interpretation (that for the minimum alternative number) and all ambiguous alternatives. The actual interpretation is always first. This method reuses the same physical input token stream used to detect the ambiguity by the original parser in the first place. This method resets/seeks within but does not alter originalParser. The trees are rooted at the node whose start..stop token indices include the start and stop indices of this ambiguity event. That is, the trees returned will always include the complete ambiguous subphrase identified by the ambiguity event. The subtrees returned will also always contain the node associated with the overridden decision. Be aware that this method does NOT notify error or parse listeners as it would trigger duplicate or otherwise unwanted events. This uses a temporary ParserATNSimulator and a ParserInterpreter so we don't mess up any statistics, event lists, etc... The parse tree constructed while identifying/making ambiguityInfo is not affected by this method as it creates a new parser interp to get the ambiguous interpretations. Nodes in the returned ambig trees are independent of the original parse tree (constructed while identifying/creating ambiguityInfo).
| |
| getLookaheadParseTrees | back to summary |
|---|---|
| public static List Return a list of parse trees, one for each alternative in a decision
given the same input.
Very similar to Notes // we must parse the entire input now with decision overrides // we cannot parse a subset because it could be that a decision // above our decision of interest needs to read way past // lookaheadInfo.stopIndex. It seems like there is no escaping // the use of a full and complete token stream if we are // resetting to token index 0 and re-parsing from the start symbol. // It's not easy to restart parsing somewhere in the middle like a // continuation because our call stack does not match the // tree stack because of left recursive rule rewriting. grrrr!
| |
| reset | back to summary |
|---|---|
| public void reset() Overrides org. Doc from org. reset the parser's state
| |
| visitDecisionState | back to summary |
|---|---|
| protected int visitDecisionState(DecisionState p) Overrides org. Override this method so that we can record which alternative was taken at each decision point. For non-left recursive rules, it's simple. Set decisionStatesThatSetOuterAltNumInContext indicates which decision states should set the outer alternative number. Left recursive rules are much more complicated to deal with: there is typically a decision for the primary alternatives and a decision to choose between the recursive operator alternatives. For example, the following left recursive rule has two primary and 2 recursive alternatives. e : e '*' e | '-' INT | e '+' e | ID ;ANTLR rewrites that rule to be e[int precedence] : ('-' INT | ID) ( {...}? '*' e[5] | {...}? '+' e[3] )* ;So, there are two decisions associated with picking the outermost alt. This complicates our tracking significantly. The outermost alternative number is a function of the decision (ATN state) within a left recursive rule and the predicted alternative coming back from adaptivePredict(). We use stateToAltsMap as a cache to avoid expensive calls to getRecursiveOpAlts().
| |
BailErrorStrategy as consume() constructs trees. We make sure
to create an error node during recovery with this strategy. We
consume() 1 token during the "bail out of rule" mechanism in recover()
and let it fall out of the rule to finish constructing trees. For
recovery in line, we throw InputMismatchException to engage recover().
| Modifier and Type | Field and Description |
|---|---|
| public int |
| Access | Constructor and Description |
|---|---|
| public |
| Modifier and Type | Method and Description |
|---|---|
| public void | recover(Parser
the parser instance recognizer, RecognitionException the recognition exception to recover from e)Overrides org. Implements org. e.
|
| public Token | recoverInline(Parser
the parser instance recognizer)Overrides org. Implements org. Parser#match.
|
| public void | sync(Parser
the parser instance recognizer)Overrides org. Implements org. ANTLRErrorStrategy#sync makes sure
that the current lookahead symbol is consistent with what were expecting
at this point in the ATN.
|
| firstErrorTokenIndex | back to summary |
|---|---|
| public int firstErrorTokenIndex | |
| BailButConsumeErrorStrategy | back to summary |
|---|---|
| public BailButConsumeErrorStrategy() | |
| recover | back to summary |
|---|---|
| public void recover(Parser recognizer, RecognitionException e) Overrides org. Implements org. Doc from org. This method is called to recover from exception The default implementation resynchronizes the parser by consuming tokens until we find one in the resynchronization set--loosely the set of tokens that can follow the current rule.
| |
| recoverInline | back to summary |
|---|---|
| public Token recoverInline(Parser recognizer) throws RecognitionException Overrides org. Implements org. Doc from org. This method is called when an unexpected symbol is encountered during an
inline match operation, such as This method handles the consumption of any tokens - the caller should
not call Note that the calling code will not report an error if this method
returns successfully. The error strategy implementation is responsible
for calling The default implementation attempts to recover from the mismatched input
by using single token insertion and deletion as described below. If the
recovery attempt fails, this method throws an
EXTRA TOKEN (single token deletion)
This recovery strategy is implemented by MISSING TOKEN (single token insertion) If current token (at This recovery strategy is implemented by EXAMPLE For example, Input stat → expr → atomand it will be trying to match the ')' at this point in the
derivation:
=> ID '=' '(' INT ')' ('+' atom)* ';'
^
The attempt to match ')' will fail when it sees ';' and
call recoverInline. To recover, it sees that LA(1)==';'
is in the set of tokens that can follow the ')' token reference
in rule atom. It can assume that you forgot the ')'.
| |
| sync | back to summary |
|---|---|
| public void sync(Parser recognizer) Overrides org. Implements org. Doc from org. The default implementation of Implements Jim Idle's magic sync mechanism in closures and optional subrules. E.g.,
a : sync ( stuff sync )* ;
sync : {consume to what can follow sync} ;
At the start of a sub rule upon error, sync performs single
token deletion, if possible. If it can't do that, it bails on the current
rule and uses the default error recovery, which consumes until the
resynchronization set of the current rule.
If the sub rule is optional ( During loop iteration, it consumes until it sees a token that can start a sub rule or what follows loop. Yes, that is pretty aggressive. We opt to stay in the loop as long as possible. ORIGINS Previous versions of ANTLR did a poor job of their recovery within loops. A single mismatch token or missing token would force the parser to bail out of the entire rules surrounding the loop. So, for rule
classDef : 'class' ID '{' member* '}'
input with an extra token between members would force the parser to
consume until it found the next class definition rather than the next
member definition of the current class.
This functionality cost a little bit of effort because the parser has to compare token set at the start of the loop and at each iteration. If for some reason speed is suffering for you, you can turn off this functionality by simply overriding this method as a blank { }.
| |