Top Description Fields Constructors Methods
org.antlr.analysis

public Class DecisionProbe

extends Object
Class Inheritance
Imports
org.antlr.grammar.v3.ANTLRParser, org.antlr.misc.MultiMap, .Utils, org.antlr.runtime.Token, org.antlr.tool.ErrorManager, .Grammar, .GrammarAST, java.util.*

Collection of information about what is wrong with a decision as discovered while building the DFA predictor. The information is collected during NFA→DFA conversion and, while some of this is available elsewhere, it is nice to have it all tracked in one spot so a great error message can be easily had. I also like the fact that this object tracks it all for later perusing to make an excellent error message instead of lots of imprecise on-the-fly warnings (during conversion). A decision normally only has one problem; e.g., some input sequence can be matched by multiple alternatives. Unfortunately, some decisions such as a : ( A | B ) | ( A | B ) | A ; have multiple problems. So in general, you should approach a decision as having multiple flaws each one uniquely identified by a DFAState. For example, statesWithSyntacticallyAmbiguousAltsSet tracks the set of all DFAStates where ANTLR has discovered a problem. Recall that a decision is represented internall with a DFA comprised of multiple states, each of which could potentially have problems. Because of this, you need to iterate over this list of DFA states. You'll note that most of the informational methods like getSampleNonDeterministicInputSequence() require a DFAState. This state will be one of the iterated states from stateToSyntacticallyAmbiguousAltsSet. This class is not thread safe due to shared use of visited maps etc... Only one thread should really need to access one DecisionProbe anyway.

Field Summary

Modifier and TypeField and Description
protected Set<Integer>
altsWithProblem

The overall list of alts within the decision that have at least one conflicting input sequence.

protected Set<DFAState>
danglingStates

The set of states w/o emanating edges and w/o resolving sem preds.

public DFA
public boolean
nonLLStarDecision

If decision with > 1 alt has recursion in > 1 alt, it's (likely) nonregular lookahead.

public static final Integer
public static final Integer
public static final Integer
protected Map<Integer, Integer>
stateReachable

Used to find paths through syntactically ambiguous DFA.

protected Set<DFAState>
statesResolvedWithSemanticPredicatesSet

Was a syntactic ambiguity resolved with predicates?

protected Set<String>
statesVisitedAtInputDepth

Used while finding a path through an NFA whose edge labels match an input sequence.

protected Set<Integer>
protected Set<DFAState>
statesWithSyntacticallyAmbiguousAltsSet

Track all DFA states with nondeterministic alternatives.

protected Map<DFAState, Map<Integer, SemanticContext>>
stateToAltSetWithSemanticPredicatesMap

Track the predicates for each alt per DFA state; more than one DFA state might have syntactically ambig alt prediction.

protected Map<DFAState, Map<Integer, Set<Token>>>
stateToIncompletelyCoveredAltsMap

Tracks alts insufficiently covered.

protected MultiMap<Integer, NFAConfiguration>
stateToRecursionOverflowConfigurationsMap

Recursion is limited to a particular depth.

protected Map<DFAState, Set<Integer>>
stateToSyntacticallyAmbiguousTokensRuleAltsMap

Track just like stateToSyntacticallyAmbiguousAltsMap, but only for nondeterminisms that arise in the Tokens rule such as keyword vs ID rule.

protected boolean
timedOut

Did ANTLR have to terminate early on the analysis of this decision?

public static boolean

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
public boolean
analysisOverflowed()

Took too long to analyze a DFA

private void
computeAltToProblemMaps(Set<Integer> dfaStatesUnaliased, Map<Integer, List<NFAConfiguration>> configurationsMap, Map<Integer, Map<String, Set<NFAState>>> altToTargetToCallSitesMap, Map<Integer, DFAState> altToDFAState)

public Set<DFAState>
getDanglingStates()

return set of states w/o emanating edges and w/o resolving sem preds.

public String
getDescription()

Return a string like "3:22: ( A {;} | B )" that describes this decision.

protected Set<DFAState>
public Set<DFAState>
getDFAStatesWithSyntacticallyAmbiguousAlts()

Return all DFA states in this DFA that have NFA configurations that conflict.

public Set<Integer>
getDisabledAlternatives(DFAState d)

Which alts were specifically turned off to resolve nondeterminisms?

public Map<Integer, Set<Token>>
getIncompletelyCoveredAlts(DFAState d)

Return a list of alts whose predicate context was insufficient to resolve a nondeterminism for state d.

public String
getInputSequenceDisplay(List<? extends Label> labels)

Given List<Label>, return a String with a useful representation of the associated input string.

protected boolean
getNFAPath(NFAState s, int labelIndex, List<? extends Label> labels, List<? super NFAState> path)

Given a sample input sequence, you usually would like to know the path taken through the NFA.

public List<? extends NFAState>
getNFAPathStatesForAlt(int firstAlt, int alt, List<? extends Label> labels)

Given an alternative associated with a nondeterministic DFA state, find the path of NFA states associated with the labels sequence.

public Set<Integer>
public List<Integer>
getNonDeterministicAltsForState(DFAState targetState)

Return the sorted list of alts that conflict within a single state.

public Set<DFAState>
public int
getNumberOfStates()

How many states does the DFA predictor have?

protected void
getSampleInputSequenceUsingStateSet(State startState, State targetState, Set<DFAState> states, List<Label> labels)

Given a start state and a final state, find a list of edge labels between the two ignoring epsilon.

public List<Label>
getSampleNonDeterministicInputSequence(DFAState targetState)

Return a List<Label> indicating an input sequence that can be matched from the start state of the DFA to the targetState (which is known to have a problem).

public SemanticContext
getSemanticContextForAlt(DFAState d, int alt)

Each state in the DFA represents a different input sequence for an alt of the decision.

protected String
getStateLabelIndexKey(int s, int i)

public String
getTokenNameForTokensRuleAlt(int alt)

From an alt number associated with artificial Tokens rule, return the name of the token that is associated with that alt.

private Set<Integer>
getUnaliasedDFAStateSet(Set<Integer> dfaStatesWithRecursionProblems)

public List<Integer>
getUnreachableAlts()

Get a list of all unreachable alternatives for this decision.

public boolean
hasPredicate()

At least one alt refs a sem or syn pred

public boolean
public boolean
isDeterministic()

If no states are dead-ends, no alts are unreachable, there are no nondeterminisms unresolved by syn preds, all is ok with decision.

public boolean
isNonLLStarDecision()

Found recursion in > 1 alt

public boolean
protected void
public void
protected boolean
reachesState(DFAState startState, DFAState targetState, Set<DFAState> states)

Given a start state and a target state, return true if start can reach target state.

public void
removeRecursiveOverflowState(DFAState d)

If a recursion overflow is resolve with predicates, then we need to shut off the warning that would be generated.

public void
reportAltPredicateContext(DFAState d, Map<Integer, ? extends SemanticContext> altPredicateContext)

Report the list of predicates found for each alternative; copy the list because this set gets altered later by the method tryToResolveWithSemanticPredicates() while flagging NFA configurations in d as resolved.

public void
reportDanglingState(DFAState d)

Report the fact that DFA state d is not a state resolved with predicates and yet it has no emanating edges.

public void
reportIncompletelyCoveredAlts(DFAState d, Map<Integer, Set<Token>> altToLocationsReachableWithoutPredicate)

public void
reportLexerRuleNondeterminism(DFAState d, Set<Integer> nondeterministicAlts)

Currently the analysis reports issues between token definitions, but we don't print out warnings in favor of just picking the first token definition found in the grammar ala lex/flex.

public void
reportNondeterminism(DFAState d, Set<Integer> nondeterministicAlts)

public void
public void
reportNonLLStarDecision(DFA dfa)

Report that at least 2 alts have recursive constructs.

public void
reportRecursionOverflow(DFAState d, NFAConfiguration recursionNFAConfiguration)

public void
protected void
stripWildCardAlts(Set<Integer> disabledAlts)

Get the last disabled alt number and check in the grammar to see if that alt is a simple wildcard.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

altsWithProblemback to summary
protected Set<Integer> altsWithProblem

The overall list of alts within the decision that have at least one conflicting input sequence.

danglingStatesback to summary
protected Set<DFAState> danglingStates

The set of states w/o emanating edges and w/o resolving sem preds.

dfaback to summary
public DFA dfa
nonLLStarDecisionback to summary
public boolean nonLLStarDecision

If decision with > 1 alt has recursion in > 1 alt, it's (likely) nonregular lookahead. The decision cannot be made with a DFA. the alts are stored in altsWithProblem.

REACHABLE_BUSYback to summary
public static final Integer REACHABLE_BUSY
REACHABLE_NOback to summary
public static final Integer REACHABLE_NO
REACHABLE_YESback to summary
public static final Integer REACHABLE_YES
stateReachableback to summary
protected Map<Integer, Integer> stateReachable

Used to find paths through syntactically ambiguous DFA. If we've seen statement number before, what did we learn?

statesResolvedWithSemanticPredicatesSetback to summary
protected Set<DFAState> statesResolvedWithSemanticPredicatesSet

Was a syntactic ambiguity resolved with predicates? Any DFA state that predicts more than one alternative, must be resolved with predicates or it should be reported to the user.

statesVisitedAtInputDepthback to summary
protected Set<String> statesVisitedAtInputDepth

Used while finding a path through an NFA whose edge labels match an input sequence. Tracks the input position we were at the last time at this node. If same input position, then we'd have reached same state without consuming input...probably an infinite loop. Stop. Set<String>. The strings look like stateNumber_labelIndex.

statesVisitedDuringSampleSequenceback to summary
protected Set<Integer> statesVisitedDuringSampleSequence
statesWithSyntacticallyAmbiguousAltsSetback to summary
protected Set<DFAState> statesWithSyntacticallyAmbiguousAltsSet

Track all DFA states with nondeterministic alternatives. By reaching the same DFA state, a path through the NFA for some input is able to reach the same NFA state by starting at more than one alternative's left edge. Though, later, we may find that predicates resolve the issue, but track info anyway. Note that from the DFA state, you can ask for which alts are nondeterministic.

stateToAltSetWithSemanticPredicatesMapback to summary
protected Map<DFAState, Map<Integer, SemanticContext>> stateToAltSetWithSemanticPredicatesMap

Track the predicates for each alt per DFA state; more than one DFA state might have syntactically ambig alt prediction. Maps DFA state to another map, mapping alt number to a SemanticContext (pred(s) to execute to resolve syntactic ambiguity).

stateToIncompletelyCoveredAltsMapback to summary
protected Map<DFAState, Map<Integer, Set<Token>>> stateToIncompletelyCoveredAltsMap

Tracks alts insufficiently covered. For example, p1||true gets reduced to true and so leaves whole alt uncovered. This maps DFA state to the set of alts

stateToRecursionOverflowConfigurationsMapback to summary
protected MultiMap<Integer, NFAConfiguration> stateToRecursionOverflowConfigurationsMap

Recursion is limited to a particular depth. If that limit is exceeded the proposed new NFAConfiguration is recorded for the associated DFA state.

stateToSyntacticallyAmbiguousTokensRuleAltsMapback to summary
protected Map<DFAState, Set<Integer>> stateToSyntacticallyAmbiguousTokensRuleAltsMap

Track just like stateToSyntacticallyAmbiguousAltsMap, but only for nondeterminisms that arise in the Tokens rule such as keyword vs ID rule. The state maps to the list of Tokens rule alts that are in conflict.

timedOutback to summary
protected boolean timedOut

Did ANTLR have to terminate early on the analysis of this decision?

verboseback to summary
public static boolean verbose

Constructor Detail

DecisionProbeback to summary
public DecisionProbe(DFA dfa)

Method Detail

analysisOverflowedback to summary
public boolean analysisOverflowed()

Took too long to analyze a DFA

computeAltToProblemMapsback to summary
private void computeAltToProblemMaps(Set<Integer> dfaStatesUnaliased, Map<Integer, List<NFAConfiguration>> configurationsMap, Map<Integer, Map<String, Set<NFAState>>> altToTargetToCallSitesMap, Map<Integer, DFAState> altToDFAState)
getDanglingStatesback to summary
public Set<DFAState> getDanglingStates()

return set of states w/o emanating edges and w/o resolving sem preds. These states come about because the analysis algorithm had to terminate early to avoid infinite recursion for example (due to left recursion perhaps).

getDescriptionback to summary
public String getDescription()

Return a string like "3:22: ( A {;} | B )" that describes this decision.

getDFAPathStatesToTargetback to summary
protected Set<DFAState> getDFAPathStatesToTarget(DFAState targetState)
getDFAStatesWithSyntacticallyAmbiguousAltsback to summary
public Set<DFAState> getDFAStatesWithSyntacticallyAmbiguousAlts()

Return all DFA states in this DFA that have NFA configurations that conflict. You must report a problem for each state in this set because each state represents a different input sequence.

getDisabledAlternativesback to summary
public Set<Integer> getDisabledAlternatives(DFAState d)

Which alts were specifically turned off to resolve nondeterminisms? This is different than the unreachable alts. Disabled doesn't mean that the alternative is totally unreachable necessarily, it just means that for this DFA state, that alt is disabled. There may be other accept states for that alt that make an alt reachable.

getIncompletelyCoveredAltsback to summary
public Map<Integer, Set<Token>> getIncompletelyCoveredAlts(DFAState d)

Return a list of alts whose predicate context was insufficient to resolve a nondeterminism for state d.

getInputSequenceDisplayback to summary
public String getInputSequenceDisplay(List<? extends Label> labels)

Given List<Label>, return a String with a useful representation of the associated input string. One could show something different for lexers and parsers, for example.

getNFAPathback to summary
protected boolean getNFAPath(NFAState s, int labelIndex, List<? extends Label> labels, List<? super NFAState> path)

Given a sample input sequence, you usually would like to know the path taken through the NFA. Return the list of NFA states visited while matching a list of labels. This cannot use the usual interpreter, which does a deterministic walk. We need to be able to take paths that are turned off during nondeterminism resolution. So, just do a depth-first walk traversing edges labeled with the current label. Return true if a path was found emanating from state s.

getNFAPathStatesForAltback to summary
public List<? extends NFAState> getNFAPathStatesForAlt(int firstAlt, int alt, List<? extends Label> labels)

Given an alternative associated with a nondeterministic DFA state, find the path of NFA states associated with the labels sequence. Useful tracing where in the NFA, a single input sequence can be matched. For different alts, you should get different NFA paths. The first NFA state for all NFA paths will be the same: the starting NFA state of the first nondeterministic alt. Imagine (A|B|A|A): 5->9-A->o | 6->10-B->o | 7->11-A->o | 8->12-A->o There are 3 nondeterministic alts. The paths should be: 5 9 ... 5 6 7 11 ... 5 6 7 8 12 ... The NFA path matching the sample input sequence (labels) is computed using states 9, 11, and 12 rather than 5, 7, 8 because state 5, for example can get to all ambig paths. Must isolate for each alt (hence, the extra state beginning each alt in my NFA structures). Here, firstAlt=1.

getNonDeterministicAltsback to summary
public Set<Integer> getNonDeterministicAlts()
getNonDeterministicAltsForStateback to summary
public List<Integer> getNonDeterministicAltsForState(DFAState targetState)

Return the sorted list of alts that conflict within a single state. Note that predicates may resolve the conflict.

getNondeterministicStatesResolvedWithSemanticPredicateback to summary
public Set<DFAState> getNondeterministicStatesResolvedWithSemanticPredicate()
getNumberOfStatesback to summary
public int getNumberOfStates()

How many states does the DFA predictor have?

getSampleInputSequenceUsingStateSetback to summary
protected void getSampleInputSequenceUsingStateSet(State startState, State targetState, Set<DFAState> states, List<Label> labels)

Given a start state and a final state, find a list of edge labels between the two ignoring epsilon. Limit your scan to a set of states passed in. This is used to show a sample input sequence that is nondeterministic with respect to this decision. Return List<Label> as a parameter. The incoming states set must be all states that lead from startState to targetState and no others so this algorithm doesn't take a path that eventually leads to a state other than targetState. Don't follow loops, leading to short (possibly shortest) path.

getSampleNonDeterministicInputSequenceback to summary
public List<Label> getSampleNonDeterministicInputSequence(DFAState targetState)

Return a List<Label> indicating an input sequence that can be matched from the start state of the DFA to the targetState (which is known to have a problem).

getSemanticContextForAltback to summary
public SemanticContext getSemanticContextForAlt(DFAState d, int alt)

Each state in the DFA represents a different input sequence for an alt of the decision. Given a DFA state, what is the semantic predicate context for a particular alt.

getStateLabelIndexKeyback to summary
protected String getStateLabelIndexKey(int s, int i)
getTokenNameForTokensRuleAltback to summary
public String getTokenNameForTokensRuleAlt(int alt)

From an alt number associated with artificial Tokens rule, return the name of the token that is associated with that alt.

getUnaliasedDFAStateSetback to summary
private Set<Integer> getUnaliasedDFAStateSet(Set<Integer> dfaStatesWithRecursionProblems)
getUnreachableAltsback to summary
public List<Integer> getUnreachableAlts()

Get a list of all unreachable alternatives for this decision. There may be multiple alternatives with ambiguous input sequences, but this is the overall list of unreachable alternatives (either due to conflict resolution or alts w/o accept states).

hasPredicateback to summary
public boolean hasPredicate()

At least one alt refs a sem or syn pred

isCyclicback to summary
public boolean isCyclic()
isDeterministicback to summary
public boolean isDeterministic()

If no states are dead-ends, no alts are unreachable, there are no nondeterminisms unresolved by syn preds, all is ok with decision.

isNonLLStarDecisionback to summary
public boolean isNonLLStarDecision()

Found recursion in > 1 alt

isReducedback to summary
public boolean isReduced()
issueRecursionWarningsback to summary
protected void issueRecursionWarnings()
issueWarningsback to summary
public void issueWarnings()
reachesStateback to summary
protected boolean reachesState(DFAState startState, DFAState targetState, Set<DFAState> states)

Given a start state and a target state, return true if start can reach target state. Also, compute the set of DFA states that are on a path from start to target; return in states parameter.

removeRecursiveOverflowStateback to summary
public void removeRecursiveOverflowState(DFAState d)

If a recursion overflow is resolve with predicates, then we need to shut off the warning that would be generated.

reportAltPredicateContextback to summary
public void reportAltPredicateContext(DFAState d, Map<Integer, ? extends SemanticContext> altPredicateContext)

Report the list of predicates found for each alternative; copy the list because this set gets altered later by the method tryToResolveWithSemanticPredicates() while flagging NFA configurations in d as resolved.

reportDanglingStateback to summary
public void reportDanglingState(DFAState d)

Report the fact that DFA state d is not a state resolved with predicates and yet it has no emanating edges. Usually this is a result of the closure/reach operations being unable to proceed

reportIncompletelyCoveredAltsback to summary
public void reportIncompletelyCoveredAlts(DFAState d, Map<Integer, Set<Token>> altToLocationsReachableWithoutPredicate)
reportLexerRuleNondeterminismback to summary
public void reportLexerRuleNondeterminism(DFAState d, Set<Integer> nondeterministicAlts)

Currently the analysis reports issues between token definitions, but we don't print out warnings in favor of just picking the first token definition found in the grammar ala lex/flex.

reportNondeterminismback to summary
public void reportNondeterminism(DFAState d, Set<Integer> nondeterministicAlts)
reportNondeterminismResolvedWithSemanticPredicateback to summary
public void reportNondeterminismResolvedWithSemanticPredicate(DFAState d)
reportNonLLStarDecisionback to summary
public void reportNonLLStarDecision(DFA dfa)

Report that at least 2 alts have recursive constructs. There is no way to build a DFA so we terminated.

reportRecursionOverflowback to summary
public void reportRecursionOverflow(DFAState d, NFAConfiguration recursionNFAConfiguration)
resetback to summary
public void reset()
stripWildCardAltsback to summary
protected void stripWildCardAlts(Set<Integer> disabledAlts)

Get the last disabled alt number and check in the grammar to see if that alt is a simple wildcard. If so, treat like an else clause and don't emit the error. Strip out the last alt if it's wildcard.