Top Description Fields Constructors Methods
org.antlr.analysis

public Class DFA

extends Object
Class Inheritance
Known Direct Subclasses
org.antlr.analysis.LL1DFA
Imports
org.antlr.codegen.CodeGenerator, org.antlr.misc.IntSet, .IntervalSet, .Utils, org.antlr.runtime.IntStream, org.stringtemplate.v4.ST, org.antlr.tool.*, java.util.*

A DFA (converted from a grammar's NFA). DFAs are used as prediction machine for alternative blocks in all kinds of recognizers (lexers, parsers, tree walkers).

Field Summary

Modifier and TypeField and Description
public Vector<Integer>
protected DFAState[]
altToAcceptState

We only want one accept state per predicted alt; track here

protected boolean
cyclic

Are there any loops in this DFA?

public static final int
public static final int
public static final int
public NFAState
decisionNFAStartState

From what NFAState did we create the DFA?

public int
decisionNumber

This DFA is being built for which decision?

public String
description

The printable grammar fragment associated with this DFA

protected int
edgeTransitionClass

The unique edge transition class number; every time we see a new set of edges emanating from a state, we number it so we can reuse if it's every seen again for another state.

public Map<List<Integer>, Integer>
edgeTransitionClassMap

Map an edge transition table to a unique set number; ordered so we can push into the output template as an ordered list of sets and then ref them from within the transition[][] table.

public Vector<Integer>
public Vector<Integer>
protected CodeGenerator
generator

Which generator to use if we're building state tables

public boolean
public Vector<Integer>
protected int
max_k

While building the DFA, track max lookahead depth if not cyclic

public static int
MAX_STATE_TRANSITIONS_FOR_TABLE

How many edges can each DFA state have before a "special" state is created that uses IF expressions instead of a table?

public static int
MAX_TIME_PER_DFA_CREATION

Set to 0 to not terminate early (time in ms)

public Vector<Integer>
protected int
public NFA
nfa

Which NFA are we converting (well, which piece of the NFA)?

protected NFAToDFAConverter
protected int
numberOfStates

count only new states not states that were rejected as already present

public boolean
predicateVisible

Track whether this DFA has at least one sem/syn pred encountered during a closure operation.

public DecisionProbe
probe

This probe tells you a lot about a decision and is useful even when there is no error such as when a syntactic nondeterminism is solved via semantic predicates.

public static final int
public static final int
public static final int
public static final int
public IntSet
recursiveAltSet

Track whether an alt discovers recursion for each alt during NFA to DFA conversion; >1 alt with recursion implies nonregular.

protected boolean
reduced

Is this DFA reduced?

public Vector<Integer>
public List<DFAState>
specialStates

List of special DFAState objects

public List<ST>
specialStateSTs

List of ST for special states.

public DFAState
startState

What's the start state for this DFA?

protected int
stateCounter

Unique state numbers per DFA

protected Vector<DFAState>
states

Maps the state number to the actual DFAState.

public Vector<Vector<Integer>>
public Vector<Integer>
transitionEdgeTables

just the Vector<Integer> indicating which unique edge table is at position i.

protected int
protected Map<DFAState, DFAState>
uniqueStates

A set of all uniquely-numbered DFA states.

protected List<Integer>
unreachableAlts

Each alt in an NFA derived from a grammar must have a DFA state that predicts it lest the parser not know what to do.

protected int
user_k

User specified max fixed lookahead.

Constructor Summary

AccessConstructor and Description
protected
DFA()

public
DFA(int decisionNumber, NFAState decisionStartState)

Method Summary

Modifier and TypeMethod and Description
pack-priv int
pack-priv boolean
pack-priv boolean
pack-priv boolean
protected DFAState
addState(DFAState d)

Add a new DFA state to this DFA if not already present.

public boolean
protected void
createEOTAndEOFTables(DFAState s)

Set up the EOT and EOF tables; we cannot put -1 min/max values so we need another way to test that in the DFA transition function.

protected void
protected void
public void
protected void
protected boolean
doesStateReachAcceptState(DFAState d)

figure out if this state eventually reaches an accept state and modify the instance variable 'reduced' to indicate if we find at least one state that cannot reach an accept state.

public void
findAllGatedSynPredsUsedInDFAAcceptStates()

Walk all accept states and find the manually-specified synpreds.

public DFAState
getAcceptState(int alt)

public boolean
public GrammarAST
getDecisionASTNode()

What GrammarAST node (derived from the grammar) is this DFA associated with?

public int
public String
public boolean
public List<? extends String>
public List<? extends String>
public List<? extends String>
public List<? extends String>
public List<? extends String>
public List<? extends String>
public List<List<? extends String>>
public int
getMaxLookaheadDepth()

Return k if decision is LL(k) for some k else return max int

public int
getMaxStateNumber()

What is the max state number ever created?

public NFAState
public int
public int
public String
public List<? extends String>
getRunLengthEncoding(List<Integer> data)

Compress the incoming data list so that runs of same number are encoded as number,value pair sequences.

public DFAState
getState(int stateNumber)

public Map<DFAState, DFAState>
public List<Integer>
getUnreachableAlts()

Return a list of Integer alt numbers for which no lookahead could be computed or for which no single DFA accept state predicts those alts.

public int
getUserMaxLookahead()

The user may specify a max, acyclic lookahead for any decision.

public boolean
hasCycle()

Compute cyclic w/o relying on state computed during analysis.

public boolean
public boolean
hasSynPred()

Count all disambiguating syn preds (ignore synpred tests for gated edges, which occur for nonambig input sequences).

protected void
public boolean
public boolean
isCyclic()

Is this DFA cyclic?

public boolean
public boolean
isReduced()

Is the DFA reduced?

public boolean
isTokensRuleDecision()

Is this DFA derived from the NFA for the Tokens rule?

public DFAState
public boolean
okToRetryDFAWithK1()

If this DFA failed to finish during construction, we might be able to retry with k=1 but we need to know whether it will potentially succeed.

public int
public void
public void
resetStateNumbersToBeContiguous()

Walk all states and reset their numbers to be a contiguous sequence of integers starting from 0.

public void
setAcceptState(int alt, DFAState acceptState)

public void
setState(int stateNumber, DFAState d)

public void
public String
toString()

Overrides java.lang.Object.toString.

Returns a string representation of the object.
public void
verify()

Once this DFA has been built, need to verify that: 1.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAllwaitwaitwait

Field Detail

acceptback to summary
public Vector<Integer> accept
altToAcceptStateback to summary
protected DFAState[] altToAcceptState

We only want one accept state per predicted alt; track here

cyclicback to summary
protected boolean cyclic

Are there any loops in this DFA? Computed by doesStateReachAcceptState()

CYCLIC_BUSYback to summary
public static final int CYCLIC_BUSY
CYCLIC_DONEback to summary
public static final int CYCLIC_DONE
CYCLIC_UNKNOWNback to summary
public static final int CYCLIC_UNKNOWN
decisionNFAStartStateback to summary
public NFAState decisionNFAStartState

From what NFAState did we create the DFA?

decisionNumberback to summary
public int decisionNumber

This DFA is being built for which decision?

descriptionback to summary
public String description

The printable grammar fragment associated with this DFA

edgeTransitionClassback to summary
protected int edgeTransitionClass

The unique edge transition class number; every time we see a new set of edges emanating from a state, we number it so we can reuse if it's every seen again for another state. For Java grammar, some of the big edge transition tables are seen about 57 times.

edgeTransitionClassMapback to summary
public Map<List<Integer>, Integer> edgeTransitionClassMap

Map an edge transition table to a unique set number; ordered so we can push into the output template as an ordered list of sets and then ref them from within the transition[][] table. Like this for C# target: public static readonly DFA30_transition0 = new short[] { 46, 46, -1, 46, 46, -1, -1, -1, -1, -1, -1, -1,...}; public static readonly DFA30_transition1 = new short[] { 21 }; public static readonly short[][] DFA30_transition = { DFA30_transition0, DFA30_transition0, DFA30_transition1, ... };

eofback to summary
public Vector<Integer> eof
eotback to summary
public Vector<Integer> eot
generatorback to summary
protected CodeGenerator generator

Which generator to use if we're building state tables

hasPredicateBlockedByActionback to summary
public boolean hasPredicateBlockedByAction
maxback to summary
public Vector<Integer> max
max_kback to summary
protected int max_k

While building the DFA, track max lookahead depth if not cyclic

MAX_STATE_TRANSITIONS_FOR_TABLEback to summary
public static int MAX_STATE_TRANSITIONS_FOR_TABLE

How many edges can each DFA state have before a "special" state is created that uses IF expressions instead of a table?

MAX_TIME_PER_DFA_CREATIONback to summary
public static int MAX_TIME_PER_DFA_CREATION

Set to 0 to not terminate early (time in ms)

minback to summary
public Vector<Integer> min
nAltsback to summary
protected int nAlts
nfaback to summary
public NFA nfa

Which NFA are we converting (well, which piece of the NFA)?

nfaConverterback to summary
protected NFAToDFAConverter nfaConverter
numberOfStatesback to summary
protected int numberOfStates

count only new states not states that were rejected as already present

predicateVisibleback to summary
public boolean predicateVisible

Track whether this DFA has at least one sem/syn pred encountered during a closure operation. This is useful for deciding whether to retry a non-LL(*) with k=1. If no pred, it will not work w/o a pred so don't bother. It would just give another error message.

probeback to summary
public DecisionProbe probe

This probe tells you a lot about a decision and is useful even when there is no error such as when a syntactic nondeterminism is solved via semantic predicates. Perhaps a GUI would want the ability to show that.

REACHABLE_BUSYback to summary
public static final int REACHABLE_BUSY
REACHABLE_NOback to summary
public static final int REACHABLE_NO
REACHABLE_UNKNOWNback to summary
public static final int REACHABLE_UNKNOWN
REACHABLE_YESback to summary
public static final int REACHABLE_YES
recursiveAltSetback to summary
public IntSet recursiveAltSet

Track whether an alt discovers recursion for each alt during NFA to DFA conversion; >1 alt with recursion implies nonregular.

reducedback to summary
protected boolean reduced

Is this DFA reduced? I.e., can all states lead to an accept state?

specialback to summary
public Vector<Integer> special
specialStatesback to summary
public List<DFAState> specialStates

List of special DFAState objects

specialStateSTsback to summary
public List<ST> specialStateSTs

List of ST for special states.

startStateback to summary
public DFAState startState

What's the start state for this DFA?

stateCounterback to summary
protected int stateCounter

Unique state numbers per DFA

statesback to summary
protected Vector<DFAState> states

Maps the state number to the actual DFAState. Use a Vector as it grows automatically when I set the ith element. This contains all states, but the states are not unique. s3 might be same as s1 so s3 → s1 in this table. This is how cycles occur. If fixed k, then these states will all be unique as states[i] always points at state i when no cycles exist. This is managed in parallel with uniqueStates and simply provides a way to go from state number to DFAState rather than via a hash lookup.

transitionback to summary
public Vector<Vector<Integer>> transition
transitionEdgeTablesback to summary
public Vector<Integer> transitionEdgeTables

just the Vector<Integer> indicating which unique edge table is at position i.

uniqueCompressedSpecialStateNumback to summary
protected int uniqueCompressedSpecialStateNum
uniqueStatesback to summary
protected Map<DFAState, DFAState> uniqueStates

A set of all uniquely-numbered DFA states. Maps hash of DFAState to the actual DFAState object. We use this to detect existing DFA states. Map<DFAState,DFAState>. Use Map so we can get old state back (Set only allows you to see if it's there). Not used during fixed k lookahead as it's a waste to fill it with a dup of states array.

unreachableAltsback to summary
protected List<Integer> unreachableAlts

Each alt in an NFA derived from a grammar must have a DFA state that predicts it lest the parser not know what to do. Nondeterminisms can lead to this situation (assuming no semantic predicates can resolve the problem) and when for some reason, I cannot compute the lookahead (which might arise from an error in the algorithm or from left-recursion etc...). This list starts out with all alts contained and then in method doesStateReachAcceptState() I remove the alts I know to be uniquely predicted.

user_kback to summary
protected int user_k

User specified max fixed lookahead. If 0, nothing specified. -1 implies we have not looked at the options table yet to set k.

Constructor Detail

DFAback to summary
protected DFA()
DFAback to summary
public DFA(int decisionNumber, NFAState decisionStartState)

Method Detail

_getMaxLookaheadDepthback to summary
pack-priv int _getMaxLookaheadDepth(DFAState d, int depth)
_hasCycleback to summary
pack-priv boolean _hasCycle(DFAState d, Map<DFAState, Integer> busy)
_hasSemPredback to summary
pack-priv boolean _hasSemPred(DFAState d, Set<DFAState> busy)
_hasSynPredback to summary
pack-priv boolean _hasSynPred(DFAState d, Set<DFAState> busy)
addStateback to summary
protected DFAState addState(DFAState d)

Add a new DFA state to this DFA if not already present. To force an acyclic, fixed maximum depth DFA, just always return the incoming state. By not reusing old states, no cycles can be created. If we're doing fixed k lookahead don't updated uniqueStates, just return incoming state, which indicates it's a new state.

canInlineDecisionback to summary
public boolean canInlineDecision()
createEOTAndEOFTablesback to summary
protected void createEOTAndEOFTables(DFAState s)

Set up the EOT and EOF tables; we cannot put -1 min/max values so we need another way to test that in the DFA transition function.

createMinMaxTablesback to summary
protected void createMinMaxTables(DFAState s)
createSpecialTableback to summary
protected void createSpecialTable(DFAState s)
createStateTablesback to summary
public void createStateTables(CodeGenerator generator)
createTransitionTableEntryForStateback to summary
protected void createTransitionTableEntryForState(DFAState s)
doesStateReachAcceptStateback to summary
protected boolean doesStateReachAcceptState(DFAState d)

figure out if this state eventually reaches an accept state and modify the instance variable 'reduced' to indicate if we find at least one state that cannot reach an accept state. This implies that the overall DFA is not reduced. This algorithm should be linear in the number of DFA states. The algorithm also tracks which alternatives have no accept state, indicating a nondeterminism. Also computes whether the DFA is cyclic.

Todo

I call getUniquelyPredicatedAlt too much; cache predicted alt

findAllGatedSynPredsUsedInDFAAcceptStatesback to summary
public void findAllGatedSynPredsUsedInDFAAcceptStates()

Walk all accept states and find the manually-specified synpreds. Gated preds are not always hoisted I used to do this in the code generator, but that is too late. This converter tries to avoid computing DFA for decisions in syntactic predicates that are not ever used such as those created by autobacktrack mode.

getAcceptStateback to summary
public DFAState getAcceptState(int alt)
getAutoBacktrackModeback to summary
public boolean getAutoBacktrackMode()
getDecisionASTNodeback to summary
public GrammarAST getDecisionASTNode()

What GrammarAST node (derived from the grammar) is this DFA associated with? It will point to the start of a block or the loop back of a (...)+ block etc...

getDecisionNumberback to summary
public int getDecisionNumber()
getDescriptionback to summary
public String getDescription()
getHasSynPredback to summary
public boolean getHasSynPred()
getJavaCompressedAcceptback to summary
public List<? extends String> getJavaCompressedAccept()
getJavaCompressedEOFback to summary
public List<? extends String> getJavaCompressedEOF()
getJavaCompressedEOTback to summary
public List<? extends String> getJavaCompressedEOT()
getJavaCompressedMaxback to summary
public List<? extends String> getJavaCompressedMax()
getJavaCompressedMinback to summary
public List<? extends String> getJavaCompressedMin()
getJavaCompressedSpecialback to summary
public List<? extends String> getJavaCompressedSpecial()
getJavaCompressedTransitionback to summary
public List<List<? extends String>> getJavaCompressedTransition()
getMaxLookaheadDepthback to summary
public int getMaxLookaheadDepth()

Return k if decision is LL(k) for some k else return max int

getMaxStateNumberback to summary
public int getMaxStateNumber()

What is the max state number ever created? This may be beyond getNumberOfStates().

getNFADecisionStartStateback to summary
public NFAState getNFADecisionStartState()
getNumberOfAltsback to summary
public int getNumberOfAlts()
getNumberOfStatesback to summary
public int getNumberOfStates()
getReasonForFailureback to summary
public String getReasonForFailure()
getRunLengthEncodingback to summary
public List<? extends String> getRunLengthEncoding(List<Integer> data)

Compress the incoming data list so that runs of same number are encoded as number,value pair sequences. 3 -1 -1 -1 28 is encoded as 1 3 3 -1 1 28. I am pretty sure this is the lossless compression that GIF files use. Transition tables are heavily compressed by this technique. I got the idea from JFlex http://jflex.de/ Return List<String> where each string is either \xyz for 8bit char and ￿ for 16bit. Hideous and specific to Java, but it is the only target bad enough to need it.

getStateback to summary
public DFAState getState(int stateNumber)
getUniqueStatesback to summary
public Map<DFAState, DFAState> getUniqueStates()
getUnreachableAltsback to summary
public List<Integer> getUnreachableAlts()

Return a list of Integer alt numbers for which no lookahead could be computed or for which no single DFA accept state predicts those alts. Must call verify() first before this makes sense.

getUserMaxLookaheadback to summary
public int getUserMaxLookahead()

The user may specify a max, acyclic lookahead for any decision. No DFA cycles are created when this value, k, is greater than 0. If this decision has no k lookahead specified, then try the grammar.

hasCycleback to summary
public boolean hasCycle()

Compute cyclic w/o relying on state computed during analysis. just check.

hasSemPredback to summary
public boolean hasSemPred()
hasSynPredback to summary
public boolean hasSynPred()

Count all disambiguating syn preds (ignore synpred tests for gated edges, which occur for nonambig input sequences). E.g., x : (X)=> (X|Y)\n" + | X\n" + ; gives .s0-X->.s1 .s0-Y&&{synpred1_t}?->:s2=>1 .s1-{synpred1_t}?->:s2=>1 .s1-{true}?->:s3=>2

initAltRelatedInfoback to summary
protected void initAltRelatedInfo()
isClassicDFAback to summary
public boolean isClassicDFA()
isCyclicback to summary
public boolean isCyclic()

Is this DFA cyclic? That is, are there any loops? If not, then the DFA is essentially an LL(k) predictor for some fixed, max k value. We can build a series of nested IF statements to match this. In the presence of cycles, we need to build a general DFA and interpret it to distinguish between alternatives.

isGreedyback to summary
public boolean isGreedy()
isReducedback to summary
public boolean isReduced()

Is the DFA reduced? I.e., does every state have a path to an accept state? If not, don't delete as we need to generate an error indicating which paths are "dead ends". Also tracks list of alts with no accept state in the DFA. Must call verify() first before this makes sense.

isTokensRuleDecisionback to summary
public boolean isTokensRuleDecision()

Is this DFA derived from the NFA for the Tokens rule?

newStateback to summary
public DFAState newState()
okToRetryDFAWithK1back to summary
public boolean okToRetryDFAWithK1()

If this DFA failed to finish during construction, we might be able to retry with k=1 but we need to know whether it will potentially succeed. Can only succeed if there is a predicate to resolve the issue. Don't try if k=1 already as it would cycle forever. Timeout can retry with k=1 even if no predicate if k!=1.

predictback to summary
public int predict(IntStream input)
removeStateback to summary
public void removeState(DFAState d)
resetStateNumbersToBeContiguousback to summary
public void resetStateNumbersToBeContiguous()

Walk all states and reset their numbers to be a contiguous sequence of integers starting from 0. Only cyclic DFA can have unused positions in states list. State i might be identical to a previous state j and will result in states[i] == states[j]. We don't want to waste a state number on this. Useful mostly for code generation in tables. At the start of this routine, states[i].stateNumber <= i by definition. If states[50].stateNumber is 50 then a cycle during conversion may try to add state 103, but we find that an identical DFA state, named 50, already exists, hence, states[103]==states[50] and both have stateNumber 50 as they point at same object. Afterwards, the set of state numbers from all states should represent a contiguous range from 0..n-1 where n is the number of unique states.

setAcceptStateback to summary
public void setAcceptState(int alt, DFAState acceptState)
setStateback to summary
public void setState(int stateNumber, DFAState d)
setUserMaxLookaheadback to summary
public void setUserMaxLookahead(int k)
toStringback to summary
public String toString()

Overrides java.lang.Object.toString.

Doc from java.lang.Object.toString.

Returns a string representation of the object.

Returns:String

a string representation of the object

Annotations
@Override
verifyback to summary
public void verify()

Once this DFA has been built, need to verify that: 1. it's reduced 2. all alts have an accept state Elsewhere, in the NFA converter, we need to verify that: 3. alts i and j have disjoint lookahead if no sem preds 4. if sem preds, nondeterministic alts must be sufficiently covered This is avoided if analysis bails out for any reason.