When a grammar fails to generate a parser the ParserStateMachine has a
null start state. When this is passed to construct a Parser quietly
avoid creating any parser specific state, esp. ParserNode which will
assert if passed a null state.
Fixes#36 "Improve performance of GrammarSymbolSet". Not reproduced
exactly because I didn't have the grammar to hand but it seems likely
this is the problem.
This works for UTF-8 encoded strings but debugging will fail when using
UTF-16 or UTF-32 as the printf() formatted string won't deal with null
characters embedded within the wider character encodings.
Committing this for now because it doesn't break any existing
functionality but does enable parsing of wider character inputs.
Labels are only used to add meaningful text to states in Graphviz DOT
graphs. But these strings can take lots of time and memory to generate
especially in large grammars like that for PostgreSQL. Therefore only
generate labels when they will be used.
The previous algorithm would loop through all states but only process
those that didn't have their processed flag set. Much faster to only
iterate through those states that haven't been processed.
Allows most metacharacters, e.g. "*+()[", to appear unescaped within a
character class. This, along with hyphens at the end, allows "[+-]" to
match a plus or a minus.
Using set<>::operator<() doesn't generate a stable comparison wrt. the
nodes in the item. This is either set<>::operator<() not behaving as
a lexicographical compare like I expect or not taking into account the
comparison object the set is initialized with.
Fixes non-deterministic lexer generation.
Previously this cast a GrammarSymbol to a ParserSymbol and then wrote
out its index -- generating a wildly wrong value. This was okay at
runtime because the lexer only checks to see whether or not there is a
symbol and never uses its value.
However it looks like a pretty nasty error to anyone looking at the
generated source code. Fixed by converting the GrammarSymbol to the
correct ParserSymbol when the lexer is written out.
Uses the algorithms described in Compilers - Principles, Techniques,
and Tools, p.241 to generate spontaneous lookaheads and propagate them
through the state machine.
Adds GrammarSymbolSet to store symbol sets as a bitmask by symbol index
and a vector of GrammarSymbol*.
Multi-threads the calculation of goto items for lookahead propagation.
