Fixes#36 "Improve performance of GrammarSymbolSet". Not reproduced
exactly because I didn't have the grammar to hand but it seems likely
this is the problem.
This works for UTF-8 encoded strings but debugging will fail when using
UTF-16 or UTF-32 as the printf() formatted string won't deal with null
characters embedded within the wider character encodings.
Committing this for now because it doesn't break any existing
functionality but does enable parsing of wider character inputs.
Labels are only used to add meaningful text to states in Graphviz DOT
graphs. But these strings can take lots of time and memory to generate
especially in large grammars like that for PostgreSQL. Therefore only
generate labels when they will be used.
The previous algorithm would loop through all states but only process
those that didn't have their processed flag set. Much faster to only
iterate through those states that haven't been processed.
Allows most metacharacters, e.g. "*+()[", to appear unescaped within a
character class. This, along with hyphens at the end, allows "[+-]" to
match a plus or a minus.
Using set<>::operator<() doesn't generate a stable comparison wrt. the
nodes in the item. This is either set<>::operator<() not behaving as
a lexicographical compare like I expect or not taking into account the
comparison object the set is initialized with.
Fixes non-deterministic lexer generation.
Previously this cast a GrammarSymbol to a ParserSymbol and then wrote
out its index -- generating a wildly wrong value. This was okay at
runtime because the lexer only checks to see whether or not there is a
symbol and never uses its value.
However it looks like a pretty nasty error to anyone looking at the
generated source code. Fixed by converting the GrammarSymbol to the
correct ParserSymbol when the lexer is written out.
Uses the algorithms described in Compilers - Principles, Techniques,
and Tools, p.241 to generate spontaneous lookaheads and propagate them
through the state machine.
Adds GrammarSymbolSet to store symbol sets as a bitmask by symbol index
and a vector of GrammarSymbol*.
Multi-threads the calculation of goto items for lookahead propagation.
Sets the precedence and associativity of an implicit terminal from the
literal or identifier that sets it. It's an error for both the terminal
and the identifier to set it.
Implicit terminals can be identified by their literal value or by the
identifier that names them. But if both forms are used to specify an
associativity and precedence then it's not clear which one is correct.
Make it an error if associativity is specified for a terminal by its
literal and its identifier.
