A common error is to write a grammar that doesn’t have an END
token. End tokens are needed when it is not clear when to stop
parsing. For example, when parsing the expression 3+5, it is
not clear after reading 3 whether to treat it as a complete
expression or whether the parser should continue reading.
Therefore the grammar for numeric expressions should include an end
token. Another example is the grammar for Lisp expressions. In
Lisp, it is always clear when you should stop parsing, so you do
not need an end token. In fact, it may be more useful not
to have an end token, so that you can read in several Lisp expressions.
If there is a chance of ambiguity, make sure to put the choices
in the order you want them checked. Usually the most specific
choice should be first. Empty sequences should usually be last.
The context sensitive scanner is not appropriate for all
grammars. You might try using the insensitive scanner with the
context-insensitive-scanner option in the grammar.
If performance turns out to be a problem, try writing a custom
scanner. The Yapps scanner is rather slow (but flexible and easy
to understand).
