Discussion
Studying more of the eXpat XML parser as vendored into Papyrix...
The primary parser state might transition to a CData state. It invokes a helper propagating any errors before deciding whether to switch to the external-entity state or that primary state. Normalizes its input, iterates over the lexed XML tokens possibly invoking another helper to help clean up, & branches over results to clean up. Primarily its looks for cdata-sect-close tokens, converting others to text.
1/?
The 2nd state invokes a custom lexer & (via a helper with its lookuptables) normalizes its input & repeatedly branches over tokens with validation. Usually it practically ignores the tokens though it might hand off to the DTD parser. Afterwords it branches over the final token to determine which state to carefully hand off to next. This branch is *very* extensive!
Once the last tag's closed the primary state hands off to an epilog parser.
3/?
This epilog parser handles remaining tokens invoking callbacks for comments & processing instructions.
There's an entity processing util dispatching to the appropriate state. Including a state for carefully looking up internal entitites.
There's an error state which merely returns an error code.
There's helpers for gathering arrays of attributes. There's helpers for gathering entities to lookup. There's a line normalization helper. There's helpers wrapping the callbacks.
4/5!
There's helpers for managing a "context" hashmap. There's more normalization helpers. There's helpers for managing a "pool" of strings, arena-allocated. Helpers to manage some "scaffolding" datastructures.
There's accessors & string utils. And there's profiling counters with debug output.
4.5/4.5 Fin for today! Tomorrow: XML roles & tokenizing!
If the DTD build flag's set it there's a whole suite of states for parsing that DTD prologue. I won't discuss further, since I've got enough already!
There's a wrapper around callbacks for handling text encodings. There's a wrapper around one of 2 XML declaration lexers which additionally copies data into memory that's not about to be freed, while handling those text encodings.
There's another wrapper around the callbacks for unknown encodings.
These are used in the starting parser state.
2/?