Repository navigation
perf: cut the lookahead scans in ComplexExpressionList - #2876
Merged
manticore-projects merged 1 commit intoOct 29, 2026
Merged
manticore-projects merged 1 commit into
manticore-projects merged 1 commit into
Conversation
`ComplexExpressionList()` parses every function argument list, and its lookaheads took 11% of parse CPU on `performance.sql`. The loop's `LOOKAHEAD(2) ","` scanned through the choice behind the comma, so every nested lookahead ran once in the scan and again when the choice was entered. And JavaCC scanned a token of `Expression()` before entering the default alternative of both choices, which a semantic lookahead on that alternative avoids. The loop now checks for a comma by token kind and the default alternatives enter `Expression()` directly. `SimpleExpressionList()` gets the same two changes. Parser CPU per parse drops from 33.5 ms to 29.8 ms (-11.5%, three interleaved pairs), and `./gradlew jmh` from 39.31 to 32.06 ms/op.
Contributor
|
I love those performance contributions, thank you so much! |
Contributor
Author
One more incoming. |
knutwannheden
added a commit
to knutwannheden/JSqlParser
that referenced
this pull request
Oct 29, 2026
`ExpressionList()` ran three 3-token lookaheads to choose between the complex, simple and parenthesed list forms, and `ParenthesedExpressionList()` scanned into the list on the parse path to tell an empty body from a non-empty one. Together they took about 5% of parser CPU on `performance.sql`. The probes decided nothing: `ComplexExpressionList()` accepts every list the other two forms do, including `()` and bracketed operands such as `( 1 + a ) / ( 1 + b )`, so it won wherever they could have. It is now called directly, and the parenthesed body is guarded by the token kind of the closing bracket. Parser CPU per parse drops from 29.0 ms to 27.7 ms (-4.5%, three interleaved pairs, on top of JSQLParser#2876), and `./gradlew jmh` from 32.67 to 28.54 ms/op.
manticore-projects
pushed a commit
that referenced
this pull request
Oct 31, 2026
`ExpressionList()` ran three 3-token lookaheads to choose between the complex, simple and parenthesed list forms, and `ParenthesedExpressionList()` scanned into the list on the parse path to tell an empty body from a non-empty one. Together they took about 5% of parser CPU on `performance.sql`. The probes decided nothing: `ComplexExpressionList()` accepts every list the other two forms do, including `()` and bracketed operands such as `( 1 + a ) / ( 1 + b )`, so it won wherever they could have. It is now called directly, and the parenthesed body is guarded by the token kind of the closing bracket. Parser CPU per parse drops from 29.0 ms to 27.7 ms (-4.5%, three interleaved pairs, on top of #2876), and `./gradlew jmh` from 32.67 to 28.54 ms/op.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parsing
performance.sql, the JMH benchmark's input, spends 43% of parser CPU in lookahead scans once lexing is factored out, and a single production accounts for a quarter of that:ComplexExpressionList(), which parses every function argument list. A CPU profile (async-profiler, cross-checked with JFR) attributed 11% of all parse time to its lookaheads, per argument, not per statement.Two scans ran for every argument:
LOOKAHEAD(2, {!interrupted}) ","scanned through the choice behind the comma. The nested lookaheads of that choice ran once inside the scan and again when the choice was entered.Expression()before entering the default alternative of both choices.Condition()already avoids that withLOOKAHEAD({ !interrupted })on its last alternative.The loop now checks for a comma by token kind and both default alternatives enter
Expression()directly.SimpleExpressionList(), the sibling used for VALUES lists and as the fallback list form, gets the same two changes. Accepted syntax is unchanged: a comma followed by something that is not an argument still fails, insideExpression()instead of at the loop.The first draft also replaced the named-parameter lookaheads with a token-kind check and broke MySQL's
f((@b := 2)), whichMySqlSetStatementTestandUpdateTestcaught: theRelObjectNameExt()scan is what keeps@b :=out ofPostgresNamedFunctionParameter(). Those lookaheads are unchanged, and the generated code already short-circuits them on the token kind../gradlew jmh -PjmhProfiler=gcon the same machine, master a075737 first:Parser-thread CPU per parse, three interleaved pairs of JVMs, 200 warmup and 300 measured parses each:
The next lookahead costs are
ExpressionList()deciding between its three list forms andInternalFunction()aroundFunctionArgumentList(). Both are 3-token decisions between differently shaped productions, not scans that a token check replaces, so they are left as they are.