Skip to content

perf: drop the list-form probes in ExpressionList - #2877

Merged
manticore-projects merged 1 commit into
JSQLParser:masterfrom
knutwannheden:perf/expression-list-lookahead
Oct 31, 2026
Merged

manticore-projects merged 1 commit into
JSQLParser:masterfrom
knutwannheden:perf/expression-list-lookahead

Conversation

@knutwannheden

@knutwannheden knutwannheden commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

After #2876, the largest remaining lookahead cost in parsing performance.sql sits in the callers of ComplexExpressionList(): ExpressionList() ran three 3-token LOOKAHEAD(3) probes to choose between the complex, simple and parenthesed list forms, and ParenthesedExpressionList() scanned into the list to decide whether its body is empty. Together they took about 5% of parser CPU, net of lexing, and both decisions have a token-level answer.

The three probes in ExpressionList() decided nothing, because ComplexExpressionList() accepts every list the other two forms do:

  • Expression() accepts every SimpleExpression(), so the simple form is a subset.
  • Its probe walks Expression() → PrimaryExpression() → "(" → ParenthesedExpressionList(), the same tokens the parenthesed probe walks, and PrimaryExpression()'s subselect alternatives are semantic guards that hold inside a scan as well.

So the complex alternative won wherever either of the others could have, and the fallback alternative behind the probes, added in ca2b3d7 for bracketed operands such as ( 1 + a ) / ( 1 + b ), was the same production once more. ExpressionList() now calls ComplexExpressionList() directly.

ParenthesedExpressionList()'s optional body is now [ LOOKAHEAD({ getToken(1).kind != CLOSING_BRACKET }) ComplexExpressionList() ]. Anything else that is not a list start fails inside Expression() instead of at the ), as in #2876. JavaCC drops that semantic guard inside nested scans, so the saving is on the parse path, where the profile samples were.

Which alternative wins matters for the AST: the block after the choice re-wraps a plain list holding one parenthesed list, but returns a parenthesed list holding one as it is. The reasoning above says the complex alternative already won in every case that parsed, and the full suite agrees.

FunctionTest.testBracketedOperandAndEmptyListArguments pins the two shapes only the removed alternatives looked responsible for, f((1 + a) / (1 + b), (c + d) * 2) and f(()), which ca2b3d7 shipped without a test, plus VALUES () for a list that reaches ExpressionList() outside a function call.

Parser-thread CPU per parse against #2876 (e8243dc), three interleaved pairs of JVMs, 200 warmup and 300 measured parses each:

pair #2876 this branch
1 29.05 ms 27.68 ms
2 28.80 ms 27.41 ms
3 29.15 ms 28.04 ms

./gradlew jmh -PjmhProfiler=gc on the same machine, #2876 first:

#2876 this branch 5.4
time 32.67 ± 1.17 ms/op 28.54 ± 0.70 ms/op (−12.6%) 49.22 / 47.43 ms/op
gc.alloc.rate.norm 7,166,820 B/op 7,165,980 B/op 7,767,975 / 7,771,902

The 5.4 control moved 3.6% between the two runs, so the CPU table above is the conservative figure; the direction is the same in both.

This branch is based on master after #2876 merged; the baseline column above was taken on #2876's branch commit e8243dc, which is the same tree as its merge commit a0e0861. It also rewords one comment that #2876 adds in ComplexExpressionList(), and the JMH pair was taken on an earlier iteration whose only difference is a fallback alternative that ran when the parser was interrupted.

`ExpressionList()` ran three 3-token lookaheads to choose between the
complex, simple and parenthesed list forms, and `ParenthesedExpressionList()`
scanned into the list on the parse path to tell an empty body from a
non-empty one. Together they took about 5% of parser CPU on
`performance.sql`.

The probes decided nothing: `ComplexExpressionList()` accepts every list
the other two forms do, including `()` and bracketed operands such as
`( 1 + a ) / ( 1 + b )`, so it won wherever they could have. It is now
called directly, and the parenthesed body is guarded by the token kind of
the closing bracket. Parser CPU per parse drops from 29.0 ms to 27.7 ms
(-4.5%, three interleaved pairs, on top of JSQLParser#2876), and `./gradlew jmh`
from 32.67 to 28.54 ms/op.
@knutwannheden
knutwannheden force-pushed the perf/expression-list-lookahead branch from 92bfd04 to 85690dc Compare October 29, 2026 11:30
@manticore-projects
manticore-projects merged commit f162cfe into JSQLParser:master Oct 31, 2026
10 checks passed
@manticore-projects

Copy link
Copy Markdown
Contributor

Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants