Repository navigation
learn: retrospective learnings - #908
peco-engineer-bot[bot] wants to merge 25 commits into
Conversation
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
peco-engineer-bot
Bot
added
the
engineer-bot-learning
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
![peco-review-bot[bot]](https://avatars.githubusercontent.com/in/3801177?s=60&v=4){kind=small}
There was a problem hiding this comment.
Verdict: 1 Low
Documentation-only PR appending auto-generated retrospective learning sections — no executable code, APIs, or tests touched, so correctness/security/test axes don't apply. Structure, ordering, and sentinel line are intact. One Low data-quality note: three consecutive sections (2026-09-24, 2026-09-28, 2026-10-07) repeat the same since 2026-09-18T17:27:19Z watermark and re-describe PR #947, suggesting the generator's window cursor didn't advance between those runs.
| - **Context:** PR #947 fixed `RedactUrlQueryParamsFilter.filter` in `src/databricks/sql/__init__.py`, which crashed on log calls with no args (`record.args is None`) and carried a latent `record.arg[k]` NameError typo hidden in the never-exercised dict-args branch. | ||
| **Rule:** A `logging.Filter` that mutates `record.args` must handle all three shapes it can take — `None` (message logged with no args), a tuple, and a dict — and tests must exercise each shape (especially the dict path) since an untested branch masked a NameError. | ||
|
|
||
| ### 2026-09-28: learnings since 2026-09-18T17:27:19Z |
There was a problem hiding this comment.
🔵 Low — Three consecutive sections carry the same watermark learnings since 2026-09-18T17:27:19Z — ### 2026-09-24, ### 2026-09-28, and ### 2026-10-07 — and all three describe the same PR #947 RedactUrlQueryParamsFilter.filter fix in near-identical wording. The log's own convention is that each daily run appends learnings for a new window; a static since timestamp across three dated runs means the retrospective's cursor was not advanced after the 09-24 and 09-28 runs, so the same window (and the same PR) was redistilled and re-appended three times.
This is a generator/data-quality issue, not a code bug, but it leaves three redundant entries that future author-phase reads will treat as three distinct lessons about the same fix. Worth confirming the retrospective advances its knowledge_log cursor to the merge/run time after each run so windows don't overlap. The 2026-10-08 and 2026-10-13 sections correctly advance the watermark, so the issue appears isolated to this run of duplicates.
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — this PR only appends dated entries to a markdown learning log, so no code-correctness, error-handling, security, or test axes apply. One low data-quality note: PR #947 is recorded three times (2026-09-24 / 2026-09-28 / 2026-10-07) all under the same 2026-09-18T17:27:19Z watermark, indicating the same window was reprocessed and the learning triplicated.
| - **Context:** PR #947 fixed `RedactUrlQueryParamsFilter.filter` in `src/databricks/sql/__init__.py`, which crashed on log calls with no args (`record.args is None`) and carried a latent `record.arg[k]` NameError typo hidden in the never-exercised dict-args branch. | ||
| **Rule:** A `logging.Filter` that mutates `record.args` must handle all three shapes it can take — `None` (message logged with no args), a tuple, and a dict — and tests must exercise each shape (especially the dict path) since an untested branch masked a NameError. | ||
|
|
||
| ### 2026-09-28: learnings since 2026-09-18T17:27:19Z |
There was a problem hiding this comment.
🔵 Low — The same learning about PR #947's RedactUrlQueryParamsFilter.filter fix is recorded three times — in the 2026-09-24, 2026-09-28, and 2026-10-07 sections — and all three share the identical watermark learnings since 2026-09-18T17:27:19Z. The retrospective appears to have re-processed the same 2026-09-18 window on three consecutive runs, emitting a near-duplicate rule each time (all describe the record.args None/tuple/dict handling and the record.arg[k] typo). Since this log is read by the engineer author phase to steer future fixes (author.knowledge_log), the triplication adds redundant context and suggests the watermark wasn't advanced after the first emission. Consider collapsing to a single entry and confirming the retrospective's since-watermark advances past 2026-09-18T17:27:19Z so the window isn't reprocessed again.
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — this PR only appends dated sections to the bot-maintained retrospective knowledge log (.claude/knowledge/learning-log.md); no executable code changes, entries are well-formed and ordered oldest→newest above the sentinel line. One low-severity content note: the 2026-09-24 / 2026-09-28 / 2026-10-07 sections triplicate the same PR #947 learning under an identical, non-advancing since 2026-09-18T17:27:19Z watermark.
| - **Context:** PR #947 fixed `RedactUrlQueryParamsFilter.filter` in `src/databricks/sql/__init__.py`, which crashed on log calls with no args (`record.args is None`) and carried a latent `record.arg[k]` NameError typo hidden in the never-exercised dict-args branch. | ||
| **Rule:** A `logging.Filter` that mutates `record.args` must handle all three shapes it can take — `None` (message logged with no args), a tuple, and a dict — and tests must exercise each shape (especially the dict path) since an untested branch masked a NameError. | ||
|
|
||
| ### 2026-09-28: learnings since 2026-09-18T17:27:19Z |
There was a problem hiding this comment.
🔵 Low — The three sections dated 2026-09-24 (L127), 2026-09-28 (L131), and 2026-10-07 (L135) all carry the identical watermark learnings since 2026-09-18T17:27:19Z and all three re-state the same PR #947 RedactUrlQueryParamsFilter.filter learning with only cosmetic wording differences.
Everywhere else in the log the since timestamp advances run-to-run (e.g. the 2026-09-17 section is since 2026-09-16, the 2026-09-24 section since 2026-09-18). Three consecutive runs frozen at 2026-09-18T17:27:19Z indicates the retrospective's "last processed" watermark did not advance after the 2026-09-24 run, so the same already-captured PR #947 learning was re-harvested and re-appended twice more.
Impact is limited to log hygiene — the engineer author phase reads this log (per the file's own header / .bot/config.yaml author.knowledge_log), so the triplicate entry wastes context budget and over-weights one lesson. Worth a one-line cleanup (drop L131 and L135, keep the original at L127) and a look at why the pipeline's watermark stalled, since the same stall would re-duplicate future learnings. Not a connector-code defect; flagging as low.
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 3 Low
Looks fine as a docs-only learning-log update, but there are 3 low-severity duplicate-entry issues. The repeated PR #947 sections all share the same since watermark, which points to a retrospective cursor that isn't advancing.
| - **Context:** PR #947 fixed `RedactUrlQueryParamsFilter.filter` in `src/databricks/sql/__init__.py`, which crashed on log calls with no args (`record.args is None`) and carried a latent `record.arg[k]` NameError typo hidden in the never-exercised dict-args branch. | ||
| **Rule:** A `logging.Filter` that mutates `record.args` must handle all three shapes it can take — `None` (message logged with no args), a tuple, and a dict — and tests must exercise each shape (especially the dict path) since an untested branch masked a NameError. | ||
|
|
||
| ### 2026-09-28: learnings since 2026-09-18T17:27:19Z |
There was a problem hiding this comment.
🔵 Low — The PR #947 RedactUrlQueryParamsFilter learning appears three times, in the 2026-09-24, 2026-09-28 and 2026-10-07 sections. All three carry the same since 2026-09-18T17:27:19Z watermark and give nearly the same rule. This looks like the retrospective's watermark did not advance on those days, so it re-distilled the same PR each time. Keep only the 2026-09-24 entry. Separately, check the retrospective cursor logic so later runs don't keep re-adding learnings that are already in the log; the author phase reads this log, so duplicates make it noisier.
| - **Context:** Engineer-bot run #39665896492 fixed GH #989 in `src/databricks/sql/utils.py::_concat_arrow_tables`, which unconditionally called `pyarrow.concat_tables(..., promote_options="default")` (fallback `promote=True`) and failed on duplicate column names. | ||
| **Rule:** When concatenating pyarrow tables, only pass a `promote`/`promote_options` argument when chunk schemas actually differ — any promote option triggers Arrow schema unification, which raises `ArrowInvalid: Can't unify schema with duplicate field names` even for a single chunk; valid SQL (`SELECT 1 AS a, 2 AS a`, self-JOINs) legitimately produces duplicate column names, so short-circuit to a plain `concat_tables` when all chunk schemas equal the first, preserving promotion only for the genuine differing-schema (e.g. nullability) case. | ||
|
|
||
| ### 2026-10-24: learnings since 2026-10-23T17:31:38Z |
There was a problem hiding this comment.
🔵 Low — The 2026-10-24 entry for PR #995 says the same thing as the 2026-10-22 entry for run #39665896492. Both cover the _concat_arrow_tables duplicate-field-name / promote_options fix, and both rules boil down to "only promote when schemas differ." Merge them into one entry, citing both the run and the PR.
| **Rule:** SEA (`use_sea=True`) is the only available backend for RT/Lakehouse warehouse types (they refuse Thrift), so deprecation/steering messaging must frame the kernel backend as the recommended path *where supported* rather than an unconditional drop-in replacement for all SEA users. | ||
|
|
||
| ### 2026-08-21: learnings since 2026-08-20T17:35:31Z | ||
| - **Context:** In PR #922 a reviewer bot benchmarked kernel `row_limit=0` semantics against the SEA backend (`sea/models/requests.py`, which treats 0 as unlimited); a human corrected that SEA is deprecated and the kernel/server matches Thrift, which treats 0 as a real zero-row limit. |
There was a problem hiding this comment.
🔵 Low — The 2026-08-21 rule ("SEA is deprecated — use Thrift as the reference backend, not SEA") repeats the 2026-08-18 rule ("treat Thrift as the authoritative reference; SEA ... must NOT be used as an alignment target"). Consider removing the second copy, or merging the PR #922 context into the first entry.
Rolling retrospective learnings
This PR accumulates one dated section of learnings per day (from merged PRs and engineer-bot author runs) until it is merged; merging it starts a fresh one. The bot never writes the canonical log directly.
Latest update 2026-10-30: 3 new learning(s) since 2026-10-29T17:26:19Z.