3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
le king fu
|
c9872fc36b |
feat(import): recognise known bank layouts and report format drift
All checks were successful
PR Check — Frontend / frontend (pull_request) Successful in 1m41s
Two failure modes, both anchored on the header row.
A KNOWN BANK IS NOW READ BY NAME. `bankSignatures.ts` declares the
documented export layout of Desjardins, RBC, Banque Nationale and
Tangerine as a set of normalized header labels plus a delimiter and a
preamble quirk. The table is evaluated BEFORE the generic dictionary and
an unknown file falls straight through to it, unchanged.
The signatures are not decorative. The generic dictionary matches
keywords as substrings, one role at a time, which reads two of these
four layouts wrong — both frozen as counterfactual test pairs, same
rows, header renamed:
- Tangerine writes `Date,Transaction,Name,Memo,Amount`. `Transaction`
is a description keyword, so every row of the file was labelled with
its direction word instead of the merchant.
- RBC writes its amount column `CAD$`, which no amount keyword
matches, next to a nearly empty `Cheque Number`. Those two are
sparse-complementary, so the shape scan paired them as debit/credit
and the one row carrying a cheque number imported as -247.95 instead
of -6.95.
A signature stays a set of PREFERENCES all the same: they are written
from documented layouts, without real statements, so every hint is
dropped the moment the data contradicts it. The single exception is the
amount mode, which outranks the sparse-complementary scan — nothing
inside an RBC file can tell that pair from a genuine one — and even that
is refused unless the declared columns are candidates the shape scan
proposed. Failing degrades to the generic path; it never breaks.
FORMAT DRIFT IS NOW REPORTED INSTEAD OF IMPORTED. Every successful
import records the normalized labels of its header row in
`import_sources.header_signature`, as a JSON array and not a hash: the
panel has to be able to name the columns that moved. On the next import,
a header that normalizes differently opens a `FormatDriftPanel` above
the preview — column by column, `Montant : 3 -> 4` — with the two
outcomes that exist: adopt the re-detected format, or keep the stored
one. A cosmetic rename (`Montant` -> `MONTANT ($)`) normalizes
identically and says nothing.
A source whose file has no header row keeps `header_signature` NULL and
drift detection is inoperative on it. Documented, not worked around:
a signature invented from the data would fire on every import.
THE REPAIR PATH IS NOW IN THE INTERFACE, in the drift panel and beside
the preview's sign flip. `findDuplicates` matches on date AND
description AND amount, so re-importing a file "now that it reads right"
does not correct the rows already written — it doubles them, and a
flipped sign produces mirror pairs that net to zero in every report. The
only safe path is deleting the faulty import from the history first.
The drift re-detection reuses `detectFormatForFile`, so there is still
exactly one detector; the static guard on its caller count moves from
two to three deliberately. Its score and bank badge are dropped straight
after: they measure the format the panel offers, not the one in use.
Resolves #330
|
||
|
le king fu
|
c88ebd862e |
test: freeze CSV detection and amount parsing behaviour on a fixture corpus
All checks were successful
PR Check — Frontend / frontend (pull_request) Successful in 1m43s
First link of the import-format stack. `autoDetectConfig`, `detectAmountMode`, `detectSingleAmount`, `preprocessQuotedCSV` and `parseFrenchAmount` had no test at all on a suite of 871, while carrying every imported amount. This adds the reference the rewrite (#323-#332) measures itself against, before any of it moves. Corpus — 11 synthetic files under `src/__fixtures__/csv/`, no real statement data, covering shapes a single real statement never contains at once: signed amount, debit/credit, debit/credit in reversed column order, unused column filled with `0,00`, preamble before the header, header carrying a number, header cell starting with digits, no header row, all-positive amounts, whole-line-quoted (Desjardins style), absolute amount + D/C indicator. Every expectation was derived by running the code, not by reading it. Four cases are frozen as DEFECTIVE, each named `KNOWN DEFECT` with the right answer in a comment and the issue that owes the fix: - reversed debit/credit — the pair is assigned by column position, never by label, so every sign is inverted while the total merely negates (#328) - unused column at `0,00` — the rule branches on `isNaN(credit)` and `"0,00"` parses to 0, so every debit imports as zero (#325) - header cell starting with digits — `parseFloat` returns the numeric prefix, `detectHeader` reads the header as data (#328/#325) - absolute amount + D/C indicator — the indicator column is ignored entirely and every credit imports as an expense (#328) `parseFrenchAmount` is pinned form by form, including the prefix-scan defect the spec flagged: `"100,00 CAD"` yields 10000 and passes `isNaN`. Per the /review-spec revision, the assertions reach the holdings call sites too, which surfaced the same x100 leak in the #245 holdings import — a price cell of `"150,25 CAD"` stores the position at 15025. The end-to-end tests replay the wizard's row-mapping rule from a mirror, since it still lives inside a `useCallback` and the repo has no jsdom. A guard test asserts the production expressions are still literally present, so the mirror cannot drift; #325 extracts `mapRow` and must then delete it. No production file is modified. 924 vitest green (was 871), build clean. Resolves #326 |
||
|
le king fu
|
12d1877870 |
test(categories): complete test coverage for migration flow (#123)
Adds unit + integration + regression tests and a QA checklist for the v2→v1 seed migration feature. - Fixtures: src/__fixtures__/profiles.ts (makeV2Profile, makeV1Profile, makeV2ProfileWithCustom) with realistic categories, keywords, suppliers, transactions, budgets. - Unit: categoryMappingService (100 cases covering every DEFAULT_MAPPINGS entry, 4-pass priority, splits, preserved/custom detection), categoryBackupService (23 cases — Tauri mocks: success, write error, integrity check, PIN-encrypted profile), categoryMigrationService (16 cases — BEGIN/COMMIT/ROLLBACK flow, backup-missing abort, journaling, custom parent creation). - Integration: full plan→backup→migrate→verify flow; rollback via SREF import; backup failure → no DB write; migration SQL failure → ROLLBACK + intact state. - Regression: parameterised v2/v1 fixtures covering auto-categorisation, budget aggregation, splits preservation. - Docs: docs/qa-refonte-seed-categories-ipc.md — manual checklist for UX, system errors, encrypted profile, custom preservation, 90-day banner, restore flow. 331 vitest tests pass (up from 193 baseline). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |