Review finding on #327. detectDescriptionColumn returned the lexically
preferred column with no check on the data, unlike the date (replayed at 0.8)
and the amount (constrained to the shape candidates). The dictionary lists
'transaction' as a description keyword, and Tangerine exports
Date,Transaction,Name,Memo,Amount where Transaction holds DEBIT/CREDIT — so the
description moved off the merchant name and keyword categorisation died.
Cardinality tells free text from an enum: a description repeats almost nothing,
an enum repeats almost everything. Average length does not — Note and Libelle
are both short, so a length veto would reject legitimate columns.
This fixes the cause. #330 had rescued the case through the Tangerine signature
alone, leaving every unrecognised file with a Transaction column broken; that
test now asserts the correct mapping with and without a signature.
Refs #327
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two failure modes, both anchored on the header row.
A KNOWN BANK IS NOW READ BY NAME. `bankSignatures.ts` declares the
documented export layout of Desjardins, RBC, Banque Nationale and
Tangerine as a set of normalized header labels plus a delimiter and a
preamble quirk. The table is evaluated BEFORE the generic dictionary and
an unknown file falls straight through to it, unchanged.
The signatures are not decorative. The generic dictionary matches
keywords as substrings, one role at a time, which reads two of these
four layouts wrong — both frozen as counterfactual test pairs, same
rows, header renamed:
- Tangerine writes `Date,Transaction,Name,Memo,Amount`. `Transaction`
is a description keyword, so every row of the file was labelled with
its direction word instead of the merchant.
- RBC writes its amount column `CAD$`, which no amount keyword
matches, next to a nearly empty `Cheque Number`. Those two are
sparse-complementary, so the shape scan paired them as debit/credit
and the one row carrying a cheque number imported as -247.95 instead
of -6.95.
A signature stays a set of PREFERENCES all the same: they are written
from documented layouts, without real statements, so every hint is
dropped the moment the data contradicts it. The single exception is the
amount mode, which outranks the sparse-complementary scan — nothing
inside an RBC file can tell that pair from a genuine one — and even that
is refused unless the declared columns are candidates the shape scan
proposed. Failing degrades to the generic path; it never breaks.
FORMAT DRIFT IS NOW REPORTED INSTEAD OF IMPORTED. Every successful
import records the normalized labels of its header row in
`import_sources.header_signature`, as a JSON array and not a hash: the
panel has to be able to name the columns that moved. On the next import,
a header that normalizes differently opens a `FormatDriftPanel` above
the preview — column by column, `Montant : 3 -> 4` — with the two
outcomes that exist: adopt the re-detected format, or keep the stored
one. A cosmetic rename (`Montant` -> `MONTANT ($)`) normalizes
identically and says nothing.
A source whose file has no header row keeps `header_signature` NULL and
drift detection is inoperative on it. Documented, not worked around:
a signature invented from the data would fire on every import.
THE REPAIR PATH IS NOW IN THE INTERFACE, in the drift panel and beside
the preview's sign flip. `findDuplicates` matches on date AND
description AND amount, so re-importing a file "now that it reads right"
does not correct the rows already written — it doubles them, and a
flipped sign produces mirror pairs that net to zero in every report. The
only safe path is deleting the faulty import from the history first.
The drift re-detection reuses `detectFormatForFile`, so there is still
exactly one detector; the static guard on its caller count moves from
two to three deliberately. Its score and bank badge are dropped straight
after: they measure the format the panel offers, not the one in use.
Resolves#330
Detection handed back a configuration it had never tested, and only ever
ran behind the magic-wand button. A source opened for the first time
therefore started on `defaultConfig` — `;`, `DD/MM/YYYY`, columns 0/1/2 —
plausible enough to import a whole file wrong rather than fail visibly.
`detectImportFormat` now REPLAYS what it just decided over every data row
of the file and returns the rate as a `DetectionScore`. The replay runs
`mapRow`, the same pure function `parseFilesInternal` runs at import time,
under the same column-level decimal arbitration: two mappers would be the
exact divergence this chantier removes — a score reading 100 % while the
import wrote different amounts. A test asserts the two agree row for row
on every fixture of the corpus.
The threshold is 90 %, and it only colours the banner. Below it the panel
warns and prints the detailed count ("132 of 150 rows read"); at or above
it the banner is neutral. Nothing is blocked, because a perfect score says
nothing about the SIGN of what was read — `all-positive` scores 100 %
while every credit imports as an expense — and the preview step (#329) is
the real net, traversed at every import.
Detection now also fires on its own, guarded on `!existing`: a source that
has never been configured. A source that HAS one is never re-detected, the
stored format wins. The condition is deliberately not `!restored` — a
stored format that fails to decode already reports its own error, and
detecting over it would replace that message with a silent guess.
The button and the automatic run share one `detectFormatForFile`, so the
button replays detection instead of running a second, drifting variant of
it. The score is cleared when its source changes, when the format is
edited by hand (compared through the codec, so a rename keeps it) and when
a template overwrites the format — a banner vouching for a configuration
nobody measured is the misinformation it exists to remove.
Finally, the sign-convention selector is hidden in debit/credit mode.
`mapRow` computes `credit - debit` on magnitudes there and never reads
`signConvention`, so the control changed nothing. Hidden, not reset: the
stored value is left untouched.
Tests: 1054 vitest (+20), build and cargo check green. No DB migration.
CHANGELOG and docs are centralized in link 10 of the stack per the plan.
Resolves#328
Detection reasoned on the shape of the data alone, so it could not tell a debit
column from a credit one: a file laid out `Date;Description;Credit;Debit` was
mapped by position and every sign of the import came out inverted — silently,
since the total is merely negated and no aggregate check notices.
A new `headerDictionary.ts` carries the FR/EN transaction dictionary (date,
description, amount, debit, credit, balance) plus the two matching helpers,
moved out of `csvAutoDetect.ts` whose holdings tables stay untouched: `montant`
is an exclusion token there and the primary amount keyword here, so the two
tables cannot be merged. Moving the generic helpers rather than exporting them
keeps the dependency one-way.
`csvAutoDetect` puts that layer in front of the shape heuristics. Labels resolve
the debit/credit order, the date, description and single-amount columns, and
give `detectHeader` a second signal for a header row carrying a bare number.
Every hint is a preference the data can veto — a labelled date column must still
parse, a labelled balance column is never excluded if it would leave nothing to
map — and a mute file (no header row, unknown labels) falls back to the shape
heuristics unchanged.
Files pairing unsigned amounts with an adjacent D/C indicator column are now
detected and REFUSED with a dedicated message, instead of being configured as
`positive_expense` and importing every deposit as an expense. Detection reports
that reason through `detectImportFormat`; `autoDetectConfig` keeps its previous
shape for the callers that only need the configuration.
Resolves#327
A detailed (by-security) account can now import its positions from a CSV
instead of adding each security one by one. An "Import CSV" button next to
"Add a title" opens a native picker; the delimiter, encoding and columns
(symbol, quantity, optional price + book_cost) are auto-detected via new
csvAutoDetect helpers and adjustable in a small mapping editor. Rows sharing
a symbol are merged (SUM qty + book_cost, first price) and the batch is merged
into the basket by normalized symbol, so no UNIQUE(snapshot_line_id,
security_id) violation can occur at save. A CSV without a price column imports
quantities with an empty unit_price (fetch/type later) — the existing atomic
save path is unchanged (no SQL/Rust change).
- csvAutoDetect: autoDetectHoldingColumns + analyzeHoldingsCsv (+ tests)
- useSnapshotEditor: holdingsFromCsvRows + IMPORT_HOLDINGS action + importHoldings (+ tests)
- HoldingsCsvImportModal: file pick -> parse -> mapping editor + preview
- i18n FR/EN under balance.snapshot.detailed.importCsv.*
- CHANGELOG (Added / Ajouté)
Resolves#245
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a "Répartitions" section below manual adjustments listing all
split transactions. Clicking a split opens the existing modal to
view, edit, or delete it.
Fix CSV auto-detect failing on files with preamble lines (e.g.
Mastercard CSVs with metadata header). Three fixes:
- Delimiter detection uses mode of column counts instead of first-line
- Detect and skip preamble rows before header/data detection
- Exclude date-like columns from amount candidates and prefer columns
with decimal values when picking the amount column
Bumps version to 0.3.4.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix column display for Desjardins-style quoted CSVs (apply preprocessQuotedCSV in header loading)
- Fix column mapping disappearing on back-navigation (generate synthetic headers when hasHeader is false)
- Fix auto-detect picking account number as amount (exclude constant-value columns, treat 0 as empty in debit/credit detection)
- Use category type instead of amount sign for dashboard pie chart and recent transactions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Analyzes imported CSV files to automatically detect delimiter, header,
date format, column mapping, amount mode, and sign convention. Excludes
running balance columns from amount mapping via consecutive-row math.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
