Compare commits

...

1 commit

Author SHA1 Message Date
le king fu
6c95e61d7f test: freeze CSV detection and amount parsing behaviour on a fixture corpus
All checks were successful
PR Check — Frontend / frontend (pull_request) Successful in 1m42s
First link of the import-format stack. `autoDetectConfig`, `detectAmountMode`,
`detectSingleAmount`, `preprocessQuotedCSV` and `parseFrenchAmount` had no test
at all on a suite of 871, while carrying every imported amount. This adds the
reference the rewrite (#323-#332) measures itself against, before any of it
moves.

Corpus — 11 synthetic files under `src/__fixtures__/csv/`, no real statement
data, covering shapes a single real statement never contains at once: signed
amount, debit/credit, debit/credit in reversed column order, unused column
filled with `0,00`, preamble before the header, header carrying a number,
header cell starting with digits, no header row, all-positive amounts,
whole-line-quoted (Desjardins style), absolute amount + D/C indicator.

Every expectation was derived by running the code, not by reading it. Four
cases are frozen as DEFECTIVE, each named `KNOWN DEFECT` with the right answer
in a comment and the issue that owes the fix:

- reversed debit/credit — the pair is assigned by column position, never by
  label, so every sign is inverted while the total merely negates (#328)
- unused column at `0,00` — the rule branches on `isNaN(credit)` and `"0,00"`
  parses to 0, so every debit imports as zero (#325)
- header cell starting with digits — `parseFloat` returns the numeric prefix,
  `detectHeader` reads the header as data (#328/#325)
- absolute amount + D/C indicator — the indicator column is ignored entirely
  and every credit imports as an expense (#328)

`parseFrenchAmount` is pinned form by form, including the prefix-scan defect
the spec flagged: `"100,00 CAD"` yields 10000 and passes `isNaN`. Per the
/review-spec revision, the assertions reach the holdings call sites too, which
surfaced the same x100 leak in the #245 holdings import — a price cell of
`"150,25 CAD"` stores the position at 15025.

The end-to-end tests replay the wizard's row-mapping rule from a mirror, since
it still lives inside a `useCallback` and the repo has no jsdom. A guard test
asserts the production expressions are still literally present, so the mirror
cannot drift; #325 extracts `mapRow` and must then delete it.

No production file is modified. 924 vitest green (was 871), build clean.

Resolves #326
2026-08-13 12:33:45 -04:00
14 changed files with 807 additions and 1 deletions

View file

@ -0,0 +1,7 @@
Date;Description;Montant;Sens
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32;D
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00;C
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18;D
22/01/2025;RESTAURANT LE BISTRO;56,75;D
27/01/2025;VIREMENT RECU;300,00;C
31/01/2025;FRAIS MENSUELS;6,95;D
1 Date Description Montant Sens
2 05/01/2025 EPICERIE METRO SAINTE-FOY 84,32 D
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00 C
4 18/01/2025 HYDRO QUEBEC PREAUTORISE 142,18 D
5 22/01/2025 RESTAURANT LE BISTRO 56,75 D
6 27/01/2025 VIREMENT RECU 300,00 C
7 31/01/2025 FRAIS MENSUELS 6,95 D

View file

@ -0,0 +1,7 @@
Date;Description;Montant
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18
22/01/2025;RESTAURANT LE BISTRO;56,75
27/01/2025;VIREMENT RECU;300,00
31/01/2025;FRAIS MENSUELS;6,95
1 Date Description Montant
2 05/01/2025 EPICERIE METRO SAINTE-FOY 84,32
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00
4 18/01/2025 HYDRO QUEBEC PREAUTORISE 142,18
5 22/01/2025 RESTAURANT LE BISTRO 56,75
6 27/01/2025 VIREMENT RECU 300,00
7 31/01/2025 FRAIS MENSUELS 6,95

View file

@ -0,0 +1,7 @@
Date;Description;Credit;Debit
05/01/2025;EPICERIE METRO SAINTE-FOY;;84,32
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00;
18/01/2025;HYDRO QUEBEC PREAUTORISE;;142,18
22/01/2025;RESTAURANT LE BISTRO;;56,75
27/01/2025;VIREMENT RECU;300,00;
31/01/2025;FRAIS MENSUELS;;6,95
1 Date Description Credit Debit
2 05/01/2025 EPICERIE METRO SAINTE-FOY 84,32
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00
4 18/01/2025 HYDRO QUEBEC PREAUTORISE 142,18
5 22/01/2025 RESTAURANT LE BISTRO 56,75
6 27/01/2025 VIREMENT RECU 300,00
7 31/01/2025 FRAIS MENSUELS 6,95

View file

@ -0,0 +1,7 @@
Date;Description;Debit;Credit
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32;
15/01/2025;DEPOT PAIE EMPLOYEUR;;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18;
22/01/2025;RESTAURANT LE BISTRO;56,75;
27/01/2025;VIREMENT RECU;;300,00
31/01/2025;FRAIS MENSUELS;6,95;
1 Date Description Debit Credit
2 05/01/2025 EPICERIE METRO SAINTE-FOY 84,32
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00
4 18/01/2025 HYDRO QUEBEC PREAUTORISE 142,18
5 22/01/2025 RESTAURANT LE BISTRO 56,75
6 27/01/2025 VIREMENT RECU 300,00
7 31/01/2025 FRAIS MENSUELS 6,95

View file

@ -0,0 +1,7 @@
"Date,""Description"",Montant"
"05/01/2025,""EPICERIE METRO SAINTE-FOY"",-84.32"
"15/01/2025,""DEPOT PAIE EMPLOYEUR"",1250.00"
"18/01/2025,""HYDRO QUEBEC PREAUTORISE"",-142.18"
"22/01/2025,""RESTAURANT LE BISTRO"",-56.75"
"27/01/2025,""VIREMENT RECU"",300.00"
"31/01/2025,""FRAIS MENSUELS"",-6.95"
1 Date,"Description",Montant
2 05/01/2025,"EPICERIE METRO SAINTE-FOY",-84.32
3 15/01/2025,"DEPOT PAIE EMPLOYEUR",1250.00
4 18/01/2025,"HYDRO QUEBEC PREAUTORISE",-142.18
5 22/01/2025,"RESTAURANT LE BISTRO",-56.75
6 27/01/2025,"VIREMENT RECU",300.00
7 31/01/2025,"FRAIS MENSUELS",-6.95

View file

@ -0,0 +1,7 @@
Date;Description;2025 Montant
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
22/01/2025;RESTAURANT LE BISTRO;-56,75
27/01/2025;VIREMENT RECU;300,00
31/01/2025;FRAIS MENSUELS;-6,95
1 Date Description 2025 Montant
2 05/01/2025 EPICERIE METRO SAINTE-FOY -84,32
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00
4 18/01/2025 HYDRO QUEBEC PREAUTORISE -142,18
5 22/01/2025 RESTAURANT LE BISTRO -56,75
6 27/01/2025 VIREMENT RECU 300,00
7 31/01/2025 FRAIS MENSUELS -6,95

View file

@ -0,0 +1,7 @@
Date;Description;Montant;Solde 2024
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32;1415,68
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00;2665,68
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18;2523,50
22/01/2025;RESTAURANT LE BISTRO;-56,75;2466,75
27/01/2025;VIREMENT RECU;300,00;2766,75
31/01/2025;FRAIS MENSUELS;-6,95;2759,80
1 Date Description Montant Solde 2024
2 05/01/2025 EPICERIE METRO SAINTE-FOY -84,32 1415,68
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00 2665,68
4 18/01/2025 HYDRO QUEBEC PREAUTORISE -142,18 2523,50
5 22/01/2025 RESTAURANT LE BISTRO -56,75 2466,75
6 27/01/2025 VIREMENT RECU 300,00 2766,75
7 31/01/2025 FRAIS MENSUELS -6,95 2759,80

View file

@ -0,0 +1,54 @@
/**
* Synthetic CSV corpus for the import-format work (issue #326).
*
* Every file in this directory is INVENTED. No real bank statement, no real
* account number, no real transaction ever lands in the repository the app is
* privacy-first and the corpus has to be readable by anyone. Synthetic is not a
* compromise here: these files deliberately cover shapes a single real statement
* never contains at once (a reversed debit/credit pair, an unused column filled
* with `0,00`, a header cell carrying a number), which is precisely the point.
*
* The corpus is the reference the detection rewrite (#323-#332) measures itself
* against. Read `csvAutoDetect.test.ts` for the frozen contract, including the
* cases that are frozen as DEFECTIVE.
*/
import { readFileSync } from "fs";
import { resolve } from "path";
/** Stable identifier of each corpus case. */
export type CsvFixtureName =
| "signed-amount"
| "debit-credit"
| "debit-credit-reversed"
| "unused-column-zero"
| "preamble"
| "header-with-number"
| "header-numeric-label"
| "no-header"
| "all-positive"
| "desjardins-quoted"
| "absolute-indicator";
/** Every case in the corpus, in the order the issue lists them. */
export const CSV_FIXTURE_NAMES: readonly CsvFixtureName[] = [
"signed-amount",
"debit-credit",
"debit-credit-reversed",
"unused-column-zero",
"preamble",
"header-with-number",
"header-numeric-label",
"no-header",
"all-positive",
"desjardins-quoted",
"absolute-indicator",
];
/**
* Read one fixture as raw text, exactly as `useImportWizard` receives it from
* the file-reading Tauri command no trimming, no normalisation.
*/
export function readCsvFixture(name: CsvFixtureName): string {
return readFileSync(resolve(import.meta.dirname, `${name}.csv`), "utf-8");
}

View file

@ -0,0 +1,6 @@
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
22/01/2025;RESTAURANT LE BISTRO;-56,75
27/01/2025;VIREMENT RECU;300,00
31/01/2025;FRAIS MENSUELS;-6,95
1 05/01/2025 EPICERIE METRO SAINTE-FOY -84,32
2 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00
3 18/01/2025 HYDRO QUEBEC PREAUTORISE -142,18
4 22/01/2025 RESTAURANT LE BISTRO -56,75
5 27/01/2025 VIREMENT RECU 300,00
6 31/01/2025 FRAIS MENSUELS -6,95

View file

@ -0,0 +1,10 @@
RELEVE DE COMPTE
Compte cheques 12345-6789
Periode couverte du 01 janvier 2025 au 31 janvier 2025
Date;Description;Montant
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
22/01/2025;RESTAURANT LE BISTRO;-56,75
27/01/2025;VIREMENT RECU;300,00
31/01/2025;FRAIS MENSUELS;-6,95
1 RELEVE DE COMPTE
2 Compte cheques 12345-6789
3 Periode couverte du 01 janvier 2025 au 31 janvier 2025
4 Date;Description;Montant
5 05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
6 15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
7 18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
8 22/01/2025;RESTAURANT LE BISTRO;-56,75
9 27/01/2025;VIREMENT RECU;300,00
10 31/01/2025;FRAIS MENSUELS;-6,95

View file

@ -0,0 +1,7 @@
Date;Description;Montant
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
22/01/2025;RESTAURANT LE BISTRO;-56,75
27/01/2025;VIREMENT RECU;300,00
31/01/2025;FRAIS MENSUELS;-6,95
1 Date Description Montant
2 05/01/2025 EPICERIE METRO SAINTE-FOY -84,32
3 15/01/2025 DEPOT PAIE EMPLOYEUR 1250,00
4 18/01/2025 HYDRO QUEBEC PREAUTORISE -142,18
5 22/01/2025 RESTAURANT LE BISTRO -56,75
6 27/01/2025 VIREMENT RECU 300,00
7 31/01/2025 FRAIS MENSUELS -6,95

View file

@ -0,0 +1,7 @@
Date;Description;Debit;Credit
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32;0,00
15/01/2025;DEPOT PAIE EMPLOYEUR;0,00;1250,00
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18;0,00
22/01/2025;RESTAURANT LE BISTRO;56,75;0,00
27/01/2025;VIREMENT RECU;0,00;300,00
31/01/2025;FRAIS MENSUELS;6,95;0,00
1 Date Description Debit Credit
2 05/01/2025 EPICERIE METRO SAINTE-FOY 84,32 0,00
3 15/01/2025 DEPOT PAIE EMPLOYEUR 0,00 1250,00
4 18/01/2025 HYDRO QUEBEC PREAUTORISE 142,18 0,00
5 22/01/2025 RESTAURANT LE BISTRO 56,75 0,00
6 27/01/2025 VIREMENT RECU 0,00 300,00
7 31/01/2025 FRAIS MENSUELS 6,95 0,00

View file

@ -0,0 +1,194 @@
// amountParser — characterization tests (issue #326).
//
// `parseFrenchAmount` had no test at all, on a codebase of 871. It is called
// from 11 sites (8 in `csvAutoDetect.ts`, 3 in `useSnapshotEditor.ts`) and sits
// under every imported amount, so this file pins what it does TODAY, before
// issue #325 hardens it.
//
// ┌── HOW TO READ THIS FILE ─────────────────────────────────────────────────┐
// │ Two kinds of test live here, and the difference is deliberate: │
// │ │
// │ `describe("… contract")` — behaviour that must SURVIVE the rewrite. │
// │ Breaking one of these is a regression. │
// │ │
// │ `describe("… KNOWN DEFECT")` — behaviour that is WRONG today and is │
// │ expected to change in #325. Every such │
// │ assertion states the right answer in a │
// │ comment. When #325 lands, these tests are │
// │ meant to fail; the fix is to update the │
// │ expectation, not to delete the test. │
// └──────────────────────────────────────────────────────────────────────────┘
//
// The defect that motivated the whole chantier: `parseFrenchAmount` ends on
// `parseFloat`, which stops at the first invalid character instead of rejecting
// the string. A trailing unit or sign therefore yields a magnitude that is off
// by a factor of 100 — and it passes `isNaN`, so it is counted as a VALID row
// everywhere downstream. `"100,00 CAD"` does not fail; it imports as 10 000.
import { describe, it, expect } from "vitest";
import { parseFrenchAmount } from "./amountParser";
import { autoDetectConfig } from "./csvAutoDetect";
import { holdingsFromCsvRows } from "../hooks/useSnapshotEditor";
import { readCsvFixture } from "../__fixtures__/csv";
describe("parseFrenchAmount — separator contract (#326)", () => {
it("reads the French decimal comma", () => {
expect(parseFrenchAmount("1234,56")).toBe(1234.56);
expect(parseFrenchAmount("12,5")).toBe(12.5);
expect(parseFrenchAmount("0,00")).toBe(0);
});
it("reads French thousand separators (space, non-breaking space, dot)", () => {
expect(parseFrenchAmount("1 234,56")).toBe(1234.56);
expect(parseFrenchAmount("1\u00A0234,56")).toBe(1234.56); // non-breaking space
expect(parseFrenchAmount("1.234,56")).toBe(1234.56);
});
it("reads English notation", () => {
expect(parseFrenchAmount("1234.56")).toBe(1234.56);
expect(parseFrenchAmount("1,234.56")).toBe(1234.56);
});
it("keeps the sign of a leading minus", () => {
expect(parseFrenchAmount("-84,32")).toBe(-84.32);
expect(parseFrenchAmount(" -84,32 ")).toBe(-84.32);
});
it("strips currency symbols on either side", () => {
expect(parseFrenchAmount("1 250,00 $")).toBe(1250);
expect(parseFrenchAmount("$1,250.00")).toBe(1250);
expect(parseFrenchAmount("€84,32")).toBe(84.32);
expect(parseFrenchAmount("£84,32")).toBe(84.32);
});
it("rejects blank and non-numeric input", () => {
expect(parseFrenchAmount("")).toBeNaN();
expect(parseFrenchAmount(" ")).toBeNaN();
expect(parseFrenchAmount("abc")).toBeNaN();
expect(parseFrenchAmount("-")).toBeNaN();
expect(parseFrenchAmount("--5")).toBeNaN();
// Letter-leading labels are rejected — this is what keeps `detectHeader`
// working on a header such as "Solde 2024" (see the twin defect below).
expect(parseFrenchAmount("Solde 2024")).toBeNaN();
// Non-string input is guarded before any parsing.
expect(parseFrenchAmount(undefined as unknown as string)).toBeNaN();
expect(parseFrenchAmount(42 as unknown as string)).toBeNaN();
});
});
describe("parseFrenchAmount — KNOWN DEFECT: parseFloat prefix scan (#326, fixed by #325)", () => {
// `parseFloat` returns the longest valid PREFIX instead of rejecting the
// whole string. Every expectation below is the bug, not the intent.
//
// #325 must anchor the validation (regex over the whole normalized string)
// and return NaN on any residual character. All of these then become NaN,
// except the accounting-parenthesis and trailing-sign forms, which #325
// explicitly adds support for.
it("returns a x100 magnitude on a trailing sign — should be -50", () => {
// "50,00-" is the trailing-minus convention of several bank exports.
// The comma stops looking like a decimal separator once "-" trails it
// (the /,\d{1,2}$/ probe fails), so the comma is dropped as a thousands
// separator and "5000-" parses as 5000.
expect(parseFrenchAmount("50,00-")).toBe(5000);
});
it("returns a x100 magnitude on a trailing indicator — should be 1234.56", () => {
expect(parseFrenchAmount("1 234,56 CR")).toBe(123456);
expect(parseFrenchAmount("1 234,56 DB")).toBe(123456);
});
it("returns a x100 magnitude on a trailing currency code — should be 100", () => {
expect(parseFrenchAmount("100,00 CAD")).toBe(10000);
expect(parseFrenchAmount("84,32 USD")).toBe(8432);
});
it("accepts a numeric prefix of a text label — should be NaN", () => {
// This is what blinds `detectHeader`: a header cell that STARTS with
// digits reads as a number, so the header row is taken for data.
expect(parseFrenchAmount("2024 Montant")).toBe(2024);
expect(parseFrenchAmount("5%")).toBe(5);
});
it("accepts JavaScript number literals a bank never emits — should be NaN", () => {
expect(parseFrenchAmount("1e3")).toBe(1000);
expect(parseFrenchAmount("Infinity")).toBe(Infinity);
});
it("keeps only the first two groups of a malformed number — should be NaN", () => {
expect(parseFrenchAmount("1,2,3")).toBe(1.2);
});
});
describe("parseFrenchAmount — KNOWN DEFECT: unsupported accounting forms (#326, fixed by #325)", () => {
it("rejects accounting parentheses — should be -50", () => {
// "(50,00)" is the standard accounting notation for a negative amount.
// Today it is NaN, so the row is dropped as "Invalid amount" rather than
// being imported with the wrong sign — a loud failure, unlike the ones
// above, but still a failure.
expect(parseFrenchAmount("(50,00)")).toBeNaN();
expect(parseFrenchAmount("(1 234,56)")).toBeNaN();
});
});
describe("parseFrenchAmount — KNOWN DEFECT: separator arbitrated per cell (#326, fixed by #325)", () => {
// The French-vs-English decision is taken on each cell in isolation
// (`/,\d{1,2}$/`), never at column level. Two cells of the SAME column can
// therefore be read under two different conventions.
//
// #325 arbitrates the separator per column, which resolves both cases.
it("reads a 3-digit group after the comma as a thousands separator", () => {
// "12,345" is 12.345 in a column of decimals, 12345 in a column of
// thousands. Nothing in the cell alone can tell.
expect(parseFrenchAmount("12,345")).toBe(12345);
});
it("reads a dot as a decimal separator when no comma is present", () => {
// In a French column, "1.234" means 1234. Read alone it yields 1.234 —
// a x1000 error in the opposite direction from the cases above.
expect(parseFrenchAmount("1.234")).toBe(1.234);
expect(parseFrenchAmount("1.234,56")).toBe(1234.56); // ...unless a comma follows
});
});
describe("parseFrenchAmount — call-site fallout (#326)", () => {
// The /review-spec revision of #326 asks the corpus to reach the holdings
// call sites too, because #325 hardens the parser GLOBALLY. These pin what
// the shared parser does to its two most exposed consumers.
it("blinds detectHeader when a header cell starts with digits", () => {
// `detectHeader` (csvAutoDetect.ts:224) treats "parses as a number" as
// proof of a data row. "2025 Montant" parses as 2025, so the header is
// taken for data.
const cfg = autoDetectConfig(readCsvFixture("header-numeric-label"))!;
expect(cfg.hasHeader).toBe(false); // DEFECT — should be true
});
it("leaks the x100 magnitude into the holdings CSV import (#245)", () => {
// `holdingsFromCsvRows` (useSnapshotEditor.ts:191-202) shares the parser.
// A price column carrying its currency code silently multiplies every
// position by 100 — a $150.25 share is stored at $15,025.
const drafts = holdingsFromCsvRows(
[
["AAPL", "10", "150,25 CAD", "1 200,00"],
["MSFT", "5", "300,50", "1 400,00 CAD"],
],
{ symbol: 0, quantity: 1, unit_price: 2, book_cost: 3 }
);
expect(drafts[0].unit_price).toBe("15025"); // DEFECT — should be "150.25"
expect(drafts[0].book_cost).toBe("1200"); // clean cell, correct today
expect(drafts[1].unit_price).toBe("300.5"); // clean cell, correct today
expect(drafts[1].book_cost).toBe("140000"); // DEFECT — should be 1400
});
it("drops an accounting-parenthesis price instead of reading it", () => {
const drafts = holdingsFromCsvRows(
[["GOOG", "2", "(140,10)", "280,20"]],
{ symbol: 0, quantity: 1, unit_price: 2, book_cost: 3 }
);
// NaN price is swallowed to an empty string — no error surfaces.
expect(drafts[0].unit_price).toBe(""); // DEFECT — should be "-140.10"
expect(drafts[0].quantity).toBe("2");
});
});

View file

@ -1,16 +1,32 @@
// csvAutoDetect — holdings-CSV detection tests (Issue #245). // csvAutoDetect — holdings-CSV detection tests (Issue #245) + transaction-CSV
// corpus contract (Issue #326).
// //
// Covers `autoDetectHoldingColumns` (column detection) and `analyzeHoldingsCsv` // Covers `autoDetectHoldingColumns` (column detection) and `analyzeHoldingsCsv`
// (delimiter/header + full analysis). These helpers are pure and DOM-free, in // (delimiter/header + full analysis). These helpers are pure and DOM-free, in
// line with the project's "test the extracted pure pieces" convention. The // line with the project's "test the extracted pure pieces" convention. The
// numeric parsing + duplicate-merge path lives in `holdingsFromCsvRows` // numeric parsing + duplicate-merge path lives in `holdingsFromCsvRows`
// (tested in useSnapshotEditor.test.ts). // (tested in useSnapshotEditor.test.ts).
//
// The second half of the file is the #326 corpus contract on the TRANSACTION
// flow (`autoDetectConfig` + `preprocessQuotedCSV`), which had no test at all.
import { describe, it, expect } from "vitest"; import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { resolve } from "path";
import Papa from "papaparse";
import { import {
autoDetectHoldingColumns, autoDetectHoldingColumns,
analyzeHoldingsCsv, analyzeHoldingsCsv,
autoDetectConfig,
preprocessQuotedCSV,
} from "./csvAutoDetect"; } from "./csvAutoDetect";
import { parseFrenchAmount } from "./amountParser";
import { parseDate } from "./dateParser";
import {
CSV_FIXTURE_NAMES,
readCsvFixture,
type CsvFixtureName,
} from "../__fixtures__/csv";
describe("autoDetectHoldingColumns (#245)", () => { describe("autoDetectHoldingColumns (#245)", () => {
it("detects symbol/quantity/price/book_cost from EN headers", () => { it("detects symbol/quantity/price/book_cost from EN headers", () => {
@ -121,3 +137,466 @@ describe("analyzeHoldingsCsv (#245)", () => {
expect(analyzeHoldingsCsv(" \n \n")).toBeNull(); expect(analyzeHoldingsCsv(" \n \n")).toBeNull();
}); });
}); });
// =============================================================================
// Issue #326 — transaction-CSV corpus contract
// =============================================================================
//
// `autoDetectConfig`, `detectAmountMode`, `detectSingleAmount` and
// `preprocessQuotedCSV` had NO test, on a codebase of 871. This block freezes
// what they do TODAY, so the detection rewrite (#323-#332) measures itself
// against an established contract rather than against an intention.
//
// ┌── HOW TO READ THIS BLOCK ────────────────────────────────────────────────┐
// │ Every expectation is what the code DOES, verified by running it — never │
// │ what it should do. Tests whose name carries `KNOWN DEFECT` encode │
// │ behaviour that is wrong and is expected to change; each states the right │
// │ answer in a comment and names the issue that owes the fix. When that │
// │ issue lands, the test is meant to fail: update the expectation, do not │
// │ delete the test. │
// └──────────────────────────────────────────────────────────────────────────┘
/**
* MIRROR of the row-mapping rule in `useImportWizard.parseFilesInternal`
* (`useImportWizard.ts:489-542`), copied verbatim.
*
* It is duplicated here rather than imported because the rule currently lives
* inside a `useCallback` of a React hook and the repository has no jsdom.
* Issue #325 extracts it as a pure `mapRow(raw, format)` in
* `src/utils/importFormat.ts` AT THAT POINT THIS MIRROR MUST BE DELETED and
* the tests re-pointed at the real `mapRow`. Until then, the guard test
* "production rule still matches the mirror" below fails the build if the
* production expression drifts, so the mirror cannot silently start lying.
*/
function mapCorpusRow(
raw: string[],
cfg: NonNullable<ReturnType<typeof autoDetectConfig>>
): { date: string; description: string; amount: number } | { error: string } {
const date = parseDate(
raw[cfg.columnMapping.date]?.trim() || "",
cfg.dateFormat
);
const description = raw[cfg.columnMapping.description]?.trim() || "";
let amount: number;
if (cfg.amountMode === "debit_credit") {
const debit = parseFrenchAmount(raw[cfg.columnMapping.debitAmount ?? 0] || "");
const credit = parseFrenchAmount(
raw[cfg.columnMapping.creditAmount ?? 0] || ""
);
amount = isNaN(credit) ? -(isNaN(debit) ? 0 : debit) : credit;
} else {
amount = parseFrenchAmount(raw[cfg.columnMapping.amount ?? 0] || "");
if (cfg.signConvention === "positive_expense" && !isNaN(amount)) {
amount = -amount;
}
}
if (!date) return { error: "Invalid date" };
if (isNaN(amount)) return { error: "Invalid amount" };
return { date, description, amount };
}
/** Full pipeline: raw file text -> signed amounts, as the wizard runs it. */
function parseFixtureEndToEnd(name: CsvFixtureName) {
const rawContent = readCsvFixture(name);
const cfg = autoDetectConfig(rawContent);
if (!cfg) throw new Error(`autoDetectConfig returned null for ${name}`);
const data = Papa.parse(preprocessQuotedCSV(rawContent), {
delimiter: cfg.delimiter,
skipEmptyLines: true,
}).data as string[][];
const startIdx = cfg.skipLines + (cfg.hasHeader ? 1 : 0);
const rows = [];
for (let i = startIdx; i < data.length; i++) {
const raw = data[i];
if (raw.length <= 1 && raw[0]?.trim() === "") continue;
rows.push(mapCorpusRow(raw, cfg));
}
return { config: cfg, rows };
}
/** Signed amounts only — the number that actually reaches the database. */
function amountsOf(name: CsvFixtureName): (number | string)[] {
return parseFixtureEndToEnd(name).rows.map((r) =>
"error" in r ? r.error : r.amount
);
}
/** The amounts every well-detected fixture in the corpus must produce. */
const REFERENCE_AMOUNTS = [-84.32, 1250, -142.18, -56.75, 300, -6.95];
describe("corpus integrity (#326)", () => {
it("exposes every declared fixture as readable, non-empty content", () => {
expect(CSV_FIXTURE_NAMES).toHaveLength(11);
for (const name of CSV_FIXTURE_NAMES) {
expect(readCsvFixture(name).trim().length).toBeGreaterThan(0);
}
});
it("detects a configuration for every fixture (none returns null)", () => {
for (const name of CSV_FIXTURE_NAMES) {
expect(autoDetectConfig(readCsvFixture(name)), name).not.toBeNull();
}
});
it("production rule still matches the mirror in mapCorpusRow", () => {
// Static contract, in the style of __integration__/transactions-transfer-icon.
// `mapCorpusRow` is a hand copy of a rule that lives inside a React hook;
// this is what stops it drifting. #325 extracts the rule as `mapRow` —
// when it does, these matchers fail, and that IS the signal to delete the
// mirror and import the real function.
const SRC = readFileSync(
resolve(import.meta.dirname, "..", "hooks", "useImportWizard.ts"),
"utf-8"
);
expect(SRC).toContain(
"amount = isNaN(credit) ? -(isNaN(debit) ? 0 : debit) : credit;"
);
expect(SRC).toContain(
'if (config.signConvention === "positive_expense" && !isNaN(amount)) {'
);
// The `?? 0` fallbacks silently read column 0 when a mapping is missing.
// #325 replaces them with an explicit row error.
expect(SRC).toContain("raw[config.columnMapping.debitAmount ?? 0]");
expect(SRC).toContain("raw[config.columnMapping.creditAmount ?? 0]");
expect(SRC).toContain("raw[config.columnMapping.amount ?? 0]");
});
});
describe("autoDetectConfig — signed single amount (#326)", () => {
it("returns the full contract for a FR semicolon file", () => {
expect(autoDetectConfig(readCsvFixture("signed-amount"))).toEqual({
delimiter: ";",
hasHeader: true,
skipLines: 0,
dateFormat: "DD/MM/YYYY",
columnMapping: { date: 0, description: 1, amount: 2 },
amountMode: "single",
signConvention: "negative_expense",
});
});
it("parses end to end to the reference amounts", () => {
expect(amountsOf("signed-amount")).toEqual(REFERENCE_AMOUNTS);
});
it("carries the description through untouched", () => {
const { rows } = parseFixtureEndToEnd("signed-amount");
expect(rows[0]).toEqual({
date: "2025-01-05",
description: "EPICERIE METRO SAINTE-FOY",
amount: -84.32,
});
expect(rows[1]).toEqual({
date: "2025-01-15",
description: "DEPOT PAIE EMPLOYEUR",
amount: 1250,
});
});
});
describe("autoDetectConfig — debit/credit pair (#326)", () => {
it("detects the sparse-complementary pair as debit_credit", () => {
expect(autoDetectConfig(readCsvFixture("debit-credit"))).toEqual({
delimiter: ";",
hasHeader: true,
skipLines: 0,
dateFormat: "DD/MM/YYYY",
columnMapping: {
date: 0,
description: 1,
debitAmount: 2,
creditAmount: 3,
},
amountMode: "debit_credit",
signConvention: "negative_expense",
});
});
it("parses end to end to the reference amounts", () => {
// Debit -> negative, credit -> positive. Correct today because the file
// happens to list Debit before Credit — see the reversed fixture.
expect(amountsOf("debit-credit")).toEqual(REFERENCE_AMOUNTS);
});
});
describe("autoDetectConfig — KNOWN DEFECT: debit/credit order guessed by position (#326, fixed by #328)", () => {
// `detectAmountMode` assigns the LEFTMOST column of a complementary pair to
// `debitAmount` and the rightmost to `creditAmount` (csvAutoDetect.ts:461-470).
// It never looks at the header labels. A file laid out `Credit;Debit` is
// therefore mapped backwards, and every sign in the import is inverted.
//
// #328 resolves the order from the header dictionary instead of the position.
it("maps Credit to debitAmount and Debit to creditAmount", () => {
const cfg = autoDetectConfig(readCsvFixture("debit-credit-reversed"))!;
// Column 2 is labelled "Credit" and column 3 "Debit" in the fixture.
expect(cfg.columnMapping.debitAmount).toBe(2); // DEFECT — should be 3
expect(cfg.columnMapping.creditAmount).toBe(3); // DEFECT — should be 2
expect(cfg.amountMode).toBe("debit_credit");
});
it("inverts every sign end to end — groceries become income", () => {
expect(amountsOf("debit-credit-reversed")).toEqual(
// DEFECT — should be REFERENCE_AMOUNTS
[84.32, -1250, 142.18, 56.75, -300, 6.95]
);
});
it("still nets to the same total, which is why the bug hides", () => {
// The signs are wrong one by one, but the sum is merely negated — no
// aggregate check catches it. Only a per-row review does.
const sum = (xs: (number | string)[]) =>
xs.reduce<number>((a, b) => a + (typeof b === "number" ? b : 0), 0);
expect(sum(amountsOf("debit-credit-reversed"))).toBeCloseTo(
-sum(REFERENCE_AMOUNTS),
6
);
});
});
describe("autoDetectConfig — KNOWN DEFECT: unused column filled with 0,00 (#326, fixed by #325)", () => {
// When the unused half of a debit/credit pair carries "0,00" instead of an
// empty cell, detection still gets the mode right — `isSparseComplementary`
// treats a 0 as absent. The PARSING is what breaks: the rule branches on
// `isNaN(credit)` (useImportWizard.ts:509), and "0,00" parses to 0, not NaN.
// Every debit row therefore imports as 0,00 and the expense vanishes.
//
// #325 replaces the nullity comparison with `amount = credit - debit` on
// magnitudes and treats a 0,00 cell as absent.
it("still detects the debit_credit mode correctly", () => {
const cfg = autoDetectConfig(readCsvFixture("unused-column-zero"))!;
expect(cfg.amountMode).toBe("debit_credit");
expect(cfg.columnMapping.debitAmount).toBe(2);
expect(cfg.columnMapping.creditAmount).toBe(3);
});
it("swallows every debit to zero end to end", () => {
expect(amountsOf("unused-column-zero")).toEqual(
// DEFECT — should be REFERENCE_AMOUNTS
[0, 1250, 0, 0, 300, 0]
);
});
it("keeps the credits intact, so only the expenses disappear", () => {
const amounts = amountsOf("unused-column-zero");
expect(amounts.filter((a) => a === 0)).toHaveLength(4);
expect(amounts.filter((a) => a !== 0)).toEqual([1250, 300]);
});
});
describe("autoDetectConfig — preamble before the header (#326)", () => {
it("counts the preamble lines into skipLines", () => {
expect(autoDetectConfig(readCsvFixture("preamble"))).toEqual({
delimiter: ";",
hasHeader: true,
skipLines: 3,
dateFormat: "DD/MM/YYYY",
columnMapping: { date: 0, description: 1, amount: 2 },
amountMode: "single",
signConvention: "negative_expense",
});
});
it("parses end to end to the reference amounts", () => {
expect(amountsOf("preamble")).toEqual(REFERENCE_AMOUNTS);
});
});
describe("autoDetectConfig — header carrying a number (#326)", () => {
// Two fixtures, opposite outcomes, same root cause — `detectHeader` decides
// "this row is data" from `parseFrenchAmount` alone (csvAutoDetect.ts:224).
it("is unharmed when the numeric header cell starts with a letter", () => {
// "Solde 2024" normalizes to "Solde2024"; parseFloat stops at the 'S' and
// returns NaN, so the header is correctly recognised. The running-balance
// column is also correctly excluded from the amount candidates.
expect(autoDetectConfig(readCsvFixture("header-with-number"))).toEqual({
delimiter: ";",
hasHeader: true,
skipLines: 0,
dateFormat: "DD/MM/YYYY",
columnMapping: { date: 0, description: 1, amount: 2 },
amountMode: "single",
signConvention: "negative_expense",
});
});
it("parses the balance-column file end to end to the reference amounts", () => {
expect(amountsOf("header-with-number")).toEqual(REFERENCE_AMOUNTS);
});
});
describe("autoDetectConfig — KNOWN DEFECT: header cell starting with digits (#326, fixed by #328)", () => {
// "2025 Montant" normalizes to "2025Montant"; parseFloat returns the 2025
// prefix, `hasNumber` flips true, and the header row is taken for data.
//
// #328 adds a lexical signal to `detectHeader`, and #325 anchors the parser
// so a numeric prefix no longer reads as a number. Either fix closes this.
it("takes the header row for a data row", () => {
const cfg = autoDetectConfig(readCsvFixture("header-numeric-label"))!;
expect(cfg.hasHeader).toBe(false); // DEFECT — should be true
// The column mapping survives because the six real rows outvote the header.
expect(cfg.columnMapping).toEqual({ date: 0, description: 1, amount: 2 });
});
it("emits the header row as an error row instead of skipping it", () => {
const { rows } = parseFixtureEndToEnd("header-numeric-label");
expect(rows).toHaveLength(7); // DEFECT — should be 6
// The header survives as far as the date check, which is the only reason
// it fails loudly rather than importing 2025,00 as a transaction.
expect(rows[0]).toEqual({ error: "Invalid date" });
expect(amountsOf("header-numeric-label").slice(1)).toEqual(
REFERENCE_AMOUNTS
);
});
});
describe("autoDetectConfig — no header row (#326)", () => {
it("recognises a headerless file and maps it positionally", () => {
expect(autoDetectConfig(readCsvFixture("no-header"))).toEqual({
delimiter: ";",
hasHeader: false,
skipLines: 0,
dateFormat: "DD/MM/YYYY",
columnMapping: { date: 0, description: 1, amount: 2 },
amountMode: "single",
signConvention: "negative_expense",
});
});
it("parses end to end to the reference amounts, losing no first row", () => {
expect(amountsOf("no-header")).toEqual(REFERENCE_AMOUNTS);
});
});
describe("autoDetectConfig — KNOWN DEFECT: all-positive amounts (#326, fixed by #329)", () => {
// `detectSingleAmount` (csvAutoDetect.ts:507-530) infers the convention from
// the share of negative values alone: no negatives -> `positive_expense` ->
// every amount is negated at parse time. On a file that mixes expenses and
// income and signs neither, the income is imported as an expense.
//
// Nothing in the file distinguishes the two cases, so detection cannot be
// "fixed" here. #329 surfaces it instead: a confidence score, a mandatory
// preview showing signed totals, and a "flip the signs" button.
it("infers positive_expense from the absence of negatives", () => {
const cfg = autoDetectConfig(readCsvFixture("all-positive"))!;
expect(cfg.signConvention).toBe("positive_expense");
expect(cfg.amountMode).toBe("single");
});
it("negates the income rows along with the expenses", () => {
expect(amountsOf("all-positive")).toEqual([
-84.32, // expense — correct
-1250, // DEFECT — payroll deposit, should be +1250
-142.18, // expense — correct
-56.75, // expense — correct
-300, // DEFECT — incoming transfer, should be +300
-6.95, // expense — correct
]);
});
it("produces a total that is plainly wrong, which the preview will expose", () => {
const total = amountsOf("all-positive").reduce<number>(
(a, b) => a + (typeof b === "number" ? b : 0),
0
);
expect(total).toBeCloseTo(-1840.2, 2); // truth is +1259.80
});
});
describe("preprocessQuotedCSV + autoDetectConfig — whole-line-quoted file (#326)", () => {
it("unwraps the outer quotes and un-escapes the inner ones", () => {
const unwrapped = preprocessQuotedCSV(readCsvFixture("desjardins-quoted"));
expect(unwrapped.split("\n")[0]).toBe('Date,"Description",Montant');
expect(unwrapped.split("\n")[1]).toBe(
'05/01/2025,"EPICERIE METRO SAINTE-FOY",-84.32'
);
});
it("leaves an ordinary file untouched", () => {
const raw = readCsvFixture("signed-amount");
expect(preprocessQuotedCSV(raw)).toBe(raw);
});
it("only fires when every non-empty line is quoted AND contains a `,\"\"`", () => {
// The guard is narrow on purpose (csvAutoDetect.ts:37-40). A semicolon
// file can never match it, because the probe looks for a comma.
const semicolonQuoted = '"Date;""Desc"";Montant"\n"05/01/2025;""A"";-1,00"';
expect(preprocessQuotedCSV(semicolonQuoted)).toBe(semicolonQuoted);
});
it("detects the comma delimiter through the unwrapping", () => {
expect(autoDetectConfig(readCsvFixture("desjardins-quoted"))).toEqual({
delimiter: ",",
hasHeader: true,
skipLines: 0,
dateFormat: "DD/MM/YYYY",
columnMapping: { date: 0, description: 1, amount: 2 },
amountMode: "single",
signConvention: "negative_expense",
});
});
it("parses end to end to the reference amounts", () => {
expect(amountsOf("desjardins-quoted")).toEqual(REFERENCE_AMOUNTS);
});
});
describe("autoDetectConfig — KNOWN DEFECT: absolute amount + D/C indicator (#326, fixed by #328)", () => {
// The third amount mode: one column of unsigned magnitudes plus a
// single-letter D/C column carrying the direction. Nothing reads that
// column. Detection sees only positive numbers, picks `positive_expense`,
// and every credit is imported as an expense.
//
// #328 must DETECT this shape and REFUSE the file with a dedicated message,
// rather than import it backwards. The spec decision is explicit: refused,
// not silently mis-imported.
it("ignores the indicator column entirely", () => {
const cfg = autoDetectConfig(readCsvFixture("absolute-indicator"))!;
expect(cfg.amountMode).toBe("single");
expect(cfg.columnMapping.amount).toBe(2);
// Column 3 ("Sens", values D/C) appears nowhere in the mapping.
expect(Object.values(cfg.columnMapping)).not.toContain(3);
expect(cfg.signConvention).toBe("positive_expense");
});
it("imports both credit rows as expenses", () => {
const amounts = amountsOf("absolute-indicator");
// Rows 2 and 5 are flagged "C" in the fixture.
expect(amounts[1]).toBe(-1250); // DEFECT — should be +1250
expect(amounts[4]).toBe(-300); // DEFECT — should be +300
expect(amounts.every((a) => typeof a === "number" && a < 0)).toBe(true);
});
it("is currently indistinguishable from the all-positive fixture", () => {
// Same detected config, same wrong outcome — which is exactly why #328
// needs the indicator column as its discriminator.
const withIndicator = autoDetectConfig(readCsvFixture("absolute-indicator"))!;
const without = autoDetectConfig(readCsvFixture("all-positive"))!;
expect(withIndicator.amountMode).toBe(without.amountMode);
expect(withIndicator.signConvention).toBe(without.signConvention);
expect(withIndicator.columnMapping).toEqual(without.columnMapping);
});
});
describe("autoDetectConfig — degenerate input (#326)", () => {
it("returns null on content with fewer than two non-empty lines", () => {
expect(autoDetectConfig("")).toBeNull();
expect(autoDetectConfig("Date;Description;Montant")).toBeNull();
expect(autoDetectConfig(" \n \n")).toBeNull();
});
it("returns null when no column parses as a date", () => {
expect(
autoDetectConfig("Nom;Ville;Montant\nAlice;Quebec;10,00\nBob;Levis;20,00")
).toBeNull();
});
});