test: freeze CSV detection and amount parsing behaviour on a fixture corpus
All checks were successful
PR Check — Frontend / frontend (pull_request) Successful in 1m43s
All checks were successful
PR Check — Frontend / frontend (pull_request) Successful in 1m43s
First link of the import-format stack. `autoDetectConfig`, `detectAmountMode`, `detectSingleAmount`, `preprocessQuotedCSV` and `parseFrenchAmount` had no test at all on a suite of 871, while carrying every imported amount. This adds the reference the rewrite (#323-#332) measures itself against, before any of it moves. Corpus — 11 synthetic files under `src/__fixtures__/csv/`, no real statement data, covering shapes a single real statement never contains at once: signed amount, debit/credit, debit/credit in reversed column order, unused column filled with `0,00`, preamble before the header, header carrying a number, header cell starting with digits, no header row, all-positive amounts, whole-line-quoted (Desjardins style), absolute amount + D/C indicator. Every expectation was derived by running the code, not by reading it. Four cases are frozen as DEFECTIVE, each named `KNOWN DEFECT` with the right answer in a comment and the issue that owes the fix: - reversed debit/credit — the pair is assigned by column position, never by label, so every sign is inverted while the total merely negates (#328) - unused column at `0,00` — the rule branches on `isNaN(credit)` and `"0,00"` parses to 0, so every debit imports as zero (#325) - header cell starting with digits — `parseFloat` returns the numeric prefix, `detectHeader` reads the header as data (#328/#325) - absolute amount + D/C indicator — the indicator column is ignored entirely and every credit imports as an expense (#328) `parseFrenchAmount` is pinned form by form, including the prefix-scan defect the spec flagged: `"100,00 CAD"` yields 10000 and passes `isNaN`. Per the /review-spec revision, the assertions reach the holdings call sites too, which surfaced the same x100 leak in the #245 holdings import — a price cell of `"150,25 CAD"` stores the position at 15025. The end-to-end tests replay the wizard's row-mapping rule from a mirror, since it still lives inside a `useCallback` and the repo has no jsdom. A guard test asserts the production expressions are still literally present, so the mirror cannot drift; #325 extracts `mapRow` and must then delete it. No production file is modified. 924 vitest green (was 871), build clean. Resolves #326
This commit is contained in:
parent
b30c9fa5c1
commit
c88ebd862e
14 changed files with 807 additions and 1 deletions
7
src/__fixtures__/csv/absolute-indicator.csv
Normal file
7
src/__fixtures__/csv/absolute-indicator.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Montant;Sens
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32;D
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00;C
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18;D
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;56,75;D
|
||||||
|
27/01/2025;VIREMENT RECU;300,00;C
|
||||||
|
31/01/2025;FRAIS MENSUELS;6,95;D
|
||||||
|
7
src/__fixtures__/csv/all-positive.csv
Normal file
7
src/__fixtures__/csv/all-positive.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Montant
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;56,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;6,95
|
||||||
|
7
src/__fixtures__/csv/debit-credit-reversed.csv
Normal file
7
src/__fixtures__/csv/debit-credit-reversed.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Credit;Debit
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;;84,32
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00;
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;;142,18
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;;56,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00;
|
||||||
|
31/01/2025;FRAIS MENSUELS;;6,95
|
||||||
|
7
src/__fixtures__/csv/debit-credit.csv
Normal file
7
src/__fixtures__/csv/debit-credit.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Debit;Credit
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32;
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18;
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;56,75;
|
||||||
|
27/01/2025;VIREMENT RECU;;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;6,95;
|
||||||
|
7
src/__fixtures__/csv/desjardins-quoted.csv
Normal file
7
src/__fixtures__/csv/desjardins-quoted.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
"Date,""Description"",Montant"
|
||||||
|
"05/01/2025,""EPICERIE METRO SAINTE-FOY"",-84.32"
|
||||||
|
"15/01/2025,""DEPOT PAIE EMPLOYEUR"",1250.00"
|
||||||
|
"18/01/2025,""HYDRO QUEBEC PREAUTORISE"",-142.18"
|
||||||
|
"22/01/2025,""RESTAURANT LE BISTRO"",-56.75"
|
||||||
|
"27/01/2025,""VIREMENT RECU"",300.00"
|
||||||
|
"31/01/2025,""FRAIS MENSUELS"",-6.95"
|
||||||
|
7
src/__fixtures__/csv/header-numeric-label.csv
Normal file
7
src/__fixtures__/csv/header-numeric-label.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;2025 Montant
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;-56,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;-6,95
|
||||||
|
7
src/__fixtures__/csv/header-with-number.csv
Normal file
7
src/__fixtures__/csv/header-with-number.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Montant;Solde 2024
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32;1415,68
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00;2665,68
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18;2523,50
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;-56,75;2466,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00;2766,75
|
||||||
|
31/01/2025;FRAIS MENSUELS;-6,95;2759,80
|
||||||
|
54
src/__fixtures__/csv/index.ts
Normal file
54
src/__fixtures__/csv/index.ts
Normal file
|
|
@ -0,0 +1,54 @@
|
||||||
|
/**
|
||||||
|
* Synthetic CSV corpus for the import-format work (issue #326).
|
||||||
|
*
|
||||||
|
* Every file in this directory is INVENTED. No real bank statement, no real
|
||||||
|
* account number, no real transaction ever lands in the repository — the app is
|
||||||
|
* privacy-first and the corpus has to be readable by anyone. Synthetic is not a
|
||||||
|
* compromise here: these files deliberately cover shapes a single real statement
|
||||||
|
* never contains at once (a reversed debit/credit pair, an unused column filled
|
||||||
|
* with `0,00`, a header cell carrying a number), which is precisely the point.
|
||||||
|
*
|
||||||
|
* The corpus is the reference the detection rewrite (#323-#332) measures itself
|
||||||
|
* against. Read `csvAutoDetect.test.ts` for the frozen contract, including the
|
||||||
|
* cases that are frozen as DEFECTIVE.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { readFileSync } from "fs";
|
||||||
|
import { resolve } from "path";
|
||||||
|
|
||||||
|
/** Stable identifier of each corpus case. */
|
||||||
|
export type CsvFixtureName =
|
||||||
|
| "signed-amount"
|
||||||
|
| "debit-credit"
|
||||||
|
| "debit-credit-reversed"
|
||||||
|
| "unused-column-zero"
|
||||||
|
| "preamble"
|
||||||
|
| "header-with-number"
|
||||||
|
| "header-numeric-label"
|
||||||
|
| "no-header"
|
||||||
|
| "all-positive"
|
||||||
|
| "desjardins-quoted"
|
||||||
|
| "absolute-indicator";
|
||||||
|
|
||||||
|
/** Every case in the corpus, in the order the issue lists them. */
|
||||||
|
export const CSV_FIXTURE_NAMES: readonly CsvFixtureName[] = [
|
||||||
|
"signed-amount",
|
||||||
|
"debit-credit",
|
||||||
|
"debit-credit-reversed",
|
||||||
|
"unused-column-zero",
|
||||||
|
"preamble",
|
||||||
|
"header-with-number",
|
||||||
|
"header-numeric-label",
|
||||||
|
"no-header",
|
||||||
|
"all-positive",
|
||||||
|
"desjardins-quoted",
|
||||||
|
"absolute-indicator",
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Read one fixture as raw text, exactly as `useImportWizard` receives it from
|
||||||
|
* the file-reading Tauri command — no trimming, no normalisation.
|
||||||
|
*/
|
||||||
|
export function readCsvFixture(name: CsvFixtureName): string {
|
||||||
|
return readFileSync(resolve(import.meta.dirname, `${name}.csv`), "utf-8");
|
||||||
|
}
|
||||||
6
src/__fixtures__/csv/no-header.csv
Normal file
6
src/__fixtures__/csv/no-header.csv
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;-56,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;-6,95
|
||||||
|
10
src/__fixtures__/csv/preamble.csv
Normal file
10
src/__fixtures__/csv/preamble.csv
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
RELEVE DE COMPTE
|
||||||
|
Compte cheques 12345-6789
|
||||||
|
Periode couverte du 01 janvier 2025 au 31 janvier 2025
|
||||||
|
Date;Description;Montant
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;-56,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;-6,95
|
||||||
|
7
src/__fixtures__/csv/signed-amount.csv
Normal file
7
src/__fixtures__/csv/signed-amount.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Montant
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;-84,32
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;-142,18
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;-56,75
|
||||||
|
27/01/2025;VIREMENT RECU;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;-6,95
|
||||||
|
7
src/__fixtures__/csv/unused-column-zero.csv
Normal file
7
src/__fixtures__/csv/unused-column-zero.csv
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
Date;Description;Debit;Credit
|
||||||
|
05/01/2025;EPICERIE METRO SAINTE-FOY;84,32;0,00
|
||||||
|
15/01/2025;DEPOT PAIE EMPLOYEUR;0,00;1250,00
|
||||||
|
18/01/2025;HYDRO QUEBEC PREAUTORISE;142,18;0,00
|
||||||
|
22/01/2025;RESTAURANT LE BISTRO;56,75;0,00
|
||||||
|
27/01/2025;VIREMENT RECU;0,00;300,00
|
||||||
|
31/01/2025;FRAIS MENSUELS;6,95;0,00
|
||||||
|
194
src/utils/amountParser.test.ts
Normal file
194
src/utils/amountParser.test.ts
Normal file
|
|
@ -0,0 +1,194 @@
|
||||||
|
// amountParser — characterization tests (issue #326).
|
||||||
|
//
|
||||||
|
// `parseFrenchAmount` had no test at all, on a codebase of 871. It is called
|
||||||
|
// from 11 sites (8 in `csvAutoDetect.ts`, 3 in `useSnapshotEditor.ts`) and sits
|
||||||
|
// under every imported amount, so this file pins what it does TODAY, before
|
||||||
|
// issue #325 hardens it.
|
||||||
|
//
|
||||||
|
// ┌── HOW TO READ THIS FILE ─────────────────────────────────────────────────┐
|
||||||
|
// │ Two kinds of test live here, and the difference is deliberate: │
|
||||||
|
// │ │
|
||||||
|
// │ `describe("… contract")` — behaviour that must SURVIVE the rewrite. │
|
||||||
|
// │ Breaking one of these is a regression. │
|
||||||
|
// │ │
|
||||||
|
// │ `describe("… KNOWN DEFECT")` — behaviour that is WRONG today and is │
|
||||||
|
// │ expected to change in #325. Every such │
|
||||||
|
// │ assertion states the right answer in a │
|
||||||
|
// │ comment. When #325 lands, these tests are │
|
||||||
|
// │ meant to fail; the fix is to update the │
|
||||||
|
// │ expectation, not to delete the test. │
|
||||||
|
// └──────────────────────────────────────────────────────────────────────────┘
|
||||||
|
//
|
||||||
|
// The defect that motivated the whole chantier: `parseFrenchAmount` ends on
|
||||||
|
// `parseFloat`, which stops at the first invalid character instead of rejecting
|
||||||
|
// the string. A trailing unit or sign therefore yields a magnitude that is off
|
||||||
|
// by a factor of 100 — and it passes `isNaN`, so it is counted as a VALID row
|
||||||
|
// everywhere downstream. `"100,00 CAD"` does not fail; it imports as 10 000.
|
||||||
|
|
||||||
|
import { describe, it, expect } from "vitest";
|
||||||
|
import { parseFrenchAmount } from "./amountParser";
|
||||||
|
import { autoDetectConfig } from "./csvAutoDetect";
|
||||||
|
import { holdingsFromCsvRows } from "../hooks/useSnapshotEditor";
|
||||||
|
import { readCsvFixture } from "../__fixtures__/csv";
|
||||||
|
|
||||||
|
describe("parseFrenchAmount — separator contract (#326)", () => {
|
||||||
|
it("reads the French decimal comma", () => {
|
||||||
|
expect(parseFrenchAmount("1234,56")).toBe(1234.56);
|
||||||
|
expect(parseFrenchAmount("12,5")).toBe(12.5);
|
||||||
|
expect(parseFrenchAmount("0,00")).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reads French thousand separators (space, non-breaking space, dot)", () => {
|
||||||
|
expect(parseFrenchAmount("1 234,56")).toBe(1234.56);
|
||||||
|
expect(parseFrenchAmount("1\u00A0234,56")).toBe(1234.56); // non-breaking space
|
||||||
|
expect(parseFrenchAmount("1.234,56")).toBe(1234.56);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reads English notation", () => {
|
||||||
|
expect(parseFrenchAmount("1234.56")).toBe(1234.56);
|
||||||
|
expect(parseFrenchAmount("1,234.56")).toBe(1234.56);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the sign of a leading minus", () => {
|
||||||
|
expect(parseFrenchAmount("-84,32")).toBe(-84.32);
|
||||||
|
expect(parseFrenchAmount(" -84,32 ")).toBe(-84.32);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("strips currency symbols on either side", () => {
|
||||||
|
expect(parseFrenchAmount("1 250,00 $")).toBe(1250);
|
||||||
|
expect(parseFrenchAmount("$1,250.00")).toBe(1250);
|
||||||
|
expect(parseFrenchAmount("€84,32")).toBe(84.32);
|
||||||
|
expect(parseFrenchAmount("£84,32")).toBe(84.32);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects blank and non-numeric input", () => {
|
||||||
|
expect(parseFrenchAmount("")).toBeNaN();
|
||||||
|
expect(parseFrenchAmount(" ")).toBeNaN();
|
||||||
|
expect(parseFrenchAmount("abc")).toBeNaN();
|
||||||
|
expect(parseFrenchAmount("-")).toBeNaN();
|
||||||
|
expect(parseFrenchAmount("--5")).toBeNaN();
|
||||||
|
// Letter-leading labels are rejected — this is what keeps `detectHeader`
|
||||||
|
// working on a header such as "Solde 2024" (see the twin defect below).
|
||||||
|
expect(parseFrenchAmount("Solde 2024")).toBeNaN();
|
||||||
|
// Non-string input is guarded before any parsing.
|
||||||
|
expect(parseFrenchAmount(undefined as unknown as string)).toBeNaN();
|
||||||
|
expect(parseFrenchAmount(42 as unknown as string)).toBeNaN();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("parseFrenchAmount — KNOWN DEFECT: parseFloat prefix scan (#326, fixed by #325)", () => {
|
||||||
|
// `parseFloat` returns the longest valid PREFIX instead of rejecting the
|
||||||
|
// whole string. Every expectation below is the bug, not the intent.
|
||||||
|
//
|
||||||
|
// #325 must anchor the validation (regex over the whole normalized string)
|
||||||
|
// and return NaN on any residual character. All of these then become NaN,
|
||||||
|
// except the accounting-parenthesis and trailing-sign forms, which #325
|
||||||
|
// explicitly adds support for.
|
||||||
|
|
||||||
|
it("returns a x100 magnitude on a trailing sign — should be -50", () => {
|
||||||
|
// "50,00-" is the trailing-minus convention of several bank exports.
|
||||||
|
// The comma stops looking like a decimal separator once "-" trails it
|
||||||
|
// (the /,\d{1,2}$/ probe fails), so the comma is dropped as a thousands
|
||||||
|
// separator and "5000-" parses as 5000.
|
||||||
|
expect(parseFrenchAmount("50,00-")).toBe(5000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns a x100 magnitude on a trailing indicator — should be 1234.56", () => {
|
||||||
|
expect(parseFrenchAmount("1 234,56 CR")).toBe(123456);
|
||||||
|
expect(parseFrenchAmount("1 234,56 DB")).toBe(123456);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns a x100 magnitude on a trailing currency code — should be 100", () => {
|
||||||
|
expect(parseFrenchAmount("100,00 CAD")).toBe(10000);
|
||||||
|
expect(parseFrenchAmount("84,32 USD")).toBe(8432);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts a numeric prefix of a text label — should be NaN", () => {
|
||||||
|
// This is what blinds `detectHeader`: a header cell that STARTS with
|
||||||
|
// digits reads as a number, so the header row is taken for data.
|
||||||
|
expect(parseFrenchAmount("2024 Montant")).toBe(2024);
|
||||||
|
expect(parseFrenchAmount("5%")).toBe(5);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts JavaScript number literals a bank never emits — should be NaN", () => {
|
||||||
|
expect(parseFrenchAmount("1e3")).toBe(1000);
|
||||||
|
expect(parseFrenchAmount("Infinity")).toBe(Infinity);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps only the first two groups of a malformed number — should be NaN", () => {
|
||||||
|
expect(parseFrenchAmount("1,2,3")).toBe(1.2);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("parseFrenchAmount — KNOWN DEFECT: unsupported accounting forms (#326, fixed by #325)", () => {
|
||||||
|
it("rejects accounting parentheses — should be -50", () => {
|
||||||
|
// "(50,00)" is the standard accounting notation for a negative amount.
|
||||||
|
// Today it is NaN, so the row is dropped as "Invalid amount" rather than
|
||||||
|
// being imported with the wrong sign — a loud failure, unlike the ones
|
||||||
|
// above, but still a failure.
|
||||||
|
expect(parseFrenchAmount("(50,00)")).toBeNaN();
|
||||||
|
expect(parseFrenchAmount("(1 234,56)")).toBeNaN();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("parseFrenchAmount — KNOWN DEFECT: separator arbitrated per cell (#326, fixed by #325)", () => {
|
||||||
|
// The French-vs-English decision is taken on each cell in isolation
|
||||||
|
// (`/,\d{1,2}$/`), never at column level. Two cells of the SAME column can
|
||||||
|
// therefore be read under two different conventions.
|
||||||
|
//
|
||||||
|
// #325 arbitrates the separator per column, which resolves both cases.
|
||||||
|
|
||||||
|
it("reads a 3-digit group after the comma as a thousands separator", () => {
|
||||||
|
// "12,345" is 12.345 in a column of decimals, 12345 in a column of
|
||||||
|
// thousands. Nothing in the cell alone can tell.
|
||||||
|
expect(parseFrenchAmount("12,345")).toBe(12345);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reads a dot as a decimal separator when no comma is present", () => {
|
||||||
|
// In a French column, "1.234" means 1234. Read alone it yields 1.234 —
|
||||||
|
// a x1000 error in the opposite direction from the cases above.
|
||||||
|
expect(parseFrenchAmount("1.234")).toBe(1.234);
|
||||||
|
expect(parseFrenchAmount("1.234,56")).toBe(1234.56); // ...unless a comma follows
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("parseFrenchAmount — call-site fallout (#326)", () => {
|
||||||
|
// The /review-spec revision of #326 asks the corpus to reach the holdings
|
||||||
|
// call sites too, because #325 hardens the parser GLOBALLY. These pin what
|
||||||
|
// the shared parser does to its two most exposed consumers.
|
||||||
|
|
||||||
|
it("blinds detectHeader when a header cell starts with digits", () => {
|
||||||
|
// `detectHeader` (csvAutoDetect.ts:224) treats "parses as a number" as
|
||||||
|
// proof of a data row. "2025 Montant" parses as 2025, so the header is
|
||||||
|
// taken for data.
|
||||||
|
const cfg = autoDetectConfig(readCsvFixture("header-numeric-label"))!;
|
||||||
|
expect(cfg.hasHeader).toBe(false); // DEFECT — should be true
|
||||||
|
});
|
||||||
|
|
||||||
|
it("leaks the x100 magnitude into the holdings CSV import (#245)", () => {
|
||||||
|
// `holdingsFromCsvRows` (useSnapshotEditor.ts:191-202) shares the parser.
|
||||||
|
// A price column carrying its currency code silently multiplies every
|
||||||
|
// position by 100 — a $150.25 share is stored at $15,025.
|
||||||
|
const drafts = holdingsFromCsvRows(
|
||||||
|
[
|
||||||
|
["AAPL", "10", "150,25 CAD", "1 200,00"],
|
||||||
|
["MSFT", "5", "300,50", "1 400,00 CAD"],
|
||||||
|
],
|
||||||
|
{ symbol: 0, quantity: 1, unit_price: 2, book_cost: 3 }
|
||||||
|
);
|
||||||
|
expect(drafts[0].unit_price).toBe("15025"); // DEFECT — should be "150.25"
|
||||||
|
expect(drafts[0].book_cost).toBe("1200"); // clean cell, correct today
|
||||||
|
expect(drafts[1].unit_price).toBe("300.5"); // clean cell, correct today
|
||||||
|
expect(drafts[1].book_cost).toBe("140000"); // DEFECT — should be "1400"
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops an accounting-parenthesis price instead of reading it", () => {
|
||||||
|
const drafts = holdingsFromCsvRows(
|
||||||
|
[["GOOG", "2", "(140,10)", "280,20"]],
|
||||||
|
{ symbol: 0, quantity: 1, unit_price: 2, book_cost: 3 }
|
||||||
|
);
|
||||||
|
// NaN price is swallowed to an empty string — no error surfaces.
|
||||||
|
expect(drafts[0].unit_price).toBe(""); // DEFECT — should be "-140.10"
|
||||||
|
expect(drafts[0].quantity).toBe("2");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
@ -1,16 +1,32 @@
|
||||||
// csvAutoDetect — holdings-CSV detection tests (Issue #245).
|
// csvAutoDetect — holdings-CSV detection tests (Issue #245) + transaction-CSV
|
||||||
|
// corpus contract (Issue #326).
|
||||||
//
|
//
|
||||||
// Covers `autoDetectHoldingColumns` (column detection) and `analyzeHoldingsCsv`
|
// Covers `autoDetectHoldingColumns` (column detection) and `analyzeHoldingsCsv`
|
||||||
// (delimiter/header + full analysis). These helpers are pure and DOM-free, in
|
// (delimiter/header + full analysis). These helpers are pure and DOM-free, in
|
||||||
// line with the project's "test the extracted pure pieces" convention. The
|
// line with the project's "test the extracted pure pieces" convention. The
|
||||||
// numeric parsing + duplicate-merge path lives in `holdingsFromCsvRows`
|
// numeric parsing + duplicate-merge path lives in `holdingsFromCsvRows`
|
||||||
// (tested in useSnapshotEditor.test.ts).
|
// (tested in useSnapshotEditor.test.ts).
|
||||||
|
//
|
||||||
|
// The second half of the file is the #326 corpus contract on the TRANSACTION
|
||||||
|
// flow (`autoDetectConfig` + `preprocessQuotedCSV`), which had no test at all.
|
||||||
|
|
||||||
import { describe, it, expect } from "vitest";
|
import { describe, it, expect } from "vitest";
|
||||||
|
import { readFileSync } from "fs";
|
||||||
|
import { resolve } from "path";
|
||||||
|
import Papa from "papaparse";
|
||||||
import {
|
import {
|
||||||
autoDetectHoldingColumns,
|
autoDetectHoldingColumns,
|
||||||
analyzeHoldingsCsv,
|
analyzeHoldingsCsv,
|
||||||
|
autoDetectConfig,
|
||||||
|
preprocessQuotedCSV,
|
||||||
} from "./csvAutoDetect";
|
} from "./csvAutoDetect";
|
||||||
|
import { parseFrenchAmount } from "./amountParser";
|
||||||
|
import { parseDate } from "./dateParser";
|
||||||
|
import {
|
||||||
|
CSV_FIXTURE_NAMES,
|
||||||
|
readCsvFixture,
|
||||||
|
type CsvFixtureName,
|
||||||
|
} from "../__fixtures__/csv";
|
||||||
|
|
||||||
describe("autoDetectHoldingColumns (#245)", () => {
|
describe("autoDetectHoldingColumns (#245)", () => {
|
||||||
it("detects symbol/quantity/price/book_cost from EN headers", () => {
|
it("detects symbol/quantity/price/book_cost from EN headers", () => {
|
||||||
|
|
@ -121,3 +137,466 @@ describe("analyzeHoldingsCsv (#245)", () => {
|
||||||
expect(analyzeHoldingsCsv(" \n \n")).toBeNull();
|
expect(analyzeHoldingsCsv(" \n \n")).toBeNull();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// =============================================================================
|
||||||
|
// Issue #326 — transaction-CSV corpus contract
|
||||||
|
// =============================================================================
|
||||||
|
//
|
||||||
|
// `autoDetectConfig`, `detectAmountMode`, `detectSingleAmount` and
|
||||||
|
// `preprocessQuotedCSV` had NO test, on a codebase of 871. This block freezes
|
||||||
|
// what they do TODAY, so the detection rewrite (#323-#332) measures itself
|
||||||
|
// against an established contract rather than against an intention.
|
||||||
|
//
|
||||||
|
// ┌── HOW TO READ THIS BLOCK ────────────────────────────────────────────────┐
|
||||||
|
// │ Every expectation is what the code DOES, verified by running it — never │
|
||||||
|
// │ what it should do. Tests whose name carries `KNOWN DEFECT` encode │
|
||||||
|
// │ behaviour that is wrong and is expected to change; each states the right │
|
||||||
|
// │ answer in a comment and names the issue that owes the fix. When that │
|
||||||
|
// │ issue lands, the test is meant to fail: update the expectation, do not │
|
||||||
|
// │ delete the test. │
|
||||||
|
// └──────────────────────────────────────────────────────────────────────────┘
|
||||||
|
|
||||||
|
/**
|
||||||
|
* MIRROR of the row-mapping rule in `useImportWizard.parseFilesInternal`
|
||||||
|
* (`useImportWizard.ts:489-542`), copied verbatim.
|
||||||
|
*
|
||||||
|
* It is duplicated here rather than imported because the rule currently lives
|
||||||
|
* inside a `useCallback` of a React hook and the repository has no jsdom.
|
||||||
|
* Issue #325 extracts it as a pure `mapRow(raw, format)` in
|
||||||
|
* `src/utils/importFormat.ts` — AT THAT POINT THIS MIRROR MUST BE DELETED and
|
||||||
|
* the tests re-pointed at the real `mapRow`. Until then, the guard test
|
||||||
|
* "production rule still matches the mirror" below fails the build if the
|
||||||
|
* production expression drifts, so the mirror cannot silently start lying.
|
||||||
|
*/
|
||||||
|
function mapCorpusRow(
|
||||||
|
raw: string[],
|
||||||
|
cfg: NonNullable<ReturnType<typeof autoDetectConfig>>
|
||||||
|
): { date: string; description: string; amount: number } | { error: string } {
|
||||||
|
const date = parseDate(
|
||||||
|
raw[cfg.columnMapping.date]?.trim() || "",
|
||||||
|
cfg.dateFormat
|
||||||
|
);
|
||||||
|
const description = raw[cfg.columnMapping.description]?.trim() || "";
|
||||||
|
|
||||||
|
let amount: number;
|
||||||
|
if (cfg.amountMode === "debit_credit") {
|
||||||
|
const debit = parseFrenchAmount(raw[cfg.columnMapping.debitAmount ?? 0] || "");
|
||||||
|
const credit = parseFrenchAmount(
|
||||||
|
raw[cfg.columnMapping.creditAmount ?? 0] || ""
|
||||||
|
);
|
||||||
|
amount = isNaN(credit) ? -(isNaN(debit) ? 0 : debit) : credit;
|
||||||
|
} else {
|
||||||
|
amount = parseFrenchAmount(raw[cfg.columnMapping.amount ?? 0] || "");
|
||||||
|
if (cfg.signConvention === "positive_expense" && !isNaN(amount)) {
|
||||||
|
amount = -amount;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!date) return { error: "Invalid date" };
|
||||||
|
if (isNaN(amount)) return { error: "Invalid amount" };
|
||||||
|
return { date, description, amount };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Full pipeline: raw file text -> signed amounts, as the wizard runs it. */
|
||||||
|
function parseFixtureEndToEnd(name: CsvFixtureName) {
|
||||||
|
const rawContent = readCsvFixture(name);
|
||||||
|
const cfg = autoDetectConfig(rawContent);
|
||||||
|
if (!cfg) throw new Error(`autoDetectConfig returned null for ${name}`);
|
||||||
|
|
||||||
|
const data = Papa.parse(preprocessQuotedCSV(rawContent), {
|
||||||
|
delimiter: cfg.delimiter,
|
||||||
|
skipEmptyLines: true,
|
||||||
|
}).data as string[][];
|
||||||
|
|
||||||
|
const startIdx = cfg.skipLines + (cfg.hasHeader ? 1 : 0);
|
||||||
|
const rows = [];
|
||||||
|
for (let i = startIdx; i < data.length; i++) {
|
||||||
|
const raw = data[i];
|
||||||
|
if (raw.length <= 1 && raw[0]?.trim() === "") continue;
|
||||||
|
rows.push(mapCorpusRow(raw, cfg));
|
||||||
|
}
|
||||||
|
return { config: cfg, rows };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Signed amounts only — the number that actually reaches the database. */
|
||||||
|
function amountsOf(name: CsvFixtureName): (number | string)[] {
|
||||||
|
return parseFixtureEndToEnd(name).rows.map((r) =>
|
||||||
|
"error" in r ? r.error : r.amount
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The amounts every well-detected fixture in the corpus must produce. */
|
||||||
|
const REFERENCE_AMOUNTS = [-84.32, 1250, -142.18, -56.75, 300, -6.95];
|
||||||
|
|
||||||
|
describe("corpus integrity (#326)", () => {
|
||||||
|
it("exposes every declared fixture as readable, non-empty content", () => {
|
||||||
|
expect(CSV_FIXTURE_NAMES).toHaveLength(11);
|
||||||
|
for (const name of CSV_FIXTURE_NAMES) {
|
||||||
|
expect(readCsvFixture(name).trim().length).toBeGreaterThan(0);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("detects a configuration for every fixture (none returns null)", () => {
|
||||||
|
for (const name of CSV_FIXTURE_NAMES) {
|
||||||
|
expect(autoDetectConfig(readCsvFixture(name)), name).not.toBeNull();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("production rule still matches the mirror in mapCorpusRow", () => {
|
||||||
|
// Static contract, in the style of __integration__/transactions-transfer-icon.
|
||||||
|
// `mapCorpusRow` is a hand copy of a rule that lives inside a React hook;
|
||||||
|
// this is what stops it drifting. #325 extracts the rule as `mapRow` —
|
||||||
|
// when it does, these matchers fail, and that IS the signal to delete the
|
||||||
|
// mirror and import the real function.
|
||||||
|
const SRC = readFileSync(
|
||||||
|
resolve(import.meta.dirname, "..", "hooks", "useImportWizard.ts"),
|
||||||
|
"utf-8"
|
||||||
|
);
|
||||||
|
expect(SRC).toContain(
|
||||||
|
"amount = isNaN(credit) ? -(isNaN(debit) ? 0 : debit) : credit;"
|
||||||
|
);
|
||||||
|
expect(SRC).toContain(
|
||||||
|
'if (config.signConvention === "positive_expense" && !isNaN(amount)) {'
|
||||||
|
);
|
||||||
|
// The `?? 0` fallbacks silently read column 0 when a mapping is missing.
|
||||||
|
// #325 replaces them with an explicit row error.
|
||||||
|
expect(SRC).toContain("raw[config.columnMapping.debitAmount ?? 0]");
|
||||||
|
expect(SRC).toContain("raw[config.columnMapping.creditAmount ?? 0]");
|
||||||
|
expect(SRC).toContain("raw[config.columnMapping.amount ?? 0]");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — signed single amount (#326)", () => {
|
||||||
|
it("returns the full contract for a FR semicolon file", () => {
|
||||||
|
expect(autoDetectConfig(readCsvFixture("signed-amount"))).toEqual({
|
||||||
|
delimiter: ";",
|
||||||
|
hasHeader: true,
|
||||||
|
skipLines: 0,
|
||||||
|
dateFormat: "DD/MM/YYYY",
|
||||||
|
columnMapping: { date: 0, description: 1, amount: 2 },
|
||||||
|
amountMode: "single",
|
||||||
|
signConvention: "negative_expense",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses end to end to the reference amounts", () => {
|
||||||
|
expect(amountsOf("signed-amount")).toEqual(REFERENCE_AMOUNTS);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("carries the description through untouched", () => {
|
||||||
|
const { rows } = parseFixtureEndToEnd("signed-amount");
|
||||||
|
expect(rows[0]).toEqual({
|
||||||
|
date: "2025-01-05",
|
||||||
|
description: "EPICERIE METRO SAINTE-FOY",
|
||||||
|
amount: -84.32,
|
||||||
|
});
|
||||||
|
expect(rows[1]).toEqual({
|
||||||
|
date: "2025-01-15",
|
||||||
|
description: "DEPOT PAIE EMPLOYEUR",
|
||||||
|
amount: 1250,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — debit/credit pair (#326)", () => {
|
||||||
|
it("detects the sparse-complementary pair as debit_credit", () => {
|
||||||
|
expect(autoDetectConfig(readCsvFixture("debit-credit"))).toEqual({
|
||||||
|
delimiter: ";",
|
||||||
|
hasHeader: true,
|
||||||
|
skipLines: 0,
|
||||||
|
dateFormat: "DD/MM/YYYY",
|
||||||
|
columnMapping: {
|
||||||
|
date: 0,
|
||||||
|
description: 1,
|
||||||
|
debitAmount: 2,
|
||||||
|
creditAmount: 3,
|
||||||
|
},
|
||||||
|
amountMode: "debit_credit",
|
||||||
|
signConvention: "negative_expense",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses end to end to the reference amounts", () => {
|
||||||
|
// Debit -> negative, credit -> positive. Correct today because the file
|
||||||
|
// happens to list Debit before Credit — see the reversed fixture.
|
||||||
|
expect(amountsOf("debit-credit")).toEqual(REFERENCE_AMOUNTS);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — KNOWN DEFECT: debit/credit order guessed by position (#326, fixed by #328)", () => {
|
||||||
|
// `detectAmountMode` assigns the LEFTMOST column of a complementary pair to
|
||||||
|
// `debitAmount` and the rightmost to `creditAmount` (csvAutoDetect.ts:461-470).
|
||||||
|
// It never looks at the header labels. A file laid out `Credit;Debit` is
|
||||||
|
// therefore mapped backwards, and every sign in the import is inverted.
|
||||||
|
//
|
||||||
|
// #328 resolves the order from the header dictionary instead of the position.
|
||||||
|
|
||||||
|
it("maps Credit to debitAmount and Debit to creditAmount", () => {
|
||||||
|
const cfg = autoDetectConfig(readCsvFixture("debit-credit-reversed"))!;
|
||||||
|
// Column 2 is labelled "Credit" and column 3 "Debit" in the fixture.
|
||||||
|
expect(cfg.columnMapping.debitAmount).toBe(2); // DEFECT — should be 3
|
||||||
|
expect(cfg.columnMapping.creditAmount).toBe(3); // DEFECT — should be 2
|
||||||
|
expect(cfg.amountMode).toBe("debit_credit");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("inverts every sign end to end — groceries become income", () => {
|
||||||
|
expect(amountsOf("debit-credit-reversed")).toEqual(
|
||||||
|
// DEFECT — should be REFERENCE_AMOUNTS
|
||||||
|
[84.32, -1250, 142.18, 56.75, -300, 6.95]
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("still nets to the same total, which is why the bug hides", () => {
|
||||||
|
// The signs are wrong one by one, but the sum is merely negated — no
|
||||||
|
// aggregate check catches it. Only a per-row review does.
|
||||||
|
const sum = (xs: (number | string)[]) =>
|
||||||
|
xs.reduce<number>((a, b) => a + (typeof b === "number" ? b : 0), 0);
|
||||||
|
expect(sum(amountsOf("debit-credit-reversed"))).toBeCloseTo(
|
||||||
|
-sum(REFERENCE_AMOUNTS),
|
||||||
|
6
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — KNOWN DEFECT: unused column filled with 0,00 (#326, fixed by #325)", () => {
|
||||||
|
// When the unused half of a debit/credit pair carries "0,00" instead of an
|
||||||
|
// empty cell, detection still gets the mode right — `isSparseComplementary`
|
||||||
|
// treats a 0 as absent. The PARSING is what breaks: the rule branches on
|
||||||
|
// `isNaN(credit)` (useImportWizard.ts:509), and "0,00" parses to 0, not NaN.
|
||||||
|
// Every debit row therefore imports as 0,00 and the expense vanishes.
|
||||||
|
//
|
||||||
|
// #325 replaces the nullity comparison with `amount = credit - debit` on
|
||||||
|
// magnitudes and treats a 0,00 cell as absent.
|
||||||
|
|
||||||
|
it("still detects the debit_credit mode correctly", () => {
|
||||||
|
const cfg = autoDetectConfig(readCsvFixture("unused-column-zero"))!;
|
||||||
|
expect(cfg.amountMode).toBe("debit_credit");
|
||||||
|
expect(cfg.columnMapping.debitAmount).toBe(2);
|
||||||
|
expect(cfg.columnMapping.creditAmount).toBe(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("swallows every debit to zero end to end", () => {
|
||||||
|
expect(amountsOf("unused-column-zero")).toEqual(
|
||||||
|
// DEFECT — should be REFERENCE_AMOUNTS
|
||||||
|
[0, 1250, 0, 0, 300, 0]
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the credits intact, so only the expenses disappear", () => {
|
||||||
|
const amounts = amountsOf("unused-column-zero");
|
||||||
|
expect(amounts.filter((a) => a === 0)).toHaveLength(4);
|
||||||
|
expect(amounts.filter((a) => a !== 0)).toEqual([1250, 300]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — preamble before the header (#326)", () => {
|
||||||
|
it("counts the preamble lines into skipLines", () => {
|
||||||
|
expect(autoDetectConfig(readCsvFixture("preamble"))).toEqual({
|
||||||
|
delimiter: ";",
|
||||||
|
hasHeader: true,
|
||||||
|
skipLines: 3,
|
||||||
|
dateFormat: "DD/MM/YYYY",
|
||||||
|
columnMapping: { date: 0, description: 1, amount: 2 },
|
||||||
|
amountMode: "single",
|
||||||
|
signConvention: "negative_expense",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses end to end to the reference amounts", () => {
|
||||||
|
expect(amountsOf("preamble")).toEqual(REFERENCE_AMOUNTS);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — header carrying a number (#326)", () => {
|
||||||
|
// Two fixtures, opposite outcomes, same root cause — `detectHeader` decides
|
||||||
|
// "this row is data" from `parseFrenchAmount` alone (csvAutoDetect.ts:224).
|
||||||
|
|
||||||
|
it("is unharmed when the numeric header cell starts with a letter", () => {
|
||||||
|
// "Solde 2024" normalizes to "Solde2024"; parseFloat stops at the 'S' and
|
||||||
|
// returns NaN, so the header is correctly recognised. The running-balance
|
||||||
|
// column is also correctly excluded from the amount candidates.
|
||||||
|
expect(autoDetectConfig(readCsvFixture("header-with-number"))).toEqual({
|
||||||
|
delimiter: ";",
|
||||||
|
hasHeader: true,
|
||||||
|
skipLines: 0,
|
||||||
|
dateFormat: "DD/MM/YYYY",
|
||||||
|
columnMapping: { date: 0, description: 1, amount: 2 },
|
||||||
|
amountMode: "single",
|
||||||
|
signConvention: "negative_expense",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses the balance-column file end to end to the reference amounts", () => {
|
||||||
|
expect(amountsOf("header-with-number")).toEqual(REFERENCE_AMOUNTS);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — KNOWN DEFECT: header cell starting with digits (#326, fixed by #328)", () => {
|
||||||
|
// "2025 Montant" normalizes to "2025Montant"; parseFloat returns the 2025
|
||||||
|
// prefix, `hasNumber` flips true, and the header row is taken for data.
|
||||||
|
//
|
||||||
|
// #328 adds a lexical signal to `detectHeader`, and #325 anchors the parser
|
||||||
|
// so a numeric prefix no longer reads as a number. Either fix closes this.
|
||||||
|
|
||||||
|
it("takes the header row for a data row", () => {
|
||||||
|
const cfg = autoDetectConfig(readCsvFixture("header-numeric-label"))!;
|
||||||
|
expect(cfg.hasHeader).toBe(false); // DEFECT — should be true
|
||||||
|
// The column mapping survives because the six real rows outvote the header.
|
||||||
|
expect(cfg.columnMapping).toEqual({ date: 0, description: 1, amount: 2 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits the header row as an error row instead of skipping it", () => {
|
||||||
|
const { rows } = parseFixtureEndToEnd("header-numeric-label");
|
||||||
|
expect(rows).toHaveLength(7); // DEFECT — should be 6
|
||||||
|
// The header survives as far as the date check, which is the only reason
|
||||||
|
// it fails loudly rather than importing 2025,00 as a transaction.
|
||||||
|
expect(rows[0]).toEqual({ error: "Invalid date" });
|
||||||
|
expect(amountsOf("header-numeric-label").slice(1)).toEqual(
|
||||||
|
REFERENCE_AMOUNTS
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — no header row (#326)", () => {
|
||||||
|
it("recognises a headerless file and maps it positionally", () => {
|
||||||
|
expect(autoDetectConfig(readCsvFixture("no-header"))).toEqual({
|
||||||
|
delimiter: ";",
|
||||||
|
hasHeader: false,
|
||||||
|
skipLines: 0,
|
||||||
|
dateFormat: "DD/MM/YYYY",
|
||||||
|
columnMapping: { date: 0, description: 1, amount: 2 },
|
||||||
|
amountMode: "single",
|
||||||
|
signConvention: "negative_expense",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses end to end to the reference amounts, losing no first row", () => {
|
||||||
|
expect(amountsOf("no-header")).toEqual(REFERENCE_AMOUNTS);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — KNOWN DEFECT: all-positive amounts (#326, fixed by #329)", () => {
|
||||||
|
// `detectSingleAmount` (csvAutoDetect.ts:507-530) infers the convention from
|
||||||
|
// the share of negative values alone: no negatives -> `positive_expense` ->
|
||||||
|
// every amount is negated at parse time. On a file that mixes expenses and
|
||||||
|
// income and signs neither, the income is imported as an expense.
|
||||||
|
//
|
||||||
|
// Nothing in the file distinguishes the two cases, so detection cannot be
|
||||||
|
// "fixed" here. #329 surfaces it instead: a confidence score, a mandatory
|
||||||
|
// preview showing signed totals, and a "flip the signs" button.
|
||||||
|
|
||||||
|
it("infers positive_expense from the absence of negatives", () => {
|
||||||
|
const cfg = autoDetectConfig(readCsvFixture("all-positive"))!;
|
||||||
|
expect(cfg.signConvention).toBe("positive_expense");
|
||||||
|
expect(cfg.amountMode).toBe("single");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("negates the income rows along with the expenses", () => {
|
||||||
|
expect(amountsOf("all-positive")).toEqual([
|
||||||
|
-84.32, // expense — correct
|
||||||
|
-1250, // DEFECT — payroll deposit, should be +1250
|
||||||
|
-142.18, // expense — correct
|
||||||
|
-56.75, // expense — correct
|
||||||
|
-300, // DEFECT — incoming transfer, should be +300
|
||||||
|
-6.95, // expense — correct
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("produces a total that is plainly wrong, which the preview will expose", () => {
|
||||||
|
const total = amountsOf("all-positive").reduce<number>(
|
||||||
|
(a, b) => a + (typeof b === "number" ? b : 0),
|
||||||
|
0
|
||||||
|
);
|
||||||
|
expect(total).toBeCloseTo(-1840.2, 2); // truth is +1259.80
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("preprocessQuotedCSV + autoDetectConfig — whole-line-quoted file (#326)", () => {
|
||||||
|
it("unwraps the outer quotes and un-escapes the inner ones", () => {
|
||||||
|
const unwrapped = preprocessQuotedCSV(readCsvFixture("desjardins-quoted"));
|
||||||
|
expect(unwrapped.split("\n")[0]).toBe('Date,"Description",Montant');
|
||||||
|
expect(unwrapped.split("\n")[1]).toBe(
|
||||||
|
'05/01/2025,"EPICERIE METRO SAINTE-FOY",-84.32'
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("leaves an ordinary file untouched", () => {
|
||||||
|
const raw = readCsvFixture("signed-amount");
|
||||||
|
expect(preprocessQuotedCSV(raw)).toBe(raw);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("only fires when every non-empty line is quoted AND contains a `,\"\"`", () => {
|
||||||
|
// The guard is narrow on purpose (csvAutoDetect.ts:37-40). A semicolon
|
||||||
|
// file can never match it, because the probe looks for a comma.
|
||||||
|
const semicolonQuoted = '"Date;""Desc"";Montant"\n"05/01/2025;""A"";-1,00"';
|
||||||
|
expect(preprocessQuotedCSV(semicolonQuoted)).toBe(semicolonQuoted);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("detects the comma delimiter through the unwrapping", () => {
|
||||||
|
expect(autoDetectConfig(readCsvFixture("desjardins-quoted"))).toEqual({
|
||||||
|
delimiter: ",",
|
||||||
|
hasHeader: true,
|
||||||
|
skipLines: 0,
|
||||||
|
dateFormat: "DD/MM/YYYY",
|
||||||
|
columnMapping: { date: 0, description: 1, amount: 2 },
|
||||||
|
amountMode: "single",
|
||||||
|
signConvention: "negative_expense",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses end to end to the reference amounts", () => {
|
||||||
|
expect(amountsOf("desjardins-quoted")).toEqual(REFERENCE_AMOUNTS);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — KNOWN DEFECT: absolute amount + D/C indicator (#326, fixed by #328)", () => {
|
||||||
|
// The third amount mode: one column of unsigned magnitudes plus a
|
||||||
|
// single-letter D/C column carrying the direction. Nothing reads that
|
||||||
|
// column. Detection sees only positive numbers, picks `positive_expense`,
|
||||||
|
// and every credit is imported as an expense.
|
||||||
|
//
|
||||||
|
// #328 must DETECT this shape and REFUSE the file with a dedicated message,
|
||||||
|
// rather than import it backwards. The spec decision is explicit: refused,
|
||||||
|
// not silently mis-imported.
|
||||||
|
|
||||||
|
it("ignores the indicator column entirely", () => {
|
||||||
|
const cfg = autoDetectConfig(readCsvFixture("absolute-indicator"))!;
|
||||||
|
expect(cfg.amountMode).toBe("single");
|
||||||
|
expect(cfg.columnMapping.amount).toBe(2);
|
||||||
|
// Column 3 ("Sens", values D/C) appears nowhere in the mapping.
|
||||||
|
expect(Object.values(cfg.columnMapping)).not.toContain(3);
|
||||||
|
expect(cfg.signConvention).toBe("positive_expense");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("imports both credit rows as expenses", () => {
|
||||||
|
const amounts = amountsOf("absolute-indicator");
|
||||||
|
// Rows 2 and 5 are flagged "C" in the fixture.
|
||||||
|
expect(amounts[1]).toBe(-1250); // DEFECT — should be +1250
|
||||||
|
expect(amounts[4]).toBe(-300); // DEFECT — should be +300
|
||||||
|
expect(amounts.every((a) => typeof a === "number" && a < 0)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("is currently indistinguishable from the all-positive fixture", () => {
|
||||||
|
// Same detected config, same wrong outcome — which is exactly why #328
|
||||||
|
// needs the indicator column as its discriminator.
|
||||||
|
const withIndicator = autoDetectConfig(readCsvFixture("absolute-indicator"))!;
|
||||||
|
const without = autoDetectConfig(readCsvFixture("all-positive"))!;
|
||||||
|
expect(withIndicator.amountMode).toBe(without.amountMode);
|
||||||
|
expect(withIndicator.signConvention).toBe(without.signConvention);
|
||||||
|
expect(withIndicator.columnMapping).toEqual(without.columnMapping);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("autoDetectConfig — degenerate input (#326)", () => {
|
||||||
|
it("returns null on content with fewer than two non-empty lines", () => {
|
||||||
|
expect(autoDetectConfig("")).toBeNull();
|
||||||
|
expect(autoDetectConfig("Date;Description;Montant")).toBeNull();
|
||||||
|
expect(autoDetectConfig(" \n \n")).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns null when no column parses as a date", () => {
|
||||||
|
expect(
|
||||||
|
autoDetectConfig("Nom;Ville;Montant\nAlice;Quebec;10,00\nBob;Levis;20,00")
|
||||||
|
).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue