feat(onboarding): create_sample_import_folders command, sample CSV templates and detection test (#354) #380

Merged
maximus merged 1 commit from issue-354-sample-import-folders into main 2026-10-04 12:51:36 +00:00
Owner

Fixes #354

Stacked on PR #379 (issue-353-transactions-budget-empty) — link 4 of the planned-2026-10-03-onboarding-phase1 chain. Only the commits above #379's head belong to this PR. The files touched here are disjoint from the previous links.

What

  • create_sample_import_folders(import_folder, today) -> Result<Vec<String>, String> in src-tauri/src/commands/fs_commands.rs, registered in lib.rs. Writes Exemple Desjardins/releve-YYYY-MM.csv and Exemple Tangerine/releve-YYYY-MM.csv into the import folder, with {{M1}} / {{M2}} rendered as MM/YYYY of the two complete months before today. Refuses a missing folder and refuses, before any write, when either sample folder already exists. Returns the two folder names.
  • Templates src/__fixtures__/samples/exemple-desjardins.csv and exemple-tangerine.csv, byte-identical to the issue body (41 lines each, trailing newline), embedded with include_str! and read by vitest from the same files. Synthetic data only.
  • src/__fixtures__/samples/index.ts: readSampleTemplate + renderSampleTemplate. src/__fixtures__/csv/index.ts and CsvFixtureName are untouched (corpus still frozen at 15).
  • src/__integration__/onboarding-samples.test.ts: detection (detectImportFormat → ok, bank desjardins / tangerine, negative_expense, single, 40/40), full parse with the production mapRow (0 rows in error, sums +740.00 / −368.46), then categorizeBatch against the 180 seed keywords read from consolidated_schema.sql → exactly 33 / 36 categorised, ≥ 80 %, and the off-seed lines are the only uncategorised ones.

Why

The onboarding checklist needs a source the user can import right away and a few lines left to classify by hand. The command never receives a file path to write: it takes the import folder and a date, and the names it writes are fixed in Rust. today comes from the webview because chrono is compiled with std + serde only (Local::now() unavailable), which also makes the command deterministic and testable.

Decisions (autopilot)

  • replaceAll avoided in index.ts (tsconfig targets ES2020, npm run build type-checks src/ tests included); regex replace(/…/g) instead.
  • The regex extracting the seed keywords is asserted against the 180 count; the seed has one uniform INSERT shape.
  • Measured categorisation on first run = 33 / 36, identical to the frozen values; no expectation was adjusted.
  • No rollback on partial write failure (decided in planning); documented in the command's doc comment.
  • docs/architecture.md and CHANGELOG are left to #357, which centralises them.

How tested

  • cargo check: clean, no warning.
  • cargo test and cargo test -- --test-threads=4: 192 passed (187 + 5 new), 0 failed.
  • npm test: 1420 passed (76 files), including the 9 new cases; csvAutoDetect.test.ts and bankSignatures.test.ts unchanged.
  • npm run build: green (tsc + vite).
  • Templates verified byte-identical to the issue body with cmp.

Generated autonomously by /autopilot run of 2026-10-03

🤖 Generated with Claude Code

Fixes #354 **Stacked on PR #379** (`issue-353-transactions-budget-empty`) — link 4 of the `planned-2026-10-03-onboarding-phase1` chain. Only the commits above #379's head belong to this PR. The files touched here are disjoint from the previous links. ## What - **`create_sample_import_folders(import_folder, today) -> Result<Vec<String>, String>`** in `src-tauri/src/commands/fs_commands.rs`, registered in `lib.rs`. Writes `Exemple Desjardins/releve-YYYY-MM.csv` and `Exemple Tangerine/releve-YYYY-MM.csv` into the import folder, with `{{M1}}` / `{{M2}}` rendered as `MM/YYYY` of the two complete months before `today`. Refuses a missing folder and refuses, **before any write**, when either sample folder already exists. Returns the two folder names. - **Templates** `src/__fixtures__/samples/exemple-desjardins.csv` and `exemple-tangerine.csv`, byte-identical to the issue body (41 lines each, trailing newline), embedded with `include_str!` and read by vitest from the same files. Synthetic data only. - **`src/__fixtures__/samples/index.ts`**: `readSampleTemplate` + `renderSampleTemplate`. `src/__fixtures__/csv/index.ts` and `CsvFixtureName` are untouched (corpus still frozen at 15). - **`src/__integration__/onboarding-samples.test.ts`**: detection (`detectImportFormat` → `ok`, bank `desjardins` / `tangerine`, `negative_expense`, `single`, 40/40), full parse with the production `mapRow` (0 rows in error, sums +740.00 / −368.46), then `categorizeBatch` against the 180 seed keywords read from `consolidated_schema.sql` → exactly 33 / 36 categorised, ≥ 80 %, and the off-seed lines are the only uncategorised ones. ## Why The onboarding checklist needs a source the user can import right away and a few lines left to classify by hand. The command never receives a file path to write: it takes the import folder and a date, and the names it writes are fixed in Rust. `today` comes from the webview because chrono is compiled with `std` + `serde` only (`Local::now()` unavailable), which also makes the command deterministic and testable. ## Decisions (autopilot) - `replaceAll` avoided in `index.ts` (tsconfig targets ES2020, `npm run build` type-checks `src/` tests included); regex `replace(/…/g)` instead. - The regex extracting the seed keywords is asserted against the 180 count; the seed has one uniform INSERT shape. - Measured categorisation on first run = 33 / 36, identical to the frozen values; no expectation was adjusted. - No rollback on partial write failure (decided in planning); documented in the command's doc comment. - `docs/architecture.md` and CHANGELOG are left to #357, which centralises them. ## How tested - `cargo check`: clean, no warning. - `cargo test` and `cargo test -- --test-threads=4`: **192 passed** (187 + 5 new), 0 failed. - `npm test`: **1420 passed** (76 files), including the 9 new cases; `csvAutoDetect.test.ts` and `bankSignatures.test.ts` unchanged. - `npm run build`: green (tsc + vite). - Templates verified byte-identical to the issue body with `cmp`. Generated autonomously by /autopilot run of 2026-10-03 🤖 Generated with [Claude Code](https://claude.com/claude-code)
maximus added 1 commit 2026-10-03 23:51:13 +00:00
feat(onboarding): create_sample_import_folders command, sample CSV templates and detection test (#354)
All checks were successful
PR Check — Frontend / frontend (pull_request) Successful in 1m46s
PR Check — Rust / rust (pull_request) Successful in 3m38s
e4a0f1042e
Add a Tauri command that writes two synthetic bank statements (Desjardins
chequing account, Tangerine credit card) into the import folder, one
sub-folder each, so a new user has sources to import on the first run.

The command takes the import folder and the webview's local date only
(chrono is compiled without `clock`), renders the `{{M1}}` / `{{M2}}`
tokens as the two complete months before that date, names each file
`releve-YYYY-MM.csv` after the most recent one, and refuses to run when
the folder is missing or when either sample folder already exists -
checked before anything is written. It returns the two folder names.

The templates live under `src/__fixtures__/samples/` so the Rust command
(`include_str!`) and vitest read the same bytes. The integration test
proves detection recognises each file as its bank with a signed single
amount and reads 40/40 rows, that the production `mapRow` parses every
row and sums to +740.00 / -368.46, and that the 180 seed keywords
categorise exactly 33 / 36 lines while leaving the hand-classified ones
untouched. The frozen detection corpus (`src/__fixtures__/csv`) is not
modified.

Five Rust tests cover the missing folder, the existing folder, the
success path, an unparseable date and the year boundary, each in its own
sandbox so they run in parallel.

CHANGELOG and architecture docs are centralised in #357.

Resolves #354

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
maximus added the
autopilot:pending-human
label 2026-10-03 23:51:29 +00:00
Author
Owner

Reviewed the full diff of PR #380 (GET /pulls/380.diff, 661 lines, 6 files) against issue #354.

Summary

The command writes only two fixed folder names and one fixed file name under the folder it is given; today is parsed as a NaiveDate and only ever re-emitted through format("%m/%Y") / format("%Y-%m"), so no byte of it reaches a path. Both templates are byte-identical to the issue body (checked with cmp on the extracted code blocks). Measured categorisation matched the frozen 33 / 36 on the first run, so no expectation was adjusted. The frozen detection corpus is untouched.

Criteria

Criterion Status Evidence
Two CSV templates, exact content, 41 lines, trailing newline OK cmp against the issue's code blocks; wc -l = 41 each
create_sample_import_folders(import_folder, today) -> Result<Vec<String>, String>, doc comment OK fs_commands.rs, registered in lib.rs after pick_folder
NaiveDate::parse_from_str, M1/M2 via checked_sub_months OK with_day(1) first so checked_sub_months never clamps on a 29-31
Err on missing folder; Err naming the folder when either exists, before any write OK both exists() checks run before the first create_dir; test refuses_when_desjardins_folder_exists_and_writes_nothing asserts Tangerine absent and the existing folder still empty
include_str! relative to src-tauri/src/commands/ OK ../../../src/__fixtures__/samples/*.csv
Writes releve-YYYY-MM.csv (M1), returns the two folder names OK writes_both_folders_for_the_two_previous_months asserts ["Exemple Desjardins", "Exemple Tangerine"] and exactly one file per folder
Five Rust cases, sandbox(name) per case, cleanup on Err paths OK cleanup runs before every unwrap_err/expect; cargo test -- --test-threads=4 192 passed
src/__fixtures__/samples/index.ts, corpus csv/index.ts untouched OK git diff on csvAutoDetect.test.ts, bankSignatures.test.ts, csv/index.ts is empty
Integration test: bank, negative_expense, single, 40/40, 0 errors, +740.00 / −368.46, 33 / 36, ≥ 80 %, off-seed not categorised OK 9 tests green; additionally asserts the off-seed lines are the ONLY uncategorised ones
check-rust and check-frontend both run on the PR In progress frontend run 673 started on e4a0f10; rust queues behind it (runner capacity 1). Both path filters match (src/ and src-tauri/)
Command never receives a file path to write OK signature is folder + date; file and folder names are constants
No CHANGELOG, no DB migration OK left to #357

Security

  • No path traversal through today: a string that is not YYYY-MM-DD fails parse_from_str, and the only derived path component is releve-YYYY-MM.csv built from the parsed date.
  • import_folder is the same trust boundary as scan_import_folder / write_export_file (a folder the webview obtained from the picker). The command cannot overwrite: create_dir fails on an existing folder, and both folders are checked first.
  • Synthetic data only: invented merchants, balances and amounts, no account number, no person.

Concerns (non-blocking)

  • No rollback when the second create_dir/write fails after the first succeeded (disk full): a single Exemple Desjardins folder may remain. Decided in planning and documented in the doc comment.
  • The seed-keyword regex in the test assumes the uniform (keyword, category_id, priority) VALUES (...) shape of consolidated_schema.sql; the test asserts the 180 count so a shape change surfaces as a failure rather than a silent shortfall.
  • The issue referenced a regex pattern at categorizationService.test.ts:9 that does not exist (line 9 is an import); a dedicated regex was written instead.

Generated autonomously by /autopilot run of 2026-10-03

## Self-review (autopilot, link 4/7) — Verdict: **APPROVE** Reviewed the full diff of PR #380 (`GET /pulls/380.diff`, 661 lines, 6 files) against issue #354. ### Summary The command writes only two fixed folder names and one fixed file name under the folder it is given; `today` is parsed as a `NaiveDate` and only ever re-emitted through `format("%m/%Y")` / `format("%Y-%m")`, so no byte of it reaches a path. Both templates are byte-identical to the issue body (checked with `cmp` on the extracted code blocks). Measured categorisation matched the frozen 33 / 36 on the first run, so no expectation was adjusted. The frozen detection corpus is untouched. ### Criteria | Criterion | Status | Evidence | |---|---|---| | Two CSV templates, exact content, 41 lines, trailing newline | OK | `cmp` against the issue's code blocks; `wc -l` = 41 each | | `create_sample_import_folders(import_folder, today) -> Result<Vec<String>, String>`, doc comment | OK | `fs_commands.rs`, registered in `lib.rs` after `pick_folder` | | `NaiveDate::parse_from_str`, M1/M2 via `checked_sub_months` | OK | `with_day(1)` first so `checked_sub_months` never clamps on a 29-31 | | `Err` on missing folder; `Err` naming the folder when either exists, before any write | OK | both `exists()` checks run before the first `create_dir`; test `refuses_when_desjardins_folder_exists_and_writes_nothing` asserts Tangerine absent and the existing folder still empty | | `include_str!` relative to `src-tauri/src/commands/` | OK | `../../../src/__fixtures__/samples/*.csv` | | Writes `releve-YYYY-MM.csv` (M1), returns the two folder names | OK | `writes_both_folders_for_the_two_previous_months` asserts `["Exemple Desjardins", "Exemple Tangerine"]` and exactly one file per folder | | Five Rust cases, `sandbox(name)` per case, cleanup on `Err` paths | OK | cleanup runs before every `unwrap_err`/`expect`; `cargo test -- --test-threads=4` 192 passed | | `src/__fixtures__/samples/index.ts`, corpus `csv/index.ts` untouched | OK | `git diff` on `csvAutoDetect.test.ts`, `bankSignatures.test.ts`, `csv/index.ts` is empty | | Integration test: bank, `negative_expense`, `single`, 40/40, 0 errors, +740.00 / −368.46, 33 / 36, ≥ 80 %, off-seed not categorised | OK | 9 tests green; additionally asserts the off-seed lines are the ONLY uncategorised ones | | `check-rust` and `check-frontend` both run on the PR | In progress | `frontend` run 673 started on `e4a0f10`; `rust` queues behind it (runner capacity 1). Both path filters match (`src/` and `src-tauri/`) | | Command never receives a file path to write | OK | signature is folder + date; file and folder names are constants | | No CHANGELOG, no DB migration | OK | left to #357 | ### Security - No path traversal through `today`: a string that is not `YYYY-MM-DD` fails `parse_from_str`, and the only derived path component is `releve-YYYY-MM.csv` built from the parsed date. - `import_folder` is the same trust boundary as `scan_import_folder` / `write_export_file` (a folder the webview obtained from the picker). The command cannot overwrite: `create_dir` fails on an existing folder, and both folders are checked first. - Synthetic data only: invented merchants, balances and amounts, no account number, no person. ### Concerns (non-blocking) - No rollback when the second `create_dir`/`write` fails after the first succeeded (disk full): a single `Exemple Desjardins` folder may remain. Decided in planning and documented in the doc comment. - The seed-keyword regex in the test assumes the uniform `(keyword, category_id, priority) VALUES (...)` shape of `consolidated_schema.sql`; the test asserts the 180 count so a shape change surfaces as a failure rather than a silent shortfall. - The issue referenced a regex pattern at `categorizationService.test.ts:9` that does not exist (line 9 is an import); a dedicated regex was written instead. Generated autonomously by /autopilot run of 2026-10-03
Author
Owner

Revue adversariale (/pr-review, passe indépendante) — Verdict : APPROVE

Revue de la PR contre sa base issue-353-transactions-budget-empty (1 commit, 6 fichiers, 8eac47f..e4a0f10). Les 5 étapes ci-dessous ont été exécutées, pas lues.

Ce qui a été vérifié

  • Gabarits : exemple-desjardins.csv et exemple-tangerine.csv identiques au byte au corps de #354 (cmp), LF, 41 lignes, saut de ligne final. Sommes recalculées depuis les fichiers : +740,00 / −368,46.
  • Seed : 180 lignes INSERT INTO keywords dans consolidated_schema.sql, les 180 matchent la regex du test, aucune ligne hors forme.
  • CI sur le head e4a0f10 : PR Check — Rust (3m38) et PR Check — Frontend (1m46) ont tous deux tourné et sont verts — la pile stackée déclenche bien la CI (aucun filtre branches:), critère d'acceptation de #354 satisfait.
  • Rejoué sur une copie hors dépôt (git archive du head) : tsc --noEmit exit 0 ; vitest onboarding-samples + csvAutoDetect + bankSignatures = 153 verts / 3 fichiers ; cargo test sample_folder_tests = 5/5, 187 filtrés (192 au total, chiffre de la PR confirmé).
  • Timeline #354 : pull_ref de #380 avec closes — l'issue se fermera au merge. Pas de secret, pas de migration, pas de chaîne UI (commande Rust seule, consommée par un maillon suivant).
  • chrono : default-features = false, features = ["serde", "std"] confirmé dans Cargo.toml:47 → l'argument today est fondé, la commande ne lit jamais l'horloge.

Mutations du relecteur (copie hors dépôt)

# Mutation Résultat
M1 Retrait de la boucle d'existence préalable (Rust) 5/5 verts — survit
M2 Inversion des jetons {{M1}}/{{M2}} (Rust) 5/5 verts — survit
M3 Nom de fichier sur M2 au lieu de M1 (Rust) 2 rouges (succès + année)
B1 Gabarit : DOLLARAMA → METRO (Tangerine) 1 rouge (expected 38 to be 36) — le gel des comptes fonctionne
B2 Inversion m1/m2 dans renderSampleTemplate 9/9 verts — survit
B3 {{M2}} laissé non rendu 8 rouges

Suggestions (non bloquantes — code correct, propriétés non épinglées)

  1. La garantie « contrôle avant toute écriture » n'est pas prouvée par les tests (M1). Le seul cas testé est « Exemple Desjardins existe » — premier de la boucle, donc fs::create_dir échoue par lui-même avec un message contenant le nom du dossier et Tangerine n'est jamais atteint : l'état observable est le même avec ou sans le contrôle. Le cas qui discrimine est « Exemple Tangerine existe → Exemple Desjardins non créé ». Sonde ajoutée dans la copie : verte sur le code de la PR, rouge sans la boucle. Le code tient ; c'est le test qui manque. Un cas de 10 lignes, à ajouter ici ou en suivi.
  2. Rien n'épingle « {{M1}} = le mois le plus récent » (M2 + B2). assert_dates_in et le test vitest months vérifient l'ensemble des deux mois, pas l'affectation. Sans effet pour l'utilisateur (mêmes 40 lignes), mais une assertion sur la première ligne (01/07/2026) et la dernière (28/08/2026) suffirait.
  3. onboarding-samples.test.ts:239 — « the seed is all priority 0, so the SQL order is the file order » est faux : COSTCO et WALMART sont à 50, COUCHE-TARD ESSENCE à 10. Le test trie bien par priority décroissante, donc le mock est juste ; seul le commentaire ment. À corriger au passage.
  4. Mineur : l'ordre des égalités de priorité repose sur la stabilité du tri JS, alors que SQLite ne garantit pas l'ordre des ex æquo sous ORDER BY priority DESC. Sans conséquence ici (aucune description des gabarits ne matche deux mots-clés de même priorité vers des catégories différentes), à garder en tête si les gabarits bougent.

Actions faites

  • Copie hors dépôt supprimée (scratchpad/pr380, fichiers issue-*.csv / head-*.csv), aucun fichier du dépôt touché, aucun résidu simpl-resultat-samples-* dans /tmp.
  • Label de #354 : déjà status:approved (self-review autopilot) — inchangé, verdict cohérent.
## Revue adversariale (`/pr-review`, passe indépendante) — Verdict : **APPROVE** Revue de la PR contre sa base `issue-353-transactions-budget-empty` (1 commit, 6 fichiers, `8eac47f..e4a0f10`). Les 5 étapes ci-dessous ont été **exécutées**, pas lues. ### Ce qui a été vérifié - **Gabarits** : `exemple-desjardins.csv` et `exemple-tangerine.csv` **identiques au byte** au corps de #354 (`cmp`), LF, 41 lignes, saut de ligne final. Sommes recalculées depuis les fichiers : **+740,00** / **−368,46**. - **Seed** : 180 lignes `INSERT INTO keywords` dans `consolidated_schema.sql`, les 180 matchent la regex du test, aucune ligne hors forme. - **CI sur le head** `e4a0f10` : `PR Check — Rust` (3m38) et `PR Check — Frontend` (1m46) ont tous deux **tourné et sont verts** — la pile stackée déclenche bien la CI (aucun filtre `branches:`), critère d'acceptation de #354 satisfait. - **Rejoué sur une copie hors dépôt** (`git archive` du head) : `tsc --noEmit` exit 0 ; vitest `onboarding-samples` + `csvAutoDetect` + `bankSignatures` = **153 verts / 3 fichiers** ; `cargo test sample_folder_tests` = **5/5, 187 filtrés** (192 au total, chiffre de la PR confirmé). - **Timeline #354** : `pull_ref` de #380 avec `closes` — l'issue se fermera au merge. Pas de secret, pas de migration, pas de chaîne UI (commande Rust seule, consommée par un maillon suivant). - `chrono` : `default-features = false, features = ["serde", "std"]` confirmé dans `Cargo.toml:47` → l'argument `today` est fondé, la commande ne lit jamais l'horloge. ### Mutations du relecteur (copie hors dépôt) | # | Mutation | Résultat | |---|---|---| | M1 | Retrait de la boucle d'existence préalable (Rust) | **5/5 verts — survit** | | M2 | Inversion des jetons `{{M1}}`/`{{M2}}` (Rust) | **5/5 verts — survit** | | M3 | Nom de fichier sur M2 au lieu de M1 (Rust) | 2 rouges (succès + année) | | B1 | Gabarit : `DOLLARAMA` → `METRO` (Tangerine) | 1 rouge (`expected 38 to be 36`) — le gel des comptes fonctionne | | B2 | Inversion `m1`/`m2` dans `renderSampleTemplate` | **9/9 verts — survit** | | B3 | `{{M2}}` laissé non rendu | 8 rouges | ### Suggestions (non bloquantes — code correct, propriétés non épinglées) 1. **La garantie « contrôle avant toute écriture » n'est pas prouvée par les tests (M1).** Le seul cas testé est « Exemple Desjardins existe » — premier de la boucle, donc `fs::create_dir` échoue par lui-même avec un message contenant le nom du dossier et Tangerine n'est jamais atteint : l'état observable est le même avec ou sans le contrôle. Le cas qui discrimine est **« Exemple Tangerine existe → Exemple Desjardins non créé »**. Sonde ajoutée dans la copie : verte sur le code de la PR, rouge sans la boucle. Le code tient ; c'est le test qui manque. Un cas de 10 lignes, à ajouter ici ou en suivi. 2. **Rien n'épingle « `{{M1}}` = le mois le plus récent » (M2 + B2).** `assert_dates_in` et le test vitest `months` vérifient l'*ensemble* des deux mois, pas l'affectation. Sans effet pour l'utilisateur (mêmes 40 lignes), mais une assertion sur la première ligne (`01/07/2026`) et la dernière (`28/08/2026`) suffirait. 3. `onboarding-samples.test.ts:239` — « the seed is all priority 0, so the SQL order is the file order » est **faux** : `COSTCO` et `WALMART` sont à 50, `COUCHE-TARD ESSENCE` à 10. Le test trie bien par `priority` décroissante, donc le mock est juste ; seul le commentaire ment. À corriger au passage. 4. Mineur : l'ordre des égalités de priorité repose sur la stabilité du tri JS, alors que SQLite ne garantit pas l'ordre des ex æquo sous `ORDER BY priority DESC`. Sans conséquence ici (aucune description des gabarits ne matche deux mots-clés de même priorité vers des catégories différentes), à garder en tête si les gabarits bougent. ### Actions faites - Copie hors dépôt supprimée (`scratchpad/pr380`, fichiers `issue-*.csv` / `head-*.csv`), aucun fichier du dépôt touché, aucun résidu `simpl-resultat-samples-*` dans `/tmp`. - Label de #354 : déjà `status:approved` (self-review autopilot) — inchangé, verdict cohérent.
maximus changed target branch from issue-353-transactions-budget-empty to main 2026-10-04 12:51:31 +00:00
maximus merged commit e4a0f1042e into main 2026-10-04 12:51:36 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: maximus/Simpl-Resultat#380
No description provided.