Skip to content

[ADD] account_statement_import_sheet_file_ux: preview the file a mapping expects - #439

Open
feg-adhoc wants to merge 1 commit into
ingadhoc:19.0from
adhoc-dev:19.0-t-71587-feg
Open

[ADD] account_statement_import_sheet_file_ux: preview the file a mapping expects#439
feg-adhoc wants to merge 1 commit into
ingadhoc:19.0from
adhoc-dev:19.0-t-71587-feg

Conversation

@feg-adhoc

@feg-adhoc feg-adhoc commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Replaces every patch we carry on top of OCA for the bank statement import, so that the base modules stay untouched and the build stops carrying them:

Patch in the build Replaced by
708bc22 / OCA #969 — fixes on top of the #958 refactoring the field and parser overrides
19.0-h-118949-rov-2auto_install on the xls/xlsx manifests the depends + auto_install of this module
a8b2a1d / OCA #979 — skip the Enterprise PDF render on import the _create_bank_statements override
9a3e5a1 / OCA #996 (closed upstream) — keep numeric xlsx cells native the _get_xlsx_row_values + _parse_decimal overrides
c19ad17 / OCA #997 — refuse an unconfigured decimal mark the _check_decimal_mark override

The only thing not carried over is #969's removal of the duplicated amount_column field definition: it is dead code (Python keeps the last definition), it cannot be undone from an external module, and it changes no behaviour.

The preview

Most of the support around the statement sheet import comes from mappings that do not describe the file the bank exports: a renamed header, a header row on the wrong line, the thousands separator the other way around. The error the user gets back ('Date' is not in list) says nothing about which part of the mapping is wrong.

  • Preview Mapping renders a sample sheet — column letters, row numbers, header in bold, ignored rows greyed out — built from the mapping as it is configured right now, with three sample transactions filled in.
  • Downloadable xlsx of that same sample. Every cell is written as text, the way the parser reads them, so the sample can be imported with that very mapping to check it end to end.
  • It reads the mapping back in plain words and warns about configurations that are going to fail: a header row number of 0 (the spreadsheet parser then reads the header row as a transaction too and the import fails), or column names on a mapping declared as having no header line.
  • The date format, the hardest field to get right, is decoded (%d/%m/%Y reads as day/month/year) and its codes are listed, highlighting the ones the mapping uses. The field help carries the same legend.

The test import

Test Import, on the mapping and next to Import and View in the wizard, reads a real file the way an import would and reports what would come in — how many transactions and between which dates — without saving anything. It mirrors import_single_statement up to, and not including, the two steps that touch the database: the creation of the statements and the write of bank_statements_source on the journal. A file that cannot be read fails with the same explained error as an import.

A failed import

A RedirectWarning that puts the explanation first, names the mapping and what is wrong with it (the missing column, the date format, the encoding, the unsupported format), keeps the raw parser error at the end behind "Technical detail", and offers a button that opens the preview of that mapping.

It has to be an ir.actions.server: the failed import rolled its transaction back, so a preview record created on that path would no longer exist by the time the user clicks.

The fixes it carries

  • Column names matched ignoring case and padding, so a bank that exports DATE one month and Date the next does not break the import.
  • Month names in Spanish: strptime only knows the English ones, so 21-Ago-2026 with %d-%b-%Y used to fail. Translated in _get_values_from_column, the smallest seam the parser offers, and only for the timestamp column.
  • Numeric xlsx cells kept native and _parse_decimal short-circuiting Decimal / int / float: stringifying a number printed python notation, and a mapping with a comma then dropped that dot and shifted the amount by ten or a hundred.
  • A decimal mark the mapping does not account for is refused rather than silently stripped.
  • Header lines skip count renamed to Header row number with a help that explains the number, and the amount columns cleared when the Amount type changes.
  • No statement PDF while importing: with Enterprise, account_accountant renders it synchronously on create, which is what makes large imports time out.
  • auto_install triggered by the sheet module, pulling xls and xlsx, so a database that can import csv statements can also import spreadsheets.

Test plan

33 tests. Because the build still carries the patches this module replaces, the ones that matter were also run against a pristine OCA 19.0 checkout shadowing the patched one in the addons path:

  • Pristine OCA with this module: 0 failed of 33.
  • Pristine OCA without the ported overrides: the amount tests fail with 15005.0 != 1500.5 and UserError not raised, and the PDF test with the statement was created without the flag that skips the PDF render — so the tests measure this module and not the patches underneath.
  • The header row lands where the parser looks for it for 0, 1, 2 and 5, and 0 produces the warning; offset columns, footer rows, concatenated columns, the three amount types, no_header with indexes and misconfigured with names, the none decimal separator and the inverse sign all render the expected cells.
  • The generated sample is parsed back with its own mapping and yields the three expected amounts, for the four amount types and with footer rows.
  • Month names parse from Ago, AGO, Set, 21 de agosto de 2026 and Aug.
  • A test import creates no statement, no line, and does not touch bank_statements_source; an empty file warns; a broken mapping raises the explained error.
  • The error carries the hint before the technical detail, and its button resolves to the preview of that mapping.
  • Installing only account_statement_import_sheet_file on an empty database leaves the base, xls, xlsx and this module installed.

Manual check pending in runbot for the visual side.

Task: https://www.adhoc.inc/odoo/project.task/71587

Copilot AI lite review requested due to automatic review settings August 21, 2026 20:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Este PR incorpora el nuevo módulo account_statement_import_sheet_file_ux para mejorar la experiencia de configuración e importación de extractos bancarios en planillas (xls/xlsx), agregando previsualización del archivo esperado, generación de un sample importable y errores de importación más accionables para guiar al usuario hacia la corrección del mapping.

Changes:

  • Agrega un wizard de “Mapping preview” que renderiza un sample visual (HTML) y permite descargar un xlsx consistente con el mapping.
  • Mejora la importación: ante fallas de parseo, eleva un RedirectWarning con contexto (mapping/columna/formato) y acceso directo al mapping.
  • Ajusta el comportamiento del parser y del mapping: matching de columnas case/trim-insensitive, parseo de decimales desde números crudos, rename de “header lines skip count” a “Header row number”, y limpieza de columnas al cambiar el Amount type.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
account_statement_import_sheet_file_ux/init.py Inicializa el módulo cargando models y wizard.
account_statement_import_sheet_file_ux/manifest.py Declara el módulo, dependencias y data (vistas + ACL).
account_statement_import_sheet_file_ux/README.rst Documenta funcionalidad, uso y notas técnicas del módulo UX.
account_statement_import_sheet_file_ux/i18n/account_statement_import_sheet_file_ux.pot Template de traducciones para los nuevos textos (Python y QWeb).
account_statement_import_sheet_file_ux/security/ir.model.access.csv ACL para el wizard de preview.
account_statement_import_sheet_file_ux/models/init.py Exporta los overrides del mapping y del parser.
account_statement_import_sheet_file_ux/models/account_statement_import_sheet_mapping.py Implementa preview layout/notes/warnings, botón de preview y mejoras UX del mapping.
account_statement_import_sheet_file_ux/models/account_statement_import_sheet_parser.py Hace el matching de headers case/trim-insensitive y acepta números crudos en _parse_decimal.
account_statement_import_sheet_file_ux/wizard/init.py Registra los wizards nuevos/extendidos.
account_statement_import_sheet_file_ux/wizard/account_statement_import.py Override de _parse_file para re-lanzar errores como RedirectWarning con hint accionable.
account_statement_import_sheet_file_ux/wizard/account_statement_import_sheet_mapping_preview.py Wizard que renderiza el HTML y genera/descarga el xlsx sample.
account_statement_import_sheet_file_ux/views/account_statement_import_sheet_mapping_views.xml Inserta el botón “Preview Mapping” en la vista del mapping.
account_statement_import_sheet_file_ux/views/account_statement_import_sheet_mapping_preview_views.xml QWeb template del preview + vista form del wizard con botón de descarga.
account_statement_import_sheet_file_ux/tests/init.py Habilita la carga del suite de tests del módulo.
account_statement_import_sheet_file_ux/tests/test_mapping_preview.py Cobertura de preview/layout, sample importable, parser fixes y RedirectWarning en imports fallidos.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@roboadhoc

Copy link
Copy Markdown
Contributor

Pull request status dashboard

@feg-adhoc feg-adhoc changed the title [ADD] account_statement_import_sheet_file_ux: preview the file a mapping expects [ADD] account_statement_import_sheet_file_ux, account_statement_import_file_ux Aug 26, 2026
@feg-adhoc feg-adhoc changed the title [ADD] account_statement_import_sheet_file_ux, account_statement_import_file_ux [ADD] account_statement_import_sheet_file_ux: preview the file a mapping expects Aug 26, 2026
@feg-adhoc
feg-adhoc force-pushed the 19.0-t-71587-feg branch 2 times, most recently from 0cb098e to 5d2ef9b Compare August 26, 2026 16:45
@feg-adhoc
feg-adhoc force-pushed the 19.0-t-71587-feg branch 6 times, most recently from 453d785 to d1e8d32 Compare September 4, 2026 17:26
Replaces every patch we carry on top of OCA for the bank statement import, so
that the base modules stay untouched.

Most of the support around the statement sheet import comes from mappings that
do not describe the file the bank exports, and the error the user gets back
('Date' is not in list) says nothing about which part of the mapping is wrong.

Adds a "Preview Mapping" button on the sheet mapping that renders a sample
sheet, with column letters and row numbers, built from the mapping as it is
configured: where the header row must be, which columns must exist, and how
dates and amounts must be written, with three sample transactions filled in.
The sample can be downloaded as xlsx; every cell is written as text, the way
the parser reads them, so the file can be imported with that very mapping to
check it end to end.

Adds a "Test Import" button, on the mapping and next to "Import and View" in
the import wizard, that reads a real file the way an import would and reports
what would come in without saving anything: no statement, no transaction, and
not even the bank_statements_source the real import writes on the journal.

Explains the date format, which is the hardest field of the mapping to get
right: the preview reads it back in words and lists the codes, highlighting the
ones the mapping uses, and the field help carries the same legend. Month names
are now read in Spanish as well as English, so a file with '21-Ago-2026' and a
format of '%d-%b-%Y' no longer fails.

A failed import explains itself: it names the mapping and what is wrong with
it, keeps the raw parser error at the end as the technical detail, and offers a
button that opens the preview of that mapping.

The rest are the fixes we used to carry as patches on the base modules, brought
in as an extension: case and padding insensitive column matching, numeric cells
of an xlsx kept native, a decimal mark the mapping does not account for refused
instead of silently shifting the amount, a clearer name and help for the header
row number, and the amount columns cleared when the amount type changes. It
depends on the xls and xlsx modules and auto installs with the sheet one, so a
database that can import csv statements can also import spreadsheets, and it
stops rendering the statement PDF while importing, which is what makes large
imports time out.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants