Case · Scientific and numerical software · Public
A battery model ran to completion on every revision and lost 2.4% of its lithium
PyBaMM, an open-source lithium-ion battery modelling library used in research and industry.
- The system said
- Solved, on every revision. The then-under-review pull request #5524 changed nothing.
- The evidence showed
- 2.44% of the lithium drifted on every revision, at every refinement. The missing term was named, and a project contributor confirmed it.
The claim tested
The model conserves lithium: the total in electrolyte and particles stays constant over a discharge and charge cycle, up to solver tolerance.
What Omega did
- Froze three revisions of the library, the v26.4.1 release, main at 258fdc8 and the head of then-under-review pull request #5524, with 1,228 files hashed before any run.
- Assembled the lithium inventory independently from the mesh, the parameters and the raw concentrations, never from the library's own totals, with a second explicit-loop implementation checked against the first on every run.
- Ran 84 jobs across two models, three revisions and seven mesh and tolerance settings, with the constant-transference model and the full modular model as controls.
- Compared the measured rate of loss, instant by instant, with the rate the equation predicts if one term is missing.
What was found
- The BasicDFN model with a concentration-dependent transference number lost 2.44% of its lithium on every revision and at every refinement. Tightening the time tolerance to 1e-10 and refining to 80 cells changed the fourth digit.
- The loss matched the rate predicted from the missing transference-gradient term to 2.9e-18 mol/s. The equation, not the discretisation, is the cause.
- The then-under-review pull request #5524 left both reported cases byte-identical to main: neither model selects the option it adds.
- Separately, the half-cell diagnostic Total lithium in electrolyte multiplies by the full-cell thickness (767 µm) where the electrolyte occupies 67 µm, overweighting that term by 11.45.
- The controls held: the constant-transference model conserved to 7.7e-15 mol and the full model to 3.5e-14 mol.
Verdict
- CONTRADICTED
- BasicDFN conserves lithium with a concentration-dependent transference number. Equation-level. A project contributor confirmed the missing term and asked for a separate report: issue #5745.
- CONTRADICTED
- The half-cell electrolyte lithium diagnostic reports moles of lithium. Wrong length scale by a factor of 11.45; confirmed and split out as issue #5746.
- CONTRADICTED
- The then-under-review pull request #5524 changes the reported cases. Byte-identical time series on all 24 completed pairs.
What this does not establish
One library, two models, one protocol, one parameter set. The full DFN conserved; nothing is said against it. Eleven half-cell jobs failed inside the solver at tight tolerance, so no half-cell convergence order is claimed. The finite-volume treatment of nonlinear source terms, which the contributor raised, was not investigated. The #5745 fix is on the default branch but not in a tagged release; #5746 is not fixed.
Upstream
- PyBaMM MERGED fixWhether the BasicDFN model conserves lithium when the transference number depends on concentration. Issue #5745, 6 September 2026, split from #5700 at a contributor's request. Warren's pull request #5747, covering the lithium and sodium BasicDFN variants, was approved and merged by the maintainer on 18 September 2026. On the default branch; in no tagged release as of 20 September 2026.
- PyBaMM MERGED fixWhether the half-cell diagnostic Total lithium in electrolyte uses the electrolyte's own thickness. Issue #5746, 6 September 2026, split from #5700 at a contributor's request. Maintainer-authored pull request #5765 corrects the half-cell lithium diagnostic and adds regression tests. Merged 2026-09-22; merge commit 9daad126258db8f6e621e4746cd3bdac062e7f1f.
Released means the fix is in a tagged release. Merged means it is on the default branch but not verified in a tagged release. Open pull request means an Omega-authored change awaits upstream action. Open issue means a report remains open without a merged fix. Maintainer acceptance is review of one change, not endorsement of Omega.
The records
Every field of each finding this case rests on, as it appears on the ledger.
OBSERVED pybamm-basicdfn-variable-transference-driftverify · faithfulness A battery model ran to completion on every revision tested and lost 2.4% of its lithium. Independent inventory accounting localised the loss to one equation, and a project contributor confirmed the missing term.
- Claim
- In PyBaMM's BasicDFN with a concentration-dependent transference number, an independently assembled lithium inventory drifted by 2.44% of the initial inventory over the reporter's 4,800-second protocol on v26.4.1, main at 258fdc8 and the head of pull request #5524, byte-identical between main and the pull request; the drift persisted under temporal tightening to 1e-10 and spatial refinement to 80 cells, and matched, instant by instant, the rate predicted from the missing transference-gradient term. The constant-transference control conserved to 7.7e-15 mol.
- Status
- OBSERVED Witnessed in one instance. No claim about how often.
- Question
- Passed is not correct
- Subject
- PyBaMM, an open-source battery modelling library (third-party system); the inventory code and the referee are Omega's. third-party subject
- Frame
- The two one-dimensional cases from issue #5700; three pinned revisions; seven mesh and tolerance settings; 84 frozen jobs.
- Method
- Lithium inventory assembled from the mesh, the parameters and the raw concentrations, never from the library's own totals; a second explicit-loop implementation checked against the vectorised one on every run; a refinement sweep; the instantaneous rate of loss compared with the rate the equation predicts.
- Oracle
- Conservation of lithium: the physical inventory computed independently of the library's diagnostics, checked against its own initial value.
- Negative control
- Present The constant-transference BasicDFN and the full modular DFN, run under the same harness, conserved to 7.7e-15 and 3.5e-14 mol; analytic referee mutations (an altered inventory, a reversed boundary sign, an omitted source, a wrong volume factor) were each detected before and after the runs.
- Denominator
- 2.44e-2 relative drift at 20 cells, 20 shells, 1e-8; 73 of 84 frozen jobs completed; main and the pull request byte-identical on all 24 completed pairs.
- Limitation
- One model in one library under one protocol and one parameter set. Nothing against PyBaMM's full DFN, whose control conserved to 3.5e-14 mol, and nothing about the finite-volume discretisation of nonlinear source terms, which a contributor raised and which was not investigated. Eleven of 84 half-cell jobs failed inside the solver at tight tolerance, so no half-cell spatial convergence order is claimed. No physical battery was measured. The #5745 fix is on the default branch but not in a tagged release; #5746 remains open.
- Source
- repowazdogz-droid/pybamm-5700-lithium-inventory @ adaa877
- Reproduce
PYBAMM_DISABLE_TELEMETRY=true python minimal_reproducer.py --output reviewer-output
- Independent reproduction
- None known.
- Upstream
- REPORTED UPSTREAM Issue #5745 was closed after Warren's pull request #5747 was approved and merged on 18 September 2026. The fix is on the default branch and in no tagged release as of 20 September 2026. Link