Project overview · branches A, B and C in progress · 18 October 2026
mdbench
A method-ladder benchmark for molecular simulation. Four scientific questions, each answered at five levels of rigour, on systems with published experimental data, all on one 6 GB laptop GPU.
Nothing here predicts anything new. Each calculation is scored against measured values (ITC binding data, crystal endpoints, smFRET distributions, mutant kcat series), and its wall-clock cost is recorded beside the score. Plotting accuracy against log cost for each branch shows where extra rigour stops paying.
“For this system class, rigour above Ln did not pay.” One such sentence per branch is the product of the repo.
The ladder
Four questions × five levels of rigour
Each row is a branch; each column adds cost and physical content. L0 is a deliberately crude null model that every level above it must beat. Branch C is built first: it is the cheapest, and it exercises the machinery (PLUMED, collective variables, convergence diagnostics, the results schema) that the other three depend on.
L0Null model
L1Single structure
L2Short MD + endpoint
L3Rigorous classical FE
L4Beyond the force field
CWhich conformations, with what probability?Built first
Boltzmann over crystal endpointsRan · ala2, AdK
Static CV valuesRan · ala2, AdK
Histogram from unbiased MDRan · ala2
Metadynamics / umbrella + MSMRan · ala2
ML-potential FES for alanine dipeptideDeferred
AHow well does this ligand bind?
Ligand heavy-atom countRan · CB7, T4L
Docking scoreRan · CB7, T4L
MM/GBSA on short MDRan · CB7, T4L
ABFE / RBFE with MBARPilot · 1 CB7 guest
ML/MM correction on the ligandDeferred
BHow does this protein move?
Anisotropic network modelRan · AdK
Normal modes from one structureRan · AdK
RMSF, contacts, PCA on short MDRan · AdK
Long or parallel MD, reversible transitionsConfigured
MACE-OFF on a small peptideDeferred
DWhat makes one enzyme faster?
kcat of nearest characterised homologPlanned
Active-site geometryPlanned
NAC populations, field at the bondPlanned
PMF along the reaction coordinatePlanned
QM/MM barrier with xTBPlanned
Null baseline: must exist and be scored before L3 runsRan, rows in results (tinted in the branch colour)Pilot onlyConfigured, deferred or planned
Verdicts so far
Where extra rigour stopped paying
The one-sentence product of each branch, as far as the runs go today. Every one of these is provisional until the branch's top level has run.
A · CB7 host–guest
Ranking peaked at L1 docking (ρ 0.53); MM/GBSA ranked worse at about 100× the cost.
15 guests; no level clears the null's 95% CI
A · T4L L99A
Ranking still rising at L2 (ρ 0.75 with MM/GBSA); no flattening yet.
7 ligands; L3 not run
B · Adenylate kinase
For the closing motion, rigour above L0 did not pay: one second of ANM captures 97% of it.
10-ns MD scale; no transitions seen
C · Alanine dipeptide
L0–L2 match L3 on the major basin; the rare αL basin needed L3 to be sampled at all.
Agreement between methods; no experimental reference yet
D · Enzymes
Not started.
Planning stage
Branch A progress · 18 October 2026
Branch A: CB7 and T4 lysozyme L99A through L2, first ABFE converged
90 rows are in results/a.parquet, each scored against a measured binding free energy: 15 CB7 guests from the benchmarksets collection (ITC, 298 K) and 7 apolar ligands of T4 lysozyme L99A (Morton 1995, 302 K). Both use Sage 2.3.0 with ff14SB, TIP3P water and 50 mM NaCl, and every level uses the same parameterised systems.
Ranking against cost Spearman ρ vs experiment with bootstrap 95% CI · grey band: the L0 null's CI
Series
Level
Spearman ρ [95% CI]
Pairwise
RMSE
Cost per ligand
Predicted against measured, per level kcal/mol · each panel on its own vertical scale · dashed: y = x where it is in rangeABFE convergence, CB7 amantadine, complex leg overlap between neighbouring λ states, log scale
No level beats the null yet. With 7–15 ligands every 95% interval overlaps the heavy-atom null's. The ranking differences are suggestive, not established.
CB7 flattens at L1. Vina ranks best (ρ 0.53, pairwise 0.71) at 13 s a guest. MM/GBSA, at 23 minutes a guest, falls back to ρ 0.28.
T4L keeps rising. ρ goes 0.36 → 0.50 → 0.75 from L0 to L2. The measured range is only 2 kcal/mol, and the null already sits at 0.67 kcal/mol RMSE.
MM/GBSA absolute values are not physical. With no entropy term it overbinds CB7 by about 30 kcal/mol and T4L by about 18, so only its ranking is compared.
An entropy term helps only between sets. Interaction entropy, from the same frames and a few minutes of CPU, lifts CB7 ρ from 0.28 to 0.50, but ranking within each benchmark set gets worse (Set 1: 0.21 → 0.12). The energy fluctuations it is built from, 3–14 kcal/mol, are beyond where the estimator is reliable. On T4L it changes nothing (ρ 0.75).
The first ABFE converged on its second schedule. Halving the electrostatics windows took the worst overlap from 0.014 to 0.14 and gave replica round trips (5 complex, 3 solvent). Amantadine: −20.6 ± 0.6 against −17.19 kcal/mol measured.
The remaining overbinding is not sampling. The schedule moved the answer by 1.4 kcal/mol; 3.4 kcal/mol is left. Fixed-charge models overbind cationic CB7 guests, and the sodium in the experiment's acetate buffer is not modelled.
Branch B progress · 17 October 2026
Branch B: adenylate kinase through L2
34 rows are in results/b.parquet. The score is how well each method's motions point along the crystal open→closed displacement (4AKE → 1AKE, C-alpha atoms): overlap 1 means the motion lies entirely in the method's modes.
Open→closed overlap captured by the softest modes 4AKE chain A → 1AKE chain ADomain angles in the three MD replicates degrees · shaded: the closed box
The closing motion is in the softest modes. Ten ANM modes capture 97% of the open→closed displacement, all-atom NMA 97%, and 3 × 10 ns of MD 93% in its top ten PCs. A random subspace of the same size captures 12%.
Direction matters. Modes of the closed structure describe opening less well (ANM 0.74, all-atom 0.83) than modes of the open structure describe closing.
No transitions. None of the three 10-ns replicates entered the closed box; the closest approach was 6.1 Å C-alpha RMSD from 1AKE. Transition fraction 0.00 ± 0.15.
B-factors don't separate methods. The null already reaches r = 0.75 on the open structure, so B-factor agreement is not evidence of a correct motion.
Progress · 10 October 2026
Branch C: alanine dipeptide through L3, adenylate kinase prepared
42 rows are in results/c.parquet. Every reference column is still empty because no published value has been verified yet, so for now the checks are agreement between levels and methods, not accuracy against experiment.
L0
L1
L2
L3
L4
Alanine dipeptide1,843 atoms, TIP3P
RanBoltzmann over minima, GBn2 · 78 s
RanStatic φ/ψ + harmonic · 121 s
Ran5 × 10 ns unbiased · 47 min
RanWT-metad, φ umbrella, MSM · 29–47 min each
Deferredneeds the mdbench-ml env
Adenylate kinase40,119 atoms, solvated
RanEndpoint energies, GBn2 · 3 min
RanDomain angles of 4AKE, 1AKE · 1 s
Configured5 × 100 ns · ≈3.3 days
Configuredmetad ≈2 d · umbrella ≈2.2 d · MSM ≈26 d
Deferrednot planned for AdK
Run, rows in resultsConfig written, not run (over the 6 h ask-first limit)
Alanine dipeptide: free energy relative to C7eq/β, by method
ΔG(αR) kcal/mol · ±1σΔG(αL/C7ax) kcal/mol · ±1σ
Each panel has its own scale. Rows with no point mean the method never sampled that basin. Cost is wall-clock on the RTX 4050; the MSM's cost includes the L2 trajectories it is built from. L0 and L1 use GBn2 implicit solvent; L2 and L3 use explicit TIP3P. To see the L2 and metadynamics runs play out, open the motion page.
Three methods agree on αR. Unbiased MD, metadynamics and the MSM all give ΔG ≈ 0.47–0.48 kcal/mol, about 30% population.
The rare basin needed L3. 50 ns of unbiased MD never visited αL (≈3%). Metadynamics and umbrella sampling agree on it at ≈1.9 kcal/mol.
Umbrella αR is unconverged. The bias is on φ only, and ψ stays slow inside each window: the two halves of the data give 21% and 5% αR. This is a CV-choice failure, as the intent file anticipated.
MSM error bars are too small. ±0.02 kcal/mol is about 3× below the spread across replicates. A bootstrap over whole trajectories is pending.
The nulls land close. L0 and L1 cost about 2 minutes and fall within 0.2 kcal/mol of the explicit-water answer for both basins. Their error bars cover only the choice of starting structures.
AdK angles separate cleanly. θNMP is 72.6° open and 44.1° closed; θLID is 145.7° and 106.2°. The L0 null calls apo AdK fully closed, which is almost certainly wrong, as a null is allowed to be.
Cost axis
What each rung costs on this machine
Wall-clock per calculation on the RTX 4050, on a log scale. Bars are the planning estimates: solid from docs/HARDWARE.md, hatched inferred from the level definitions. Dots are measured costs from the parquet rows, coloured by branch. Throughput so far: about 1,660 ns/day for alanine dipeptide (885 with PLUMED attached), 940 ns/day for a CB7 complex, 387 ns/day for a 24k-atom water box, about 190 ns/day for T4 lysozyme (35k atoms) and about 153 ns/day for solvated adenylate kinase (40k atoms, GPU shared at the time).
Systems
Shared systems across branches
The systems are chosen so that one prepared structure serves several branches, which reuses the preparation work and keeps the cross-branch comparison controlled.
System
A
B
C
D
Size
Role
TEM-1 β-lactamase· 4
~260 res
Serves all four. Scoped to reproducing published variant effects only.
T4 lysozyme L99A· 3
—
164 res
The ABFE benchmark; the cavity also breathes, giving a B/C system for free.
Adenylate kinase· 2½
—
214 res
Domain-motion reference. Its rate-limiting step is the conformational change itself.
Alanine dipeptide· 1
—
—
—
22 atoms
Correctness check. If the Ramachandran FES is wrong, everything above it is wrong.
CB7 host–guest· 1
—
—
—
~3.7k atoms
ABFE that finishes in hours, so the alchemy machinery is debugged before it costs days.
Hollow circle: partial fit, marked “~” in docs/SYSTEMS.md. Branch D's own systems, ketosteroid isomerase and chorismate mutase, appear only in its intent file.
Branches
Scope and finish line
CConformational distributions
Makes the ensemble the computed object. Alanine dipeptide first, then metadynamics and umbrella sampling on the ADK domain coordinate, then an MSM sharing trajectories with B.
Done when
Alanine dipeptide FES matches published surfaces
Metadynamics and umbrella FES for ADK agree, or the gap is explained
MSM with validated implied timescales gives populations
Open: how much simulation does the ADK FES need before it stops moving?
Now: alanine dipeptide through L3; AdK L0–L1 run, long runs configured (days, need approval).
ABinding affinity
Relative and absolute binding free energies at every level. CB7 to debug alchemy in hours, then T4L L99A as the real benchmark; a congeneric RBFE series only if both land.
Done when
L0–L3 run on CB7 and T4L with uncertainty and wall-clock
Spearman, pairwise accuracy and RMSE with bootstrap CIs
Flattening point named; disk under 40 GB
Open: is L2→L3 worth 100× the compute for ranking specifically?
Now: L0–L2 on CB7 and T4L, scored; first ABFE converged; CB7 campaign (~38 h) awaits approval.
BConformational change
Does each method recover a known motion? Adenylate kinase open/closed, plus fast folders (Trp-cage, chignolin) whose reversible folding is reachable on this GPU.
Done when
ANM motion for ADK scored before any MD runs
Replicate MD, with the number that saw a transition recorded
Protein-only XTC, branch under 30 GB
Open: does seconds of ANM capture most of what microseconds of MD show?
Now: at the 10-ns scale, yes (97% vs 93%). L3 waits on branch C's 100-ns runs.
DEnzymatic properties
Reproduces measured structure–activity relationships: KSI active-site field vs kcat across mutants, and the chorismate mutase barrier with distal-mutation data.
Done when
KSI field–kcat correlation reproduced with a CI
Chorismate mutase PMF at L3 and at L4 (xTB QM/MM)
Unreproduced distal effects written up as results
Open: does a non-polarisable force field get the KSI field right at all?
Now: planning stage.
House rules
What keeps the comparison fair
L0 before L3. No expensive run until the null has a recorded score.
Ground truth is adopted, not curated. Every reference value carries a DOI.
Ranking first. Spearman ρ and pairwise accuracy outweigh RMSE.
Noise floor ~0.4 kcal/mol. No claimed resolution below experimental reproducibility.
One parquet row per calculation. Results are kept; trajectories are disposable.
Hardware held fixed. 6 GB VRAM ceiling; any rented run records its GPU.