# Handover: dynamic AdaPBS calibration, experiments on the cluster

Written 2026-11-16 by Claude (laptop session with Fabien Maussion) for a Claude session on
the cluster. Read this first, then `claude/dev-notes.md` (full project log, decisions
with dates) and `CLAUDE.md` (Fabien's rules for this repo). Those three files are not in
git: Fabien copies them over.

## 1. Context in five lines

- `oggm-adapbs` is a small GPL package: the AdaPBS algorithm (adaptive multiple
  importance sampling, ported from MuSA, Aalstad et al. 2026) plus OGGM tools.
- New module `oggm_adapbs/dynamic.py`: **one AdaPBS run per glacier replaces OGGM's
  static MB calibration + regional inversion + dynamic calibration** (spinup + melt_f
  recalibration). This is Chloe Hancock's PhD workflow (Bristol); she will use it for
  GLUE-style uncertainty later. It is experimental.
- It has been tested on one glacier only (Vernagtferner, RGI60-11.00719, 8.6 km2), in
  the notebook `example_workflows/dynamic_calibration.ipynb`. Your job: a few more
  experiments, to see whether the conclusions hold elsewhere and what things cost.
- Fabien decides. Ask before anything nontrivial (CLAUDE.md, "Code rules"). Never push
  to GitHub, never create or switch git branches without asking.
- Keep code changes out of `adapbs.py` (no OGGM import there, stays close to MuSA).

## 2. The forward model (one particle = one parameter set)

`run_dynamic_particle(gdir, params, filesuffix, ...)`:

1. **Inversion** at the RGI date with OGGM's tasks (`apparent_mb_from_any_mb`,
   `prepare_for_inversion`, `mass_conservation_inversion(glen_a=A, fs=0)`,
   `filter_inversion_output`, `init_present_time_glacier`), with the particle's MB
   parameters and its own Glen A. Files are written with the particle's suffix and
   deleted at the end.
2. **Growth period**: `growth_years` (default 20) in a constant climate (mean climate of
   the spinup period, `ConstantMassBalance`, as in OGGM's `run_dynamic_spinup`) with
   `temp_bias + growth_temp_offset`. This makes a starting glacier for the historical
   run. The growth years have no calendar meaning.
3. **Historical run**, in memory, from `state_year - spinup_years` (default 30 years:
   1974 for RGI date 2003) to 2020, with the particle's `temp_bias` (no offset).

Parameters (`DYNAMIC_PARAMS`): `melt_f`, `prcp_fac`, `temp_bias`,
`log10_glen_a_factor` (A = 2.4e-24 * 10**x, OGGM's default A, *not* the regionally
calibrated A of the glacier directory), `growth_temp_offset` (K).

Targets (`dynamic_calibration_adapbs` defaults), all at `state_year` = RGI date + 1:

| target | model | sigma |
|---|---|---|
| geodetic MB 2000-2020 (Hugonnet) | mass change / RGI area / 20 | 0.2 x Hugonnet error (`mb_err_factor`) |
| RGI area | `area_min_h` (ice > 2 m) | 5% + discretisation term (`area_error_km2`, `terminus_step_km2`) |
| IceBoost v2 mean thickness | volume / total area | 20% (`volume_rel_err=0.2`, `volume_target='thickness'`) |

The IceBoost errors come from the `*_v20260705` tables (OGGM itself uses `v20260701`,
same volumes, no errors). They are huge (66% on Vernagtferner, median 49%): that is why
the error is 20% of the thickness instead.

Particles whose *inverted* glacier is > 5 sigma from the thickness target are rejected
before the dynamical runs (`reject_sigma`). Failed runs (mostly `Glacier exceeds domain
boundaries`) and rejected ones get NaN predictions, hence zero weight.

The output is an xarray Dataset: all particles (`particle`), weights, `pred`
(`obs` = dmdtda, area, thickness), Glen A, inverted volume, volume after growth,
`volume_km3` / `area_km2` / `mass_kg` time series (dimension `time`, starting with the
growth period), `status`, `run_time`, AdaPBS diagnostics per iteration, prior and
log evidence in the attributes. `ds.to_netcdf()` works.

## 3. What we found on Vernagtferner (your reference)

Sensitivity setup ("wide priors", Fabien: this is a sensitivity experiment, not a
calibration): logit-normal, sd 1.4 in normal space, centred mid-range (near uniform):
melt_f 1.5-17, prcp_fac 0.1-10 (OGGM bounds), temp_bias informed centre +-5 K,
log10 A factor +-1, growth offset +-3 K. 500 particles, max 20 iterations, seed 0.

| | with growth (5 params) | without growth (4 params) |
|---|---|---|
| iterations / runs | 19 / 9500 (271 s on 6 workers) | 10 / 5000 (133 s) |
| log10 A factor (prior 5-95% -0.82 to 0.81) | 0.12 [-0.73, 0.79] | **0.37 [-0.28, 0.90]** |
| melt_f | 4.69 [2.22, 11.41] | 4.57 [2.47, 11.67] |
| prcp_fac | 2.14 [0.55, 7.21] | 2.50 [0.61, 6.71] |
| growth offset (prior +-2.45) | 0.13 [-2.20, 1.28] | - |
| log evidence | -15.06 | **-14.22** |

- All three targets are fitted. OGGM's `run_dynamic_melt_f_calibration` (same spinup
  start, same MB error) lies inside the AdaPBS ranges.
- **Glen A is identifiable only without the growth period.** With it, the growth offset
  compensates A: stiff ice -> thick inversion -> warm growth thins it again. Without
  growth, A is 2.3x the default (median), close to OGGM's own value for this glacier.
- **The data do not ask for the growth period**: log Z higher without it (factor ~2.3,
  weak; AMIS evidence with ~150 effective particles is noisy).
- The MB parameters stay equifinal (as with the geodetic MB alone).
- Known problem: calibrated parameter sets match the MB 2000-2019 but are 300-400
  kg m-2 yr-1 too positive in 1979-1993 on Vernagtferner (checked against WGMS, not in
  the notebook). The 30-year spinup runs through that period. Cause unknown (climate
  data or MB temperature sensitivity).

To check your setup reproduces this, rerun the two calibrations of the notebook
(Section 5 snippet, Vernagtferner): same numbers with seed 0 and 500 particles.

## 4. Experiments to run (in this order; ask Fabien if you want to change the plan)

**E1. Do the conclusions hold on other glaciers?** (most important)
- With and without the growth period, wide priors, same settings as above.
- A sample of ~10-20 glaciers with different sizes: e.g. from the Alps (RGI60-11),
  Hintereisferner (11.00897), Great Aletsch (11.01450, Chloe's glacier, large), and
  several small ones (< 1 km2, where the area discretisation term matters). Agree on the
  list with Fabien before running.
- Per glacier and variant, report: iterations, runs, run time, failed/rejected runs,
  final Neff/Ne, log Z, posterior 5/50/95% of each parameter (and of the prior), fit of
  the three targets, OGGM's dynamic calibration for comparison (see notebook cell with
  `run_dynamic_melt_f_calibration`, run on a *copy* of the gdir:
  `tasks.copy_to_basedir(gdir, base_dir=..., setup='all')`, `ys=state_year - 30`,
  `ref_mb_err_scaling_factor=0.2`, `kwargs_run_function={'min_ice_thickness': 2.}`
  and same for `kwargs_fallback_function`, after `ref.settings['min_ice_thick_for_area'] = 2.`).
- Questions: is A constrained without growth, and not with it? Does log Z still prefer
  no growth? Is the A posterior (without growth) close to OGGM's regional A?

**E2. Aletsch cost.** Time one `run_dynamic_particle` first (it is much larger and
longer than Vernagtferner; Vernagtferner is 0.13 s per particle). Extrapolate to
~10 000 runs before launching. Check the domain: wide priors make glaciers grow, and
on a large glacier `Glacier exceeds domain boundaries` may be frequent (prepro border
80; check whether larger borders exist for 2025.6 before switching).

**E3. Spinup length.** `spinup_years` 30 / 20 / 10 (without growth, and with growth if
E1 still makes it worth it), on Vernagtferner and 2-3 other glaciers. Motivation: the
1974-1993 MB problem above. Does the A posterior or log Z change?

**E4. Convergence.** Vernagtferner, without and with growth: 3 seeds x {500, 1000}
particles (and perhaps `neff_threshold` 0.5). How much do the posterior quantiles and
log Z move? The posterior histograms with ~150 effective particles are noisy: is the
log Z difference between growth / no growth larger than the seed spread?

**E5 (only if time).** Thickness error 20% vs 30% (Fabien's range), on the E1 sample.

Optional, ask Fabien first: the 1980s MB problem on other WGMS glaciers (diagnostic
only: the workflow itself must not use WGMS data, it must work for any glacier).

## 5. How to run

Environment: OGGM **master** (the package needs `gdir.settings`, `read_store`,
day-based float years, the vectorised MB). Then `pip install -e .` in this repo. The
module is not committed yet: check with Fabien that the cluster copy has
`oggm_adapbs/dynamic.py` and `oggm_adapbs/tests/test_dynamic.py`. Run the tests once:
`pytest --run-slow oggm_adapbs` (14 pass; the MuSA comparison is skipped without
`MUSA_DIR`; the tests use OGGM's Hintereisferner test fixtures).

Data (downloaded through OGGM's cache):
- L3 directories (climate from 1901, GSWP3-W5E5 before 1979):
  `https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6/L3-L5_files/2025.6/elev_bands/W5E5/per_glacier/`,
  `prepro_border=80`. Use L3 (L5 has no inversion flowlines).
- Temperature bias file (centre of the temp_bias prior):
  `https://cluster.klima.uni-bremen.de/~oggm/ref_mb_params/oggm_v1.6/w5e5_rgi6_perglacier_temp_bias_v2025.6.2.csv`

Minimal script (one glacier, both variants):

```python
import sys, time
import numpy as np
from oggm import cfg, utils, workflow
from oggm_adapbs.dynamic import dynamic_calibration_adapbs, dynamic_prior

BASE_URL = ('https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6/'
            'L3-L5_files/2025.6/elev_bands/W5E5/per_glacier/')
TB_FILE = ('https://cluster.klima.uni-bremen.de/~oggm/ref_mb_params/oggm_v1.6/'
           'w5e5_rgi6_perglacier_temp_bias_v2025.6.2.csv')

if __name__ == '__main__':  # needed: the particles run in a spawn pool
    rgi_id, wd, out, n_workers = sys.argv[1], sys.argv[2], sys.argv[3], int(sys.argv[4])
    cfg.initialize(logging_level='WARNING')
    cfg.PATHS['working_dir'] = wd
    gdir = workflow.init_glacier_directories([rgi_id], from_prepro_level=3,
                                             prepro_border=80,
                                             prepro_base_url=BASE_URL)[0]
    # Wide priors (the notebook's)
    tb = dynamic_prior(gdir, temp_bias_file_path=TB_FILE)['centre']['temp_bias']
    bounds = {'melt_f': (gdir.settings['melt_f_min'], gdir.settings['melt_f_max']),
              'prcp_fac': (gdir.settings['prcp_fac_min'], gdir.settings['prcp_fac_max']),
              'temp_bias': (tb - 5, tb + 5), 'log10_glen_a_factor': (-1, 1),
              'growth_temp_offset': (-3, 3)}
    centre = {p: (lo + hi) / 2 for p, (lo, hi) in bounds.items()}
    kw = dict(prior_centre=centre, prior_bounds=bounds, prior_sd=1.4,
              temp_bias_file_path=TB_FILE, n_particles=500, max_iterations=20,
              seed=0, n_workers=n_workers, spinup_years=30)
    for gy in [20, 0]:
        params = ['melt_f', 'prcp_fac', 'temp_bias', 'log10_glen_a_factor']
        if gy:
            params.append('growth_temp_offset')
        t0 = time.time()
        ds = dynamic_calibration_adapbs(gdir, growth_years=gy, calibrate_params=params, **kw)
        ds.attrs['wall_time_s'] = time.time() - t0
        ds.to_netcdf(f'{out}/{rgi_id}_growth{gy}.nc')
```

Parallelism:
- `n_workers` makes a spawn pool *inside* the task. Do not combine it with OGGM's
  multiprocessing (`use_multiprocessing=True` + `execute_entity_task` over glaciers):
  daemonic workers cannot start a pool. Either one job per glacier with
  `n_workers` = cores of the job, or OGGM multiprocessing over glaciers with
  `n_workers=1`. With small glaciers and many cores, the second wastes less.
- Pool start-up costs a few seconds (each worker imports OGGM).

Gotchas:
- **Never run two calibrations on the same glacier directory at the same time**: the
  particle file suffixes (`_adapbs_dyn_it{j}_p{k}`) would collide. One working
  directory per experiment / variant running concurrently.
- `init_glacier_directories(..., from_prepro_level=...)` re-extracts the directory and
  overwrites it. To reuse one, open it without the prepro arguments.
- The first `init_glacier_directories` from a URL: multiprocessing off.
- With wide priors, half of the first iteration fails or is rejected: expected.
- Float years in OGGM are day-based: `utils.date_to_floatyear(y, m)`, never
  `y + (m - 1) / 12`.
- Check the cluster's scheduler, paths and core counts with Fabien or the cluster docs;
  this handover does not know them.

## 6. What to deliver

- A `claude/cluster_results.md` (or similar, ask) with, per experiment: setup, a table
  per glacier and variant (numbers listed in E1), and short conclusions. Date the
  entries; say plainly what did not work.
- The netCDF files of the runs, in a results folder outside the repo.
- Add the decisions and the main results to `claude/dev-notes.md` (dated one-liners,
  keep its "State" section current), as CLAUDE.md asks.
- Do not change the package without asking. If you find a bug, report it with a minimal
  reproduction. Tests: run the relevant ones (`pytest --run-slow path::Class::test`) and
  say which ones you ran.

Open decisions waiting for these results (Fabien's): drop the growth period (or make
`growth_years=0` the default)? What to do about the 1974-2000 MB? Thickness error 20%
or 30%?
