{ "cells": [ { "cell_type": "markdown", "id": "0", "metadata": {}, "source": [ "# Static MB calibration against WGMS data with AdaPBS\n", "\n", "OGGM's `mb_calibration_to_rmsd` finds the mass balance (MB) parameters\n", "(`melt_f`, `prcp_fac`, `temp_bias`) which minimise the RMSD between the modelled\n", "and the observed (WGMS) annual MB of a glacier. It returns one parameter set.\n", "\n", "Here we do the Bayesian version of this with the adaptive particle batch smoother\n", "(AdaPBS): given a prior distribution of the parameters and observations with\n", "errors, it returns weighted samples of the **posterior** distribution. This tells\n", "us not only the best parameters, but also how well the observations constrain\n", "them, and which combinations of parameters are equally good (equifinality).\n", "\n", "The geometry is fixed (the inversion flowlines), as in `mb_calibration_to_rmsd`:\n", "one model run is a few milliseconds, so we can afford thousands of them.\n", "\n", "The notebook has two parts:\n", "\n", "1. **One target: the annual MB series.** The AdaPBS equivalent of\n", " `mb_calibration_to_rmsd`. We use it to explain how AdaPBS works, and check\n", " that it finds the same solution as the deterministic calibration.\n", "2. **Several targets: the winter and summer MB.** Where the Bayesian approach\n", " becomes useful: we combine observations of different kinds, and see what each\n", " of them constrains.\n", "\n", "AdaPBS is the work of K. Aalstad and E. Alonso-González (MuSA). If you use it,\n", "please cite Aalstad et al. (2026, https://doi.org/10.5194/gmd-19-8565-2026) and\n", "Alonso-González et al. (2022, https://doi.org/10.5194/gmd-15-9127-2022)." ] }, { "cell_type": "code", "execution_count": null, "id": "1", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "from oggm import cfg, utils, workflow\n", "from oggm.core.massbalance import mb_calibration_to_rmsd\n", "\n", "from oggm_adapbs import run_adapbs, glogit, gexpit, weighted_quantile, weighted_summary\n", "from oggm_adapbs.massbalance import mb_prior, static_mb_model, seasonal_specific_mb" ] }, { "cell_type": "markdown", "id": "2", "metadata": {}, "source": [ "## Glacier and data\n", "\n", "We use Vernagtferner (Ötztal Alps, Austria), which has 40 years of winter and\n", "summer MB in OGGM's WGMS data over the period of the W5E5 climate data\n", "(1979-2019). Its neighbour Hintereisferner (`RGI60-11.00897`) works too, but has\n", "seasonal data since 2013 only.\n", "\n", "We start from the OGGM preprocessed directories at level 3 (W5E5 climate,\n", "elevation band flowlines)." ] }, { "cell_type": "code", "execution_count": null, "id": "3", "metadata": {}, "outputs": [], "source": [ "cfg.initialize(logging_level='WARNING')\n", "cfg.PATHS['working_dir'] = utils.gettempdir('adapbs_wgms_calibration', reset=True)\n", "\n", "rgi_id = 'RGI60-11.00719' # Vernagtferner\n", "base_url = ('https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6/'\n", " 'L3-L5_files/2025.6/elev_bands/W5E5/per_glacier/')\n", "gdir = workflow.init_glacier_directories([rgi_id], from_prepro_level=3,\n", " prepro_border=80,\n", " prepro_base_url=base_url)[0]\n", "\n", "# The temperature bias file used by OGGM to inform the temp_bias prior\n", "temp_bias_file = ('https://cluster.klima.uni-bremen.de/~oggm/ref_mb_params/oggm_v1.6/'\n", " 'w5e5_rgi6_perglacier_temp_bias_v2025.6.2.csv')\n", "gdir.name" ] }, { "cell_type": "markdown", "id": "4", "metadata": {}, "source": [ "The WGMS MB is given for hydrological years (October to September), with a\n", "winter (October to April) and a summer (May to September) balance. We keep the\n", "years with all three values. The climate data end in 2019." ] }, { "cell_type": "code", "execution_count": null, "id": "5", "metadata": {}, "outputs": [], "source": [ "ref = gdir.get_ref_mb_data(y0=1980, y1=2019)\n", "ref = ref[['WINTER_BALANCE', 'SUMMER_BALANCE', 'ANNUAL_BALANCE']].dropna()\n", "ref.columns = ['winter', 'summer', 'annual']\n", "years = ref.index.values\n", "print(f'{len(years)} years: {years[0]}-{years[-1]}')\n", "ref.describe().round(0)" ] }, { "cell_type": "markdown", "id": "6", "metadata": {}, "source": [ "# Part 1: one target, the annual MB series\n", "\n", "## Deterministic reference: `mb_calibration_to_rmsd`\n", "\n", "We calibrate the three parameters together, with OGGM's default bounds. Like\n", "AdaPBS below, `mb_calibration_to_rmsd` compares the WGMS data with the modelled\n", "MB of hydrological years." ] }, { "cell_type": "code", "execution_count": null, "id": "7", "metadata": {}, "outputs": [], "source": [ "params = ['melt_f', 'prcp_fac', 'temp_bias']\n", "det = mb_calibration_to_rmsd(gdir, ref_df=ref['annual'], calibrate_params=tuple(params),\n", " write_to_gdir=False, optimisation_kwargs={'seed': 0})\n", "print({p: round(det[p], 2) for p in params}, f\"RMSD: {det['rmsd']:.0f} kg m-2\")" ] }, { "cell_type": "markdown", "id": "8", "metadata": {}, "source": [ "## The prior\n", "\n", "### Why do we need a prior?\n", "\n", "In Bayesian inference, the posterior distribution of the parameters is\n", "proportional to the **prior** times the **likelihood**:\n", "\n", "$$p(\\theta \\mid \\mathrm{obs}) \\propto p(\\theta) \\; p(\\mathrm{obs} \\mid \\theta)$$\n", "\n", "The likelihood says how well a parameter set $\\theta$ = (`melt_f`, `prcp_fac`,\n", "`temp_bias`) explains the observations. The prior says which parameter values we\n", "find plausible *before* looking at the observations: for example, that `prcp_fac`\n", "is probably close to the value estimated from the winter precipitation, and\n", "certainly not 0.01 or 50.\n", "\n", "We need it for two reasons:\n", "\n", "- **The observations alone are often not enough.** When several parameter sets\n", " fit the observations equally well, the prior decides between them. The\n", " deterministic calibration has a prior too, in disguise: the bounds of the\n", " optimisation, and the values at which the parameters which are not calibrated\n", " are fixed.\n", "- **AdaPBS needs somewhere to start**: it draws its first batch of particles\n", " (parameter sets) from the prior.\n", "\n", "The prior does matter for the result. We will see an example of this at the\n", "end of this notebook (section *How much does the prior matter?*).\n", "\n", "### The prior in practice\n", "\n", "Each parameter has a **logit-normal** prior. This is a Gaussian distribution\n", "in a transformed space (the \"normal space\"), which becomes a bounded distribution\n", "in the parameter space: values outside the bounds are impossible. The transform\n", "is a generalised logit function, `glogit`, and its inverse is `gexpit`.\n", "\n", "`mb_prior` (from `oggm_adapbs.massbalance`) defines the centre and bounds of the\n", "prior of each parameter. The centres are OGGM's informed guesses:\n", "\n", "- `melt_f`: 5 (`cfg.PARAMS['melt_f']`)\n", "- `prcp_fac`: from the winter precipitation (as in OGGM's informed three-step\n", " calibration)\n", "- `temp_bias`: from the temperature bias file\n", "\n", "For the comparison with `mb_calibration_to_rmsd`, we give the prior the same\n", "bounds as the deterministic calibration (OGGM's default bounds, from the glacier\n", "settings), with the `prior_bounds` argument. `mb_prior` has narrower default\n", "bounds, which we will use at the end of the notebook.\n", "\n", "The width of the prior is given by its standard deviation in the normal space:\n", "we use 1 for all parameters (more on this below)." ] }, { "cell_type": "code", "execution_count": null, "id": "9", "metadata": {}, "outputs": [], "source": [ "# OGGM's default bounds, as used by mb_calibration_to_rmsd\n", "oggm_bounds = {p: (gdir.settings[f'{p}_min'], gdir.settings[f'{p}_max']) for p in params}\n", "\n", "# The centre and bounds of the prior of each parameter\n", "prior = mb_prior(gdir, params, temp_bias_file_path=temp_bias_file,\n", " prior_bounds=oggm_bounds)\n", "x_min, x_max = prior['x_min'], prior['x_max']\n", "prior" ] }, { "cell_type": "markdown", "id": "10", "metadata": {}, "source": [ "`glogit` transforms values from the parameter space to the normal space, and\n", "`gexpit` does the opposite. The middle of the bounds is 0 in the normal space,\n", "the bounds themselves are at $-\\infty$ and $+\\infty$:" ] }, { "cell_type": "code", "execution_count": null, "id": "11", "metadata": {}, "outputs": [], "source": [ "# The prior centre, the middle of the bounds, and values close to the bounds\n", "x = np.array([prior['x_centre'], (x_min + x_max) / 2, x_min + 0.01, x_max - 0.01])\n", "z = glogit(x, x_min, x_max)\n", "pd.DataFrame(z, columns=params, index=['centre', 'middle', 'near min', 'near max']).round(2)" ] }, { "cell_type": "markdown", "id": "12", "metadata": {}, "source": [ "The prior is a Gaussian centred on the normal-space value of the prior centre,\n", "with a standard deviation of 1. To see what this means for the parameters, we\n", "draw many samples of it in the normal space and transform them back with\n", "`gexpit`.\n", "\n", "`weighted_summary` computes quantiles and the mean of each parameter from samples\n", "and their weights: here all samples have the same weight. We will use it again\n", "for the posterior, where the weights differ." ] }, { "cell_type": "code", "execution_count": null, "id": "13", "metadata": {}, "outputs": [], "source": [ "prior_mean = glogit(prior['x_centre'], x_min, x_max) # the centre, in normal space\n", "prior_sd = 1.\n", "\n", "# Draw 10000 samples in normal space, and transform them to parameter space\n", "rng = np.random.default_rng(0)\n", "prior_theta = gexpit(prior_mean + prior_sd * rng.standard_normal((10000, 3)), x_min, x_max)\n", "\n", "# Quantiles of the prior, with equal weights\n", "prior_df = weighted_summary(prior_theta, np.ones(10000) / 10000, params)\n", "prior_df['centre'], prior_df['min'], prior_df['max'] = prior['x_centre'], x_min, x_max\n", "prior_df.round(2)" ] }, { "cell_type": "markdown", "id": "14", "metadata": {}, "source": [ "The median of the prior is its centre, and 90% of the prior lies between the\n", "`q5` and `q95` values. With these wide bounds, the prior is not very informative:\n", "it only says \"probably not too far from the centre\".\n", "\n", "A standard deviation of 1 in normal space gives a broad, single-peaked\n", "distribution. Centred in the middle of the bounds, a logit-normal with a\n", "standard deviation of ~1.4 looks almost uniform; above $\\sqrt{2}$ it becomes\n", "bimodal, with most of its mass close to the bounds, which is rarely what we\n", "want." ] }, { "cell_type": "markdown", "id": "15", "metadata": {}, "source": [ "## The observation error\n", "\n", "The likelihood compares the predicted and the observed values, and needs to know\n", "how close we expect them to be: the **observation error**. We assume\n", "independent Gaussian errors. The error must include the measurement error *and*\n", "the model error (the MB model with gridded climate data cannot reproduce every\n", "year): the latter dominates.\n", "\n", "We use the RMSD of the deterministic calibration as the error of each annual\n", "value, $\\sigma_a$. With this choice, the most likely parameters of AdaPBS are\n", "those which minimise the RMSD: the two calibrations should agree.\n", "\n", "This is a choice: the width of the posterior scales with it. Also, the model\n", "errors of consecutive years are not independent in reality, which makes the\n", "posterior too narrow." ] }, { "cell_type": "code", "execution_count": null, "id": "16", "metadata": {}, "outputs": [], "source": [ "n = len(years)\n", "sigma_a = det['rmsd']\n", "f'sigma_a = {sigma_a:.0f} kg m-2'" ] }, { "cell_type": "markdown", "id": "17", "metadata": {}, "source": [ "For part 2, we define the observations and errors of each experiment with two\n", "small helper functions: `observe` picks the observed quantities out of a table of\n", "winter, summer and annual MB (we use it for both the observations and the model\n", "output), and `obs_errors` gives their errors. The seasonal errors are explained\n", "in part 2." ] }, { "cell_type": "code", "execution_count": null, "id": "18", "metadata": {}, "outputs": [], "source": [ "sigma = {'annual': sigma_a, 'winter': sigma_a / np.sqrt(2), 'summer': sigma_a / np.sqrt(2)}\n", "\n", "def observe(df, target):\n", " # The quantities used by an experiment, from a DataFrame with the columns\n", " # winter, summer, annual (one row per year). E.g. 'winter+summer': the\n", " # winter series, followed by the summer series\n", " return np.concatenate([df[k].values for k in target.split('+')])\n", "\n", "def obs_errors(target):\n", " # The errors (1 sigma) of these quantities\n", " return np.concatenate([np.repeat(sigma[k], n) for k in target.split('+')])\n", "\n", "# For example: 39 annual values, or 39 + 39 seasonal values\n", "print(len(observe(ref, 'annual')), len(observe(ref, 'winter+summer')))" ] }, { "cell_type": "markdown", "id": "19", "metadata": {}, "source": [ "## The forward model\n", "\n", "AdaPBS does not know anything about glaciers: it only needs a **forward\n", "model**, a function which takes a batch of parameter sets and returns the\n", "predicted observations for each of them. Here, the forward model is OGGM's MB\n", "model on a fixed geometry.\n", "\n", "`static_mb_model` creates the MB model (here `MonthlyTIModel`) of the glacier and\n", "reads its inversion flowlines. We start it with the prior centre and change\n", "the parameters later." ] }, { "cell_type": "code", "execution_count": null, "id": "20", "metadata": {}, "outputs": [], "source": [ "mb_mod, fls = static_mb_model(gdir, prior['centre'])\n", "mb_mod.melt_f, mb_mod.prcp_fac, mb_mod.temp_bias" ] }, { "cell_type": "markdown", "id": "21", "metadata": {}, "source": [ "OGGM's `get_specific_mb` gives the MB of calendar years, but the WGMS data are\n", "given for hydrological years, with seasonal values. `seasonal_specific_mb` sums\n", "the monthly MB of the model into winter (October to April), summer (May to\n", "September) and annual (October to September) values, in kg m-2:" ] }, { "cell_type": "code", "execution_count": null, "id": "22", "metadata": {}, "outputs": [], "source": [ "# The MB of the glacier with the parameters at the prior centre\n", "seasonal_specific_mb(mb_mod, fls, years).head()" ] }, { "cell_type": "markdown", "id": "23", "metadata": {}, "source": [ "Now the forward function itself. AdaPBS calls it as `forward(theta, iteration)`:\n", "\n", "- `theta` is an array with one row per particle and one column per parameter\n", " (in the order of `params`), in the parameter space;\n", "- `iteration` is the number of the AdaPBS iteration (starting at 0). Our MB model\n", " does not need it, but some forward models do: for example, a dynamical model\n", " run which writes output files needs unique file names for each particle of\n", " each iteration. It can also be used to print progress;\n", "- it must return the predicted observations, one row per particle (NaN for a\n", " failed model run: such a particle gets a zero weight).\n", "\n", "It can also return a dictionary of \"extras\", which AdaPBS stores with the\n", "results: we use it to keep the full winter, summer and annual series of each\n", "particle, to look at the fit afterwards.\n", "\n", "We create one forward function per experiment with `make_forward`." ] }, { "cell_type": "code", "execution_count": null, "id": "24", "metadata": {}, "outputs": [], "source": [ "seasons = ['winter', 'summer', 'annual']\n", "\n", "def make_forward(target):\n", " def forward(theta, iteration):\n", " pred, extras = [], {k: [] for k in seasons}\n", " for th in theta: # one particle (parameter set) at a time\n", " # Set the parameters of the MB model\n", " for p, v in zip(params, th):\n", " setattr(mb_mod, p, v)\n", " # Run the model and keep what we need\n", " df = seasonal_specific_mb(mb_mod, fls, years)\n", " for k in seasons:\n", " extras[k].append(df[k].values)\n", " pred.append(observe(df, target))\n", " return np.array(pred), {k: np.array(v) for k, v in extras.items()}\n", " return forward\n", "\n", "# Test it with three particles: the mean of the predicted annual MB of each\n", "forward = make_forward('annual')\n", "pred, extras = forward(np.array([[5, 3, 1.5], [5, 3, 2.5], [5, 4, 1.5]]), 0)\n", "pred.shape, pred.mean(axis=1)" ] }, { "cell_type": "markdown", "id": "25", "metadata": {}, "source": [ "39 predicted values (years) for each of the 3 particles. Warmer (higher\n", "`temp_bias`) means more negative MB, more precipitation (higher `prcp_fac`)\n", "means more positive MB: the model responds as expected.\n", "\n", "## Running AdaPBS\n", "\n", "AdaPBS works in iterations:\n", "\n", "1. draw a batch of particles (300 here) from a proposal distribution: the prior\n", " at the first iteration;\n", "2. run the forward model for each particle;\n", "3. compute the weight of *all* the particles so far, from all iterations: how\n", " likely they are under the posterior (prior times likelihood), relative to how\n", " likely they were to be drawn (importance sampling);\n", "4. if the weights are concentrated on a few particles, build a new proposal\n", " (a Gaussian in normal space) from the best particles, and go back to 1.\n", " Otherwise, stop.\n", "\n", "The result is the set of all particles and their weights: a weighted sample of\n", "the posterior.\n", "\n", "How concentrated the weights are is measured by the **effective sample size**,\n", "$N_{eff} = 1 / \\sum w_i^2$: it is the number of particles if all have the same\n", "weight, and 1 if one particle has all the weight. AdaPBS stops when $N_{eff}$ is\n", "above a fraction (here 30%) of the number of particles per iteration.\n", "\n", "`run_adapbs` does all this. Its main arguments are the forward function, the\n", "observations and their errors, the bounds of the parameters and the prior\n", "(centre and standard deviation, in normal space). We fix the random seed for\n", "reproducible results." ] }, { "cell_type": "code", "execution_count": null, "id": "26", "metadata": {}, "outputs": [], "source": [ "res = run_adapbs(\n", " make_forward('annual'), # the forward model\n", " observe(ref, 'annual'), # the observations\n", " obs_errors('annual'), # their errors (1 sigma)\n", " x_min, x_max, # the parameter bounds\n", " prior_mean=prior_mean, # the prior centre (normal space)\n", " prior_sd=prior_sd, # the prior standard deviation (normal space)\n", " n_particles=300, # particles per iteration\n", " max_iterations=10, # stop after 10 iterations at most\n", " neff_threshold=0.3, # stop when Neff > 30% of n_particles\n", " rng=np.random.default_rng(0), # random generator (reproducible)\n", ")" ] }, { "cell_type": "markdown", "id": "27", "metadata": {}, "source": [ "`run_adapbs` returns a dictionary. Its main entries are:\n", "\n", "- `theta`: all the particles of all iterations (one row each, parameter space)\n", "- `weights`: their posterior weights (they sum to 1)\n", "- `iteration`: the iteration of each particle (starting at 1)\n", "- `pred`: their predicted observations\n", "- `extras`: what our forward function returned as extras\n", "- `posterior_theta`: 300 particles resampled according to the weights (see\n", " below)\n", "- `diagnostics`: a table with one row per iteration" ] }, { "cell_type": "code", "execution_count": null, "id": "28", "metadata": {}, "outputs": [], "source": [ "res['diagnostics'][['iteration', 'n_history', 'neff', 'neff_frac']]" ] }, { "cell_type": "markdown", "id": "29", "metadata": {}, "source": [ "At the first iteration (particles drawn from the prior), Neff = 1: a single\n", "particle carries almost all the weight. The observations (39 values with an\n", "error of 251 kg m-2 each) are much more precise than the prior, so very few\n", "samples of the prior fall where the observations allow. AdaPBS then builds new\n", "proposals from the best particles: they move towards the posterior, and Neff\n", "grows until it passes 30% of 300 (90), at the 6th iteration. In total: 1800\n", "model runs, in about a second.\n", "\n", "The figure shows this in the `melt_f` - `prcp_fac` plane:" ] }, { "cell_type": "code", "execution_count": null, "id": "30", "metadata": {}, "outputs": [], "source": [ "n_it = len(res['diagnostics'])\n", "fig, axes = plt.subplots(1, n_it, figsize=(3 * n_it, 3.2), sharex=True, sharey=True)\n", "for j, ax in enumerate(axes):\n", " it = res['iteration']\n", " ax.scatter(prior_theta[:1000, 0], prior_theta[:1000, 1], s=3, color='0.85')\n", " # Particles of the previous iterations\n", " old = it < j + 1\n", " ax.scatter(res['theta'][old, 0], res['theta'][old, 1], s=4, color='0.55')\n", " # Particles drawn at this iteration\n", " new = it == j + 1\n", " ax.scatter(res['theta'][new, 0], res['theta'][new, 1], s=6, color='#2a78d6')\n", " ax.scatter(det['melt_f'], det['prcp_fac'], marker='X', s=80, color='k',\n", " edgecolor='w', zorder=5)\n", " neff = res['diagnostics']['neff'].iloc[j]\n", " ax.set_title(f'iteration {j + 1}\\nNeff = {neff:.0f}', fontsize=10)\n", " ax.set_xlabel('melt_f')\n", "axes[0].set_ylabel('prcp_fac')\n", "axes[0].set_xlim(1.5, 12)\n", "axes[0].set_ylim(0, 6)\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "id": "31", "metadata": {}, "source": [ "Light grey: samples of the prior. Blue: the particles drawn at each iteration.\n", "Dark grey: the particles of the previous iterations. Black cross:\n", "`mb_calibration_to_rmsd`.\n", "\n", "### Weights and resampled particles\n", "\n", "The weights of the last iteration are the posterior weights of all the\n", "particles, from all iterations. How many of them really count?" ] }, { "cell_type": "code", "execution_count": null, "id": "32", "metadata": {}, "outputs": [], "source": [ "w = np.sort(res['weights'])[::-1]\n", "n90 = np.searchsorted(np.cumsum(w), 0.9) + 1\n", "print(f'{len(w)} particles, {n90} of them carry 90% of the weight')" ] }, { "cell_type": "markdown", "id": "33", "metadata": {}, "source": [ "Most particles have a weight close to zero: the first iterations were mostly\n", "exploring the prior.\n", "\n", "Weighted particles are not always convenient (for example, to run a glacier\n", "model with each of them). `posterior_theta` contains 300 particles resampled\n", "with replacement according to the weights: each of them has the same weight,\n", "and together they are a sample of the posterior. Particles with a large weight\n", "can appear several times.\n", "\n", "## The posterior\n", "\n", "We summarise the posterior of each parameter by its median and 5-95% range,\n", "computed from the weighted particles with `weighted_summary`:" ] }, { "cell_type": "code", "execution_count": null, "id": "34", "metadata": {}, "outputs": [], "source": [ "def fmt(row):\n", " return f\"{row['q50']:.2f} [{row['q5']:.2f}, {row['q95']:.2f}]\"\n", "\n", "table = pd.DataFrame({'prior': prior_df.apply(fmt, axis=1),\n", " 'AdaPBS annual': weighted_summary(res['theta'], res['weights'],\n", " params).apply(fmt, axis=1),\n", " 'mb_calibration_to_rmsd': [f'{det[p]:.2f}' for p in params]}).T\n", "table" ] }, { "cell_type": "code", "execution_count": null, "id": "35", "metadata": {}, "outputs": [], "source": [ "labels = {'melt_f': 'melt_f (kg m-2 day-1 K-1)', 'prcp_fac': 'prcp_fac',\n", " 'temp_bias': 'temp_bias (K)'}\n", "plot_range = {'melt_f': (1.5, 12), 'prcp_fac': (0, 6), 'temp_bias': (-4, 7)}\n", "\n", "def plot_marginals(results, colors):\n", " # Weighted histograms of the particles, one panel per parameter\n", " fig, axes = plt.subplots(1, 3, figsize=(13, 3.8))\n", " for k, (ax, p) in enumerate(zip(axes, params)):\n", " bins = np.linspace(*plot_range[p], 60)\n", " ax.hist(prior_theta[:, k], bins=bins, density=True, color='0.8', label='prior')\n", " for t, r in results.items():\n", " ax.hist(r['theta'][:, k], bins=bins, weights=r['weights'], density=True,\n", " histtype='step', lw=2, color=colors[t], label=t)\n", " ax.axvline(det[p], color='k', ls='--', lw=1.5, label='mb_calibration_to_rmsd')\n", " ax.set_xlabel(labels[p])\n", " ax.set_yticks([])\n", " axes[0].set_ylabel('density')\n", " axes[-1].legend(loc='upper left', bbox_to_anchor=(1.02, 1), frameon=False)\n", " plt.tight_layout()\n", "\n", "def plot_pairs(results, colors):\n", " # The resampled particles, for each pair of parameters\n", " pairs = [('melt_f', 'prcp_fac'), ('melt_f', 'temp_bias'), ('prcp_fac', 'temp_bias')]\n", " fig, axes = plt.subplots(1, 3, figsize=(13, 4))\n", " for ax, (px, py) in zip(axes, pairs):\n", " ix, iy = params.index(px), params.index(py)\n", " ax.scatter(prior_theta[:2000, ix], prior_theta[:2000, iy], s=4, color='0.85',\n", " label='prior')\n", " for t, r in results.items():\n", " post = r['posterior_theta']\n", " ax.scatter(post[:, ix], post[:, iy], s=8, color=colors[t], alpha=0.6,\n", " label=t)\n", " ax.scatter(det[px], det[py], marker='X', s=120, color='k',\n", " edgecolor='w', label='mb_calibration_to_rmsd', zorder=5)\n", " ax.set_xlim(plot_range[px])\n", " ax.set_ylim(plot_range[py])\n", " ax.set_xlabel(labels[px])\n", " ax.set_ylabel(labels[py])\n", " axes[-1].legend(loc='upper left', bbox_to_anchor=(1.02, 1), frameon=False,\n", " markerscale=2)\n", " plt.tight_layout()\n", "\n", "colors = {'annual': '#eb6834', 'winter': '#1baf7a', 'summer': '#eda100',\n", " 'winter+summer': '#e87ba4'}\n", "results = {'annual': res}\n", "plot_marginals(results, colors)\n", "plot_pairs(results, colors)" ] }, { "cell_type": "markdown", "id": "36", "metadata": {}, "source": [ "AdaPBS finds the same solution as `mb_calibration_to_rmsd`: the deterministic\n", "parameters are very close to the posterior median. In addition, it tells us how\n", "well the annual MB constrains the parameters: the posterior is much narrower\n", "than the prior. The pair plots show that the parameters are correlated: a higher\n", "`melt_f` goes with a lower `temp_bias` (more melt per degree, fewer degrees),\n", "and a higher `prcp_fac` with a higher `temp_bias` (more snow, compensated by\n", "more melt)." ] }, { "cell_type": "markdown", "id": "37", "metadata": {}, "source": [ "## Fit to the observations\n", "\n", "How well does the model fit the observations with the posterior parameters?\n", "The extras give us the modelled series of each particle. For each year, we\n", "compute the 5-95% range of the modelled MB over the weighted particles with\n", "`weighted_quantile` (the posterior predictive distribution), and compare it to\n", "the WGMS data. We also look at the winter and summer MB, which we did not use\n", "for the calibration." ] }, { "cell_type": "code", "execution_count": null, "id": "38", "metadata": {}, "outputs": [], "source": [ "def band(r, season, q):\n", " # Weighted quantile q of the modelled MB, for each year\n", " x = r['extras'][season] # one row per particle, one column per year\n", " return np.array([weighted_quantile(x[:, i], r['weights'], q)\n", " for i in range(x.shape[1])])\n", "\n", "# The MB with the parameters of the deterministic calibration\n", "for p in params:\n", " setattr(mb_mod, p, det[p])\n", "det_mb = seasonal_specific_mb(mb_mod, fls, years)\n", "\n", "def plot_fit(results, colors):\n", " fig, axes = plt.subplots(3, 1, figsize=(11, 9), sharex=True)\n", " for ax, s in zip(axes, seasons):\n", " for t, r in results.items():\n", " ax.fill_between(years, band(r, s, 0.05), band(r, s, 0.95),\n", " color=colors[t], alpha=0.35, lw=0, label=f'AdaPBS {t} (5-95%)')\n", " ax.plot(years, det_mb[s], color='k', ls='--', lw=1.5, label='mb_calibration_to_rmsd')\n", " ax.plot(years, ref[s], 'o', color='k', ms=5, label='WGMS')\n", " ax.set_ylabel(f'{s} MB (kg m-2)')\n", " axes[0].legend(ncol=2, loc='upper left', frameon=False, fontsize=9)\n", " axes[0].set_ylim(top=axes[0].get_ylim()[1] + 600)\n", " plt.tight_layout()\n", "\n", "def fit_table(results):\n", " # RMSD and mean bias of the posterior mean MB, per season\n", " rows = {}\n", " for t, r in results.items():\n", " for s in seasons:\n", " m = np.average(r['extras'][s], axis=0, weights=r['weights'])\n", " rows[(t, s)] = {'RMSD': np.sqrt(np.mean((m - ref[s]) ** 2)),\n", " 'bias': np.mean(m - ref[s])}\n", " for s in seasons:\n", " rows[('mb_calibration_to_rmsd', s)] = {\n", " 'RMSD': np.sqrt(np.mean((det_mb[s] - ref[s]) ** 2)),\n", " 'bias': np.mean(det_mb[s] - ref[s])}\n", " return pd.DataFrame(rows).T.unstack().round(0)\n", "\n", "plot_fit(results, colors)\n", "fit_table(results)" ] }, { "cell_type": "markdown", "id": "39", "metadata": {}, "source": [ "Both calibrations fit the annual MB equally well (RMSD 250 kg m-2). But they\n", "get the seasons wrong: the winter MB is too low by ~530-550 kg m-2, and the\n", "summer MB too high by the same amount. The model reaches the right annual MB\n", "with too little accumulation and too little melt. The annual MB cannot see\n", "this: it only constrains the difference between accumulation and melt. This is\n", "where part 2 comes in." ] }, { "cell_type": "markdown", "id": "40", "metadata": {}, "source": [ "# Part 2: several targets, the winter and summer MB\n", "\n", "The winter MB depends mostly on the accumulation (`prcp_fac`), the summer MB\n", "mostly on the melt (`melt_f` and `temp_bias`). Using them as observations should\n", "help where the annual MB cannot. We run three experiments:\n", "\n", "| experiment | observations |\n", "|---|---|\n", "| `winter` | the winter MB time series |\n", "| `summer` | the summer MB time series |\n", "| `winter+summer` | both seasonal series together |\n", "\n", "Errors: $\\sigma_a / \\sqrt{2}$ for each seasonal value, so that the two seasons\n", "together carry the same error as the annual MB (if the two errors are\n", "independent). The prior is the same as in part 1." ] }, { "cell_type": "code", "execution_count": null, "id": "41", "metadata": {}, "outputs": [], "source": [ "{k: round(v) for k, v in sigma.items()}" ] }, { "cell_type": "code", "execution_count": null, "id": "42", "metadata": {}, "outputs": [], "source": [ "for target in ['winter', 'summer', 'winter+summer']:\n", " results[target] = run_adapbs(make_forward(target), observe(ref, target),\n", " obs_errors(target), x_min, x_max,\n", " prior_mean=prior_mean, prior_sd=prior_sd,\n", " n_particles=300, max_iterations=10,\n", " neff_threshold=0.3, rng=np.random.default_rng(0))\n", "\n", "pd.DataFrame({t: {'iterations': len(r['diagnostics']),\n", " 'model runs': len(r['weights']),\n", " 'final Neff': r['diagnostics']['neff'].iloc[-1]}\n", " for t, r in results.items()}).T.round(0)" ] }, { "cell_type": "code", "execution_count": null, "id": "43", "metadata": {}, "outputs": [], "source": [ "table = {'prior': prior_df.apply(fmt, axis=1)}\n", "for t, r in results.items():\n", " table[t] = weighted_summary(r['theta'], r['weights'], params).apply(fmt, axis=1)\n", "table = pd.DataFrame(table).T\n", "table.loc['mb_calibration_to_rmsd'] = [f'{det[p]:.2f}' for p in params]\n", "table" ] }, { "cell_type": "code", "execution_count": null, "id": "44", "metadata": {}, "outputs": [], "source": [ "plot_marginals(results, colors)" ] }, { "cell_type": "code", "execution_count": null, "id": "45", "metadata": {}, "outputs": [], "source": [ "plot_pairs({t: results[t] for t in ['annual', 'winter+summer']}, colors)" ] }, { "cell_type": "code", "execution_count": null, "id": "46", "metadata": {}, "outputs": [], "source": [ "plot_fit({t: results[t] for t in ['annual', 'winter+summer']}, colors)\n", "fit_table({t: results[t] for t in ['annual', 'winter+summer']})" ] }, { "cell_type": "markdown", "id": "47", "metadata": {}, "source": [ "What does each experiment constrain?\n", "\n", "- **`winter`** constrains `prcp_fac` (2.56 [2.41, 2.67]), but not `melt_f`: its\n", " posterior is the prior. `temp_bias` is only loosely constrained (it changes\n", " the fraction of solid precipitation): its posterior extends beyond the plot,\n", " down to -10 K.\n", "- **`summer`** constrains `melt_f` and `temp_bias`, and also prefers a low\n", " `prcp_fac` (summer snowfall is part of the summer balance).\n", "- **`winter+summer`** constrains all three parameters: `prcp_fac` is more than\n", " twice the value of the `annual` experiment. The pair plots show two clearly\n", " separate solutions. `melt_f` and `temp_bias` still trade off along a ridge,\n", " but a narrower one.\n", "\n", "The fit: the winter and summer biases drop from +-530 to less than 60 kg m-2.\n", "The annual RMSD increases from 250 to 316 kg m-2: this is the price for a model\n", "which gets both seasons right." ] }, { "cell_type": "markdown", "id": "48", "metadata": {}, "source": [ "# How much does the prior matter?\n", "\n", "So far, the prior had wide bounds, the same as the deterministic calibration.\n", "`mb_prior` has narrower default bounds, closer to what we consider physically\n", "plausible: `prcp_fac` between half and twice its centre, `temp_bias` +- 3 K\n", "around its centre (`melt_f` keeps OGGM's bounds). Let's run the `annual` and\n", "`winter+summer` experiments again with this prior." ] }, { "cell_type": "code", "execution_count": null, "id": "49", "metadata": {}, "outputs": [], "source": [ "narrow = mb_prior(gdir, params, temp_bias_file_path=temp_bias_file)\n", "narrow_mean = glogit(narrow['x_centre'], narrow['x_min'], narrow['x_max'])\n", "print('prcp_fac bounds:', narrow['x_min'][1].round(2), narrow['x_max'][1].round(2))\n", "print('temp_bias bounds:', narrow['x_min'][2].round(2), narrow['x_max'][2].round(2))\n", "\n", "results_narrow = {}\n", "for target in ['annual', 'winter+summer']:\n", " results_narrow[target] = run_adapbs(make_forward(target), observe(ref, target),\n", " obs_errors(target), narrow['x_min'], narrow['x_max'],\n", " prior_mean=narrow_mean, prior_sd=prior_sd,\n", " n_particles=300, max_iterations=10,\n", " neff_threshold=0.3, rng=np.random.default_rng(0))\n", "\n", "table = {}\n", "for t in ['annual', 'winter+summer']:\n", " for name, rr in [('wide prior', results), ('narrow prior', results_narrow)]:\n", " r = rr[t]\n", " table[(t, name)] = weighted_summary(r['theta'], r['weights'], params).apply(fmt, axis=1)\n", "pd.DataFrame(table).T" ] }, { "cell_type": "markdown", "id": "50", "metadata": {}, "source": [ "With the narrow prior, the `annual` posterior of `prcp_fac` piles up against\n", "its lower bound (1.68): median 1.97, against 1.13 with the wide prior. And\n", "`temp_bias` moves to compensate (2.48 against 1.49). The annual MB wants a low\n", "`prcp_fac`, the prior does not allow it: the result is a compromise decided by\n", "the prior.\n", "\n", "The `winter+summer` posterior does not change: the winter MB constrains\n", "`prcp_fac` well inside both priors.\n", "\n", "This is general: the prior matters most for the parameters (or combinations of\n", "parameters) which the observations do not constrain well. When a posterior\n", "piles up against a prior bound, the prior is doing part of the job: ask whether\n", "the bound is physically justified, or whether you need more observations.\n", "\n", "A more systematic look at the role of the prior (its centre, width and bounds)\n", "will be the topic of another notebook." ] }, { "cell_type": "markdown", "id": "51", "metadata": {}, "source": [ "# Take-aways\n", "\n", "- With one target (the annual MB series), AdaPBS reproduces\n", " `mb_calibration_to_rmsd`: the same parameters, within the posterior range, and\n", " the same RMSD. In addition, it gives the uncertainty of the parameters and\n", " their correlations.\n", "- It needed 1800 model runs (6 iterations) for this: about a second on a\n", " laptop with the static MB model.\n", "- The annual MB alone gives the right annual MB with too small a seasonal\n", " amplitude (too little accumulation and melt, seasonal biases of about\n", " +-530 kg m-2), and cannot tell.\n", "- The winter and summer MB together constrain all three parameters, and the\n", " model fits both seasons. Combining observations of different kinds is where\n", " the Bayesian approach is most useful.\n", "- The prior matters where the observations are not informative enough: here,\n", " for `prcp_fac` with the annual MB only.\n", "\n", "## Caveats\n", "\n", "- The observation errors are a choice (see above), and the years are treated\n", " as independent: the posteriors are probably too narrow.\n", "- The prior matters, especially where the observations do not constrain the\n", " parameters, or where the posterior reaches the prior bounds.\n", "- WGMS seasonal balances are measured either at fixed dates or with the\n", " stratigraphic method: the model's October-April / May-September split is an\n", " approximation.\n", "- The geometry is fixed: this calibrates the MB model, not the glacier evolution." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.15" } }, "nbformat": 4, "nbformat_minor": 5 }