Skip to content

API Reference

Inference

sbi.inference.base.infer(simulator, prior, method, num_simulations, num_workers=1)

Return posterior distribution by running simulation-based inference.

This function provides a simple interface to run sbi. Inference is run for a single round and hence the returned posterior p(\theta|x) can be sampled and evaluated for any x (i.e. it is amortized).

The scope of this function is limited to the most essential features of sbi. For more flexibility (e.g. multi-round inference, different density estimators) please use the flexible interface described here: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters \theta and maps them to simulations, or observations, x, \mathrm{sim}(\theta)\to x. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
method str

What inference method to use. Either of SNPE, SNLE or SNRE.

required
num_simulations int

Number of simulation calls. More simulations means a longer runtime, but a better posterior estimate.

required
num_workers int

Number of parallel workers to use for simulations.

1

Returns: Posterior over parameters conditional on observations (amortized).

Source code in sbi/inference/base.py
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def infer(
    simulator: Callable, prior, method: str, num_simulations: int, num_workers: int = 1
) -> NeuralPosterior:
    r"""
    Return posterior distribution by running simulation-based inference.

    This function provides a simple interface to run sbi. Inference is run for a single
    round and hence the returned posterior $p(\theta|x)$ can be sampled and evaluated
    for any $x$ (i.e. it is amortized).

    The scope of this function is limited to the most essential features of sbi. For
    more flexibility (e.g. multi-round inference, different density estimators) please
    use the flexible interface described here:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\mathrm{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        method: What inference method to use. Either of SNPE, SNLE or SNRE.
        num_simulations: Number of simulation calls. More simulations means a longer
            runtime, but a better posterior estimate.
        num_workers: Number of parallel workers to use for simulations.

    Returns: Posterior over parameters conditional on observations (amortized).
    """

    try:
        method_fun: Callable = getattr(sbi.inference, method.upper())
    except AttributeError:
        raise NameError(
            "Method not available. `method` must be one of 'SNPE', 'SNLE', 'SNRE'."
        )

    simulator, prior = prepare_for_sbi(simulator, prior)

    inference = method_fun(prior)
    theta, x = simulate_for_sbi(
        simulator=simulator,
        proposal=prior,
        num_simulations=num_simulations,
        num_workers=num_workers,
    )
    _ = inference.append_simulations(theta, x).train()
    posterior = inference.build_posterior()

    return posterior

sbi.utils.user_input_checks.prepare_for_sbi(simulator, prior)

Prepare simulator, prior and for usage in sbi.

One of the goals is to allow you to use sbi with inputs computed in numpy.

Attempts to meet the following requirements by reshaping and type-casting:

  • the simulator function receives as input and returns a Tensor.
  • the simulator can simulate batches of parameters and return batches of data.
  • the prior does not produce batches and samples and evaluates to Tensor.
  • the output shape is a torch.Size((1,N)) (i.e, has a leading batch dimension 1).

If this is not possible, a suitable exception will be raised.

Parameters:

Name Type Description Default
simulator Callable

Simulator as provided by the user.

required
prior

Prior as provided by the user.

required

Returns:

Type Description
Tuple[Callable, torch.distributions.distribution.Distribution]

Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.

Source code in sbi/utils/user_input_checks.py
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
def prepare_for_sbi(simulator: Callable, prior) -> Tuple[Callable, Distribution]:
    """Prepare simulator, prior and for usage in sbi.

    One of the goals is to allow you to use sbi with inputs computed in numpy.

    Attempts to meet the following requirements by reshaping and type-casting:

    - the simulator function receives as input and returns a Tensor.<br/>
    - the simulator can simulate batches of parameters and return batches of data.<br/>
    - the prior does not produce batches and samples and evaluates to Tensor.<br/>
    - the output shape is a `torch.Size((1,N))` (i.e, has a leading batch dimension 1).

    If this is not possible, a suitable exception will be raised.

    Args:
        simulator: Simulator as provided by the user.
        prior: Prior as provided by the user.

    Returns:
        Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.
    """

    # Check prior, return PyTorch prior.
    prior, _, prior_returns_numpy = process_prior(prior)

    # Check simulator, returns PyTorch simulator able to simulate batches.
    simulator = process_simulator(simulator, prior, prior_returns_numpy)

    # Consistency check after making ready for sbi.
    check_sbi_inputs(simulator, prior)

    return simulator, prior

sbi.inference.base.simulate_for_sbi(simulator, proposal, num_simulations, num_workers=1, simulation_batch_size=1, show_progress_bar=True)

Returns (\theta, x) pairs obtained from sampling the proposal and simulating.

This function performs two steps:

  • Sample parameters \theta from the proposal.
  • Simulate these parameters to obtain x.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters \theta and maps them to simulations, or observations, x, \text{sim}(\theta)\to x. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
proposal Any

Probability distribution that the parameters \theta are sampled from.

required
num_simulations int

Number of simulations that are run.

required
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bar bool

Whether to show a progress bar for simulating. This will not affect whether there will be a progressbar while drawing samples from the proposal.

True

Returns: Sampled parameters \theta and simulation-outputs x.

Source code in sbi/inference/base.py
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
def simulate_for_sbi(
    simulator: Callable,
    proposal: Any,
    num_simulations: int,
    num_workers: int = 1,
    simulation_batch_size: int = 1,
    show_progress_bar: bool = True,
) -> Tuple[Tensor, Tensor]:
    r"""
    Returns ($\theta, x$) pairs obtained from sampling the proposal and simulating.

    This function performs two steps:

    - Sample parameters $\theta$ from the `proposal`.
    - Simulate these parameters to obtain $x$.

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\text{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        proposal: Probability distribution that the parameters $\theta$ are sampled
            from.
        num_simulations: Number of simulations that are run.
        num_workers: Number of parallel workers to use for simulations.
        simulation_batch_size: Number of parameter sets that the simulator
            maps to data x at once. If None, we simulate all parameter sets at the
            same time. If >= 1, the simulator has to process data of shape
            (simulation_batch_size, parameter_dimension).
        show_progress_bar: Whether to show a progress bar for simulating. This will not
            affect whether there will be a progressbar while drawing samples from the
            proposal.

    Returns: Sampled parameters $\theta$ and simulation-outputs $x$.
    """

    check_if_proposal_has_default_x(proposal)

    theta = proposal.sample((num_simulations,))

    x = simulate_in_batches(
        simulator,
        theta,
        simulation_batch_size,
        num_workers,
        show_progress_bar,
    )

    return theta, x

sbi.inference.snpe.snpe_c.SNPE_C

__init__(self, prior=None, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special

SNPE-C / APT [1].

[1] Automatic Posterior Transformation for Likelihood-free Inference, Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

This class implements two loss variants of SNPE-C: the non-atomic and the atomic version. The atomic loss of SNPE-C can be used for any density estimator, i.e. also for normalizing flows. However, it suffers from leakage issues. On the other hand, the non-atomic loss can only be used only if the proposal distribution is a mixture of Gaussians, the density estimator is a mixture of Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from leakage issues. At the beginning of each round, we print whether the non-atomic or the atomic version is used.

In this codebase, we will automatically switch to the non-atomic loss if the following criteria are fulfilled:
- proposal is a DirectPosterior with density_estimator mdn, as built with utils.sbi.posterior_nn().
- the density estimator is a mdn, as built with utils.sbi.posterior_nn().
- isinstance(prior, MultivariateNormal) (from torch.distributions) or isinstance(prior, sbi.utils.BoxUniform)

Note that custom implementations of any of these densities (or estimators) will not trigger the non-atomic loss, and the algorithm will fall back onto using the atomic loss.

Parameters:

Name Type Description Default
prior Optional[Any]

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

None
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during training.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snpe/snpe_c.py
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
def __init__(
    self,
    prior: Optional[Any] = None,
    density_estimator: Union[str, Callable] = "maf",
    device: str = "cpu",
    logging_level: Union[int, str] = "WARNING",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args,
):
    r"""SNPE-C / APT [1].

    [1] _Automatic Posterior Transformation for Likelihood-free Inference_,
        Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

    This class implements two loss variants of SNPE-C: the non-atomic and the atomic
    version. The atomic loss of SNPE-C can be used for any density estimator,
    i.e. also for normalizing flows. However, it suffers from leakage issues. On
    the other hand, the non-atomic loss can only be used only if the proposal
    distribution is a mixture of Gaussians, the density estimator is a mixture of
    Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from
    leakage issues. At the beginning of each round, we print whether the non-atomic
    or the atomic version is used.

    In this codebase, we will automatically switch to the non-atomic loss if the
    following criteria are fulfilled:<br/>
    - proposal is a `DirectPosterior` with density_estimator `mdn`, as built
        with `utils.sbi.posterior_nn()`.<br/>
    - the density estimator is a `mdn`, as built with
        `utils.sbi.posterior_nn()`.<br/>
    - `isinstance(prior, MultivariateNormal)` (from `torch.distributions`) or
        `isinstance(prior, sbi.utils.BoxUniform)`

    Note that custom implementations of any of these densities (or estimators) will
    not trigger the non-atomic loss, and the algorithm will fall back onto using
    the atomic loss.

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        density_estimator: If it is a string, use a pre-configured network of the
            provided type (one of nsf, maf, mdn, made). Alternatively, a function
            that builds a custom neural network can be provided. The function will
            be called with the first batch of simulations (theta, x), which can
            thus be used for shape inference and potentially for z-scoring. It
            needs to return a PyTorch `nn.Module` implementing the density
            estimator. The density estimator needs to provide the methods
            `.log_prob` and `.sample()`.
        device: torch device on which to compute, e.g. gpu, cpu.
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during training.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, proposal=None) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \theta, x, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
proposal Optional[Any]

The distribution that the parameters \theta were sampled from. Pass None if the parameters were sampled from the prior. If not None, it will trigger a different loss-function.

None

Returns:

Type Description
PosteriorEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snpe/snpe_c.py
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
def append_simulations(
    self, theta: Tensor, x: Tensor, proposal: Optional[Any] = None
) -> "PosteriorEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        proposal: The distribution that the parameters $\theta$ were sampled from.
            Pass `None` if the parameters were sampled from the prior. If not
            `None`, it will trigger a different loss-function.

    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)
    self._check_proposal(proposal)

    if (
        proposal is None
        or proposal is self._prior
        or (
            isinstance(proposal, RestrictedPrior) and proposal._prior is self._prior
        )
    ):
        # The `_data_round_index` will later be used to infer if one should train
        # with MLE loss or with atomic loss (see, in `train()`:
        # self._round = max(self._data_round_index))
        self._data_round_index.append(0)
        self._prior_masks.append(mask_sims_from_prior(0, theta.size(0)))
    else:
        if not self._data_round_index:
            # This catches a pretty specific case: if, in the first round, one
            # passes data that does not come from the prior.
            self._data_round_index.append(1)
        else:
            self._data_round_index.append(max(self._data_round_index) + 1)
        self._prior_masks.append(mask_sims_from_prior(1, theta.size(0)))

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._proposal_roundwise.append(proposal)

    if self._prior is None or isinstance(self._prior, ImproperEmpirical):
        if proposal is not None:
            raise ValueError(
                "You had not passed a prior at initialization, but now you "
                "passed a proposal. If you want to run multi-round SNPE, you have "
                "to specify a prior (set the `.prior` argument or re-initialize "
                "the object with a prior distribution). If the samples you passed "
                "to `append_simulations()` were sampled from the prior, you can "
                "run single-round inference with "
                "`append_simulations(..., proposal=None)`."
            )
        theta_prior = self.get_simulations()[0]
        self._prior = ImproperEmpirical(theta_prior, ones(theta_prior.shape[0]))

    return self

build_posterior(self, density_estimator=None, rejection_sampling_parameters=None, sample_with_mcmc=False, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

For SNPE, the posterior distribution that is returned here implements the following functionality over the raw neural density estimator:

  • correct the calculation of the log probability such that it compensates for the leakage.
  • reject samples that lie outside of the prior bounds.
  • alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

None
sample_with_mcmc bool

Whether to sample with MCMC. MCMC can be used to deal with high leakage.

False
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
DirectPosterior

Posterior p(\theta|x) with .sample() and .log_prob() methods.

Source code in sbi/inference/snpe/snpe_c.py
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    sample_with_mcmc: bool = False,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> DirectPosterior:
    r"""
    Build posterior from the neural density estimator.

    For SNPE, the posterior distribution that is returned here implements the
    following functionality over the raw neural density estimator:

    - correct the calculation of the log probability such that it compensates for
        the leakage.
    - reject samples that lie outside of the prior bounds.
    - alternatively, if leakage is very high (which can happen for multi-round
        SNPE), sample from the posterior with MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `max_sampling_batch_size` to set the batch size for drawing new
            samples from the candidate distribution, e.g., the posterior. Larger
            batch size speeds up sampling.
        sample_with_mcmc: Whether to sample with MCMC. MCMC can be used to deal
            with high leakage.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods.
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device

    self._posterior = DirectPosterior(
        method_family="snpe",
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        rejection_sampling_parameters=rejection_sampling_parameters,
        sample_with_mcmc=sample_with_mcmc,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snpe/snpe_c.py
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \theta, x, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snpe/snpe_c.py
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \theta and x to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snpe/snpe_c.py
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, calibration_kernel=None, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, use_combined_loss=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return density estimator that approximates the distribution p(\theta|x).

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
calibration_kernel Optional[Callable]

A function to calibrate the loss with respect to the simulations x. See Lueckmann, Gonçalves et al., NeurIPS 2017.

None
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
use_combined_loss bool

Whether to train the neural net also on prior samples using maximum likelihood in addition to training it on all samples using atomic loss. The extra MLE loss helps prevent density leaking with bounded priors.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
DirectPosterior

Density estimator that approximates the distribution p(\theta|x).

Source code in sbi/inference/snpe/snpe_c.py
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
def train(
    self,
    num_atoms: int = 10,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    calibration_kernel: Optional[Callable] = None,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    use_combined_loss: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> DirectPosterior:
    r"""
    Return density estimator that approximates the distribution $p(\theta|x)$.

    Args:
        num_atoms: Number of atoms to use for classification.
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        calibration_kernel: A function to calibrate the loss with respect to the
            simulations `x`. See Lueckmann, Gonçalves et al., NeurIPS 2017.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        use_combined_loss: Whether to train the neural net also on prior samples
            using maximum likelihood in addition to training it on all samples using
            atomic loss. The extra MLE loss helps prevent density leaking with
            bounded priors.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Density estimator that approximates the distribution $p(\theta|x)$.
    """

    # WARNING: sneaky trick ahead. We proxy the parent's `train` here,
    # requiring the signature to have `num_atoms`, save it for use below, and
    # continue. It's sneaky because we are using the object (self) as a namespace
    # to pass arguments between functions, and that's implicit state management.
    self._num_atoms = num_atoms
    self._use_combined_loss = use_combined_loss
    kwargs = del_entries(
        locals(), entries=("self", "__class__", "num_atoms", "use_combined_loss")
    )

    self._round = max(self._data_round_index)

    if self._round > 0:
        # Set the proposal to the last proposal that was passed by the user. For
        # atomic SNPE, it does not matter what the proposal is. For non-atomic
        # SNPE, we only use the latest data that was passed, i.e. the one from the
        # last proposal.
        proposal = self._proposal_roundwise[-1]
        if hasattr(proposal, "net"):
            self.use_non_atomic_loss = (
                isinstance(proposal.net._distribution, mdn)
                and isinstance(self._neural_net._distribution, mdn)
                and check_dist_class(
                    self._prior, class_to_check=(Uniform, MultivariateNormal)
                )[0]
            )
        else:
            self.use_non_atomic_loss = False

        algorithm = "non-atomic" if self.use_non_atomic_loss else "atomic"
        print(f"Using SNPE-C with {algorithm} loss")

        if self.use_non_atomic_loss:
            # Take care of z-scoring, pre-compute and store prior terms.
            self._set_state_for_mog_proposal()

    return super().train(**kwargs)

sbi.inference.snle.snle_a.SNLE_A

__init__(self, prior, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special

Sequential Neural Likelihood [1].

[1] Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows_, Papamakarios et al., AISTATS 2019, https://arxiv.org/abs/1805.07226

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snle/snle_a.py
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def __init__(
    self,
    prior,
    density_estimator: Union[str, Callable] = "maf",
    device: str = "cpu",
    logging_level: Union[int, str] = "WARNING",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args,
):
    r"""Sequential Neural Likelihood [1].

    [1] Sequential Neural Likelihood: Fast Likelihood-free Inference with
    Autoregressive Flows_, Papamakarios et al., AISTATS 2019,
    https://arxiv.org/abs/1805.07226

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        density_estimator: If it is a string, use a pre-configured network of the
            provided type (one of nsf, maf, mdn, made). Alternatively, a function
            that builds a custom neural network can be provided. The function will
            be called with the first batch of simulations (theta, x), which can
            thus be used for shape inference and potentially for z-scoring. It
            needs to return a PyTorch `nn.Module` implementing the density
            estimator. The density estimator needs to provide the methods
            `.log_prob` and `.sample()`.
        device: torch device on which to compute, e.g. gpu, cpu.
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \theta, x, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNLE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
LikelihoodEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snle/snle_a.py
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
def append_simulations(
    self,
    theta: Tensor,
    x: Tensor,
    from_round: int = 0,
) -> "LikelihoodEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        from_round: Which round the data stemmed from. Round 0 means from the prior.
            With default settings, this is not used at all for `SNLE`. Only when
            the user later on requests `.train(discard_prior_samples=True)`, we
            use these indices to find which training data stemmed from the prior.

    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0)))
    self._data_round_index.append(int(from_round))

    return self

build_posterior(self, density_estimator=None, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

SNLE trains a neural network to approximate the likelihood p(x|\theta). The LikelihoodBasedPosterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability p(\theta|x) \propto p(x|\theta) \cdot p(\theta) and draw samples from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
LikelihoodBasedPosterior

Posterior p(\theta|x) with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snle/snle_a.py
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> LikelihoodBasedPosterior:
    r"""
    Build posterior from the neural density estimator.

    SNLE trains a neural network to approximate the likelihood $p(x|\theta)$. The
    `LikelihoodBasedPosterior` class wraps the trained network such that one can
    directly evaluate the unnormalized posterior log probability
    $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the
    posterior with MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods
        (the returned log-probability is unnormalized).
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device

    self._posterior = LikelihoodBasedPosterior(
        method_family="snle",
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snle/snle_a.py
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \theta, x, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snle/snle_a.py
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \theta and x to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snle/snle_a.py
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return density estimator that approximates the distribution p(x|\theta).

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Density estimator that approximates the distribution p(x|\theta).

Source code in sbi/inference/snle/snle_a.py
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def train(
    self,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> NeuralPosterior:
    r"""
    Return density estimator that approximates the distribution $p(x|\theta)$.

    Args:
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Density estimator that approximates the distribution $p(x|\theta)$.
    """
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    return super().train(**kwargs)

sbi.inference.snre.snre_a.SNRE_A

__init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special

AALR[1], here known as SNRE_A.

[1] Likelihood-free MCMC with Amortized Approximate Likelihood Ratios, Hermans et al., ICML 2020, https://arxiv.org/abs/1903.04057

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_a.py
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
def __init__(
    self,
    prior,
    classifier: Union[str, Callable] = "resnet",
    device: str = "cpu",
    logging_level: Union[int, str] = "warning",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args
):
    r"""AALR[1], here known as SNRE_A.

    [1] _Likelihood-free MCMC with Amortized Approximate Likelihood Ratios_, Hermans
        et al., ICML 2020, https://arxiv.org/abs/1903.04057

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        classifier: Classifier trained to approximate likelihood ratios. If it is
            a string, use a pre-configured network of the provided type (one of
            linear, mlp, resnet). Alternatively, a function that builds a custom
            neural network can be provided. The function will be called with the
            first batch of simulations (theta, x), which can thus be used for shape
            inference and potentially for z-scoring. It needs to return a PyTorch
            `nn.Module` implementing the classifier.
        device: torch device on which to compute, e.g. gpu, cpu.
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \theta, x, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_a.py
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
def append_simulations(
    self,
    theta: Tensor,
    x: Tensor,
    from_round: int = 0,
) -> "RatioEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        from_round: Which round the data stemmed from. Round 0 means from the prior.
            With default settings, this is not used at all for `SNRE`. Only when
            the user later on requests `.train(discard_prior_samples=True)`, we
            use these indices to find which training data stemmed from the prior.
    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0)))
    self._data_round_index.append(int(from_round))

    return self

build_posterior(self, density_estimator=None, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior p(\theta|x) \propto p(x|\theta) \cdot p(\theta). The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability p(\theta|x) \propto p(x|\theta) \cdot p(\theta) and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
RatioBasedPosterior

Posterior p(\theta|x) with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_a.py
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> RatioBasedPosterior:
    r"""
    Build posterior from the neural density estimator.

    SNRE trains a neural network to approximate likelihood ratios, which in turn
    can be used obtain an unnormalized posterior
    $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here
    wraps the trained network such that one can directly evaluate the unnormalized
    posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and
    draw samples from the posterior with MCMC. Note that, in the case of
    single-round SNRE_A / AALR, it is possible to evaluate the log-probability of
    the **normalized** posterior, but sampling still requires MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods
        (the returned log-probability is unnormalized).
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device

    self._posterior = RatioBasedPosterior(
        method_family=self.__class__.__name__.lower(),
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_a.py
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \theta, x, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_a.py
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \theta and x to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_a.py
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return classifier that approximates the ratio p(\theta,x)/p(\theta)p(x).

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio p(\theta,x)/p(\theta)p(x).

Source code in sbi/inference/snre/snre_a.py
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def train(
    self,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> NeuralPosterior:
    r"""
    Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

    Args:
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.
    """

    # AALR is defined for `num_atoms=2`.
    # Proxy to `super().__call__` to ensure right parameter.
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    return super().train(**kwargs, num_atoms=2)

sbi.inference.snre.snre_b.SNRE_B

__init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special

SRE[1], here known as SNRE_B.

[1] On Contrastive Learning for Likelihood-free Inference, Durkan et al., ICML 2020, https://arxiv.org/pdf/2002.03712

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_b.py
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
def __init__(
    self,
    prior,
    classifier: Union[str, Callable] = "resnet",
    device: str = "cpu",
    logging_level: Union[int, str] = "warning",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args
):
    r"""SRE[1], here known as SNRE_B.

    [1] _On Contrastive Learning for Likelihood-free Inference_, Durkan et al.,
        ICML 2020, https://arxiv.org/pdf/2002.03712

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        classifier: Classifier trained to approximate likelihood ratios. If it is
            a string, use a pre-configured network of the provided type (one of
            linear, mlp, resnet). Alternatively, a function that builds a custom
            neural network can be provided. The function will be called with the
            first batch of simulations (theta, x), which can thus be used for shape
            inference and potentially for z-scoring. It needs to return a PyTorch
            `nn.Module` implementing the classifier.
        device: torch device on which to compute, e.g. gpu, cpu.
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \theta, x, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_b.py
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
def append_simulations(
    self,
    theta: Tensor,
    x: Tensor,
    from_round: int = 0,
) -> "RatioEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        from_round: Which round the data stemmed from. Round 0 means from the prior.
            With default settings, this is not used at all for `SNRE`. Only when
            the user later on requests `.train(discard_prior_samples=True)`, we
            use these indices to find which training data stemmed from the prior.
    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0)))
    self._data_round_index.append(int(from_round))

    return self

build_posterior(self, density_estimator=None, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior p(\theta|x) \propto p(x|\theta) \cdot p(\theta). The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability p(\theta|x) \propto p(x|\theta) \cdot p(\theta) and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
RatioBasedPosterior

Posterior p(\theta|x) with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_b.py
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> RatioBasedPosterior:
    r"""
    Build posterior from the neural density estimator.

    SNRE trains a neural network to approximate likelihood ratios, which in turn
    can be used obtain an unnormalized posterior
    $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here
    wraps the trained network such that one can directly evaluate the unnormalized
    posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and
    draw samples from the posterior with MCMC. Note that, in the case of
    single-round SNRE_A / AALR, it is possible to evaluate the log-probability of
    the **normalized** posterior, but sampling still requires MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods
        (the returned log-probability is unnormalized).
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device

    self._posterior = RatioBasedPosterior(
        method_family=self.__class__.__name__.lower(),
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_b.py
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \theta, x, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_b.py
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \theta and x to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_b.py
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return classifier that approximates the ratio p(\theta,x)/p(\theta)p(x).

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio p(\theta,x)/p(\theta)p(x).

Source code in sbi/inference/snre/snre_b.py
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
def train(
    self,
    num_atoms: int = 10,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> NeuralPosterior:
    r"""
    Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

    Args:
        num_atoms: Number of atoms to use for classification.
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.
    """
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    return super().train(**kwargs)

sbi.inference.abc.mcabc.MCABC

__call__(self, x_o, num_simulations, eps=None, quantile=None, return_distances=False, return_x_accepted=False, lra=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1) special

Run MCABC.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_simulations int

Number of simulations to run.

required
eps Optional[float]

Acceptance threshold \epsilon for distance between observed and simulated data.

None
quantile Optional[float]

Upper quantile of smallest distances for which the corresponding parameters are returned, e.g, q=0.01 will return the top 1%. Exactly one of quantile or eps have to be passed.

None
return_distances bool

Whether to return the distances corresponding to the accepted parameters.

False
return_distances bool

Whether to return the simulated data corresponding to the accepted parameters.

False
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1

Returns:

Type Description
Union[torch.distributions.distribution.Distribution, Tuple[torch.distributions.distribution.Distribution, torch.Tensor]]

posterior: Empirical distribution based on selected parameters. distances: Tensor of distances of the selected parameters.

Source code in sbi/inference/abc/mcabc.py
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
def __call__(
    self,
    x_o: Union[Tensor, ndarray],
    num_simulations: int,
    eps: Optional[float] = None,
    quantile: Optional[float] = None,
    return_distances: bool = False,
    return_x_accepted: bool = False,
    lra: bool = False,
    sass: bool = False,
    sass_fraction: float = 0.25,
    sass_expansion_degree: int = 1,
) -> Union[Distribution, Tuple[Distribution, Tensor]]:
    r"""Run MCABC.

    Args:
        x_o: Observed data.
        num_simulations: Number of simulations to run.
        eps: Acceptance threshold $\epsilon$ for distance between observed and
            simulated data.
        quantile: Upper quantile of smallest distances for which the corresponding
            parameters are returned, e.g, q=0.01 will return the top 1%. Exactly
            one of quantile or `eps` have to be passed.
        return_distances: Whether to return the distances corresponding to
            the accepted parameters.
        return_distances: Whether to return the simulated data corresponding to
            the accepted parameters.
        lra: Whether to run linear regression adjustment as in Beaumont et al. 2002
        sass: Whether to determine semi-automatic summary statistics as in
            Fearnhead & Prangle 2012.
        sass_fraction: Fraction of simulation budget used for the initial sass run.
        sass_expansion_degree: Degree of the polynomial feature expansion for the
            sass regression, default 1 - no expansion.

    Returns:
        posterior: Empirical distribution based on selected parameters.
        distances: Tensor of distances of the selected parameters.
    """
    # Exactly one of eps or quantile need to be passed.
    assert (eps is not None) ^ (
        quantile is not None
    ), "Eps or quantile must be passed, but not both."

    # Run SASS and change the simulator and x_o accordingly.
    if sass:
        num_pilot_simulations = int(sass_fraction * num_simulations)
        self.logger.info(
            f"Running SASS with {num_pilot_simulations} pilot samples."
        )
        num_simulations -= num_pilot_simulations

        pilot_theta = self.prior.sample((num_pilot_simulations,))
        pilot_x = self._batched_simulator(pilot_theta)

        sass_transform = self.get_sass_transform(
            pilot_theta, pilot_x, sass_expansion_degree
        )

        simulator = lambda theta: sass_transform(self._batched_simulator(theta))
        x_o = sass_transform(x_o)
    else:
        simulator = self._batched_simulator

    # Simulate and calculate distances.
    theta = self.prior.sample((num_simulations,))
    x = simulator(theta)

    # Infer shape of x to test and set x_o.
    self.x_shape = x[0].unsqueeze(0).shape
    self.x_o = process_x(x_o, self.x_shape)

    distances = self.distance(self.x_o, x)

    # Select based on acceptance threshold epsilon.
    if eps is not None:
        is_accepted = distances < eps
        num_accepted = is_accepted.sum().item()
        assert num_accepted > 0, f"No parameters accepted, eps={eps} too small"

        theta_accepted = theta[is_accepted]
        distances_accepted = distances[is_accepted]
        x_accepted = x[is_accepted]

    # Select based on quantile on sorted distances.
    elif quantile is not None:
        num_top_samples = int(num_simulations * quantile)
        sort_idx = torch.argsort(distances)
        theta_accepted = theta[sort_idx][:num_top_samples]
        distances_accepted = distances[sort_idx][:num_top_samples]
        x_accepted = x[sort_idx][:num_top_samples]

    else:
        raise ValueError("One of epsilon or quantile has to be passed.")

    # Maybe adjust theta with LRA.
    if lra:
        self.logger.info("Running Linear regression adjustment.")
        theta_adjusted = self.run_lra(
            theta_accepted, x_accepted, observation=self.x_o
        )
    else:
        theta_adjusted = theta_accepted

    posterior = Empirical(theta_adjusted, log_weights=ones(theta_accepted.shape[0]))

    if return_distances and return_x_accepted:
        return posterior, distances_accepted, x_accepted
    if return_distances:
        return posterior, distances_accepted
    if return_x_accepted:
        return posterior, x_accepted
    else:
        return posterior

__init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True) special

Monte-Carlo Approximate Bayesian Computation (Rejection ABC) [1].

[1] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W. (1999). Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $ heta$ and maps them to simulations, or observations, x, \mathrm{sim}( heta) o x. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
Source code in sbi/inference/abc/mcabc.py
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def __init__(
    self,
    simulator: Callable,
    prior,
    distance: Union[str, Callable] = "l2",
    num_workers: int = 1,
    simulation_batch_size: int = 1,
    show_progress_bars: bool = True,
):
    """Monte-Carlo Approximate Bayesian Computation (Rejection ABC) [1].

    [1] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W.
    (1999). Population growth of human Y chromosomes: a study of Y chromosome
    microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\mathrm{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        distance: Distance function to compare observed and simulated data. Can be
            a custom function or one of `l1`, `l2`, `mse`.
        num_workers: Number of parallel workers to use for simulations.
        simulation_batch_size: Number of parameter sets that the simulator
            maps to data x at once. If None, we simulate all parameter sets at the
            same time. If >= 1, the simulator has to process data of shape
            (simulation_batch_size, parameter_dimension).
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
    """

    super().__init__(
        simulator=simulator,
        prior=prior,
        distance=distance,
        num_workers=num_workers,
        simulation_batch_size=simulation_batch_size,
        show_progress_bars=show_progress_bars,
    )

choose_distance_function(distance_type='l2') inherited

Return distance function for given distance type.

Source code in sbi/inference/abc/mcabc.py
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
@staticmethod
def choose_distance_function(distance_type: str = "l2") -> Callable:
    """Return distance function for given distance type."""

    if distance_type == "mse":
        distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1)
    elif distance_type == "l2":
        distance = lambda xo, x: torch.norm((xo - x), dim=-1)
    elif distance_type == "l1":
        distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1)
    else:
        raise ValueError(r"Distance {distance_type} not supported.")

    def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor:
        """Return distance over batch dimension.

        Args:
            observed_data: Observed data, could be 1D.
            simulated_data: Batch of simulated data, has batch dimension.

        Returns:
            Torch tensor with batch of distances.
        """
        assert simulated_data.ndim == 2, "simulated data needs batch dimension"

        return distance(observed_data, simulated_data)

    return distance_fun

get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Following implementation in https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity and https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic

Source code in sbi/inference/abc/mcabc.py
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
@staticmethod
def get_sass_transform(
    theta: torch.Tensor,
    x: torch.Tensor,
    expansion_degree: int = 1,
    sample_weight=None,
) -> Callable:
    """Return semi-automatic summary statitics function.

    Running weighted linear regressin as in
    Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

    Following implementation in
    https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity
    and
    https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic
    """
    expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False)
    # Transform x, remove intercept.
    x_expanded = expansion.fit_transform(x)
    sumstats_map = np.zeros((x_expanded.shape[1], theta.shape[1]))

    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight
        )
        sumstats_map[:, parameter_idx] = regression_model.coef_

    sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32)

    def sumstats_transform(x):
        x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32)
        return x_expanded.mm(sumstats_map)

    return sumstats_transform

run_lra(theta, x, observation, sample_weight=None) inherited

Return parameters adjusted with linear regression adjustment.

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/mcabc.py
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
@staticmethod
def run_lra(
    theta: torch.Tensor,
    x: torch.Tensor,
    observation: torch.Tensor,
    sample_weight=None,
) -> torch.Tensor:
    """Return parameters adjusted with linear regression adjustment.

    Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254
    """

    theta_adjusted = theta
    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x,
            y=theta[:, parameter_idx],
            sample_weight=sample_weight,
        )
        theta_adjusted[:, parameter_idx] += regression_model.predict(
            observation.reshape(1, -1)
        )
        theta_adjusted[:, parameter_idx] -= regression_model.predict(x)

    return theta_adjusted

sbi.inference.abc.smcabc.SMCABC

__call__(self, x_o, num_particles, num_initial_pop, num_simulations, epsilon_decay, distance_based_decay=False, ess_min=None, kernel_variance_scale=1.0, use_last_pop_samples=True, return_summary=False, lra=False, lra_with_weights=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1) special

Run SMCABC.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_particles int

Number of particles in each population.

required
num_initial_pop int

Number of simulations used for initial population.

required
num_simulations int

Total number of possible simulations.

required
epsilon_decay float

Factor with which the acceptance threshold \epsilon decays.

required
distance_based_decay bool

Whether the \epsilon decay is constant over populations or calculated from the previous populations distribution of distances.

False
ess_min Optional[float]

Threshold of effective sampling size for resampling weights. Not used when None (default).

None
kernel_variance_scale float

Factor for scaling the perturbation kernel variance.

1.0
use_last_pop_samples bool

Whether to fill up the current population with samples from the previous population when the budget is used up. If False, the current population is discarded and the previous population is returned.

True
return_summary bool

Whether to return a dictionary with all accepted particles, weights, etc. at the end.

False
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
lra_with_weights bool

Whether to run lra as weighted linear regression with SMC weights

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1

Returns:

Type Description
Union[torch.distributions.distribution.Distribution, Tuple[torch.distributions.distribution.Distribution, dict]]

posterior: Empirical posterior distribution defined by the accepted particles and their weights. summary (optional): A dictionary containing particles, weights, epsilons and distances of each population.

Source code in sbi/inference/abc/smcabc.py
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
def __call__(
    self,
    x_o: Union[Tensor, ndarray],
    num_particles: int,
    num_initial_pop: int,
    num_simulations: int,
    epsilon_decay: float,
    distance_based_decay: bool = False,
    ess_min: Optional[float] = None,
    kernel_variance_scale: float = 1.0,
    use_last_pop_samples: bool = True,
    return_summary: bool = False,
    lra: bool = False,
    lra_with_weights: bool = False,
    sass: bool = False,
    sass_fraction: float = 0.25,
    sass_expansion_degree: int = 1,
) -> Union[Distribution, Tuple[Distribution, dict]]:
    r"""Run SMCABC.

    Args:
        x_o: Observed data.
        num_particles: Number of particles in each population.
        num_initial_pop: Number of simulations used for initial population.
        num_simulations: Total number of possible simulations.
        epsilon_decay: Factor with which the acceptance threshold $\epsilon$ decays.
        distance_based_decay: Whether the $\epsilon$ decay is constant over
            populations or calculated from the previous populations distribution of
            distances.
        ess_min: Threshold of effective sampling size for resampling weights. Not
            used when None (default).
        kernel_variance_scale: Factor for scaling the perturbation kernel variance.
        use_last_pop_samples: Whether to fill up the current population with
            samples from the previous population when the budget is used up. If
            False, the current population is discarded and the previous population
            is returned.
        return_summary: Whether to return a dictionary with all accepted particles,
            weights, etc. at the end.
        lra: Whether to run linear regression adjustment as in Beaumont et al. 2002
        lra_with_weights: Whether to run lra as weighted linear regression with SMC
            weights
        sass: Whether to determine semi-automatic summary statistics as in
            Fearnhead & Prangle 2012.
        sass_fraction: Fraction of simulation budget used for the initial sass run.
        sass_expansion_degree: Degree of the polynomial feature expansion for the
            sass regression, default 1 - no expansion.

    Returns:
        posterior: Empirical posterior distribution defined by the accepted
            particles and their weights.
        summary (optional): A dictionary containing particles, weights, epsilons
            and distances of each population.
    """

    pop_idx = 0
    self.num_simulations = num_simulations

    # Pilot run for SASS.
    if sass:
        num_pilot_simulations = int(sass_fraction * num_simulations)
        self.logger.info(
            f"Running SASS with {num_pilot_simulations} pilot samples."
        )
        sass_transform = self.run_sass_set_xo(
            num_particles, num_pilot_simulations, x_o, lra, sass_expansion_degree
        )
        # Udpate simulator and xo
        x_o = sass_transform(self.x_o)

        def sass_simulator(theta):
            self.simulation_counter += theta.shape[0]
            return sass_transform(self._batched_simulator(theta))

        self._simulate_with_budget = sass_simulator

    # run initial population
    particles, epsilon, distances, x = self._set_xo_and_sample_initial_population(
        x_o, num_particles, num_initial_pop
    )
    log_weights = torch.log(1 / num_particles * ones(num_particles))

    self.logger.info(
        (
            f"population={pop_idx}, eps={epsilon}, ess={1.0}, "
            f"num_sims={num_initial_pop}"
        )
    )

    all_particles = [particles]
    all_log_weights = [log_weights]
    all_distances = [distances]
    all_epsilons = [epsilon]
    all_x = [x]

    while self.simulation_counter < self.num_simulations:

        pop_idx += 1
        # Decay based on quantile of distances from previous pop.
        if distance_based_decay:
            epsilon = self._get_next_epsilon(
                all_distances[pop_idx - 1], epsilon_decay
            )
        # Constant decay.
        else:
            epsilon *= epsilon_decay

        # Get kernel variance from previous pop.
        self.kernel_variance = self.get_kernel_variance(
            all_particles[pop_idx - 1],
            torch.exp(all_log_weights[pop_idx - 1]),
            samples_per_dim=500,
            kernel_variance_scale=kernel_variance_scale,
        )
        particles, log_weights, distances, x = self._sample_next_population(
            particles=all_particles[pop_idx - 1],
            log_weights=all_log_weights[pop_idx - 1],
            distances=all_distances[pop_idx - 1],
            epsilon=epsilon,
            x=all_x[pop_idx - 1],
            use_last_pop_samples=use_last_pop_samples,
        )

        # Resample population if effective sampling size is too small.
        if ess_min is not None:
            particles, log_weights = self.resample_if_ess_too_small(
                particles, log_weights, ess_min, pop_idx
            )

        self.logger.info(
            (
                f"population={pop_idx} done: eps={epsilon:.6f},"
                f" num_sims={self.simulation_counter}."
            )
        )

        # collect results
        all_particles.append(particles)
        all_log_weights.append(log_weights)
        all_distances.append(distances)
        all_epsilons.append(epsilon)
        all_x.append(x)

    # Maybe run LRA and adjust weights.
    if lra:
        self.logger.info("Running Linear regression adjustment.")
        adjusted_particels, adjusted_weights = self.run_lra_update_weights(
            particles=all_particles[-1],
            xs=all_x[-1],
            observation=x_o,
            log_weights=all_log_weights[-1],
            lra_with_weights=lra_with_weights,
        )
        posterior = Empirical(adjusted_particels, log_weights=adjusted_weights)
    else:
        posterior = Empirical(all_particles[-1], log_weights=all_log_weights[-1])

    if return_summary:
        return (
            posterior,
            dict(
                particles=all_particles,
                weights=all_log_weights,
                epsilons=all_epsilons,
                distances=all_distances,
                xs=all_x,
            ),
        )
    else:
        return posterior

__init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True, kernel='gaussian', algorithm_variant='C') special

Sequential Monte Carlo Approximate Bayesian Computation.

We distinguish between three different SMC methods here: - A: Toni et al. 2010 (Phd Thesis) - B: Sisson et al. 2007 (with correction from 2009) - C: Beaumont et al. 2009

In Toni et al. 2010 we find an overview of the differences on page 34: - B: same as A except for resampling of weights if the effective sampling size is too small. - C: same as A except for calculation of the covariance of the perturbation kernel: the kernel covariance is a scaled version of the covariance of the previous population.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters \theta and maps them to simulations, or observations, x, \mathrm{sim}(\theta)\to x. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior Distribution

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
kernel Optional[str]

Perturbation kernel.

'gaussian'
algorithm_variant str

Indicating the choice of algorithm variant, A, B, or C.

'C'
Source code in sbi/inference/abc/smcabc.py
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
def __init__(
    self,
    simulator: Callable,
    prior: Distribution,
    distance: Union[str, Callable] = "l2",
    num_workers: int = 1,
    simulation_batch_size: int = 1,
    show_progress_bars: bool = True,
    kernel: Optional[str] = "gaussian",
    algorithm_variant: str = "C",
):
    r"""Sequential Monte Carlo Approximate Bayesian Computation.

    We distinguish between three different SMC methods here:
        - A: Toni et al. 2010 (Phd Thesis)
        - B: Sisson et al. 2007 (with correction from 2009)
        - C: Beaumont et al. 2009

    In Toni et al. 2010 we find an overview of the differences on page 34:
        - B: same as A except for resampling of weights if the effective sampling
            size is too small.
        - C: same as A except for calculation of the covariance of the perturbation
            kernel: the kernel covariance is a scaled version of the covariance of
            the previous population.

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\mathrm{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        distance: Distance function to compare observed and simulated data. Can be
            a custom function or one of `l1`, `l2`, `mse`.
        num_workers: Number of parallel workers to use for simulations.
        simulation_batch_size: Number of parameter sets that the simulator
            maps to data x at once. If None, we simulate all parameter sets at the
            same time. If >= 1, the simulator has to process data of shape
            (simulation_batch_size, parameter_dimension).
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        kernel: Perturbation kernel.
        algorithm_variant: Indicating the choice of algorithm variant, A, B, or C.

    """

    super().__init__(
        simulator=simulator,
        prior=prior,
        distance=distance,
        num_workers=num_workers,
        simulation_batch_size=simulation_batch_size,
        show_progress_bars=show_progress_bars,
    )

    kernels = ("gaussian", "uniform")
    assert (
        kernel in kernels
    ), f"Kernel '{kernel}' not supported. Choose one from {kernels}."
    self.kernel = kernel

    algorithm_variants = ("A", "B", "C")
    assert algorithm_variant in algorithm_variants, (
        f"SMCABC variant '{algorithm_variant}' not supported, choose one from"
        " {algorithm_variants}."
    )
    self.algorithm_variant = algorithm_variant
    self.distance_to_x0 = None
    self.simulation_counter = 0
    self.num_simulations = 0

    # Define simulator that keeps track of budget.
    def simulate_with_budget(theta):
        self.simulation_counter += theta.shape[0]
        return self._batched_simulator(theta)

    self._simulate_with_budget = simulate_with_budget

choose_distance_function(distance_type='l2') inherited

Return distance function for given distance type.

Source code in sbi/inference/abc/smcabc.py
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
@staticmethod
def choose_distance_function(distance_type: str = "l2") -> Callable:
    """Return distance function for given distance type."""

    if distance_type == "mse":
        distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1)
    elif distance_type == "l2":
        distance = lambda xo, x: torch.norm((xo - x), dim=-1)
    elif distance_type == "l1":
        distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1)
    else:
        raise ValueError(r"Distance {distance_type} not supported.")

    def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor:
        """Return distance over batch dimension.

        Args:
            observed_data: Observed data, could be 1D.
            simulated_data: Batch of simulated data, has batch dimension.

        Returns:
            Torch tensor with batch of distances.
        """
        assert simulated_data.ndim == 2, "simulated data needs batch dimension"

        return distance(observed_data, simulated_data)

    return distance_fun

get_new_kernel(self, thetas)

Return new kernel distribution for a given set of paramters.

Source code in sbi/inference/abc/smcabc.py
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
def get_new_kernel(self, thetas: Tensor) -> Distribution:
    """Return new kernel distribution for a given set of paramters."""

    if self.kernel == "gaussian":
        assert self.kernel_variance.ndim == 2
        return MultivariateNormal(
            loc=thetas, covariance_matrix=self.kernel_variance
        )

    elif self.kernel == "uniform":
        low = thetas - self.kernel_variance
        high = thetas + self.kernel_variance
        # Move batch shape to event shape to get Uniform that is multivariate in
        # parameter dimension.
        return Uniform(low=low, high=high).to_event(1)
    else:
        raise ValueError(f"Kernel, '{self.kernel}' not supported.")

get_particle_ranges(self, particles, weights, samples_per_dim=100)

Return range of particles in each parameter dimension.

Source code in sbi/inference/abc/smcabc.py
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
def get_particle_ranges(
    self, particles: Tensor, weights: Tensor, samples_per_dim: int = 100
) -> Tensor:
    """Return range of particles in each parameter dimension."""

    # get weighted samples
    samples = self.sample_from_population_with_weights(
        particles,
        weights,
        num_samples=samples_per_dim * particles.shape[1],
    )

    # Variance spans the range of particles for every dimension.
    particle_ranges = tensor([max(column) - min(column) for column in samples.T])
    assert particle_ranges.ndim < 2
    return particle_ranges

get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Following implementation in https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity and https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic

Source code in sbi/inference/abc/smcabc.py
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
@staticmethod
def get_sass_transform(
    theta: torch.Tensor,
    x: torch.Tensor,
    expansion_degree: int = 1,
    sample_weight=None,
) -> Callable:
    """Return semi-automatic summary statitics function.

    Running weighted linear regressin as in
    Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

    Following implementation in
    https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity
    and
    https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic
    """
    expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False)
    # Transform x, remove intercept.
    x_expanded = expansion.fit_transform(x)
    sumstats_map = np.zeros((x_expanded.shape[1], theta.shape[1]))

    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight
        )
        sumstats_map[:, parameter_idx] = regression_model.coef_

    sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32)

    def sumstats_transform(x):
        x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32)
        return x_expanded.mm(sumstats_map)

    return sumstats_transform

resample_if_ess_too_small(self, particles, log_weights, ess_min, pop_idx)

Return resampled particles and uniform weights if effectice sampling size is too small.

Source code in sbi/inference/abc/smcabc.py
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
def resample_if_ess_too_small(
    self,
    particles: Tensor,
    log_weights: Tensor,
    ess_min: float,
    pop_idx: int,
) -> Tuple[Tensor, Tensor]:
    """Return resampled particles and uniform weights if effectice sampling size is
    too small.
    """

    num_particles = particles.shape[0]
    ess = (1 / torch.sum(torch.exp(2.0 * log_weights), dim=0)) / num_particles
    # Resampling of weights for low ESS only for Sisson et al. 2007.
    if ess < ess_min:
        self.logger.info(f"ESS={ess:.2f} too low, resampling pop {pop_idx}...")
        # First resample, then set to uniform weights as in Sisson et al. 2007.
        particles = self.sample_from_population_with_weights(
            particles, torch.exp(log_weights), num_samples=num_particles
        )
        log_weights = torch.log(1 / num_particles * ones(num_particles))

    return particles, log_weights

run_lra(theta, x, observation, sample_weight=None) inherited

Return parameters adjusted with linear regression adjustment.

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/smcabc.py
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
@staticmethod
def run_lra(
    theta: torch.Tensor,
    x: torch.Tensor,
    observation: torch.Tensor,
    sample_weight=None,
) -> torch.Tensor:
    """Return parameters adjusted with linear regression adjustment.

    Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254
    """

    theta_adjusted = theta
    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x,
            y=theta[:, parameter_idx],
            sample_weight=sample_weight,
        )
        theta_adjusted[:, parameter_idx] += regression_model.predict(
            observation.reshape(1, -1)
        )
        theta_adjusted[:, parameter_idx] -= regression_model.predict(x)

    return theta_adjusted

run_lra_update_weights(self, particles, xs, observation, log_weights, lra_with_weights)

Return particles and weights adjusted with LRA.

Runs (weighted) linear regression from xs onto particles to adjust the particles.

Updates the SMC weights according to the new particles.

Source code in sbi/inference/abc/smcabc.py
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
def run_lra_update_weights(
    self,
    particles: Tensor,
    xs: Tensor,
    observation: Tensor,
    log_weights: Tensor,
    lra_with_weights: bool,
) -> Tuple[Tensor, Tensor]:
    """Return particles and weights adjusted with LRA.

    Runs (weighted) linear regression from xs onto particles to adjust the
    particles.

    Updates the SMC weights according to the new particles.
    """

    adjusted_particels = self.run_lra(
        theta=particles,
        x=xs,
        observation=observation,
        sample_weight=log_weights.exp() if lra_with_weights else None,
    )

    # Update SMC weights with LRA adjusted weights
    adjusted_log_weights = self._calculate_new_log_weights(
        new_particles=adjusted_particels,
        old_particles=particles,
        old_log_weights=log_weights,
    )

    return adjusted_particels, adjusted_log_weights

run_sass_set_xo(self, num_particles, num_pilot_simulations, x_o, lra=False, sass_expansion_degree=1)

Return transform for semi-automatic summary statistics.

Runs an single round of rejection abc with fixed budget and accepts num_particles simulations to run the regression for sass.

Sets self.x_o once the x_shape can be derived from simulations.

Source code in sbi/inference/abc/smcabc.py
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
def run_sass_set_xo(
    self,
    num_particles: int,
    num_pilot_simulations: int,
    x_o,
    lra: bool = False,
    sass_expansion_degree: int = 1,
) -> Callable:
    """Return transform for semi-automatic summary statistics.

    Runs an single round of rejection abc with fixed budget and accepts
    num_particles simulations to run the regression for sass.

    Sets self.x_o once the x_shape can be derived from simulations.
    """
    (pilot_particles, _, _, pilot_xs,) = self._set_xo_and_sample_initial_population(
        x_o, num_particles, num_pilot_simulations
    )
    # Adjust with LRA.
    if lra:
        pilot_particles = self.run_lra(pilot_particles, pilot_xs, self.x_o)
    sass_transform = self.get_sass_transform(
        pilot_particles,
        pilot_xs,
        expansion_degree=sass_expansion_degree,
        sample_weight=None,
    )
    return sass_transform

sample_from_population_with_weights(particles, weights, num_samples=1) staticmethod

Return samples from particles sampled with weights.

Source code in sbi/inference/abc/smcabc.py
466
467
468
469
470
471
472
473
474
475
476
477
478
479
@staticmethod
def sample_from_population_with_weights(
    particles: Tensor, weights: Tensor, num_samples: int = 1
) -> Tensor:
    """Return samples from particles sampled with weights."""

    # define multinomial with weights as probs
    multi = Multinomial(probs=weights)
    # sample num samples, with replacement
    samples = multi.sample(sample_shape=(num_samples,))
    # get indices of success trials
    indices = torch.where(samples)[1]
    # return those indices from trace
    return particles[indices]

Posteriors

sbi.inference.posteriors.direct_posterior.DirectPosterior

Posterior p(\theta|x) with log_prob() and sample() methods, obtained with SNPE.

SNPE trains a neural network to directly approximate the posterior distribution. However, for bounded priors, the neural network can have leakage: it puts non-zero mass in regions where the prior is zero. The DirectPosterior class wraps the trained network to deal with these cases.

Specifically, this class offers the following functionality:
- correct the calculation of the log probability such that it compensates for the leakage.
- reject samples that lie outside of the prior bounds.
- alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

mcmc_method: str inherited property writable

Returns MCMC method.

mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

rejection_sampling_parameters: dict property writable

Returns rejection sampling parameter.

sample_with_mcmc: bool property writable

Return True if NeuralPosterior instance should use MCMC in .sample().

__init__(self, method_family, neural_net, prior, x_shape, rejection_sampling_parameters=None, sample_with_mcmc=True, mcmc_method='slice_np', mcmc_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of a single simulator output.

required
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

None
sample_with_mcmc bool

Whether to sample with MCMC. Will always be True for SRE and SNL, but can also be set to True for SNPE if MCMC is preferred to deal with leakage over rejection sampling.

True
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
device str

Training device, e.g., cpu or cuda:0

'cpu'
Source code in sbi/inference/posteriors/direct_posterior.py
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def __init__(
    self,
    method_family: str,
    neural_net: nn.Module,
    prior,
    x_shape: torch.Size,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    sample_with_mcmc: bool = True,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    device: str = "cpu",
):
    """
    Args:
        method_family: One of snpe, snl, snre_a or snre_b.
        neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
        prior: Prior distribution with `.log_prob()` and `.sample()`.
        x_shape: Shape of a single simulator output.
        rejection_sampling_parameters: Dictonary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `max_sampling_batch_size` to set the batch size for drawing new
            samples from the candidate distribution, e.g., the posterior. Larger
            batch size speeds up sampling.
        sample_with_mcmc: Whether to sample with MCMC. Will always be `True` for SRE
            and SNL, but can also be set to `True` for SNPE if MCMC is preferred to
            deal with leakage over rejection sampling.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        device: Training device, e.g., cpu or cuda:0
    """

    kwargs = del_entries(
        locals(),
        entries=(
            "self",
            "__class__",
            "sample_with_mcmc",
            "rejection_sampling_parameters",
        ),
    )
    super().__init__(**kwargs)

    self.set_sample_with_mcmc(sample_with_mcmc)
    self.set_rejection_sampling_parameters(rejection_sampling_parameters)
    self._purpose = (
        "It allows to .sample() and .log_prob() the posterior and wraps the "
        "output of the .net to avoid leakage into regions with 0 prior probability."
    )

copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

  • Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
  • default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
NeuralPosterior

Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/direct_posterior.py
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def copy_hyperparameters_from(
    self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
    """
    Copies the hyperparameters from a given posterior to `self`.

    The hyperparameters that are copied are:

    - Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
    - `default_x` at which to evaluate the posterior.

    Args:
        posterior: Posterior that the hyperparameters are copied from.

    Returns:
        Posterior object with the same hyperparameters as the passed posterior.
        This makes the call chainable:
        `posterior = infer.build_posterior().copy_hyperparameters_from(proposal)`
    """

    assert isinstance(
        posterior, NeuralPosterior
    ), "`copy_state_from` must be a `NeuralPosterior`."

    self.set_mcmc_method(posterior._mcmc_method)
    self.set_mcmc_parameters(posterior._mcmc_parameters)
    self.set_default_x(posterior.default_x)
    self._mcmc_init_params = posterior._mcmc_init_params
    if hasattr(self, "_sample_with_mcmc"):
        self.set_sample_with_mcmc(posterior._sample_with_mcmc)
    if hasattr(self, "_rejection_sampling_parameters"):
        self.set_rejection_sampling_parameters(
            posterior._rejection_sampling_parameters
        )

    return self

leakage_correction(self, x, num_rejection_samples=10000, force_update=False, show_progress_bars=False, rejection_sampling_batch_size=10000)

Return leakage correction factor for a leaky posterior density estimate.

The factor is estimated from the acceptance probability during rejection sampling from the posterior.

This is to avoid re-estimating the acceptance probability from scratch whenever log_prob is called and norm_posterior=True. Here, it is estimated only once for self.default_x and saved for later. We re-evaluate only whenever a new x is passed.

Parameters:

Name Type Description Default
x Tensor

Conditioning context for posterior p(\theta|x).

required
num_rejection_samples int

Number of samples used to estimate correction factor.

10000
force_update bool

Whether to force a reevaluation of the leakage correction even if the context x is the same as self.default_x. This is useful to enforce a new leakage estimate for rounds after the first (2, 3,..).

False
show_progress_bars bool

Whether to show a progress bar during sampling.

False
rejection_sampling_batch_size int

Batch size for rejection sampling.

10000

Returns:

Type Description
Tensor

Saved or newly-estimated correction factor (as a scalar Tensor).

Source code in sbi/inference/posteriors/direct_posterior.py
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
@torch.no_grad()
def leakage_correction(
    self,
    x: Tensor,
    num_rejection_samples: int = 10_000,
    force_update: bool = False,
    show_progress_bars: bool = False,
    rejection_sampling_batch_size: int = 10_000,
) -> Tensor:
    r"""Return leakage correction factor for a leaky posterior density estimate.

    The factor is estimated from the acceptance probability during rejection
    sampling from the posterior.

    This is to avoid re-estimating the acceptance probability from scratch
    whenever `log_prob` is called and `norm_posterior=True`. Here, it
    is estimated only once for `self.default_x` and saved for later. We
    re-evaluate only whenever a new `x` is passed.

    Arguments:
        x: Conditioning context for posterior $p(\theta|x)$.
        num_rejection_samples: Number of samples used to estimate correction factor.
        force_update: Whether to force a reevaluation of the leakage correction even
            if the context `x` is the same as `self.default_x`. This is useful to
            enforce a new leakage estimate for rounds after the first (2, 3,..).
        show_progress_bars: Whether to show a progress bar during sampling.
        rejection_sampling_batch_size: Batch size for rejection sampling.

    Returns:
        Saved or newly-estimated correction factor (as a scalar `Tensor`).
    """

    def acceptance_at(x: Tensor) -> Tensor:
        return utils.sample_posterior_within_prior(
            self.net,
            self._prior,
            x.to(self._device),
            num_rejection_samples,
            show_progress_bars,
            sample_for_correction_factor=True,
            max_sampling_batch_size=rejection_sampling_batch_size,
        )[1]

    # Check if the provided x matches the default x (short-circuit on identity).
    is_new_x = self.default_x is None or (
        x is not self.default_x and (x != self.default_x).any()
    )

    not_saved_at_default_x = self._leakage_density_correction_factor is None

    if is_new_x:  # Calculate at x; don't save.
        return acceptance_at(x)
    elif not_saved_at_default_x or force_update:  # Calculate at default_x; save.
        self._leakage_density_correction_factor = acceptance_at(self.default_x)

    return self._leakage_density_correction_factor  # type:ignore

log_prob(self, theta, x=None, norm_posterior=True, track_gradients=False, leakage_correction_params=None)

Returns the log-probability of the posterior p(\theta|x).

Parameters:

Name Type Description Default
theta Tensor

Parameters \theta.

required
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
norm_posterior bool

Whether to enforce a normalized posterior density. Renormalization of the posterior is useful when some probability falls out or leaks out of the prescribed prior support. The normalizing factor is calculated via rejection sampling, so if you need speedier but unnormalized log posterior estimates set here norm_posterior=False. The returned log posterior is set to -∞ outside of the prior support regardless of this setting.

True
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False
leakage_correction_params Optional[dict]

A dict of keyword arguments to override the default values of leakage_correction(). Possible options are: num_rejection_samples, force_update, show_progress_bars, and rejection_sampling_batch_size. These parameters only have an effect if norm_posterior=True.

None

Returns:

Type Description
Tensor

(len(θ),)-shaped log posterior probability \log p(\theta|x) for θ in the support of the prior, -∞ (corresponding to 0 probability) outside.

Source code in sbi/inference/posteriors/direct_posterior.py
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
def log_prob(
    self,
    theta: Tensor,
    x: Optional[Tensor] = None,
    norm_posterior: bool = True,
    track_gradients: bool = False,
    leakage_correction_params: Optional[dict] = None,
) -> Tensor:
    r"""
    Returns the log-probability of the posterior $p(\theta|x).$

    Args:
        theta: Parameters $\theta$.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        norm_posterior: Whether to enforce a normalized posterior density.
            Renormalization of the posterior is useful when some
            probability falls out or leaks out of the prescribed prior support.
            The normalizing factor is calculated via rejection sampling, so if you
            need speedier but unnormalized log posterior estimates set here
            `norm_posterior=False`. The returned log posterior is set to
            -∞ outside of the prior support regardless of this setting.
        track_gradients: Whether the returned tensor supports tracking gradients.
            This can be helpful for e.g. sensitivity analysis, but increases memory
            consumption.
        leakage_correction_params: A `dict` of keyword arguments to override the
            default values of `leakage_correction()`. Possible options are:
            `num_rejection_samples`, `force_update`, `show_progress_bars`, and
            `rejection_sampling_batch_size`.
            These parameters only have an effect if `norm_posterior=True`.

    Returns:
        `(len(θ),)`-shaped log posterior probability $\log p(\theta|x)$ for θ in the
        support of the prior, -∞ (corresponding to 0 probability) outside.

    """

    # TODO Train exited here, entered after sampling?
    self.net.eval()

    theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)
    theta_repeated, x_repeated = self._match_theta_and_x_batch_shapes(theta, x)

    with torch.set_grad_enabled(track_gradients):

        # Evaluate on device, move back to cpu for comparison with prior.
        unnorm_log_prob = self.net.log_prob(
            theta_repeated.to(self._device), x_repeated.to(self._device)
        ).cpu()

        # Force probability to be zero outside prior support.
        in_prior_support = within_support(self._prior, theta)

        masked_log_prob = torch.where(
            in_prior_support,
            unnorm_log_prob,
            torch.tensor(float("-inf"), dtype=torch.float32),
        )

        if leakage_correction_params is None:
            leakage_correction_params = dict()  # use defaults
        log_factor = (
            log(
                self.leakage_correction(
                    x=batched_first_of_batch(x), **leakage_correction_params
                )
            )
            if norm_posterior
            else 0
        )

        return masked_log_prob - log_factor

map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=1000, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior p( heta|x). If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

1000
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/direct_posterior.py
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
def map(
    self,
    x: Optional[Tensor] = None,
    num_iter: int = 1000,
    learning_rate: float = 1e-2,
    init_method: Union[str, Tensor] = "posterior",
    num_init_samples: int = 1_000,
    num_to_optimize: int = 100,
    save_best_every: int = 10,
    show_progress_bars: bool = True,
) -> Tensor:
    """
    Returns the maximum-a-posteriori estimate (MAP).

    The method can be interrupted (Ctrl-C) when the user sees that the
    log-probability converges. The best estimate will be saved in `self.map_`.

    The MAP is obtained by running gradient ascent from a given number of starting
    positions (samples from the posterior with the highest log-probability). After
    the optimization is done, we select the parameter set that has the highest
    log-probability after the optimization.

    Warning: The default values used by this function are not well-tested. They
    might require hand-tuning for the problem at hand.

    Args:
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        num_iter: Number of optimization steps that the algorithm takes
            to find the MAP.
        learning_rate: Learning rate of the optimizer.
        init_method: How to select the starting parameters for the optimization. If
            it is a string, it can be either [`posterior`, `prior`], which samples
            the respective distribution `num_init_samples` times. If it is a,
            the tensor will be used as init locations.
        num_init_samples: Draw this number of samples from the posterior and
            evaluate the log-probability of all of them.
        num_to_optimize: From the drawn `num_init_samples`, use the
            `num_to_optimize` with highest log-probability as the initial points
            for the optimization.
        save_best_every: The best log-probability is computed, saved in the
            `map`-attribute, and printed every `print_best_every`-th iteration.
            Computing the best log-probability creates a significant overhead
            (thus, the default is `10`.)
        show_progress_bars: Whether or not to show a progressbar for sampling from
            the posterior.

    Returns:
        The MAP estimate.
    """
    return super().map(
        x=x,
        num_iter=num_iter,
        learning_rate=learning_rate,
        init_method=init_method,
        num_init_samples=num_init_samples,
        num_to_optimize=num_to_optimize,
        save_best_every=save_best_every,
        show_progress_bars=show_progress_bars,
        log_prob_kwargs={"norm_posterior": False},
    )

sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with_mcmc=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from posterior distribution p(\theta|x).

Samples are obtained either with rejection sampling or MCMC. Rejection sampling will be a lot faster if leakage is rather low. If leakage is high (e.g. over 99%, which can happen in multi-round SNPE), MCMC can be faster than rejection sampling.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with_mcmc Optional[bool]

Optional parameter to override self.sample_with_mcmc.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
def sample(
    self,
    sample_shape: Shape = torch.Size(),
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    sample_with_mcmc: Optional[bool] = None,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from posterior distribution $p(\theta|x)$.

    Samples are obtained either with rejection sampling or MCMC. Rejection sampling
    will be a lot faster if leakage is rather low. If leakage is high (e.g. over
    99%, which can happen in multi-round SNPE), MCMC can be faster than rejection
    sampling.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        sample_with_mcmc: Optional parameter to override `self.sample_with_mcmc`.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `max_sampling_batch_size` to set the batch size for drawing new
            samples from the candidate distribution, e.g., the posterior. Larger
            batch size speeds up sampling.
    Returns:
        Samples from posterior.
    """

    x, num_samples, mcmc_method, mcmc_parameters = self._prepare_for_sample(
        x, sample_shape, mcmc_method, mcmc_parameters
    )

    sample_with_mcmc = (
        sample_with_mcmc if sample_with_mcmc is not None else self.sample_with_mcmc
    )

    self.net.eval()

    if sample_with_mcmc:
        potential_fn_provider = PotentialFunctionProvider()
        samples = self._sample_posterior_mcmc(
            num_samples=num_samples,
            potential_fn=potential_fn_provider(
                self._prior, self.net, x, mcmc_method
            ),
            init_fn=self._build_mcmc_init_fn(
                self._prior,
                potential_fn_provider(self._prior, self.net, x, "slice_np"),
                **mcmc_parameters,
            ),
            mcmc_method=mcmc_method,
            show_progress_bars=show_progress_bars,
            **mcmc_parameters,
        )
    else:
        # Rejection sampling.
        samples, _ = utils.sample_posterior_within_prior(
            self.net,
            self._prior,
            x,
            num_samples=num_samples,
            show_progress_bars=show_progress_bars,
            **rejection_sampling_parameters
            if (rejection_sampling_parameters is not None)
            else self.rejection_sampling_parameters,
        )

    self.net.train(True)

    return samples.reshape((*sample_shape, -1))

sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, show_progress_bars=True, mcmc_method=None, mcmc_parameters=None)

Return samples from conditional posterior p(\theta_i|\theta_j, x).

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
def sample_conditional(
    self,
    sample_shape: Shape,
    condition: Tensor,
    dims_to_sample: List[int],
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

    In this function, we do not sample from the full posterior, but instead only
    from a few parameter dimensions while the other parameter dimensions are kept
    fixed at values specified in `condition`.

    Samples are obtained with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_sample: Which dimensions to sample from. The dimensions not
            specified in `dims_to_sample` will be fixed to values given in
            `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        Samples from conditional posterior.
    """

    return super().sample_conditional(
        PotentialFunctionProvider(),
        sample_shape,
        condition,
        dims_to_sample,
        x,
        show_progress_bars,
        mcmc_method,
        mcmc_parameters,
    )

set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior p(theta|x).

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/direct_posterior.py
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
    """Set new default x for `.sample(), .log_prob` to use as conditioning context.

    This is a pure convenience to avoid having to repeatedly specify `x` in calls to
    `.sample()` and `.log_prob()` - only θ needs to be passed.

    This convenience is particularly useful when the posterior is focused, i.e.
    has been trained over multiple rounds to be accurate in the vicinity of a
    particular `x=x_o` (you can check if your posterior object is focused by
    printing it).

    NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
    that calls like `posterior.set_default_x(my_x).sample(mytheta)` are possible.

    Args:
        x: The default observation to set for the posterior $p(theta|x)$.

    Returns:
        `NeuralPosterior` that will use a default `x` when not explicitly passed.
    """
    self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x)
    self._num_iid_trials = self._x.shape[0]

    return self

set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
147
148
149
150
151
152
153
154
155
156
157
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
    """Sets sampling method to for MCMC and returns `NeuralPosterior`.

    Args:
        method: Method to use.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_method = method
    return self

set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
    """Sets parameters for MCMC and returns `NeuralPosterior`.

    Args:
        parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_parameters = parameters
    return self

set_rejection_sampling_parameters(self, parameters)

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to the set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

required

Returns:

Type Description
NeuralPosterior

`NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
def set_rejection_sampling_parameters(
    self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
    """Sets parameters for rejection sampling and returns `NeuralPosterior`.

    Args:
        parameters: Dictonary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `max_sampling_batch_size` to the set the batch size for drawing new
            samples from the candidate distribution, e.g., the posterior. Larger
            batch size speeds up sampling.

    Returns:
        `NeuralPosterior for chainable calls.
    """
    self._rejection_sampling_parameters = parameters
    return self

set_sample_with_mcmc(self, use_mcmc)

Turns MCMC sampling on or off and returns NeuralPosterior.

Parameters:

Name Type Description Default
use_mcmc bool

Flag to set whether or not MCMC sampling is used.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/direct_posterior.py
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
def set_sample_with_mcmc(self, use_mcmc: bool) -> "NeuralPosterior":
    """Turns MCMC sampling on or off and returns `NeuralPosterior`.

    Args:
        use_mcmc: Flag to set whether or not MCMC sampling is used.

    Returns:
        `NeuralPosterior` for chainable calls.

    Raises:
        ValueError: on attempt to turn off MCMC sampling for family of methods that
            do not support rejection sampling.
    """
    self._sample_with_mcmc = use_mcmc
    return self

sbi.inference.posteriors.likelihood_based_posterior.LikelihoodBasedPosterior

Posterior p(\theta|x) with log_prob() and sample() methods, obtained with SNLE.

SNLE trains a neural network to approximate the likelihood p(x|\theta). The SNLE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability p(\theta|x) \propto p(x|\theta) \cdot p(\theta) and draw samples from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

mcmc_method: str inherited property writable

Returns MCMC method.

mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

__init__(self, method_family, neural_net, prior, x_shape, mcmc_method='slice_np', mcmc_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations.

required
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
device str

Training device, e.g., cpu or cuda:0.

'cpu'
Source code in sbi/inference/posteriors/likelihood_based_posterior.py
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def __init__(
    self,
    method_family: str,
    neural_net: nn.Module,
    prior,
    x_shape: torch.Size,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    device: str = "cpu",
):
    """
    Args:
        method_family: One of snpe, snl, snre_a or snre_b.
        neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
        prior: Prior distribution with `.log_prob()` and `.sample()`.
        x_shape: Shape of the simulated data. It can differ from the
            observed data the posterior is conditioned on later in the batch
            dimension. If it differs, the additional entries are interpreted as
            independent and identically distributed data / trials. I.e., the data is
            assumed to be generated based on the same (unknown) model parameters or
            experimental condations.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        device: Training device, e.g., cpu or cuda:0.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__"))
    super().__init__(**kwargs)

    self._purpose = (
        "It provides MCMC to .sample() from the posterior and "
        "can evaluate the _unnormalized_ posterior density with .log_prob()."
    )

copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

  • Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
  • default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
NeuralPosterior

Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def copy_hyperparameters_from(
    self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
    """
    Copies the hyperparameters from a given posterior to `self`.

    The hyperparameters that are copied are:

    - Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
    - `default_x` at which to evaluate the posterior.

    Args:
        posterior: Posterior that the hyperparameters are copied from.

    Returns:
        Posterior object with the same hyperparameters as the passed posterior.
        This makes the call chainable:
        `posterior = infer.build_posterior().copy_hyperparameters_from(proposal)`
    """

    assert isinstance(
        posterior, NeuralPosterior
    ), "`copy_state_from` must be a `NeuralPosterior`."

    self.set_mcmc_method(posterior._mcmc_method)
    self.set_mcmc_parameters(posterior._mcmc_parameters)
    self.set_default_x(posterior.default_x)
    self._mcmc_init_params = posterior._mcmc_init_params
    if hasattr(self, "_sample_with_mcmc"):
        self.set_sample_with_mcmc(posterior._sample_with_mcmc)
    if hasattr(self, "_rejection_sampling_parameters"):
        self.set_rejection_sampling_parameters(
            posterior._rejection_sampling_parameters
        )

    return self

log_prob(self, theta, x=None, track_gradients=False)

Returns the log-probability of p(x|\theta) \cdot p(\theta).

This corresponds to an unnormalized posterior log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters \theta.

required
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability \log(p(x|\theta) \cdot p(\theta)).

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def log_prob(
    self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False
) -> Tensor:
    r"""
    Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

    This corresponds to an **unnormalized** posterior log-probability.

    Args:
        theta: Parameters $\theta$.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        track_gradients: Whether the returned tensor supports tracking gradients.
            This can be helpful for e.g. sensitivity analysis, but increases memory
            consumption.

    Returns:
        `(len(θ),)`-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

    """

    # TODO Train exited here, entered after sampling?
    self.net.eval()

    theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)

    # Calculate likelihood over trials and in one batch.
    warn(
        "The log probability from SNL is only correct up to a normalizing constant."
    )
    log_likelihood_trial_sum = self._log_likelihoods_over_trials(
        x=x.to(self._device),
        theta=theta.to(self._device),
        net=self.net,
        track_gradients=track_gradients,
    )

    # Move to cpu for comparison with prior.
    return log_likelihood_trial_sum.cpu() + self._prior.log_prob(theta)

map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior p( heta|x). If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
def map(
    self,
    x: Optional[Tensor] = None,
    num_iter: int = 1000,
    learning_rate: float = 1e-2,
    init_method: Union[str, Tensor] = "posterior",
    num_init_samples: int = 500,
    num_to_optimize: int = 100,
    save_best_every: int = 10,
    show_progress_bars: bool = True,
) -> Tensor:
    """
    Returns the maximum-a-posteriori estimate (MAP).

    The method can be interrupted (Ctrl-C) when the user sees that the
    log-probability converges. The best estimate will be saved in `self.map_`.

    The MAP is obtained by running gradient ascent from a given number of starting
    positions (samples from the posterior with the highest log-probability). After
    the optimization is done, we select the parameter set that has the highest
    log-probability after the optimization.

    Warning: The default values used by this function are not well-tested. They
    might require hand-tuning for the problem at hand.

    Args:
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        num_iter: Number of optimization steps that the algorithm takes
            to find the MAP.
        learning_rate: Learning rate of the optimizer.
        init_method: How to select the starting parameters for the optimization. If
            it is a string, it can be either [`posterior`, `prior`], which samples
            the respective distribution `num_init_samples` times. If it is a,
            the tensor will be used as init locations.
        num_init_samples: Draw this number of samples from the posterior and
            evaluate the log-probability of all of them.
        num_to_optimize: From the drawn `num_init_samples`, use the
            `num_to_optimize` with highest log-probability as the initial points
            for the optimization.
        save_best_every: The best log-probability is computed, saved in the
            `map`-attribute, and printed every `print_best_every`-th iteration.
            Computing the best log-probability creates a significant overhead
            (thus, the default is `10`.)
        show_progress_bars: Whether or not to show a progressbar for sampling from
            the posterior.

    Returns:
        The MAP estimate.
    """
    return super().map(
        x=x,
        num_iter=num_iter,
        learning_rate=learning_rate,
        init_method=init_method,
        num_init_samples=num_init_samples,
        num_to_optimize=num_to_optimize,
        save_best_every=save_best_every,
        show_progress_bars=show_progress_bars,
    )

sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with_mcmc=None, mcmc_method=None, mcmc_parameters=None)

Return samples from posterior distribution p(\theta|x) with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with_mcmc Optional[bool]

Optional parameter to override self.sample_with_mcmc.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
def sample(
    self,
    sample_shape: Shape = torch.Size(),
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    sample_with_mcmc: Optional[bool] = None,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from posterior distribution $p(\theta|x)$ with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        sample_with_mcmc: Optional parameter to override `self.sample_with_mcmc`.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        Samples from posterior.
    """

    x, num_samples, mcmc_method, mcmc_parameters = self._prepare_for_sample(
        x, sample_shape, mcmc_method, mcmc_parameters
    )

    self.net.eval()

    potential_fn_provider = PotentialFunctionProvider()
    samples = self._sample_posterior_mcmc(
        num_samples=num_samples,
        potential_fn=potential_fn_provider(self._prior, self.net, x, mcmc_method),
        init_fn=self._build_mcmc_init_fn(
            self._prior,
            potential_fn_provider(self._prior, self.net, x, "slice_np"),
            **mcmc_parameters,
        ),
        mcmc_method=mcmc_method,
        show_progress_bars=show_progress_bars,
        **mcmc_parameters,
    )

    self.net.train(True)

    return samples.reshape((*sample_shape, -1))

sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, show_progress_bars=True, mcmc_method=None, mcmc_parameters=None)

Return samples from conditional posterior p(\theta_i|\theta_j, x).

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
def sample_conditional(
    self,
    sample_shape: Shape,
    condition: Tensor,
    dims_to_sample: List[int],
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

    In this function, we do not sample from the full posterior, but instead only
    from a few parameter dimensions while the other parameter dimensions are kept
    fixed at values specified in `condition`.

    Samples are obtained with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_sample: Which dimensions to sample from. The dimensions not
            specified in `dims_to_sample` will be fixed to values given in
            `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        Samples from conditional posterior.
    """

    return super().sample_conditional(
        PotentialFunctionProvider(),
        sample_shape,
        condition,
        dims_to_sample,
        x,
        show_progress_bars,
        mcmc_method,
        mcmc_parameters,
    )

set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior p(theta|x).

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
    """Set new default x for `.sample(), .log_prob` to use as conditioning context.

    This is a pure convenience to avoid having to repeatedly specify `x` in calls to
    `.sample()` and `.log_prob()` - only θ needs to be passed.

    This convenience is particularly useful when the posterior is focused, i.e.
    has been trained over multiple rounds to be accurate in the vicinity of a
    particular `x=x_o` (you can check if your posterior object is focused by
    printing it).

    NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
    that calls like `posterior.set_default_x(my_x).sample(mytheta)` are possible.

    Args:
        x: The default observation to set for the posterior $p(theta|x)$.

    Returns:
        `NeuralPosterior` that will use a default `x` when not explicitly passed.
    """
    self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x)
    self._num_iid_trials = self._x.shape[0]

    return self

set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
147
148
149
150
151
152
153
154
155
156
157
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
    """Sets sampling method to for MCMC and returns `NeuralPosterior`.

    Args:
        method: Method to use.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_method = method
    return self

set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
    """Sets parameters for MCMC and returns `NeuralPosterior`.

    Args:
        parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_parameters = parameters
    return self

sbi.inference.posteriors.ratio_based_posterior.RatioBasedPosterior

Posterior p(\theta|x) with log_prob() and sample() methods, obtained with SNRE.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior p(\theta|x) \propto p(x|\theta) \cdot p(\theta). The SNRE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability p(\theta|x) \propto p(x|\theta) \cdot p(\theta) and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

The neural network itself can be accessed via the .net attribute.

default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

mcmc_method: str inherited property writable

Returns MCMC method.

mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

__init__(self, method_family, neural_net, prior, x_shape, mcmc_method='slice_np', mcmc_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of a single simulator output.

required
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
device str

Training device, e.g., cpu or cuda:0.

'cpu'
Source code in sbi/inference/posteriors/ratio_based_posterior.py
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
def __init__(
    self,
    method_family: str,
    neural_net: nn.Module,
    prior,
    x_shape: torch.Size,
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    device: str = "cpu",
):
    """
    Args:
        method_family: One of snpe, snl, snre_a or snre_b.
        neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
        prior: Prior distribution with `.log_prob()` and `.sample()`.
        x_shape: Shape of a single simulator output.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        device: Training device, e.g., cpu or cuda:0.
    """
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    super().__init__(**kwargs)

copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

  • Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
  • default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
NeuralPosterior

Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/ratio_based_posterior.py
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def copy_hyperparameters_from(
    self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
    """
    Copies the hyperparameters from a given posterior to `self`.

    The hyperparameters that are copied are:

    - Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
    - `default_x` at which to evaluate the posterior.

    Args:
        posterior: Posterior that the hyperparameters are copied from.

    Returns:
        Posterior object with the same hyperparameters as the passed posterior.
        This makes the call chainable:
        `posterior = infer.build_posterior().copy_hyperparameters_from(proposal)`
    """

    assert isinstance(
        posterior, NeuralPosterior
    ), "`copy_state_from` must be a `NeuralPosterior`."

    self.set_mcmc_method(posterior._mcmc_method)
    self.set_mcmc_parameters(posterior._mcmc_parameters)
    self.set_default_x(posterior.default_x)
    self._mcmc_init_params = posterior._mcmc_init_params
    if hasattr(self, "_sample_with_mcmc"):
        self.set_sample_with_mcmc(posterior._sample_with_mcmc)
    if hasattr(self, "_rejection_sampling_parameters"):
        self.set_rejection_sampling_parameters(
            posterior._rejection_sampling_parameters
        )

    return self

log_prob(self, theta, x=None, track_gradients=False)

Returns the log-probability of p(x|\theta) \cdot p(\theta).

This corresponds to an unnormalized posterior log-probability. Only for single-round SNRE_A / AALR, the returned log-probability will correspond to the normalized log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters \theta.

required
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability \log(p(x|\theta) \cdot p(\theta)).

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
def log_prob(
    self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False
) -> Tensor:
    r"""Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

    This corresponds to an **unnormalized** posterior log-probability. Only for
    single-round SNRE_A / AALR, the returned log-probability will correspond to the
    **normalized** log-probability.

    Args:
        theta: Parameters $\theta$.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        track_gradients: Whether the returned tensor supports tracking gradients.
            This can be helpful for e.g. sensitivity analysis, but increases memory
            consumption.

    Returns:
        `(len(θ),)`-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

    """

    # TODO Train exited here, entered after sampling?
    self.net.eval()

    theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)

    self._warn_log_prob_snre()

    # Sum log ratios over x batch of iid trials.
    log_ratio = self._log_ratios_over_trials(
        x.to(self._device),
        theta.to(self._device),
        self.net,
        track_gradients=track_gradients,
    )

    return log_ratio.cpu() + self._prior.log_prob(theta)

map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior p( heta|x). If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Maximum Number of optimization steps that the algorithm takes to find the MAP.

1000
early_stop_at

If None, it will optimize for max_num_iter iterations. If float, the optimization will stop as soon as the steps taken by the optimizer are smaller than early_stop_at times the standard deviation of the initial guesses.

required
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
def map(
    self,
    x: Optional[Tensor] = None,
    num_iter: int = 1000,
    learning_rate: float = 1e-2,
    init_method: Union[str, Tensor] = "posterior",
    num_init_samples: int = 500,
    num_to_optimize: int = 100,
    save_best_every: int = 10,
    show_progress_bars: bool = True,
) -> Tensor:
    """
    Returns the maximum-a-posteriori estimate (MAP).

    The method can be interrupted (Ctrl-C) when the user sees that the
    log-probability converges. The best estimate will be saved in `self.map_`.

    The MAP is obtained by running gradient ascent from a given number of starting
    positions (samples from the posterior with the highest log-probability). After
    the optimization is done, we select the parameter set that has the highest
    log-probability after the optimization.

    Warning: The default values used by this function are not well-tested. They
    might require hand-tuning for the problem at hand.

    Args:
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        num_iter: Maximum Number of optimization steps that the algorithm takes
            to find the MAP.
        early_stop_at: If `None`, it will optimize for `max_num_iter` iterations.
            If `float`, the optimization will stop as soon as the steps taken by
            the optimizer are smaller than `early_stop_at` times the standard
            deviation of the initial guesses.
        learning_rate: Learning rate of the optimizer.
        init_method: How to select the starting parameters for the optimization. If
            it is a string, it can be either [`posterior`, `prior`], which samples
            the respective distribution `num_init_samples` times. If it is a,
            the tensor will be used as init locations.
        num_init_samples: Draw this number of samples from the posterior and
            evaluate the log-probability of all of them.
        num_to_optimize: From the drawn `num_init_samples`, use the
            `num_to_optimize` with highest log-probability as the initial points
            for the optimization.
        save_best_every: The best log-probability is computed, saved in the
            `map`-attribute, and printed every `print_best_every`-th iteration.
            Computing the best log-probability creates a significant overhead
            (thus, the default is `10`.)
        show_progress_bars: Whether or not to show a progressbar for sampling from
            the posterior.

    Returns:
        The MAP estimate.
    """
    return super().map(
        x=x,
        num_iter=num_iter,
        learning_rate=learning_rate,
        init_method=init_method,
        num_init_samples=num_init_samples,
        num_to_optimize=num_to_optimize,
        save_best_every=save_best_every,
        show_progress_bars=show_progress_bars,
    )

sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with_mcmc=None, mcmc_method=None, mcmc_parameters=None)

Return samples from posterior distribution p(\theta|x) with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with_mcmc Optional[bool]

Optional parameter to override self.sample_with_mcmc.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
def sample(
    self,
    sample_shape: Shape = torch.Size(),
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    sample_with_mcmc: Optional[bool] = None,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from posterior distribution $p(\theta|x)$ with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        sample_with_mcmc: Optional parameter to override `self.sample_with_mcmc`.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        Samples from posterior.
    """

    x, num_samples, mcmc_method, mcmc_parameters = self._prepare_for_sample(
        x, sample_shape, mcmc_method, mcmc_parameters
    )

    self.net.eval()

    potential_fn_provider = PotentialFunctionProvider()
    samples = self._sample_posterior_mcmc(
        num_samples=num_samples,
        potential_fn=potential_fn_provider(self._prior, self.net, x, mcmc_method),
        init_fn=self._build_mcmc_init_fn(
            self._prior,
            potential_fn_provider(self._prior, self.net, x, "slice_np"),
            **mcmc_parameters,
        ),
        mcmc_method=mcmc_method,
        show_progress_bars=show_progress_bars,
        **mcmc_parameters,
    )

    self.net.train(True)

    return samples.reshape((*sample_shape, -1))

sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, show_progress_bars=True, mcmc_method=None, mcmc_parameters=None)

Return samples from conditional posterior p(\theta_i|\theta_j, x).

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior p(\theta|x). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
def sample_conditional(
    self,
    sample_shape: Shape,
    condition: Tensor,
    dims_to_sample: List[int],
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

    In this function, we do not sample from the full posterior, but instead only
    from a few parameter dimensions while the other parameter dimensions are kept
    fixed at values specified in `condition`.

    Samples are obtained with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_sample: Which dimensions to sample from. The dimensions not
            specified in `dims_to_sample` will be fixed to values given in
            `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        Samples from conditional posterior.
    """

    return super().sample_conditional(
        PotentialFunctionProvider(),
        sample_shape,
        condition,
        dims_to_sample,
        x,
        show_progress_bars,
        mcmc_method,
        mcmc_parameters,
    )

set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior p(theta|x).

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
    """Set new default x for `.sample(), .log_prob` to use as conditioning context.

    This is a pure convenience to avoid having to repeatedly specify `x` in calls to
    `.sample()` and `.log_prob()` - only θ needs to be passed.

    This convenience is particularly useful when the posterior is focused, i.e.
    has been trained over multiple rounds to be accurate in the vicinity of a
    particular `x=x_o` (you can check if your posterior object is focused by
    printing it).

    NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
    that calls like `posterior.set_default_x(my_x).sample(mytheta)` are possible.

    Args:
        x: The default observation to set for the posterior $p(theta|x)$.

    Returns:
        `NeuralPosterior` that will use a default `x` when not explicitly passed.
    """
    self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x)
    self._num_iid_trials = self._x.shape[0]

    return self

set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
147
148
149
150
151
152
153
154
155
156
157
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
    """Sets sampling method to for MCMC and returns `NeuralPosterior`.

    Args:
        method: Method to use.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_method = method
    return self

set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
    """Sets parameters for MCMC and returns `NeuralPosterior`.

    Args:
        parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_parameters = parameters
    return self

Models

sbi.utils.get_nn_models.posterior_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10)

Returns a function that builds a density estimator for learning the posterior.

This function will usually be used for SNPE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters \theta before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs x before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for simulation outputs x. This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
def posterior_nn(
    model: str,
    z_score_theta: bool = True,
    z_score_x: bool = True,
    hidden_features: int = 50,
    num_transforms: int = 5,
    num_bins: int = 10,
    embedding_net: nn.Module = nn.Identity(),
    num_components: int = 10,
) -> Callable:
    r"""
    Returns a function that builds a density estimator for learning the posterior.

    This function will usually be used for SNPE. The returned function is to be passed
    to the inference class when using the flexible interface.

    Args:
        model: The type of density estimator that will be created. One of [`mdn`,
            `made`, `maf`, `nsf`].
        z_score_theta: Whether to z-score parameters $\theta$ before passing them into
            the network.
        z_score_x: Whether to z-score simulation outputs $x$ before passing them into
            the network.
        hidden_features: Number of hidden features.
        num_transforms: Number of transforms when a flow is used. Only relevant if
            density estimator is a normalizing flow (i.e. currently either a `maf` or a
            `nsf`). Ignored if density estimator is a `mdn` or `made`.
        num_bins: Number of bins used for the splines in `nsf`. Ignored if density
            estimator not `nsf`.
        embedding_net: Optional embedding network for simulation outputs $x$. This
            embedding net allows to learn features from potentially high-dimensional
            simulation outputs.
        num_components: Number of mixture components for a mixture of Gaussians.
            Ignored if density estimator is not an mdn.
    """

    kwargs = dict(
        zip(
            (
                "z_score_x",
                "z_score_y",
                "hidden_features",
                "num_transforms",
                "num_bins",
                "embedding_net",
                "num_components",
            ),
            (
                z_score_theta,
                z_score_x,
                hidden_features,
                num_transforms,
                num_bins,
                embedding_net,
                num_components,
            ),
        )
    )

    if model == "mdn_snpe_a":
        if num_components != 10:
            raise ValueError(
                "You set `num_components`. For SNPE-A, this has to be done at "
                "instantiation of the inference object, i.e. "
                "`inference = SNPE_A(..., num_components=20)`"
            )
        kwargs.pop("num_components")

        def build_fn(batch_theta, batch_x, num_components):
            # Extract the number of components from the kwargs, such that
            # they are exposed as a kwargs, offering the possibility to later
            # override this kwarg with functools.partial. This is necessary
            # in order to make sure that the MDN in SNPE-A only has one
            # component when running the Algorithm 1 part.
            return build_mdn(
                batch_x=batch_theta,
                batch_y=batch_x,
                num_components=num_components,
                **kwargs
            )

    else:

        def build_fn(batch_theta, batch_x):
            if model == "mdn":
                return build_mdn(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            elif model == "made":
                return build_made(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            elif model == "maf":
                return build_maf(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            elif model == "nsf":
                return build_nsf(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            else:
                raise NotImplementedError

    return build_fn

sbi.utils.get_nn_models.likelihood_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10)

Returns a function that builds a density estimator for learning the likelihood.

This function will usually be used for SNLE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters \theta before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs x before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for parameters \theta.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
def likelihood_nn(
    model: str,
    z_score_theta: bool = True,
    z_score_x: bool = True,
    hidden_features: int = 50,
    num_transforms: int = 5,
    num_bins: int = 10,
    embedding_net: nn.Module = nn.Identity(),
    num_components: int = 10,
) -> Callable:
    r"""
    Returns a function that builds a density estimator for learning the likelihood.

    This function will usually be used for SNLE. The returned function is to be passed
    to the inference class when using the flexible interface.

    Args:
        model: The type of density estimator that will be created. One of [`mdn`,
            `made`, `maf`, `nsf`].
        z_score_theta: Whether to z-score parameters $\theta$ before passing them into
            the network.
        z_score_x: Whether to z-score simulation outputs $x$ before passing them into
            the network.
        hidden_features: Number of hidden features.
        num_transforms: Number of transforms when a flow is used. Only relevant if
            density estimator is a normalizing flow (i.e. currently either a `maf` or a
            `nsf`). Ignored if density estimator is a `mdn` or `made`.
        num_bins: Number of bins used for the splines in `nsf`. Ignored if density
            estimator not `nsf`.
        embedding_net: Optional embedding network for parameters $\theta$.
        num_components: Number of mixture components for a mixture of Gaussians.
            Ignored if density estimator is not an mdn.
    """

    kwargs = dict(
        zip(
            (
                "z_score_x",
                "z_score_y",
                "hidden_features",
                "num_transforms",
                "num_bins",
                "embedding_net",
                "num_components",
            ),
            (
                z_score_x,
                z_score_theta,
                hidden_features,
                num_transforms,
                num_bins,
                embedding_net,
                num_components,
            ),
        )
    )

    def build_fn(batch_theta, batch_x):
        if model == "mdn":
            return build_mdn(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        if model == "made":
            return build_made(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        if model == "maf":
            return build_maf(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        elif model == "nsf":
            return build_nsf(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        else:
            raise NotImplementedError

    return build_fn

sbi.utils.get_nn_models.classifier_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, embedding_net_theta=Identity(), embedding_net_x=Identity())

Returns a function that builds a classifier for learning density ratios.

This function will usually be used for SNRE. The returned function is to be passed to the inference class when using the flexible interface.

Note that in the view of the SNRE classifier we build below, x=theta and y=x.

Parameters:

Name Type Description Default
model str

The type of classifier that will be created. One of [linear, mlp, resnet].

required
z_score_theta bool

Whether to z-score parameters \theta before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs x before passing them into the network.

True
hidden_features int

Number of hidden features.

50
embedding_net_theta Module

Optional embedding network for parameters \theta.

Identity()
embedding_net_x Module

Optional embedding network for simulation outputs x. This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
Source code in sbi/utils/get_nn_models.py
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
def classifier_nn(
    model: str,
    z_score_theta: bool = True,
    z_score_x: bool = True,
    hidden_features: int = 50,
    embedding_net_theta: nn.Module = nn.Identity(),
    embedding_net_x: nn.Module = nn.Identity(),
) -> Callable:
    r"""
    Returns a function that builds a classifier for learning density ratios.

    This function will usually be used for SNRE. The returned function is to be passed
    to the inference class when using the flexible interface.

    Note that in the view of the SNRE classifier we build below, x=theta and y=x.

    Args:
        model: The type of classifier that will be created. One of [`linear`, `mlp`,
            `resnet`].
        z_score_theta: Whether to z-score parameters $\theta$ before passing them into
            the network.
        z_score_x: Whether to z-score simulation outputs $x$ before passing them into
            the network.
        hidden_features: Number of hidden features.
        embedding_net_theta:  Optional embedding network for parameters $\theta$.
        embedding_net_x:  Optional embedding network for simulation outputs $x$. This
            embedding net allows to learn features from potentially high-dimensional
            simulation outputs.
    """

    kwargs = dict(
        zip(
            (
                "z_score_x",
                "z_score_y",
                "hidden_features",
                "embedding_net_x",
                "embedding_net_y",
            ),
            (
                z_score_theta,
                z_score_x,
                hidden_features,
                embedding_net_theta,
                embedding_net_x,
            ),
        )
    )

    def build_fn(batch_theta, batch_x):
        if model == "linear":
            return build_linear_classifier(
                batch_x=batch_theta, batch_y=batch_x, **kwargs
            )
        if model == "mlp":
            return build_mlp_classifier(batch_x=batch_theta, batch_y=batch_x, **kwargs)
        if model == "resnet":
            return build_resnet_classifier(
                batch_x=batch_theta, batch_y=batch_x, **kwargs
            )
        else:
            raise NotImplementedError

    return build_fn

Analysis

sbi.analysis.plot.pairplot(samples=None, points=None, limits=None, subset=None, upper='hist', diag='hist', figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs)

Plot samples in a 2D grid showing marginals and pairwise marginals.

Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution that the samples were drawn from. Each upper-diagonal plot can be interpreted as a 2D-marginal of the distribution.

Parameters:

Name Type Description Default
samples Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

Samples used to build the histogram.

None
points Optional[Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]]

List of additional points to scatter.

None
limits Optional[Union[List, torch.Tensor]]

Array containing the plot xlim for each parameter dimension. If None, just use the min and max of the passed samples

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on).

None
upper Optional[str]

Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.

'hist'
diag Optional[str]

Plotting style for diagonal, {hist, cond, None}.

'hist'
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
def pairplot(
    samples: Union[
        List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor
    ] = None,
    points: Optional[
        Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor]
    ] = None,
    limits: Optional[Union[List, torch.Tensor]] = None,
    subset: List[int] = None,
    upper: Optional[str] = "hist",
    diag: Optional[str] = "hist",
    figsize: Tuple = (10, 10),
    labels: Optional[List[str]] = None,
    ticks: Union[List, torch.Tensor] = [],
    points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"],
    **kwargs
):
    """
    Plot samples in a 2D grid showing marginals and pairwise marginals.

    Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution
    that the samples were drawn from. Each upper-diagonal plot can be interpreted as a
    2D-marginal of the distribution.

    Args:
        samples: Samples used to build the histogram.
        points: List of additional points to scatter.
        limits: Array containing the plot xlim for each parameter dimension. If None,
            just use the min and max of the passed samples
        subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot
            plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and,
            if they exist, the 4th, 5th and so on).
        upper: Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.
        diag: Plotting style for diagonal, {hist, cond, None}.
        figsize: Size of the entire figure.
        labels: List of strings specifying the names of the parameters.
        ticks: Position of the ticks.
        points_colors: Colors of the `points`.
        **kwargs: Additional arguments to adjust the plot, see the source code in
            `_get_default_opts()` in `sbi.utils.plot` for more details.

    Returns: figure and axis of posterior distribution plot
    """
    return utils_pairplot(
        samples=samples,
        points=points,
        limits=limits,
        subset=subset,
        upper=upper,
        diag=diag,
        figsize=figsize,
        labels=labels,
        ticks=ticks,
        points_colors=points_colors,
        warn_about_deprecation=False,
        **kwargs,
    )

sbi.analysis.plot.conditional_pairplot(density, condition, limits, points=None, subset=None, resolution=50, figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs)

Plot conditional distribution given all other parameters.

The conditionals can be interpreted as slices through the density at a location given by condition.

For example: Say we have a 3D density with parameters \theta_0, \theta_1, \theta_2 and a condition c passed by the user in the condition argument. For the plot of \theta_0 on the diagonal, this will plot the conditional p(\theta_0 | \theta_1=c[1], \theta_2=c[2]). For the upper diagonal of \theta_1 and \theta_2, it will plot p(\theta_1, \theta_2 | \theta_0=c[0]). All other diagonals and upper-diagonals are built in the corresponding way.

Parameters:

Name Type Description Default
density Any

Probability density with a log_prob() method.

required
condition Tensor

Condition that all but the one/two regarded parameters are fixed to. The condition should be of shape (1, dim_theta), i.e. it could e.g. be a sample from the posterior distribution.

required
limits Union[List, torch.Tensor]

Limits in between which each parameter will be evaluated.

required
points Optional[Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]]

Additional points to scatter.

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on)

None
resolution int

Resolution of the grid at which we evaluate the pdf.

50
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
def conditional_pairplot(
    density: Any,
    condition: torch.Tensor,
    limits: Union[List, torch.Tensor],
    points: Optional[
        Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor]
    ] = None,
    subset: List[int] = None,
    resolution: int = 50,
    figsize: Tuple = (10, 10),
    labels: Optional[List[str]] = None,
    ticks: Union[List, torch.Tensor] = [],
    points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"],
    **kwargs
):
    r"""
    Plot conditional distribution given all other parameters.

    The conditionals can be interpreted as slices through the `density` at a location
    given by `condition`.

    For example:
    Say we have a 3D density with parameters $\theta_0$, $\theta_1$, $\theta_2$ and
    a condition $c$ passed by the user in the `condition` argument.
    For the plot of $\theta_0$ on the diagonal, this will plot the conditional
    $p(\theta_0 | \theta_1=c[1], \theta_2=c[2])$. For the upper
    diagonal of $\theta_1$ and $\theta_2$, it will plot
    $p(\theta_1, \theta_2 | \theta_0=c[0])$. All other diagonals and upper-diagonals
    are built in the corresponding way.

    Args:
        density: Probability density with a `log_prob()` method.
        condition: Condition that all but the one/two regarded parameters are fixed to.
            The condition should be of shape (1, dim_theta), i.e. it could e.g. be
            a sample from the posterior distribution.
        limits: Limits in between which each parameter will be evaluated.
        points: Additional points to scatter.
        subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot
            plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and,
            if they exist, the 4th, 5th and so on)
        resolution: Resolution of the grid at which we evaluate the `pdf`.
        figsize: Size of the entire figure.
        labels: List of strings specifying the names of the parameters.
        ticks: Position of the ticks.
        points_colors: Colors of the `points`.
        **kwargs: Additional arguments to adjust the plot, see the source code in
            `_get_default_opts()` in `sbi.utils.plot` for more details.

    Returns: figure and axis of posterior distribution plot
    """
    return utils_conditional_pairplot(
        density=density,
        condition=condition,
        limits=limits,
        points=points,
        subset=subset,
        resolution=resolution,
        figsize=figsize,
        labels=labels,
        ticks=ticks,
        points_colors=points_colors,
        warn_about_deprecation=False,
        **kwargs,
    )

sbi.analysis.conditional_density.conditional_corrcoeff(density, limits, condition, subset=None, resolution=50)

Returns the conditional correlation matrix of a distribution.

To compute the conditional distribution, we condition all but two parameters to values from condition, and then compute the Pearson correlation coefficient \rho between the remaining two parameters under the distribution density. We do so for any pair of parameters specified in subset, thus creating a matrix containing conditional correlations between any pair of parameters.

If condition is a batch of conditions, this function computes the conditional correlation matrix for each one of them and returns the mean.

Parameters:

Name Type Description Default
density Any

Probability density function with .log_prob() function.

required
limits Tensor

Limits within which to evaluate the density.

required
condition Tensor

Values to condition the density on. If a batch of conditions is passed, we compute the conditional correlation matrix for each of them and return the average conditional correlation matrix.

required
subset Optional[List[int]]

Evaluate the conditional distribution only on a subset of dimensions. If None this function uses all dimensions.

None
resolution int

Number of grid points on which the conditional distribution is evaluated. A higher value increases the accuracy of the estimated correlation but also increases the computational cost.

50

Returns: Average conditional correlation matrix of shape either (num_dim, num_dim) or (len(subset), len(subset)) if subset was specified.

Source code in sbi/analysis/conditional_density.py
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
def conditional_corrcoeff(
    density: Any,
    limits: Tensor,
    condition: Tensor,
    subset: Optional[List[int]] = None,
    resolution: int = 50,
) -> Tensor:
    r"""
    Returns the conditional correlation matrix of a distribution.

    To compute the conditional distribution, we condition all but two parameters to
    values from `condition`, and then compute the Pearson correlation
    coefficient $\rho$ between the remaining two parameters under the distribution
    `density`. We do so for any pair of parameters specified in `subset`, thus
    creating a matrix containing conditional correlations between any pair of
    parameters.

    If `condition` is a batch of conditions, this function computes the conditional
    correlation matrix for each one of them and returns the mean.

    Args:
        density: Probability density function with `.log_prob()` function.
        limits: Limits within which to evaluate the `density`.
        condition: Values to condition the `density` on. If a batch of conditions is
            passed, we compute the conditional correlation matrix for each of them and
            return the average conditional correlation matrix.
        subset: Evaluate the conditional distribution only on a subset of dimensions.
            If `None` this function uses all dimensions.
        resolution: Number of grid points on which the conditional distribution is
            evaluated. A higher value increases the accuracy of the estimated
            correlation but also increases the computational cost.

    Returns: Average conditional correlation matrix of shape either `(num_dim, num_dim)`
    or `(len(subset), len(subset))` if `subset` was specified.
    """
    return utils_conditional_corrcoeff(
        density=density,
        limits=limits,
        condition=condition,
        subset=subset,
        resolution=resolution,
        warn_about_deprecation=False,
    )