# API Reference¶

## Inference¶

### sbi.inference.base.infer(simulator, prior, method, num_simulations, num_workers=1) ¶

Return posterior distribution by running simulation-based inference.

This function provides a simple interface to run sbi. Inference is run for a single round and hence the returned posterior $p(\theta|x)$ can be sampled and evaluated for any $x$ (i.e. it is amortized).

The scope of this function is limited to the most essential features of sbi. For more flexibility (e.g. multi-round inference, different density estimators) please use the flexible interface described here: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
method str

What inference method to use. Either of SNPE, SNLE or SNRE.

required
num_simulations int

Number of simulation calls. More simulations means a longer runtime, but a better posterior estimate.

required
num_workers int

Number of parallel workers to use for simulations.

1

Returns: Posterior over parameters conditional on observations (amortized).

Source code in sbi/inference/base.py
 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 def infer( simulator: Callable, prior, method: str, num_simulations: int, num_workers: int = 1 ) -> NeuralPosterior: r""" Return posterior distribution by running simulation-based inference. This function provides a simple interface to run sbi. Inference is run for a single round and hence the returned posterior $p(\theta|x)$ can be sampled and evaluated for any $x$ (i.e. it is amortized). The scope of this function is limited to the most essential features of sbi. For more flexibility (e.g. multi-round inference, different density estimators) please use the flexible interface described here: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/ Args: simulator: A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used. prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. method: What inference method to use. Either of SNPE, SNLE or SNRE. num_simulations: Number of simulation calls. More simulations means a longer runtime, but a better posterior estimate. num_workers: Number of parallel workers to use for simulations. Returns: Posterior over parameters conditional on observations (amortized). """ try: method_fun: Callable = getattr(sbi.inference, method.upper()) except AttributeError: raise NameError( "Method not available. method must be one of 'SNPE', 'SNLE', 'SNRE'." ) simulator, prior = prepare_for_sbi(simulator, prior) inference = method_fun(prior) theta, x = simulate_for_sbi( simulator=simulator, proposal=prior, num_simulations=num_simulations, num_workers=num_workers, ) _ = inference.append_simulations(theta, x).train() posterior = inference.build_posterior() return posterior

### sbi.utils.user_input_checks.prepare_for_sbi(simulator, prior) ¶

Prepare simulator, prior and for usage in sbi.

One of the goals is to allow you to use sbi with inputs computed in numpy.

Attempts to meet the following requirements by reshaping and type-casting:

• the simulator function receives as input and returns a Tensor.
• the simulator can simulate batches of parameters and return batches of data.
• the prior does not produce batches and samples and evaluates to Tensor.
• the output shape is a torch.Size((1,N)) (i.e, has a leading batch dimension 1).

If this is not possible, a suitable exception will be raised.

Parameters:

Name Type Description Default
simulator Callable

Simulator as provided by the user.

required
prior

Prior as provided by the user.

required

Returns:

Type Description
Tuple[Callable, torch.distributions.distribution.Distribution]

Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.

Source code in sbi/utils/user_input_checks.py
 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 def prepare_for_sbi(simulator: Callable, prior) -> Tuple[Callable, Distribution]: """Prepare simulator, prior and for usage in sbi. One of the goals is to allow you to use sbi with inputs computed in numpy. Attempts to meet the following requirements by reshaping and type-casting: - the simulator function receives as input and returns a Tensor.
- the simulator can simulate batches of parameters and return batches of data.
- the prior does not produce batches and samples and evaluates to Tensor.
- the output shape is a torch.Size((1,N)) (i.e, has a leading batch dimension 1). If this is not possible, a suitable exception will be raised. Args: simulator: Simulator as provided by the user. prior: Prior as provided by the user. Returns: Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi. """ # Check prior, return PyTorch prior. prior, _, prior_returns_numpy = process_prior(prior) # Check simulator, returns PyTorch simulator able to simulate batches. simulator = process_simulator(simulator, prior, prior_returns_numpy) # Consistency check after making ready for sbi. check_sbi_inputs(simulator, prior) return simulator, prior

### sbi.inference.base.simulate_for_sbi(simulator, proposal, num_simulations, num_workers=1, simulation_batch_size=1, show_progress_bar=True) ¶

Returns ($\theta, x$) pairs obtained from sampling the proposal and simulating.

This function performs two steps:

• Sample parameters $\theta$ from the proposal.
• Simulate these parameters to obtain $x$.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\text{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
proposal Any

Probability distribution that the parameters $\theta$ are sampled from.

required
num_simulations int

Number of simulations that are run.

required
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bar bool

Whether to show a progress bar for simulating. This will not affect whether there will be a progressbar while drawing samples from the proposal.

True

Returns: Sampled parameters $\theta$ and simulation-outputs $x$.

Source code in sbi/inference/base.py
 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 def simulate_for_sbi( simulator: Callable, proposal: Any, num_simulations: int, num_workers: int = 1, simulation_batch_size: int = 1, show_progress_bar: bool = True, ) -> Tuple[Tensor, Tensor]: r""" Returns ($\theta, x$) pairs obtained from sampling the proposal and simulating. This function performs two steps: - Sample parameters $\theta$ from the proposal. - Simulate these parameters to obtain $x$. Args: simulator: A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\text{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used. proposal: Probability distribution that the parameters $\theta$ are sampled from. num_simulations: Number of simulations that are run. num_workers: Number of parallel workers to use for simulations. simulation_batch_size: Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension). show_progress_bar: Whether to show a progress bar for simulating. This will not affect whether there will be a progressbar while drawing samples from the proposal. Returns: Sampled parameters $\theta$ and simulation-outputs $x$. """ check_if_proposal_has_default_x(proposal) theta = proposal.sample((num_simulations,)) x = simulate_in_batches( simulator, theta, simulation_batch_size, num_workers, show_progress_bar, ) return theta, x

### sbi.inference.snpe.snpe_c.SNPE_C¶

#### __init__(self, prior=None, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special

SNPE-C / APT [1].

[1] Automatic Posterior Transformation for Likelihood-free Inference, Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

This class implements two loss variants of SNPE-C: the non-atomic and the atomic version. The atomic loss of SNPE-C can be used for any density estimator, i.e. also for normalizing flows. However, it suffers from leakage issues. On the other hand, the non-atomic loss can only be used only if the proposal distribution is a mixture of Gaussians, the density estimator is a mixture of Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from leakage issues. At the beginning of each round, we print whether the non-atomic or the atomic version is used.

In this codebase, we will automatically switch to the non-atomic loss if the following criteria are fulfilled:
- proposal is a DirectPosterior with density_estimator mdn, as built with utils.sbi.posterior_nn().
- the density estimator is a mdn, as built with utils.sbi.posterior_nn().
- isinstance(prior, MultivariateNormal) (from torch.distributions) or isinstance(prior, sbi.utils.BoxUniform)

Note that custom implementations of any of these densities (or estimators) will not trigger the non-atomic loss, and the algorithm will fall back onto using the atomic loss.

Parameters:

Name Type Description Default
prior Optional[Any]

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

None
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during training.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snpe/snpe_c.py
 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 def __init__( self, prior: Optional[Any] = None, density_estimator: Union[str, Callable] = "maf", device: str = "cpu", logging_level: Union[int, str] = "WARNING", summary_writer: Optional[TensorboardSummaryWriter] = None, show_progress_bars: bool = True, **unused_args, ): r"""SNPE-C / APT [1]. [1] _Automatic Posterior Transformation for Likelihood-free Inference_, Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488. This class implements two loss variants of SNPE-C: the non-atomic and the atomic version. The atomic loss of SNPE-C can be used for any density estimator, i.e. also for normalizing flows. However, it suffers from leakage issues. On the other hand, the non-atomic loss can only be used only if the proposal distribution is a mixture of Gaussians, the density estimator is a mixture of Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from leakage issues. At the beginning of each round, we print whether the non-atomic or the atomic version is used. In this codebase, we will automatically switch to the non-atomic loss if the following criteria are fulfilled:
- proposal is a DirectPosterior with density_estimator mdn, as built with utils.sbi.posterior_nn().
- the density estimator is a mdn, as built with utils.sbi.posterior_nn().
- isinstance(prior, MultivariateNormal) (from torch.distributions) or isinstance(prior, sbi.utils.BoxUniform) Note that custom implementations of any of these densities (or estimators) will not trigger the non-atomic loss, and the algorithm will fall back onto using the atomic loss. Args: prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. density_estimator: If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample(). device: torch device on which to compute, e.g. gpu, cpu. logging_level: Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL. summary_writer: A tensorboard SummaryWriter to control, among others, log file location (default is /logs.) show_progress_bars: Whether to show a progressbar during training. unused_args: Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument. """ kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args")) super().__init__(**kwargs, **unused_args)

#### append_simulations(self, theta, x, proposal=None) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
proposal Optional[Any]

The distribution that the parameters $\theta$ were sampled from. Pass None if the parameters were sampled from the prior. If not None, it will trigger a different loss-function.

None

Returns:

Type Description
PosteriorEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snpe/snpe_c.py
 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 def append_simulations( self, theta: Tensor, x: Tensor, proposal: Optional[Any] = None ) -> "PosteriorEstimator": r""" Store parameters and simulation outputs to use them for later training. Data are stored as entries in lists for each type of variable (parameter/data). Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from. Args: theta: Parameter sets. x: Simulation outputs. proposal: The distribution that the parameters $\theta$ were sampled from. Pass None if the parameters were sampled from the prior. If not None, it will trigger a different loss-function. Returns: NeuralInference object (returned so that this function is chainable). """ theta, x = validate_theta_and_x(theta, x, training_device=self._device) self._check_proposal(proposal) if ( proposal is None or proposal is self._prior or ( isinstance(proposal, RestrictedPrior) and proposal._prior is self._prior ) ): # The _data_round_index will later be used to infer if one should train # with MLE loss or with atomic loss (see, in train(): # self._round = max(self._data_round_index)) self._data_round_index.append(0) self._prior_masks.append(mask_sims_from_prior(0, theta.size(0))) else: if not self._data_round_index: # This catches a pretty specific case: if, in the first round, one # passes data that does not come from the prior. self._data_round_index.append(1) else: self._data_round_index.append(max(self._data_round_index) + 1) self._prior_masks.append(mask_sims_from_prior(1, theta.size(0))) self._theta_roundwise.append(theta) self._x_roundwise.append(x) self._proposal_roundwise.append(proposal) if self._prior is None or isinstance(self._prior, ImproperEmpirical): if proposal is not None: raise ValueError( "You had not passed a prior at initialization, but now you " "passed a proposal. If you want to run multi-round SNPE, you have " "to specify a prior (set the .prior argument or re-initialize " "the object with a prior distribution). If the samples you passed " "to append_simulations() were sampled from the prior, you can " "run single-round inference with " "append_simulations(..., proposal=None)." ) theta_prior = self.get_simulations()[0] self._prior = ImproperEmpirical(theta_prior, ones(theta_prior.shape[0])) return self

#### build_posterior(self, density_estimator=None, rejection_sampling_parameters=None, sample_with_mcmc=False, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

For SNPE, the posterior distribution that is returned here implements the following functionality over the raw neural density estimator:

• correct the calculation of the log probability such that it compensates for the leakage.
• reject samples that lie outside of the prior bounds.
• alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

None
sample_with_mcmc bool

Whether to sample with MCMC. MCMC can be used to deal with high leakage.

False
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
DirectPosterior

Posterior $p(\theta|x)$ with .sample() and .log_prob() methods.

Source code in sbi/inference/snpe/snpe_c.py
 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 def build_posterior( self, density_estimator: Optional[TorchModule] = None, rejection_sampling_parameters: Optional[Dict[str, Any]] = None, sample_with_mcmc: bool = False, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> DirectPosterior: r""" Build posterior from the neural density estimator. For SNPE, the posterior distribution that is returned here implements the following functionality over the raw neural density estimator: - correct the calculation of the log probability such that it compensates for the leakage. - reject samples that lie outside of the prior bounds. - alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC. Args: density_estimator: The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained. rejection_sampling_parameters: Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling. sample_with_mcmc: Whether to sample with MCMC. MCMC can be used to deal with high leakage. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Posterior $p(\theta|x)$ with .sample() and .log_prob() methods. """ if density_estimator is None: density_estimator = self._neural_net # If internal net is used device is defined. device = self._device else: # Otherwise, infer it from the device of the net parameters. device = next(density_estimator.parameters()).device self._posterior = DirectPosterior( method_family="snpe", neural_net=density_estimator, prior=self._prior, x_shape=self._x_shape, rejection_sampling_parameters=rejection_sampling_parameters, sample_with_mcmc=sample_with_mcmc, mcmc_method=mcmc_method, mcmc_parameters=mcmc_parameters, device=device, ) self._posterior._num_trained_rounds = self._round + 1 # Store models at end of each round. self._model_bank.append(deepcopy(self._posterior)) self._model_bank[-1].net.eval() return deepcopy(self._posterior)

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snpe/snpe_c.py

#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snpe/snpe_c.py
 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 def get_simulations( self, starting_round: int = 0, exclude_invalid_x: bool = True, warn_on_invalid: bool = True, ) -> Tuple[Tensor, Tensor, Tensor]: r""" Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round. If requested, do not return invalid data. Args: starting_round: The earliest round to return samples from (we start counting from zero). exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. warn_on_invalid: Whether to give out a warning if invalid simulations were found. Returns: Parameters, simulation outputs, prior masks. """ theta = get_simulations_since_round( self._theta_roundwise, self._data_round_index, starting_round ) x = get_simulations_since_round( self._x_roundwise, self._data_round_index, starting_round ) prior_masks = get_simulations_since_round( self._prior_masks, self._data_round_index, starting_round ) # Check for NaNs in simulations. is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x) # Check for problematic z-scoring warn_if_zscoring_changes_data(x) if warn_on_invalid: warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x) warn_on_invalid_x_for_snpec_leakage( num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round ) return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

#### provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snpe/snpe_c.py
 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 def provide_presimulated( self, theta: Tensor, x: Tensor, from_round: int = 0 ) -> None: r""" Deprecated since sbi 0.14.0. Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/ Provide external $\theta$ and $x$ to be used for training later on. Args: theta: Parameter sets used to generate presimulated data. x: Simulation outputs of presimulated data. from_round: Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior. """ raise NameError( f"Deprecated since sbi 0.14.0. " f"Instead of using this, please use .append_simulations(). Please " f"consult release notes to see how you can update your code: " f"https://github.com/mackelab/sbi/releases/tag/v0.14.0" f"More information can be found under the corresponding pull request on " f"github: " f"https://github.com/mackelab/sbi/pull/378" f"and tutorials: " f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/", )

#### train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, calibration_kernel=None, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, use_combined_loss=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return density estimator that approximates the distribution $p(\theta|x)$.

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
calibration_kernel Optional[Callable]

A function to calibrate the loss with respect to the simulations x. See Lueckmann, Gonçalves et al., NeurIPS 2017.

None
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
use_combined_loss bool

Whether to train the neural net also on prior samples using maximum likelihood in addition to training it on all samples using atomic loss. The extra MLE loss helps prevent density leaking with bounded priors.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
DirectPosterior

Density estimator that approximates the distribution $p(\theta|x)$.

Source code in sbi/inference/snpe/snpe_c.py
 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 def train( self, num_atoms: int = 10, training_batch_size: int = 50, learning_rate: float = 5e-4, validation_fraction: float = 0.1, stop_after_epochs: int = 20, max_num_epochs: Optional[int] = None, clip_max_norm: Optional[float] = 5.0, calibration_kernel: Optional[Callable] = None, exclude_invalid_x: bool = True, resume_training: bool = False, discard_prior_samples: bool = False, use_combined_loss: bool = False, retrain_from_scratch_each_round: bool = False, show_train_summary: bool = False, dataloader_kwargs: Optional[Dict] = None, ) -> DirectPosterior: r""" Return density estimator that approximates the distribution $p(\theta|x)$. Args: num_atoms: Number of atoms to use for classification. training_batch_size: Training batch size. learning_rate: Learning rate for Adam optimizer. validation_fraction: The fraction of data to use for validation. stop_after_epochs: The number of epochs to wait for improvement on the validation set before terminating training. max_num_epochs: Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs). clip_max_norm: Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping. calibration_kernel: A function to calibrate the loss with respect to the simulations x. See Lueckmann, Gonçalves et al., NeurIPS 2017. exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False. resume_training: Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called. discard_prior_samples: Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples. use_combined_loss: Whether to train the neural net also on prior samples using maximum likelihood in addition to training it on all samples using atomic loss. The extra MLE loss helps prevent density leaking with bounded priors. retrain_from_scratch_each_round: Whether to retrain the conditional density estimator for the posterior from scratch each round. show_train_summary: Whether to print the number of epochs and validation loss and leakage after the training. dataloader_kwargs: Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn) Returns: Density estimator that approximates the distribution $p(\theta|x)$. """ # WARNING: sneaky trick ahead. We proxy the parent's train here, # requiring the signature to have num_atoms, save it for use below, and # continue. It's sneaky because we are using the object (self) as a namespace # to pass arguments between functions, and that's implicit state management. self._num_atoms = num_atoms self._use_combined_loss = use_combined_loss kwargs = del_entries( locals(), entries=("self", "__class__", "num_atoms", "use_combined_loss") ) self._round = max(self._data_round_index) if self._round > 0: # Set the proposal to the last proposal that was passed by the user. For # atomic SNPE, it does not matter what the proposal is. For non-atomic # SNPE, we only use the latest data that was passed, i.e. the one from the # last proposal. proposal = self._proposal_roundwise[-1] if hasattr(proposal, "net"): self.use_non_atomic_loss = ( isinstance(proposal.net._distribution, mdn) and isinstance(self._neural_net._distribution, mdn) and check_dist_class( self._prior, class_to_check=(Uniform, MultivariateNormal) )[0] ) else: self.use_non_atomic_loss = False algorithm = "non-atomic" if self.use_non_atomic_loss else "atomic" print(f"Using SNPE-C with {algorithm} loss") if self.use_non_atomic_loss: # Take care of z-scoring, pre-compute and store prior terms. self._set_state_for_mog_proposal() return super().train(**kwargs)

### sbi.inference.snle.snle_a.SNLE_A¶

#### __init__(self, prior, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special

Sequential Neural Likelihood [1].

[1] Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows_, Papamakarios et al., AISTATS 2019, https://arxiv.org/abs/1805.07226

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snle/snle_a.py
 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 def __init__( self, prior, density_estimator: Union[str, Callable] = "maf", device: str = "cpu", logging_level: Union[int, str] = "WARNING", summary_writer: Optional[TensorboardSummaryWriter] = None, show_progress_bars: bool = True, **unused_args, ): r"""Sequential Neural Likelihood [1]. [1] Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows_, Papamakarios et al., AISTATS 2019, https://arxiv.org/abs/1805.07226 Args: prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. density_estimator: If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample(). device: torch device on which to compute, e.g. gpu, cpu. logging_level: Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL. summary_writer: A tensorboard SummaryWriter to control, among others, log file location (default is /logs.) show_progress_bars: Whether to show a progressbar during simulation and sampling. unused_args: Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument. """ kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args")) super().__init__(**kwargs, **unused_args)

#### append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNLE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
LikelihoodEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snle/snle_a.py
 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 def append_simulations( self, theta: Tensor, x: Tensor, from_round: int = 0, ) -> "LikelihoodEstimator": r""" Store parameters and simulation outputs to use them for later training. Data are stored as entries in lists for each type of variable (parameter/data). Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from. Args: theta: Parameter sets. x: Simulation outputs. from_round: Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNLE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior. Returns: NeuralInference object (returned so that this function is chainable). """ theta, x = validate_theta_and_x(theta, x, training_device=self._device) self._theta_roundwise.append(theta) self._x_roundwise.append(x) self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0))) self._data_round_index.append(int(from_round)) return self

#### build_posterior(self, density_estimator=None, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

SNLE trains a neural network to approximate the likelihood $p(x|\theta)$. The LikelihoodBasedPosterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
LikelihoodBasedPosterior

Posterior $p(\theta|x)$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snle/snle_a.py
 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 def build_posterior( self, density_estimator: Optional[TorchModule] = None, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> LikelihoodBasedPosterior: r""" Build posterior from the neural density estimator. SNLE trains a neural network to approximate the likelihood $p(x|\theta)$. The LikelihoodBasedPosterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC. Args: density_estimator: The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Posterior $p(\theta|x)$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized). """ if density_estimator is None: density_estimator = self._neural_net # If internal net is used device is defined. device = self._device else: # Otherwise, infer it from the device of the net parameters. device = next(density_estimator.parameters()).device self._posterior = LikelihoodBasedPosterior( method_family="snle", neural_net=density_estimator, prior=self._prior, x_shape=self._x_shape, mcmc_method=mcmc_method, mcmc_parameters=mcmc_parameters, device=device, ) self._posterior._num_trained_rounds = self._round + 1 # Store models at end of each round. self._model_bank.append(deepcopy(self._posterior)) self._model_bank[-1].net.eval() return deepcopy(self._posterior)

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snle/snle_a.py

#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snle/snle_a.py
 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 def get_simulations( self, starting_round: int = 0, exclude_invalid_x: bool = True, warn_on_invalid: bool = True, ) -> Tuple[Tensor, Tensor, Tensor]: r""" Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round. If requested, do not return invalid data. Args: starting_round: The earliest round to return samples from (we start counting from zero). exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. warn_on_invalid: Whether to give out a warning if invalid simulations were found. Returns: Parameters, simulation outputs, prior masks. """ theta = get_simulations_since_round( self._theta_roundwise, self._data_round_index, starting_round ) x = get_simulations_since_round( self._x_roundwise, self._data_round_index, starting_round ) prior_masks = get_simulations_since_round( self._prior_masks, self._data_round_index, starting_round ) # Check for NaNs in simulations. is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x) # Check for problematic z-scoring warn_if_zscoring_changes_data(x) if warn_on_invalid: warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x) warn_on_invalid_x_for_snpec_leakage( num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round ) return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

#### provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snle/snle_a.py
 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 def provide_presimulated( self, theta: Tensor, x: Tensor, from_round: int = 0 ) -> None: r""" Deprecated since sbi 0.14.0. Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/ Provide external $\theta$ and $x$ to be used for training later on. Args: theta: Parameter sets used to generate presimulated data. x: Simulation outputs of presimulated data. from_round: Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior. """ raise NameError( f"Deprecated since sbi 0.14.0. " f"Instead of using this, please use .append_simulations(). Please " f"consult release notes to see how you can update your code: " f"https://github.com/mackelab/sbi/releases/tag/v0.14.0" f"More information can be found under the corresponding pull request on " f"github: " f"https://github.com/mackelab/sbi/pull/378" f"and tutorials: " f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/", )

#### train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return density estimator that approximates the distribution $p(x|\theta)$.

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Density estimator that approximates the distribution $p(x|\theta)$.

Source code in sbi/inference/snle/snle_a.py
 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 def train( self, training_batch_size: int = 50, learning_rate: float = 5e-4, validation_fraction: float = 0.1, stop_after_epochs: int = 20, max_num_epochs: Optional[int] = None, clip_max_norm: Optional[float] = 5.0, exclude_invalid_x: bool = True, resume_training: bool = False, discard_prior_samples: bool = False, retrain_from_scratch_each_round: bool = False, show_train_summary: bool = False, dataloader_kwargs: Optional[Dict] = None, ) -> NeuralPosterior: r""" Return density estimator that approximates the distribution $p(x|\theta)$. Args: training_batch_size: Training batch size. learning_rate: Learning rate for Adam optimizer. validation_fraction: The fraction of data to use for validation. stop_after_epochs: The number of epochs to wait for improvement on the validation set before terminating training. max_num_epochs: Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs). clip_max_norm: Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping. exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False. resume_training: Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called. discard_prior_samples: Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples. retrain_from_scratch_each_round: Whether to retrain the conditional density estimator for the posterior from scratch each round. show_train_summary: Whether to print the number of epochs and validation loss and leakage after the training. dataloader_kwargs: Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn) Returns: Density estimator that approximates the distribution $p(x|\theta)$. """ kwargs = del_entries(locals(), entries=("self", "__class__")) return super().train(**kwargs)

### sbi.inference.snre.snre_a.SNRE_A¶

#### __init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special

AALR[1], here known as SNRE_A.

[1] Likelihood-free MCMC with Amortized Approximate Likelihood Ratios, Hermans et al., ICML 2020, https://arxiv.org/abs/1903.04057

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_a.py
 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 def __init__( self, prior, classifier: Union[str, Callable] = "resnet", device: str = "cpu", logging_level: Union[int, str] = "warning", summary_writer: Optional[TensorboardSummaryWriter] = None, show_progress_bars: bool = True, **unused_args ): r"""AALR[1], here known as SNRE_A. [1] _Likelihood-free MCMC with Amortized Approximate Likelihood Ratios_, Hermans et al., ICML 2020, https://arxiv.org/abs/1903.04057 Args: prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. classifier: Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier. device: torch device on which to compute, e.g. gpu, cpu. logging_level: Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL. summary_writer: A tensorboard SummaryWriter to control, among others, log file location (default is /logs.) show_progress_bars: Whether to show a progressbar during simulation and sampling. unused_args: Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument. """ kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args")) super().__init__(**kwargs, **unused_args)

#### append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_a.py
 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 def append_simulations( self, theta: Tensor, x: Tensor, from_round: int = 0, ) -> "RatioEstimator": r""" Store parameters and simulation outputs to use them for later training. Data are stored as entries in lists for each type of variable (parameter/data). Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from. Args: theta: Parameter sets. x: Simulation outputs. from_round: Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior. Returns: NeuralInference object (returned so that this function is chainable). """ theta, x = validate_theta_and_x(theta, x, training_device=self._device) self._theta_roundwise.append(theta) self._x_roundwise.append(x) self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0))) self._data_round_index.append(int(from_round)) return self

#### build_posterior(self, density_estimator=None, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
RatioBasedPosterior

Posterior $p(\theta|x)$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_a.py
 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 def build_posterior( self, density_estimator: Optional[TorchModule] = None, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> RatioBasedPosterior: r""" Build posterior from the neural density estimator. SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the **normalized** posterior, but sampling still requires MCMC. Args: density_estimator: The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Posterior $p(\theta|x)$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized). """ if density_estimator is None: density_estimator = self._neural_net # If internal net is used device is defined. device = self._device else: # Otherwise, infer it from the device of the net parameters. device = next(density_estimator.parameters()).device self._posterior = RatioBasedPosterior( method_family=self.__class__.__name__.lower(), neural_net=density_estimator, prior=self._prior, x_shape=self._x_shape, mcmc_method=mcmc_method, mcmc_parameters=mcmc_parameters, device=device, ) self._posterior._num_trained_rounds = self._round + 1 # Store models at end of each round. self._model_bank.append(deepcopy(self._posterior)) self._model_bank[-1].net.eval() return deepcopy(self._posterior)

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_a.py

#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_a.py
 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 def get_simulations( self, starting_round: int = 0, exclude_invalid_x: bool = True, warn_on_invalid: bool = True, ) -> Tuple[Tensor, Tensor, Tensor]: r""" Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round. If requested, do not return invalid data. Args: starting_round: The earliest round to return samples from (we start counting from zero). exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. warn_on_invalid: Whether to give out a warning if invalid simulations were found. Returns: Parameters, simulation outputs, prior masks. """ theta = get_simulations_since_round( self._theta_roundwise, self._data_round_index, starting_round ) x = get_simulations_since_round( self._x_roundwise, self._data_round_index, starting_round ) prior_masks = get_simulations_since_round( self._prior_masks, self._data_round_index, starting_round ) # Check for NaNs in simulations. is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x) # Check for problematic z-scoring warn_if_zscoring_changes_data(x) if warn_on_invalid: warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x) warn_on_invalid_x_for_snpec_leakage( num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round ) return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

#### provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_a.py
 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 def provide_presimulated( self, theta: Tensor, x: Tensor, from_round: int = 0 ) -> None: r""" Deprecated since sbi 0.14.0. Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/ Provide external $\theta$ and $x$ to be used for training later on. Args: theta: Parameter sets used to generate presimulated data. x: Simulation outputs of presimulated data. from_round: Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior. """ raise NameError( f"Deprecated since sbi 0.14.0. " f"Instead of using this, please use .append_simulations(). Please " f"consult release notes to see how you can update your code: " f"https://github.com/mackelab/sbi/releases/tag/v0.14.0" f"More information can be found under the corresponding pull request on " f"github: " f"https://github.com/mackelab/sbi/pull/378" f"and tutorials: " f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/", )

#### train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

Source code in sbi/inference/snre/snre_a.py
 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 def train( self, training_batch_size: int = 50, learning_rate: float = 5e-4, validation_fraction: float = 0.1, stop_after_epochs: int = 20, max_num_epochs: Optional[int] = None, clip_max_norm: Optional[float] = 5.0, exclude_invalid_x: bool = True, resume_training: bool = False, discard_prior_samples: bool = False, retrain_from_scratch_each_round: bool = False, show_train_summary: bool = False, dataloader_kwargs: Optional[Dict] = None, ) -> NeuralPosterior: r""" Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$. Args: training_batch_size: Training batch size. learning_rate: Learning rate for Adam optimizer. validation_fraction: The fraction of data to use for validation. stop_after_epochs: The number of epochs to wait for improvement on the validation set before terminating training. max_num_epochs: Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs). clip_max_norm: Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping. exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False. resume_training: Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called. discard_prior_samples: Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples. retrain_from_scratch_each_round: Whether to retrain the conditional density estimator for the posterior from scratch each round. show_train_summary: Whether to print the number of epochs and validation loss and leakage after the training. dataloader_kwargs: Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn) Returns: Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$. """ # AALR is defined for num_atoms=2. # Proxy to super().__call__ to ensure right parameter. kwargs = del_entries(locals(), entries=("self", "__class__")) return super().train(**kwargs, num_atoms=2)

### sbi.inference.snre.snre_b.SNRE_B¶

#### __init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special

SRE[1], here known as SNRE_B.

[1] On Contrastive Learning for Likelihood-free Inference, Durkan et al., ICML 2020, https://arxiv.org/pdf/2002.03712

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

torch device on which to compute, e.g. gpu, cpu.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_b.py
 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 def __init__( self, prior, classifier: Union[str, Callable] = "resnet", device: str = "cpu", logging_level: Union[int, str] = "warning", summary_writer: Optional[TensorboardSummaryWriter] = None, show_progress_bars: bool = True, **unused_args ): r"""SRE[1], here known as SNRE_B. [1] _On Contrastive Learning for Likelihood-free Inference_, Durkan et al., ICML 2020, https://arxiv.org/pdf/2002.03712 Args: prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. classifier: Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier. device: torch device on which to compute, e.g. gpu, cpu. logging_level: Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL. summary_writer: A tensorboard SummaryWriter to control, among others, log file location (default is /logs.) show_progress_bars: Whether to show a progressbar during simulation and sampling. unused_args: Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument. """ kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args")) super().__init__(**kwargs, **unused_args)

#### append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_b.py
 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 def append_simulations( self, theta: Tensor, x: Tensor, from_round: int = 0, ) -> "RatioEstimator": r""" Store parameters and simulation outputs to use them for later training. Data are stored as entries in lists for each type of variable (parameter/data). Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from. Args: theta: Parameter sets. x: Simulation outputs. from_round: Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior. Returns: NeuralInference object (returned so that this function is chainable). """ theta, x = validate_theta_and_x(theta, x, training_device=self._device) self._theta_roundwise.append(theta) self._x_roundwise.append(x) self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0))) self._data_round_index.append(int(from_round)) return self

#### build_posterior(self, density_estimator=None, mcmc_method='slice_np', mcmc_parameters=None) inherited

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
RatioBasedPosterior

Posterior $p(\theta|x)$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_b.py
 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 def build_posterior( self, density_estimator: Optional[TorchModule] = None, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> RatioBasedPosterior: r""" Build posterior from the neural density estimator. SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the **normalized** posterior, but sampling still requires MCMC. Args: density_estimator: The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Posterior $p(\theta|x)$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized). """ if density_estimator is None: density_estimator = self._neural_net # If internal net is used device is defined. device = self._device else: # Otherwise, infer it from the device of the net parameters. device = next(density_estimator.parameters()).device self._posterior = RatioBasedPosterior( method_family=self.__class__.__name__.lower(), neural_net=density_estimator, prior=self._prior, x_shape=self._x_shape, mcmc_method=mcmc_method, mcmc_parameters=mcmc_parameters, device=device, ) self._posterior._num_trained_rounds = self._round + 1 # Store models at end of each round. self._model_bank.append(deepcopy(self._posterior)) self._model_bank[-1].net.eval() return deepcopy(self._posterior)

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_b.py

#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_b.py
 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 def get_simulations( self, starting_round: int = 0, exclude_invalid_x: bool = True, warn_on_invalid: bool = True, ) -> Tuple[Tensor, Tensor, Tensor]: r""" Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round. If requested, do not return invalid data. Args: starting_round: The earliest round to return samples from (we start counting from zero). exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. warn_on_invalid: Whether to give out a warning if invalid simulations were found. Returns: Parameters, simulation outputs, prior masks. """ theta = get_simulations_since_round( self._theta_roundwise, self._data_round_index, starting_round ) x = get_simulations_since_round( self._x_roundwise, self._data_round_index, starting_round ) prior_masks = get_simulations_since_round( self._prior_masks, self._data_round_index, starting_round ) # Check for NaNs in simulations. is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x) # Check for problematic z-scoring warn_if_zscoring_changes_data(x) if warn_on_invalid: warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x) warn_on_invalid_x_for_snpec_leakage( num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round ) return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

#### provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_b.py
 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 def provide_presimulated( self, theta: Tensor, x: Tensor, from_round: int = 0 ) -> None: r""" Deprecated since sbi 0.14.0. Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/ Provide external $\theta$ and $x$ to be used for training later on. Args: theta: Parameter sets used to generate presimulated data. x: Simulation outputs of presimulated data. from_round: Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior. """ raise NameError( f"Deprecated since sbi 0.14.0. " f"Instead of using this, please use .append_simulations(). Please " f"consult release notes to see how you can update your code: " f"https://github.com/mackelab/sbi/releases/tag/v0.14.0" f"More information can be found under the corresponding pull request on " f"github: " f"https://github.com/mackelab/sbi/pull/378" f"and tutorials: " f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/", )

#### train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

Source code in sbi/inference/snre/snre_b.py
 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 def train( self, num_atoms: int = 10, training_batch_size: int = 50, learning_rate: float = 5e-4, validation_fraction: float = 0.1, stop_after_epochs: int = 20, max_num_epochs: Optional[int] = None, clip_max_norm: Optional[float] = 5.0, exclude_invalid_x: bool = True, resume_training: bool = False, discard_prior_samples: bool = False, retrain_from_scratch_each_round: bool = False, show_train_summary: bool = False, dataloader_kwargs: Optional[Dict] = None, ) -> NeuralPosterior: r""" Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$. Args: num_atoms: Number of atoms to use for classification. training_batch_size: Training batch size. learning_rate: Learning rate for Adam optimizer. validation_fraction: The fraction of data to use for validation. stop_after_epochs: The number of epochs to wait for improvement on the validation set before terminating training. max_num_epochs: Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs). clip_max_norm: Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping. exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False. resume_training: Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called. discard_prior_samples: Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples. retrain_from_scratch_each_round: Whether to retrain the conditional density estimator for the posterior from scratch each round. show_train_summary: Whether to print the number of epochs and validation loss and leakage after the training. dataloader_kwargs: Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn) Returns: Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$. """ kwargs = del_entries(locals(), entries=("self", "__class__")) return super().train(**kwargs)

### sbi.inference.abc.mcabc.MCABC¶

#### __call__(self, x_o, num_simulations, eps=None, quantile=None, return_distances=False, return_x_accepted=False, lra=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1) special

Run MCABC.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_simulations int

Number of simulations to run.

required
eps Optional[float]

Acceptance threshold $\epsilon$ for distance between observed and simulated data.

None
quantile Optional[float]

Upper quantile of smallest distances for which the corresponding parameters are returned, e.g, q=0.01 will return the top 1%. Exactly one of quantile or eps have to be passed.

None
return_distances bool

Whether to return the distances corresponding to the accepted parameters.

False
return_distances bool

Whether to return the simulated data corresponding to the accepted parameters.

False
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1

Returns:

Type Description
Union[torch.distributions.distribution.Distribution, Tuple[torch.distributions.distribution.Distribution, torch.Tensor]]

posterior: Empirical distribution based on selected parameters. distances: Tensor of distances of the selected parameters.

Source code in sbi/inference/abc/mcabc.py
 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 def __call__( self, x_o: Union[Tensor, ndarray], num_simulations: int, eps: Optional[float] = None, quantile: Optional[float] = None, return_distances: bool = False, return_x_accepted: bool = False, lra: bool = False, sass: bool = False, sass_fraction: float = 0.25, sass_expansion_degree: int = 1, ) -> Union[Distribution, Tuple[Distribution, Tensor]]: r"""Run MCABC. Args: x_o: Observed data. num_simulations: Number of simulations to run. eps: Acceptance threshold $\epsilon$ for distance between observed and simulated data. quantile: Upper quantile of smallest distances for which the corresponding parameters are returned, e.g, q=0.01 will return the top 1%. Exactly one of quantile or eps have to be passed. return_distances: Whether to return the distances corresponding to the accepted parameters. return_distances: Whether to return the simulated data corresponding to the accepted parameters. lra: Whether to run linear regression adjustment as in Beaumont et al. 2002 sass: Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012. sass_fraction: Fraction of simulation budget used for the initial sass run. sass_expansion_degree: Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion. Returns: posterior: Empirical distribution based on selected parameters. distances: Tensor of distances of the selected parameters. """ # Exactly one of eps or quantile need to be passed. assert (eps is not None) ^ ( quantile is not None ), "Eps or quantile must be passed, but not both." # Run SASS and change the simulator and x_o accordingly. if sass: num_pilot_simulations = int(sass_fraction * num_simulations) self.logger.info( f"Running SASS with {num_pilot_simulations} pilot samples." ) num_simulations -= num_pilot_simulations pilot_theta = self.prior.sample((num_pilot_simulations,)) pilot_x = self._batched_simulator(pilot_theta) sass_transform = self.get_sass_transform( pilot_theta, pilot_x, sass_expansion_degree ) simulator = lambda theta: sass_transform(self._batched_simulator(theta)) x_o = sass_transform(x_o) else: simulator = self._batched_simulator # Simulate and calculate distances. theta = self.prior.sample((num_simulations,)) x = simulator(theta) # Infer shape of x to test and set x_o. self.x_shape = x[0].unsqueeze(0).shape self.x_o = process_x(x_o, self.x_shape) distances = self.distance(self.x_o, x) # Select based on acceptance threshold epsilon. if eps is not None: is_accepted = distances < eps num_accepted = is_accepted.sum().item() assert num_accepted > 0, f"No parameters accepted, eps={eps} too small" theta_accepted = theta[is_accepted] distances_accepted = distances[is_accepted] x_accepted = x[is_accepted] # Select based on quantile on sorted distances. elif quantile is not None: num_top_samples = int(num_simulations * quantile) sort_idx = torch.argsort(distances) theta_accepted = theta[sort_idx][:num_top_samples] distances_accepted = distances[sort_idx][:num_top_samples] x_accepted = x[sort_idx][:num_top_samples] else: raise ValueError("One of epsilon or quantile has to be passed.") # Maybe adjust theta with LRA. if lra: self.logger.info("Running Linear regression adjustment.") theta_adjusted = self.run_lra( theta_accepted, x_accepted, observation=self.x_o ) else: theta_adjusted = theta_accepted posterior = Empirical(theta_adjusted, log_weights=ones(theta_accepted.shape[0])) if return_distances and return_x_accepted: return posterior, distances_accepted, x_accepted if return_distances: return posterior, distances_accepted if return_x_accepted: return posterior, x_accepted else: return posterior

#### __init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True) special

Monte-Carlo Approximate Bayesian Computation (Rejection ABC) [1].

[1] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W. (1999). Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $heta$ and maps them to simulations, or observations, x, $\mathrm{sim}( heta) o x$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
Source code in sbi/inference/abc/mcabc.py
 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 def __init__( self, simulator: Callable, prior, distance: Union[str, Callable] = "l2", num_workers: int = 1, simulation_batch_size: int = 1, show_progress_bars: bool = True, ): """Monte-Carlo Approximate Bayesian Computation (Rejection ABC) [1]. [1] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W. (1999). Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Molecular biology and evolution, 16(12), 1791-1798. Args: simulator: A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used. prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. distance: Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse. num_workers: Number of parallel workers to use for simulations. simulation_batch_size: Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension). show_progress_bars: Whether to show a progressbar during simulation and sampling. """ super().__init__( simulator=simulator, prior=prior, distance=distance, num_workers=num_workers, simulation_batch_size=simulation_batch_size, show_progress_bars=show_progress_bars, )

#### choose_distance_function(distance_type='l2') inherited

Return distance function for given distance type.

Source code in sbi/inference/abc/mcabc.py
 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 @staticmethod def choose_distance_function(distance_type: str = "l2") -> Callable: """Return distance function for given distance type.""" if distance_type == "mse": distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1) elif distance_type == "l2": distance = lambda xo, x: torch.norm((xo - x), dim=-1) elif distance_type == "l1": distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1) else: raise ValueError(r"Distance {distance_type} not supported.") def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor: """Return distance over batch dimension. Args: observed_data: Observed data, could be 1D. simulated_data: Batch of simulated data, has batch dimension. Returns: Torch tensor with batch of distances. """ assert simulated_data.ndim == 2, "simulated data needs batch dimension" return distance(observed_data, simulated_data) return distance_fun

#### get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Source code in sbi/inference/abc/mcabc.py
 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 @staticmethod def get_sass_transform( theta: torch.Tensor, x: torch.Tensor, expansion_degree: int = 1, sample_weight=None, ) -> Callable: """Return semi-automatic summary statitics function. Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112 Following implementation in https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity and https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic """ expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False) # Transform x, remove intercept. x_expanded = expansion.fit_transform(x) sumstats_map = np.zeros((x_expanded.shape[1], theta.shape[1])) for parameter_idx in range(theta.shape[1]): regression_model = LinearRegression(fit_intercept=True) regression_model.fit( X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight ) sumstats_map[:, parameter_idx] = regression_model.coef_ sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32) def sumstats_transform(x): x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32) return x_expanded.mm(sumstats_map) return sumstats_transform

#### run_lra(theta, x, observation, sample_weight=None) inherited

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/mcabc.py
 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 @staticmethod def run_lra( theta: torch.Tensor, x: torch.Tensor, observation: torch.Tensor, sample_weight=None, ) -> torch.Tensor: """Return parameters adjusted with linear regression adjustment. Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254 """ theta_adjusted = theta for parameter_idx in range(theta.shape[1]): regression_model = LinearRegression(fit_intercept=True) regression_model.fit( X=x, y=theta[:, parameter_idx], sample_weight=sample_weight, ) theta_adjusted[:, parameter_idx] += regression_model.predict( observation.reshape(1, -1) ) theta_adjusted[:, parameter_idx] -= regression_model.predict(x) return theta_adjusted

### sbi.inference.abc.smcabc.SMCABC¶

#### __call__(self, x_o, num_particles, num_initial_pop, num_simulations, epsilon_decay, distance_based_decay=False, ess_min=None, kernel_variance_scale=1.0, use_last_pop_samples=True, return_summary=False, lra=False, lra_with_weights=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1) special

Run SMCABC.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_particles int

Number of particles in each population.

required
num_initial_pop int

Number of simulations used for initial population.

required
num_simulations int

Total number of possible simulations.

required
epsilon_decay float

Factor with which the acceptance threshold $\epsilon$ decays.

required
distance_based_decay bool

Whether the $\epsilon$ decay is constant over populations or calculated from the previous populations distribution of distances.

False
ess_min Optional[float]

Threshold of effective sampling size for resampling weights. Not used when None (default).

None
kernel_variance_scale float

Factor for scaling the perturbation kernel variance.

1.0
use_last_pop_samples bool

Whether to fill up the current population with samples from the previous population when the budget is used up. If False, the current population is discarded and the previous population is returned.

True
return_summary bool

Whether to return a dictionary with all accepted particles, weights, etc. at the end.

False
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
lra_with_weights bool

Whether to run lra as weighted linear regression with SMC weights

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1

Returns:

Type Description
Union[torch.distributions.distribution.Distribution, Tuple[torch.distributions.distribution.Distribution, dict]]

posterior: Empirical posterior distribution defined by the accepted particles and their weights. summary (optional): A dictionary containing particles, weights, epsilons and distances of each population.

Source code in sbi/inference/abc/smcabc.py
 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 def __call__( self, x_o: Union[Tensor, ndarray], num_particles: int, num_initial_pop: int, num_simulations: int, epsilon_decay: float, distance_based_decay: bool = False, ess_min: Optional[float] = None, kernel_variance_scale: float = 1.0, use_last_pop_samples: bool = True, return_summary: bool = False, lra: bool = False, lra_with_weights: bool = False, sass: bool = False, sass_fraction: float = 0.25, sass_expansion_degree: int = 1, ) -> Union[Distribution, Tuple[Distribution, dict]]: r"""Run SMCABC. Args: x_o: Observed data. num_particles: Number of particles in each population. num_initial_pop: Number of simulations used for initial population. num_simulations: Total number of possible simulations. epsilon_decay: Factor with which the acceptance threshold $\epsilon$ decays. distance_based_decay: Whether the $\epsilon$ decay is constant over populations or calculated from the previous populations distribution of distances. ess_min: Threshold of effective sampling size for resampling weights. Not used when None (default). kernel_variance_scale: Factor for scaling the perturbation kernel variance. use_last_pop_samples: Whether to fill up the current population with samples from the previous population when the budget is used up. If False, the current population is discarded and the previous population is returned. return_summary: Whether to return a dictionary with all accepted particles, weights, etc. at the end. lra: Whether to run linear regression adjustment as in Beaumont et al. 2002 lra_with_weights: Whether to run lra as weighted linear regression with SMC weights sass: Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012. sass_fraction: Fraction of simulation budget used for the initial sass run. sass_expansion_degree: Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion. Returns: posterior: Empirical posterior distribution defined by the accepted particles and their weights. summary (optional): A dictionary containing particles, weights, epsilons and distances of each population. """ pop_idx = 0 self.num_simulations = num_simulations # Pilot run for SASS. if sass: num_pilot_simulations = int(sass_fraction * num_simulations) self.logger.info( f"Running SASS with {num_pilot_simulations} pilot samples." ) sass_transform = self.run_sass_set_xo( num_particles, num_pilot_simulations, x_o, lra, sass_expansion_degree ) # Udpate simulator and xo x_o = sass_transform(self.x_o) def sass_simulator(theta): self.simulation_counter += theta.shape[0] return sass_transform(self._batched_simulator(theta)) self._simulate_with_budget = sass_simulator # run initial population particles, epsilon, distances, x = self._set_xo_and_sample_initial_population( x_o, num_particles, num_initial_pop ) log_weights = torch.log(1 / num_particles * ones(num_particles)) self.logger.info( ( f"population={pop_idx}, eps={epsilon}, ess={1.0}, " f"num_sims={num_initial_pop}" ) ) all_particles = [particles] all_log_weights = [log_weights] all_distances = [distances] all_epsilons = [epsilon] all_x = [x] while self.simulation_counter < self.num_simulations: pop_idx += 1 # Decay based on quantile of distances from previous pop. if distance_based_decay: epsilon = self._get_next_epsilon( all_distances[pop_idx - 1], epsilon_decay ) # Constant decay. else: epsilon *= epsilon_decay # Get kernel variance from previous pop. self.kernel_variance = self.get_kernel_variance( all_particles[pop_idx - 1], torch.exp(all_log_weights[pop_idx - 1]), samples_per_dim=500, kernel_variance_scale=kernel_variance_scale, ) particles, log_weights, distances, x = self._sample_next_population( particles=all_particles[pop_idx - 1], log_weights=all_log_weights[pop_idx - 1], distances=all_distances[pop_idx - 1], epsilon=epsilon, x=all_x[pop_idx - 1], use_last_pop_samples=use_last_pop_samples, ) # Resample population if effective sampling size is too small. if ess_min is not None: particles, log_weights = self.resample_if_ess_too_small( particles, log_weights, ess_min, pop_idx ) self.logger.info( ( f"population={pop_idx} done: eps={epsilon:.6f}," f" num_sims={self.simulation_counter}." ) ) # collect results all_particles.append(particles) all_log_weights.append(log_weights) all_distances.append(distances) all_epsilons.append(epsilon) all_x.append(x) # Maybe run LRA and adjust weights. if lra: self.logger.info("Running Linear regression adjustment.") adjusted_particels, adjusted_weights = self.run_lra_update_weights( particles=all_particles[-1], xs=all_x[-1], observation=x_o, log_weights=all_log_weights[-1], lra_with_weights=lra_with_weights, ) posterior = Empirical(adjusted_particels, log_weights=adjusted_weights) else: posterior = Empirical(all_particles[-1], log_weights=all_log_weights[-1]) if return_summary: return ( posterior, dict( particles=all_particles, weights=all_log_weights, epsilons=all_epsilons, distances=all_distances, xs=all_x, ), ) else: return posterior

#### __init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True, kernel='gaussian', algorithm_variant='C') special

Sequential Monte Carlo Approximate Bayesian Computation.

We distinguish between three different SMC methods here: - A: Toni et al. 2010 (Phd Thesis) - B: Sisson et al. 2007 (with correction from 2009) - C: Beaumont et al. 2009

In Toni et al. 2010 we find an overview of the differences on page 34: - B: same as A except for resampling of weights if the effective sampling size is too small. - C: same as A except for calculation of the covariance of the perturbation kernel: the kernel covariance is a scaled version of the covariance of the previous population.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior Distribution

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
kernel Optional[str]

Perturbation kernel.

'gaussian'
algorithm_variant str

Indicating the choice of algorithm variant, A, B, or C.

'C'
Source code in sbi/inference/abc/smcabc.py
 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 def __init__( self, simulator: Callable, prior: Distribution, distance: Union[str, Callable] = "l2", num_workers: int = 1, simulation_batch_size: int = 1, show_progress_bars: bool = True, kernel: Optional[str] = "gaussian", algorithm_variant: str = "C", ): r"""Sequential Monte Carlo Approximate Bayesian Computation. We distinguish between three different SMC methods here: - A: Toni et al. 2010 (Phd Thesis) - B: Sisson et al. 2007 (with correction from 2009) - C: Beaumont et al. 2009 In Toni et al. 2010 we find an overview of the differences on page 34: - B: same as A except for resampling of weights if the effective sampling size is too small. - C: same as A except for calculation of the covariance of the perturbation kernel: the kernel covariance is a scaled version of the covariance of the previous population. Args: simulator: A function that takes parameters $\theta$ and maps them to simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any regular Python callable (i.e. function or class with __call__ method) can be used. prior: A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used. distance: Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse. num_workers: Number of parallel workers to use for simulations. simulation_batch_size: Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension). show_progress_bars: Whether to show a progressbar during simulation and sampling. kernel: Perturbation kernel. algorithm_variant: Indicating the choice of algorithm variant, A, B, or C. """ super().__init__( simulator=simulator, prior=prior, distance=distance, num_workers=num_workers, simulation_batch_size=simulation_batch_size, show_progress_bars=show_progress_bars, ) kernels = ("gaussian", "uniform") assert ( kernel in kernels ), f"Kernel '{kernel}' not supported. Choose one from {kernels}." self.kernel = kernel algorithm_variants = ("A", "B", "C") assert algorithm_variant in algorithm_variants, ( f"SMCABC variant '{algorithm_variant}' not supported, choose one from" " {algorithm_variants}." ) self.algorithm_variant = algorithm_variant self.distance_to_x0 = None self.simulation_counter = 0 self.num_simulations = 0 # Define simulator that keeps track of budget. def simulate_with_budget(theta): self.simulation_counter += theta.shape[0] return self._batched_simulator(theta) self._simulate_with_budget = simulate_with_budget

#### choose_distance_function(distance_type='l2') inherited

Return distance function for given distance type.

Source code in sbi/inference/abc/smcabc.py
 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 @staticmethod def choose_distance_function(distance_type: str = "l2") -> Callable: """Return distance function for given distance type.""" if distance_type == "mse": distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1) elif distance_type == "l2": distance = lambda xo, x: torch.norm((xo - x), dim=-1) elif distance_type == "l1": distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1) else: raise ValueError(r"Distance {distance_type} not supported.") def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor: """Return distance over batch dimension. Args: observed_data: Observed data, could be 1D. simulated_data: Batch of simulated data, has batch dimension. Returns: Torch tensor with batch of distances. """ assert simulated_data.ndim == 2, "simulated data needs batch dimension" return distance(observed_data, simulated_data) return distance_fun

#### get_new_kernel(self, thetas)

Return new kernel distribution for a given set of paramters.

Source code in sbi/inference/abc/smcabc.py
 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 def get_new_kernel(self, thetas: Tensor) -> Distribution: """Return new kernel distribution for a given set of paramters.""" if self.kernel == "gaussian": assert self.kernel_variance.ndim == 2 return MultivariateNormal( loc=thetas, covariance_matrix=self.kernel_variance ) elif self.kernel == "uniform": low = thetas - self.kernel_variance high = thetas + self.kernel_variance # Move batch shape to event shape to get Uniform that is multivariate in # parameter dimension. return Uniform(low=low, high=high).to_event(1) else: raise ValueError(f"Kernel, '{self.kernel}' not supported.")

#### get_particle_ranges(self, particles, weights, samples_per_dim=100)

Return range of particles in each parameter dimension.

Source code in sbi/inference/abc/smcabc.py
 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 def get_particle_ranges( self, particles: Tensor, weights: Tensor, samples_per_dim: int = 100 ) -> Tensor: """Return range of particles in each parameter dimension.""" # get weighted samples samples = self.sample_from_population_with_weights( particles, weights, num_samples=samples_per_dim * particles.shape[1], ) # Variance spans the range of particles for every dimension. particle_ranges = tensor([max(column) - min(column) for column in samples.T]) assert particle_ranges.ndim < 2 return particle_ranges

#### get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Source code in sbi/inference/abc/smcabc.py
 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 @staticmethod def get_sass_transform( theta: torch.Tensor, x: torch.Tensor, expansion_degree: int = 1, sample_weight=None, ) -> Callable: """Return semi-automatic summary statitics function. Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112 Following implementation in https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity and https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic """ expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False) # Transform x, remove intercept. x_expanded = expansion.fit_transform(x) sumstats_map = np.zeros((x_expanded.shape[1], theta.shape[1])) for parameter_idx in range(theta.shape[1]): regression_model = LinearRegression(fit_intercept=True) regression_model.fit( X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight ) sumstats_map[:, parameter_idx] = regression_model.coef_ sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32) def sumstats_transform(x): x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32) return x_expanded.mm(sumstats_map) return sumstats_transform

#### resample_if_ess_too_small(self, particles, log_weights, ess_min, pop_idx)

Return resampled particles and uniform weights if effectice sampling size is too small.

Source code in sbi/inference/abc/smcabc.py
 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 def resample_if_ess_too_small( self, particles: Tensor, log_weights: Tensor, ess_min: float, pop_idx: int, ) -> Tuple[Tensor, Tensor]: """Return resampled particles and uniform weights if effectice sampling size is too small. """ num_particles = particles.shape[0] ess = (1 / torch.sum(torch.exp(2.0 * log_weights), dim=0)) / num_particles # Resampling of weights for low ESS only for Sisson et al. 2007. if ess < ess_min: self.logger.info(f"ESS={ess:.2f} too low, resampling pop {pop_idx}...") # First resample, then set to uniform weights as in Sisson et al. 2007. particles = self.sample_from_population_with_weights( particles, torch.exp(log_weights), num_samples=num_particles ) log_weights = torch.log(1 / num_particles * ones(num_particles)) return particles, log_weights

#### run_lra(theta, x, observation, sample_weight=None) inherited

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/smcabc.py
 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 @staticmethod def run_lra( theta: torch.Tensor, x: torch.Tensor, observation: torch.Tensor, sample_weight=None, ) -> torch.Tensor: """Return parameters adjusted with linear regression adjustment. Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254 """ theta_adjusted = theta for parameter_idx in range(theta.shape[1]): regression_model = LinearRegression(fit_intercept=True) regression_model.fit( X=x, y=theta[:, parameter_idx], sample_weight=sample_weight, ) theta_adjusted[:, parameter_idx] += regression_model.predict( observation.reshape(1, -1) ) theta_adjusted[:, parameter_idx] -= regression_model.predict(x) return theta_adjusted

#### run_lra_update_weights(self, particles, xs, observation, log_weights, lra_with_weights)

Return particles and weights adjusted with LRA.

Runs (weighted) linear regression from xs onto particles to adjust the particles.

Updates the SMC weights according to the new particles.

Source code in sbi/inference/abc/smcabc.py
 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 def run_lra_update_weights( self, particles: Tensor, xs: Tensor, observation: Tensor, log_weights: Tensor, lra_with_weights: bool, ) -> Tuple[Tensor, Tensor]: """Return particles and weights adjusted with LRA. Runs (weighted) linear regression from xs onto particles to adjust the particles. Updates the SMC weights according to the new particles. """ adjusted_particels = self.run_lra( theta=particles, x=xs, observation=observation, sample_weight=log_weights.exp() if lra_with_weights else None, ) # Update SMC weights with LRA adjusted weights adjusted_log_weights = self._calculate_new_log_weights( new_particles=adjusted_particels, old_particles=particles, old_log_weights=log_weights, ) return adjusted_particels, adjusted_log_weights

#### run_sass_set_xo(self, num_particles, num_pilot_simulations, x_o, lra=False, sass_expansion_degree=1)

Return transform for semi-automatic summary statistics.

Runs an single round of rejection abc with fixed budget and accepts num_particles simulations to run the regression for sass.

Sets self.x_o once the x_shape can be derived from simulations.

Source code in sbi/inference/abc/smcabc.py
 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 def run_sass_set_xo( self, num_particles: int, num_pilot_simulations: int, x_o, lra: bool = False, sass_expansion_degree: int = 1, ) -> Callable: """Return transform for semi-automatic summary statistics. Runs an single round of rejection abc with fixed budget and accepts num_particles simulations to run the regression for sass. Sets self.x_o once the x_shape can be derived from simulations. """ (pilot_particles, _, _, pilot_xs,) = self._set_xo_and_sample_initial_population( x_o, num_particles, num_pilot_simulations ) # Adjust with LRA. if lra: pilot_particles = self.run_lra(pilot_particles, pilot_xs, self.x_o) sass_transform = self.get_sass_transform( pilot_particles, pilot_xs, expansion_degree=sass_expansion_degree, sample_weight=None, ) return sass_transform

#### sample_from_population_with_weights(particles, weights, num_samples=1) staticmethod

Return samples from particles sampled with weights.

Source code in sbi/inference/abc/smcabc.py
 466 467 468 469 470 471 472 473 474 475 476 477 478 479 @staticmethod def sample_from_population_with_weights( particles: Tensor, weights: Tensor, num_samples: int = 1 ) -> Tensor: """Return samples from particles sampled with weights.""" # define multinomial with weights as probs multi = Multinomial(probs=weights) # sample num samples, with replacement samples = multi.sample(sample_shape=(num_samples,)) # get indices of success trials indices = torch.where(samples)[1] # return those indices from trace return particles[indices]

## Posteriors¶

### sbi.inference.posteriors.direct_posterior.DirectPosterior¶

Posterior $p(\theta|x)$ with log_prob() and sample() methods, obtained with SNPE.

SNPE trains a neural network to directly approximate the posterior distribution. However, for bounded priors, the neural network can have leakage: it puts non-zero mass in regions where the prior is zero. The DirectPosterior class wraps the trained network to deal with these cases.

Specifically, this class offers the following functionality:
- correct the calculation of the log probability such that it compensates for the leakage.
- reject samples that lie outside of the prior bounds.
- alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

#### default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

#### mcmc_method: str inherited property writable

Returns MCMC method.

#### mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

#### rejection_sampling_parameters: dict property writable

Returns rejection sampling parameter.

#### sample_with_mcmc: bool property writable

Return True if NeuralPosterior instance should use MCMC in .sample().

#### __init__(self, method_family, neural_net, prior, x_shape, rejection_sampling_parameters=None, sample_with_mcmc=True, mcmc_method='slice_np', mcmc_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of a single simulator output.

required
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

None
sample_with_mcmc bool

Whether to sample with MCMC. Will always be True for SRE and SNL, but can also be set to True for SNPE if MCMC is preferred to deal with leakage over rejection sampling.

True
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
device str

Training device, e.g., cpu or cuda:0

'cpu'
Source code in sbi/inference/posteriors/direct_posterior.py
 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 def __init__( self, method_family: str, neural_net: nn.Module, prior, x_shape: torch.Size, rejection_sampling_parameters: Optional[Dict[str, Any]] = None, sample_with_mcmc: bool = True, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, device: str = "cpu", ): """ Args: method_family: One of snpe, snl, snre_a or snre_b. neural_net: A classifier for SNRE, a density estimator for SNPE and SNL. prior: Prior distribution with .log_prob() and .sample(). x_shape: Shape of a single simulator output. rejection_sampling_parameters: Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling. sample_with_mcmc: Whether to sample with MCMC. Will always be True for SRE and SNL, but can also be set to True for SNPE if MCMC is preferred to deal with leakage over rejection sampling. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. device: Training device, e.g., cpu or cuda:0 """ kwargs = del_entries( locals(), entries=( "self", "__class__", "sample_with_mcmc", "rejection_sampling_parameters", ), ) super().__init__(**kwargs) self.set_sample_with_mcmc(sample_with_mcmc) self.set_rejection_sampling_parameters(rejection_sampling_parameters) self._purpose = ( "It allows to .sample() and .log_prob() the posterior and wraps the " "output of the .net to avoid leakage into regions with 0 prior probability." )

#### copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

• Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
• default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
NeuralPosterior

Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/direct_posterior.py
 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 def copy_hyperparameters_from( self, posterior: "NeuralPosterior" ) -> "NeuralPosterior": """ Copies the hyperparameters from a given posterior to self. The hyperparameters that are copied are: - Sampling parameters (MCMC for all methods, rejection sampling for SNPE). - default_x at which to evaluate the posterior. Args: posterior: Posterior that the hyperparameters are copied from. Returns: Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal) """ assert isinstance( posterior, NeuralPosterior ), "copy_state_from must be a NeuralPosterior." self.set_mcmc_method(posterior._mcmc_method) self.set_mcmc_parameters(posterior._mcmc_parameters) self.set_default_x(posterior.default_x) self._mcmc_init_params = posterior._mcmc_init_params if hasattr(self, "_sample_with_mcmc"): self.set_sample_with_mcmc(posterior._sample_with_mcmc) if hasattr(self, "_rejection_sampling_parameters"): self.set_rejection_sampling_parameters( posterior._rejection_sampling_parameters ) return self

#### leakage_correction(self, x, num_rejection_samples=10000, force_update=False, show_progress_bars=False, rejection_sampling_batch_size=10000)

Return leakage correction factor for a leaky posterior density estimate.

The factor is estimated from the acceptance probability during rejection sampling from the posterior.

This is to avoid re-estimating the acceptance probability from scratch whenever log_prob is called and norm_posterior=True. Here, it is estimated only once for self.default_x and saved for later. We re-evaluate only whenever a new x is passed.

Parameters:

Name Type Description Default
x Tensor

Conditioning context for posterior $p(\theta|x)$.

required
num_rejection_samples int

Number of samples used to estimate correction factor.

10000
force_update bool

Whether to force a reevaluation of the leakage correction even if the context x is the same as self.default_x. This is useful to enforce a new leakage estimate for rounds after the first (2, 3,..).

False
show_progress_bars bool

Whether to show a progress bar during sampling.

False
rejection_sampling_batch_size int

Batch size for rejection sampling.

10000

Returns:

Type Description
Tensor

Saved or newly-estimated correction factor (as a scalar Tensor).

Source code in sbi/inference/posteriors/direct_posterior.py
 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 @torch.no_grad() def leakage_correction( self, x: Tensor, num_rejection_samples: int = 10_000, force_update: bool = False, show_progress_bars: bool = False, rejection_sampling_batch_size: int = 10_000, ) -> Tensor: r"""Return leakage correction factor for a leaky posterior density estimate. The factor is estimated from the acceptance probability during rejection sampling from the posterior. This is to avoid re-estimating the acceptance probability from scratch whenever log_prob is called and norm_posterior=True. Here, it is estimated only once for self.default_x and saved for later. We re-evaluate only whenever a new x is passed. Arguments: x: Conditioning context for posterior $p(\theta|x)$. num_rejection_samples: Number of samples used to estimate correction factor. force_update: Whether to force a reevaluation of the leakage correction even if the context x is the same as self.default_x. This is useful to enforce a new leakage estimate for rounds after the first (2, 3,..). show_progress_bars: Whether to show a progress bar during sampling. rejection_sampling_batch_size: Batch size for rejection sampling. Returns: Saved or newly-estimated correction factor (as a scalar Tensor). """ def acceptance_at(x: Tensor) -> Tensor: return utils.sample_posterior_within_prior( self.net, self._prior, x.to(self._device), num_rejection_samples, show_progress_bars, sample_for_correction_factor=True, max_sampling_batch_size=rejection_sampling_batch_size, )[1] # Check if the provided x matches the default x (short-circuit on identity). is_new_x = self.default_x is None or ( x is not self.default_x and (x != self.default_x).any() ) not_saved_at_default_x = self._leakage_density_correction_factor is None if is_new_x: # Calculate at x; don't save. return acceptance_at(x) elif not_saved_at_default_x or force_update: # Calculate at default_x; save. self._leakage_density_correction_factor = acceptance_at(self.default_x) return self._leakage_density_correction_factor # type:ignore

#### log_prob(self, theta, x=None, norm_posterior=True, track_gradients=False, leakage_correction_params=None)

Returns the log-probability of the posterior $p(\theta|x).$

Parameters:

Name Type Description Default
theta Tensor

Parameters $\theta$.

required
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
norm_posterior bool

Whether to enforce a normalized posterior density. Renormalization of the posterior is useful when some probability falls out or leaks out of the prescribed prior support. The normalizing factor is calculated via rejection sampling, so if you need speedier but unnormalized log posterior estimates set here norm_posterior=False. The returned log posterior is set to -∞ outside of the prior support regardless of this setting.

True

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False
leakage_correction_params Optional[dict]

A dict of keyword arguments to override the default values of leakage_correction(). Possible options are: num_rejection_samples, force_update, show_progress_bars, and rejection_sampling_batch_size. These parameters only have an effect if norm_posterior=True.

None

Returns:

Type Description
Tensor

(len(θ),)-shaped log posterior probability $\log p(\theta|x)$ for θ in the support of the prior, -∞ (corresponding to 0 probability) outside.

Source code in sbi/inference/posteriors/direct_posterior.py
 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 def log_prob( self, theta: Tensor, x: Optional[Tensor] = None, norm_posterior: bool = True, track_gradients: bool = False, leakage_correction_params: Optional[dict] = None, ) -> Tensor: r""" Returns the log-probability of the posterior $p(\theta|x).$ Args: theta: Parameters $\theta$. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). norm_posterior: Whether to enforce a normalized posterior density. Renormalization of the posterior is useful when some probability falls out or leaks out of the prescribed prior support. The normalizing factor is calculated via rejection sampling, so if you need speedier but unnormalized log posterior estimates set here norm_posterior=False. The returned log posterior is set to -∞ outside of the prior support regardless of this setting. track_gradients: Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption. leakage_correction_params: A dict of keyword arguments to override the default values of leakage_correction(). Possible options are: num_rejection_samples, force_update, show_progress_bars, and rejection_sampling_batch_size. These parameters only have an effect if norm_posterior=True. Returns: (len(θ),)-shaped log posterior probability $\log p(\theta|x)$ for θ in the support of the prior, -∞ (corresponding to 0 probability) outside. """ # TODO Train exited here, entered after sampling? self.net.eval() theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x) theta_repeated, x_repeated = self._match_theta_and_x_batch_shapes(theta, x) with torch.set_grad_enabled(track_gradients): # Evaluate on device, move back to cpu for comparison with prior. unnorm_log_prob = self.net.log_prob( theta_repeated.to(self._device), x_repeated.to(self._device) ).cpu() # Force probability to be zero outside prior support. in_prior_support = within_support(self._prior, theta) masked_log_prob = torch.where( in_prior_support, unnorm_log_prob, torch.tensor(float("-inf"), dtype=torch.float32), ) if leakage_correction_params is None: leakage_correction_params = dict() # use defaults log_factor = ( log( self.leakage_correction( x=batched_first_of_batch(x), **leakage_correction_params ) ) if norm_posterior else 0 ) return masked_log_prob - log_factor

#### map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=1000, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior $p( heta|x)$. If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

1000
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/direct_posterior.py
 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 def map( self, x: Optional[Tensor] = None, num_iter: int = 1000, learning_rate: float = 1e-2, init_method: Union[str, Tensor] = "posterior", num_init_samples: int = 1_000, num_to_optimize: int = 100, save_best_every: int = 10, show_progress_bars: bool = True, ) -> Tensor: """ Returns the maximum-a-posteriori estimate (MAP). The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_. The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization. Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand. Args: x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). num_iter: Number of optimization steps that the algorithm takes to find the MAP. learning_rate: Learning rate of the optimizer. init_method: How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations. num_init_samples: Draw this number of samples from the posterior and evaluate the log-probability of all of them. num_to_optimize: From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization. save_best_every: The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.) show_progress_bars: Whether or not to show a progressbar for sampling from the posterior. Returns: The MAP estimate. """ return super().map( x=x, num_iter=num_iter, learning_rate=learning_rate, init_method=init_method, num_init_samples=num_init_samples, num_to_optimize=num_to_optimize, save_best_every=save_best_every, show_progress_bars=show_progress_bars, log_prob_kwargs={"norm_posterior": False}, )

#### sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with_mcmc=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from posterior distribution $p(\theta|x)$.

Samples are obtained either with rejection sampling or MCMC. Rejection sampling will be a lot faster if leakage is rather low. If leakage is high (e.g. over 99%, which can happen in multi-round SNPE), MCMC can be faster than rejection sampling.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with_mcmc Optional[bool]

Optional parameter to override self.sample_with_mcmc.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 def sample( self, sample_shape: Shape = torch.Size(), x: Optional[Tensor] = None, show_progress_bars: bool = True, sample_with_mcmc: Optional[bool] = None, mcmc_method: Optional[str] = None, mcmc_parameters: Optional[Dict[str, Any]] = None, rejection_sampling_parameters: Optional[Dict[str, Any]] = None, ) -> Tensor: r""" Return samples from posterior distribution $p(\theta|x)$. Samples are obtained either with rejection sampling or MCMC. Rejection sampling will be a lot faster if leakage is rather low. If leakage is high (e.g. over 99%, which can happen in multi-round SNPE), MCMC can be faster than rejection sampling. Args: sample_shape: Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). show_progress_bars: Whether to show sampling progress monitor. sample_with_mcmc: Optional parameter to override self.sample_with_mcmc. mcmc_method: Optional parameter to override self.mcmc_method. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. rejection_sampling_parameters: Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling. Returns: Samples from posterior. """ x, num_samples, mcmc_method, mcmc_parameters = self._prepare_for_sample( x, sample_shape, mcmc_method, mcmc_parameters ) sample_with_mcmc = ( sample_with_mcmc if sample_with_mcmc is not None else self.sample_with_mcmc ) self.net.eval() if sample_with_mcmc: potential_fn_provider = PotentialFunctionProvider() samples = self._sample_posterior_mcmc( num_samples=num_samples, potential_fn=potential_fn_provider( self._prior, self.net, x, mcmc_method ), init_fn=self._build_mcmc_init_fn( self._prior, potential_fn_provider(self._prior, self.net, x, "slice_np"), **mcmc_parameters, ), mcmc_method=mcmc_method, show_progress_bars=show_progress_bars, **mcmc_parameters, ) else: # Rejection sampling. samples, _ = utils.sample_posterior_within_prior( self.net, self._prior, x, num_samples=num_samples, show_progress_bars=show_progress_bars, **rejection_sampling_parameters if (rejection_sampling_parameters is not None) else self.rejection_sampling_parameters, ) self.net.train(True) return samples.reshape((*sample_shape, -1))

#### sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, show_progress_bars=True, mcmc_method=None, mcmc_parameters=None)

Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 def sample_conditional( self, sample_shape: Shape, condition: Tensor, dims_to_sample: List[int], x: Optional[Tensor] = None, show_progress_bars: bool = True, mcmc_method: Optional[str] = None, mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> Tensor: r""" Return samples from conditional posterior $p(\theta_i|\theta_j, x)$. In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition. Samples are obtained with MCMC. Args: sample_shape: Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape. condition: Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored. dims_to_sample: Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). show_progress_bars: Whether to show sampling progress monitor. mcmc_method: Optional parameter to override self.mcmc_method. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Samples from conditional posterior. """ return super().sample_conditional( PotentialFunctionProvider(), sample_shape, condition, dims_to_sample, x, show_progress_bars, mcmc_method, mcmc_parameters, )

#### set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior $p(theta|x)$.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/direct_posterior.py
 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 def set_default_x(self, x: Tensor) -> "NeuralPosterior": """Set new default x for .sample(), .log_prob to use as conditioning context. This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed. This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it). NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible. Args: x: The default observation to set for the posterior $p(theta|x)$. Returns: NeuralPosterior that will use a default x when not explicitly passed. """ self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x) self._num_iid_trials = self._x.shape[0] return self

#### set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
 147 148 149 150 151 152 153 154 155 156 157 def set_mcmc_method(self, method: str) -> "NeuralPosterior": """Sets sampling method to for MCMC and returns NeuralPosterior. Args: method: Method to use. Returns: NeuralPosterior for chainable calls. """ self._mcmc_method = method return self

#### set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior": """Sets parameters for MCMC and returns NeuralPosterior. Args: parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: NeuralPosterior for chainable calls. """ self._mcmc_parameters = parameters return self

#### set_rejection_sampling_parameters(self, parameters)

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to the set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 def set_rejection_sampling_parameters( self, parameters: Dict[str, Any] ) -> "NeuralPosterior": """Sets parameters for rejection sampling and returns NeuralPosterior. Args: parameters: Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: max_sampling_batch_size to the set the batch size for drawing new samples from the candidate distribution, e.g., the posterior. Larger batch size speeds up sampling. Returns: NeuralPosterior for chainable calls. """ self._rejection_sampling_parameters = parameters return self

#### set_sample_with_mcmc(self, use_mcmc)

Turns MCMC sampling on or off and returns NeuralPosterior.

Parameters:

Name Type Description Default
use_mcmc bool

Flag to set whether or not MCMC sampling is used.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/direct_posterior.py
 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 def set_sample_with_mcmc(self, use_mcmc: bool) -> "NeuralPosterior": """Turns MCMC sampling on or off and returns NeuralPosterior. Args: use_mcmc: Flag to set whether or not MCMC sampling is used. Returns: NeuralPosterior for chainable calls. Raises: ValueError: on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling. """ self._sample_with_mcmc = use_mcmc return self

### sbi.inference.posteriors.likelihood_based_posterior.LikelihoodBasedPosterior¶

Posterior $p(\theta|x)$ with log_prob() and sample() methods, obtained with SNLE.

SNLE trains a neural network to approximate the likelihood $p(x|\theta)$. The SNLE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

#### default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

#### mcmc_method: str inherited property writable

Returns MCMC method.

#### mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

#### __init__(self, method_family, neural_net, prior, x_shape, mcmc_method='slice_np', mcmc_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations.

required
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
device str

Training device, e.g., cpu or cuda:0.

'cpu'
Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 def __init__( self, method_family: str, neural_net: nn.Module, prior, x_shape: torch.Size, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, device: str = "cpu", ): """ Args: method_family: One of snpe, snl, snre_a or snre_b. neural_net: A classifier for SNRE, a density estimator for SNPE and SNL. prior: Prior distribution with .log_prob() and .sample(). x_shape: Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. device: Training device, e.g., cpu or cuda:0. """ kwargs = del_entries(locals(), entries=("self", "__class__")) super().__init__(**kwargs) self._purpose = ( "It provides MCMC to .sample() from the posterior and " "can evaluate the _unnormalized_ posterior density with .log_prob()." )

#### copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

• Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
• default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
NeuralPosterior

Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 def copy_hyperparameters_from( self, posterior: "NeuralPosterior" ) -> "NeuralPosterior": """ Copies the hyperparameters from a given posterior to self. The hyperparameters that are copied are: - Sampling parameters (MCMC for all methods, rejection sampling for SNPE). - default_x at which to evaluate the posterior. Args: posterior: Posterior that the hyperparameters are copied from. Returns: Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal) """ assert isinstance( posterior, NeuralPosterior ), "copy_state_from must be a NeuralPosterior." self.set_mcmc_method(posterior._mcmc_method) self.set_mcmc_parameters(posterior._mcmc_parameters) self.set_default_x(posterior.default_x) self._mcmc_init_params = posterior._mcmc_init_params if hasattr(self, "_sample_with_mcmc"): self.set_sample_with_mcmc(posterior._sample_with_mcmc) if hasattr(self, "_rejection_sampling_parameters"): self.set_rejection_sampling_parameters( posterior._rejection_sampling_parameters ) return self

Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

This corresponds to an unnormalized posterior log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters $\theta$.

required
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 def log_prob( self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False ) -> Tensor: r""" Returns the log-probability of $p(x|\theta) \cdot p(\theta).$ This corresponds to an **unnormalized** posterior log-probability. Args: theta: Parameters $\theta$. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). track_gradients: Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption. Returns: (len(θ),)-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$. """ # TODO Train exited here, entered after sampling? self.net.eval() theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x) # Calculate likelihood over trials and in one batch. warn( "The log probability from SNL is only correct up to a normalizing constant." ) log_likelihood_trial_sum = self._log_likelihoods_over_trials( x=x.to(self._device), theta=theta.to(self._device), net=self.net, track_gradients=track_gradients, ) # Move to cpu for comparison with prior. return log_likelihood_trial_sum.cpu() + self._prior.log_prob(theta)

#### map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior $p( heta|x)$. If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 def map( self, x: Optional[Tensor] = None, num_iter: int = 1000, learning_rate: float = 1e-2, init_method: Union[str, Tensor] = "posterior", num_init_samples: int = 500, num_to_optimize: int = 100, save_best_every: int = 10, show_progress_bars: bool = True, ) -> Tensor: """ Returns the maximum-a-posteriori estimate (MAP). The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_. The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization. Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand. Args: x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). num_iter: Number of optimization steps that the algorithm takes to find the MAP. learning_rate: Learning rate of the optimizer. init_method: How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations. num_init_samples: Draw this number of samples from the posterior and evaluate the log-probability of all of them. num_to_optimize: From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization. save_best_every: The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.) show_progress_bars: Whether or not to show a progressbar for sampling from the posterior. Returns: The MAP estimate. """ return super().map( x=x, num_iter=num_iter, learning_rate=learning_rate, init_method=init_method, num_init_samples=num_init_samples, num_to_optimize=num_to_optimize, save_best_every=save_best_every, show_progress_bars=show_progress_bars, )

#### sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with_mcmc=None, mcmc_method=None, mcmc_parameters=None)

Return samples from posterior distribution $p(\theta|x)$ with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with_mcmc Optional[bool]

Optional parameter to override self.sample_with_mcmc.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 def sample( self, sample_shape: Shape = torch.Size(), x: Optional[Tensor] = None, show_progress_bars: bool = True, sample_with_mcmc: Optional[bool] = None, mcmc_method: Optional[str] = None, mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> Tensor: r""" Return samples from posterior distribution $p(\theta|x)$ with MCMC. Args: sample_shape: Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). show_progress_bars: Whether to show sampling progress monitor. sample_with_mcmc: Optional parameter to override self.sample_with_mcmc. mcmc_method: Optional parameter to override self.mcmc_method. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Samples from posterior. """ x, num_samples, mcmc_method, mcmc_parameters = self._prepare_for_sample( x, sample_shape, mcmc_method, mcmc_parameters ) self.net.eval() potential_fn_provider = PotentialFunctionProvider() samples = self._sample_posterior_mcmc( num_samples=num_samples, potential_fn=potential_fn_provider(self._prior, self.net, x, mcmc_method), init_fn=self._build_mcmc_init_fn( self._prior, potential_fn_provider(self._prior, self.net, x, "slice_np"), **mcmc_parameters, ), mcmc_method=mcmc_method, show_progress_bars=show_progress_bars, **mcmc_parameters, ) self.net.train(True) return samples.reshape((*sample_shape, -1))

#### sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, show_progress_bars=True, mcmc_method=None, mcmc_parameters=None)

Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 def sample_conditional( self, sample_shape: Shape, condition: Tensor, dims_to_sample: List[int], x: Optional[Tensor] = None, show_progress_bars: bool = True, mcmc_method: Optional[str] = None, mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> Tensor: r""" Return samples from conditional posterior $p(\theta_i|\theta_j, x)$. In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition. Samples are obtained with MCMC. Args: sample_shape: Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape. condition: Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored. dims_to_sample: Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). show_progress_bars: Whether to show sampling progress monitor. mcmc_method: Optional parameter to override self.mcmc_method. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Samples from conditional posterior. """ return super().sample_conditional( PotentialFunctionProvider(), sample_shape, condition, dims_to_sample, x, show_progress_bars, mcmc_method, mcmc_parameters, )

#### set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior $p(theta|x)$.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 def set_default_x(self, x: Tensor) -> "NeuralPosterior": """Set new default x for .sample(), .log_prob to use as conditioning context. This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed. This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it). NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible. Args: x: The default observation to set for the posterior $p(theta|x)$. Returns: NeuralPosterior that will use a default x when not explicitly passed. """ self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x) self._num_iid_trials = self._x.shape[0] return self

#### set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 147 148 149 150 151 152 153 154 155 156 157 def set_mcmc_method(self, method: str) -> "NeuralPosterior": """Sets sampling method to for MCMC and returns NeuralPosterior. Args: method: Method to use. Returns: NeuralPosterior for chainable calls. """ self._mcmc_method = method return self

#### set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior": """Sets parameters for MCMC and returns NeuralPosterior. Args: parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: NeuralPosterior for chainable calls. """ self._mcmc_parameters = parameters return self

### sbi.inference.posteriors.ratio_based_posterior.RatioBasedPosterior¶

Posterior $p(\theta|x)$ with log_prob() and sample() methods, obtained with SNRE.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The SNRE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

The neural network itself can be accessed via the .net attribute.

#### default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

#### mcmc_method: str inherited property writable

Returns MCMC method.

#### mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

#### __init__(self, method_family, neural_net, prior, x_shape, mcmc_method='slice_np', mcmc_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of a single simulator output.

required
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
device str

Training device, e.g., cpu or cuda:0.

'cpu'
Source code in sbi/inference/posteriors/ratio_based_posterior.py
 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 def __init__( self, method_family: str, neural_net: nn.Module, prior, x_shape: torch.Size, mcmc_method: str = "slice_np", mcmc_parameters: Optional[Dict[str, Any]] = None, device: str = "cpu", ): """ Args: method_family: One of snpe, snl, snre_a or snre_b. neural_net: A classifier for SNRE, a density estimator for SNPE and SNL. prior: Prior distribution with .log_prob() and .sample(). x_shape: Shape of a single simulator output. mcmc_method: Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. device: Training device, e.g., cpu or cuda:0. """ kwargs = del_entries(locals(), entries=("self", "__class__")) super().__init__(**kwargs)

#### copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

• Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
• default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
NeuralPosterior

Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 def copy_hyperparameters_from( self, posterior: "NeuralPosterior" ) -> "NeuralPosterior": """ Copies the hyperparameters from a given posterior to self. The hyperparameters that are copied are: - Sampling parameters (MCMC for all methods, rejection sampling for SNPE). - default_x at which to evaluate the posterior. Args: posterior: Posterior that the hyperparameters are copied from. Returns: Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable: posterior = infer.build_posterior().copy_hyperparameters_from(proposal) """ assert isinstance( posterior, NeuralPosterior ), "copy_state_from must be a NeuralPosterior." self.set_mcmc_method(posterior._mcmc_method) self.set_mcmc_parameters(posterior._mcmc_parameters) self.set_default_x(posterior.default_x) self._mcmc_init_params = posterior._mcmc_init_params if hasattr(self, "_sample_with_mcmc"): self.set_sample_with_mcmc(posterior._sample_with_mcmc) if hasattr(self, "_rejection_sampling_parameters"): self.set_rejection_sampling_parameters( posterior._rejection_sampling_parameters ) return self

Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

This corresponds to an unnormalized posterior log-probability. Only for single-round SNRE_A / AALR, the returned log-probability will correspond to the normalized log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters $\theta$.

required
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 def log_prob( self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False ) -> Tensor: r"""Returns the log-probability of $p(x|\theta) \cdot p(\theta).$ This corresponds to an **unnormalized** posterior log-probability. Only for single-round SNRE_A / AALR, the returned log-probability will correspond to the **normalized** log-probability. Args: theta: Parameters $\theta$. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). track_gradients: Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption. Returns: (len(θ),)-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$. """ # TODO Train exited here, entered after sampling? self.net.eval() theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x) self._warn_log_prob_snre() # Sum log ratios over x batch of iid trials. log_ratio = self._log_ratios_over_trials( x.to(self._device), theta.to(self._device), self.net, track_gradients=track_gradients, ) return log_ratio.cpu() + self._prior.log_prob(theta)

#### map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior $p( heta|x)$. If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Maximum Number of optimization steps that the algorithm takes to find the MAP.

1000
early_stop_at

If None, it will optimize for max_num_iter iterations. If float, the optimization will stop as soon as the steps taken by the optimizer are smaller than early_stop_at times the standard deviation of the initial guesses.

required
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 def map( self, x: Optional[Tensor] = None, num_iter: int = 1000, learning_rate: float = 1e-2, init_method: Union[str, Tensor] = "posterior", num_init_samples: int = 500, num_to_optimize: int = 100, save_best_every: int = 10, show_progress_bars: bool = True, ) -> Tensor: """ Returns the maximum-a-posteriori estimate (MAP). The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_. The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization. Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand. Args: x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). num_iter: Maximum Number of optimization steps that the algorithm takes to find the MAP. early_stop_at: If None, it will optimize for max_num_iter iterations. If float, the optimization will stop as soon as the steps taken by the optimizer are smaller than early_stop_at times the standard deviation of the initial guesses. learning_rate: Learning rate of the optimizer. init_method: How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations. num_init_samples: Draw this number of samples from the posterior and evaluate the log-probability of all of them. num_to_optimize: From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization. save_best_every: The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.) show_progress_bars: Whether or not to show a progressbar for sampling from the posterior. Returns: The MAP estimate. """ return super().map( x=x, num_iter=num_iter, learning_rate=learning_rate, init_method=init_method, num_init_samples=num_init_samples, num_to_optimize=num_to_optimize, save_best_every=save_best_every, show_progress_bars=show_progress_bars, )

#### sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with_mcmc=None, mcmc_method=None, mcmc_parameters=None)

Return samples from posterior distribution $p(\theta|x)$ with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with_mcmc Optional[bool]

Optional parameter to override self.sample_with_mcmc.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 def sample( self, sample_shape: Shape = torch.Size(), x: Optional[Tensor] = None, show_progress_bars: bool = True, sample_with_mcmc: Optional[bool] = None, mcmc_method: Optional[str] = None, mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> Tensor: r""" Return samples from posterior distribution $p(\theta|x)$ with MCMC. Args: sample_shape: Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). show_progress_bars: Whether to show sampling progress monitor. sample_with_mcmc: Optional parameter to override self.sample_with_mcmc. mcmc_method: Optional parameter to override self.mcmc_method. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Samples from posterior. """ x, num_samples, mcmc_method, mcmc_parameters = self._prepare_for_sample( x, sample_shape, mcmc_method, mcmc_parameters ) self.net.eval() potential_fn_provider = PotentialFunctionProvider() samples = self._sample_posterior_mcmc( num_samples=num_samples, potential_fn=potential_fn_provider(self._prior, self.net, x, mcmc_method), init_fn=self._build_mcmc_init_fn( self._prior, potential_fn_provider(self._prior, self.net, x, "slice_np"), **mcmc_parameters, ), mcmc_method=mcmc_method, show_progress_bars=show_progress_bars, **mcmc_parameters, ) self.net.train(True) return samples.reshape((*sample_shape, -1))

#### sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, show_progress_bars=True, mcmc_method=None, mcmc_parameters=None)

Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 def sample_conditional( self, sample_shape: Shape, condition: Tensor, dims_to_sample: List[int], x: Optional[Tensor] = None, show_progress_bars: bool = True, mcmc_method: Optional[str] = None, mcmc_parameters: Optional[Dict[str, Any]] = None, ) -> Tensor: r""" Return samples from conditional posterior $p(\theta_i|\theta_j, x)$. In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition. Samples are obtained with MCMC. Args: sample_shape: Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape. condition: Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored. dims_to_sample: Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition. x: Conditioning context for posterior $p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). show_progress_bars: Whether to show sampling progress monitor. mcmc_method: Optional parameter to override self.mcmc_method. mcmc_parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: Samples from conditional posterior. """ return super().sample_conditional( PotentialFunctionProvider(), sample_shape, condition, dims_to_sample, x, show_progress_bars, mcmc_method, mcmc_parameters, )

#### set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior $p(theta|x)$.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 def set_default_x(self, x: Tensor) -> "NeuralPosterior": """Set new default x for .sample(), .log_prob to use as conditioning context. This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed. This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it). NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible. Args: x: The default observation to set for the posterior $p(theta|x)$. Returns: NeuralPosterior that will use a default x when not explicitly passed. """ self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x) self._num_iid_trials = self._x.shape[0] return self

#### set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 147 148 149 150 151 152 153 154 155 156 157 def set_mcmc_method(self, method: str) -> "NeuralPosterior": """Sets sampling method to for MCMC and returns NeuralPosterior. Args: method: Method to use. Returns: NeuralPosterior for chainable calls. """ self._mcmc_method = method return self

#### set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior": """Sets parameters for MCMC and returns NeuralPosterior. Args: parameters: Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. Returns: NeuralPosterior for chainable calls. """ self._mcmc_parameters = parameters return self

## Models¶

### sbi.utils.get_nn_models.posterior_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10) ¶

Returns a function that builds a density estimator for learning the posterior.

This function will usually be used for SNPE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters $\theta$ before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs $x$ before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for simulation outputs $x$. This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 def posterior_nn( model: str, z_score_theta: bool = True, z_score_x: bool = True, hidden_features: int = 50, num_transforms: int = 5, num_bins: int = 10, embedding_net: nn.Module = nn.Identity(), num_components: int = 10, ) -> Callable: r""" Returns a function that builds a density estimator for learning the posterior. This function will usually be used for SNPE. The returned function is to be passed to the inference class when using the flexible interface. Args: model: The type of density estimator that will be created. One of [mdn, made, maf, nsf]. z_score_theta: Whether to z-score parameters $\theta$ before passing them into the network. z_score_x: Whether to z-score simulation outputs $x$ before passing them into the network. hidden_features: Number of hidden features. num_transforms: Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made. num_bins: Number of bins used for the splines in nsf. Ignored if density estimator not nsf. embedding_net: Optional embedding network for simulation outputs $x$. This embedding net allows to learn features from potentially high-dimensional simulation outputs. num_components: Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn. """ kwargs = dict( zip( ( "z_score_x", "z_score_y", "hidden_features", "num_transforms", "num_bins", "embedding_net", "num_components", ), ( z_score_theta, z_score_x, hidden_features, num_transforms, num_bins, embedding_net, num_components, ), ) ) if model == "mdn_snpe_a": if num_components != 10: raise ValueError( "You set num_components. For SNPE-A, this has to be done at " "instantiation of the inference object, i.e. " "inference = SNPE_A(..., num_components=20)" ) kwargs.pop("num_components") def build_fn(batch_theta, batch_x, num_components): # Extract the number of components from the kwargs, such that # they are exposed as a kwargs, offering the possibility to later # override this kwarg with functools.partial. This is necessary # in order to make sure that the MDN in SNPE-A only has one # component when running the Algorithm 1 part. return build_mdn( batch_x=batch_theta, batch_y=batch_x, num_components=num_components, **kwargs ) else: def build_fn(batch_theta, batch_x): if model == "mdn": return build_mdn(batch_x=batch_theta, batch_y=batch_x, **kwargs) elif model == "made": return build_made(batch_x=batch_theta, batch_y=batch_x, **kwargs) elif model == "maf": return build_maf(batch_x=batch_theta, batch_y=batch_x, **kwargs) elif model == "nsf": return build_nsf(batch_x=batch_theta, batch_y=batch_x, **kwargs) else: raise NotImplementedError return build_fn

### sbi.utils.get_nn_models.likelihood_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10) ¶

Returns a function that builds a density estimator for learning the likelihood.

This function will usually be used for SNLE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters $\theta$ before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs $x$ before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for parameters $\theta$.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 def likelihood_nn( model: str, z_score_theta: bool = True, z_score_x: bool = True, hidden_features: int = 50, num_transforms: int = 5, num_bins: int = 10, embedding_net: nn.Module = nn.Identity(), num_components: int = 10, ) -> Callable: r""" Returns a function that builds a density estimator for learning the likelihood. This function will usually be used for SNLE. The returned function is to be passed to the inference class when using the flexible interface. Args: model: The type of density estimator that will be created. One of [mdn, made, maf, nsf]. z_score_theta: Whether to z-score parameters $\theta$ before passing them into the network. z_score_x: Whether to z-score simulation outputs $x$ before passing them into the network. hidden_features: Number of hidden features. num_transforms: Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made. num_bins: Number of bins used for the splines in nsf. Ignored if density estimator not nsf. embedding_net: Optional embedding network for parameters $\theta$. num_components: Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn. """ kwargs = dict( zip( ( "z_score_x", "z_score_y", "hidden_features", "num_transforms", "num_bins", "embedding_net", "num_components", ), ( z_score_x, z_score_theta, hidden_features, num_transforms, num_bins, embedding_net, num_components, ), ) ) def build_fn(batch_theta, batch_x): if model == "mdn": return build_mdn(batch_x=batch_x, batch_y=batch_theta, **kwargs) if model == "made": return build_made(batch_x=batch_x, batch_y=batch_theta, **kwargs) if model == "maf": return build_maf(batch_x=batch_x, batch_y=batch_theta, **kwargs) elif model == "nsf": return build_nsf(batch_x=batch_x, batch_y=batch_theta, **kwargs) else: raise NotImplementedError return build_fn

### sbi.utils.get_nn_models.classifier_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, embedding_net_theta=Identity(), embedding_net_x=Identity()) ¶

Returns a function that builds a classifier for learning density ratios.

This function will usually be used for SNRE. The returned function is to be passed to the inference class when using the flexible interface.

Note that in the view of the SNRE classifier we build below, x=theta and y=x.

Parameters:

Name Type Description Default
model str

The type of classifier that will be created. One of [linear, mlp, resnet].

required
z_score_theta bool

Whether to z-score parameters $\theta$ before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs $x$ before passing them into the network.

True
hidden_features int

Number of hidden features.

50
embedding_net_theta Module

Optional embedding network for parameters $\theta$.

Identity()
embedding_net_x Module

Optional embedding network for simulation outputs $x$. This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
Source code in sbi/utils/get_nn_models.py
 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 def classifier_nn( model: str, z_score_theta: bool = True, z_score_x: bool = True, hidden_features: int = 50, embedding_net_theta: nn.Module = nn.Identity(), embedding_net_x: nn.Module = nn.Identity(), ) -> Callable: r""" Returns a function that builds a classifier for learning density ratios. This function will usually be used for SNRE. The returned function is to be passed to the inference class when using the flexible interface. Note that in the view of the SNRE classifier we build below, x=theta and y=x. Args: model: The type of classifier that will be created. One of [linear, mlp, resnet]. z_score_theta: Whether to z-score parameters $\theta$ before passing them into the network. z_score_x: Whether to z-score simulation outputs $x$ before passing them into the network. hidden_features: Number of hidden features. embedding_net_theta: Optional embedding network for parameters $\theta$. embedding_net_x: Optional embedding network for simulation outputs $x$. This embedding net allows to learn features from potentially high-dimensional simulation outputs. """ kwargs = dict( zip( ( "z_score_x", "z_score_y", "hidden_features", "embedding_net_x", "embedding_net_y", ), ( z_score_theta, z_score_x, hidden_features, embedding_net_theta, embedding_net_x, ), ) ) def build_fn(batch_theta, batch_x): if model == "linear": return build_linear_classifier( batch_x=batch_theta, batch_y=batch_x, **kwargs ) if model == "mlp": return build_mlp_classifier(batch_x=batch_theta, batch_y=batch_x, **kwargs) if model == "resnet": return build_resnet_classifier( batch_x=batch_theta, batch_y=batch_x, **kwargs ) else: raise NotImplementedError return build_fn

## Analysis¶

### sbi.analysis.plot.pairplot(samples=None, points=None, limits=None, subset=None, upper='hist', diag='hist', figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs) ¶

Plot samples in a 2D grid showing marginals and pairwise marginals.

Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution that the samples were drawn from. Each upper-diagonal plot can be interpreted as a 2D-marginal of the distribution.

Parameters:

Name Type Description Default
samples Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

Samples used to build the histogram.

None
points Optional[Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]]

List of additional points to scatter.

None
limits Optional[Union[List, torch.Tensor]]

Array containing the plot xlim for each parameter dimension. If None, just use the min and max of the passed samples

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on).

None
upper Optional[str]

Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.

'hist'
diag Optional[str]

Plotting style for diagonal, {hist, cond, None}.

'hist'
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 def pairplot( samples: Union[ List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor ] = None, points: Optional[ Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor] ] = None, limits: Optional[Union[List, torch.Tensor]] = None, subset: List[int] = None, upper: Optional[str] = "hist", diag: Optional[str] = "hist", figsize: Tuple = (10, 10), labels: Optional[List[str]] = None, ticks: Union[List, torch.Tensor] = [], points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"], **kwargs ): """ Plot samples in a 2D grid showing marginals and pairwise marginals. Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution that the samples were drawn from. Each upper-diagonal plot can be interpreted as a 2D-marginal of the distribution. Args: samples: Samples used to build the histogram. points: List of additional points to scatter. limits: Array containing the plot xlim for each parameter dimension. If None, just use the min and max of the passed samples subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on). upper: Plotting style for upper diagonal, {hist, scatter, contour, cond, None}. diag: Plotting style for diagonal, {hist, cond, None}. figsize: Size of the entire figure. labels: List of strings specifying the names of the parameters. ticks: Position of the ticks. points_colors: Colors of the points. **kwargs: Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details. Returns: figure and axis of posterior distribution plot """ return utils_pairplot( samples=samples, points=points, limits=limits, subset=subset, upper=upper, diag=diag, figsize=figsize, labels=labels, ticks=ticks, points_colors=points_colors, warn_about_deprecation=False, **kwargs, )

### sbi.analysis.plot.conditional_pairplot(density, condition, limits, points=None, subset=None, resolution=50, figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs) ¶

Plot conditional distribution given all other parameters.

The conditionals can be interpreted as slices through the density at a location given by condition.

For example: Say we have a 3D density with parameters $\theta_0$, $\theta_1$, $\theta_2$ and a condition $c$ passed by the user in the condition argument. For the plot of $\theta_0$ on the diagonal, this will plot the conditional $p(\theta_0 | \theta_1=c[1], \theta_2=c[2])$. For the upper diagonal of $\theta_1$ and $\theta_2$, it will plot $p(\theta_1, \theta_2 | \theta_0=c[0])$. All other diagonals and upper-diagonals are built in the corresponding way.

Parameters:

Name Type Description Default
density Any

Probability density with a log_prob() method.

required
condition Tensor

Condition that all but the one/two regarded parameters are fixed to. The condition should be of shape (1, dim_theta), i.e. it could e.g. be a sample from the posterior distribution.

required
limits Union[List, torch.Tensor]

Limits in between which each parameter will be evaluated.

required
points Optional[Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]]

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on)

None
resolution int

Resolution of the grid at which we evaluate the pdf.

50
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 def conditional_pairplot( density: Any, condition: torch.Tensor, limits: Union[List, torch.Tensor], points: Optional[ Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor] ] = None, subset: List[int] = None, resolution: int = 50, figsize: Tuple = (10, 10), labels: Optional[List[str]] = None, ticks: Union[List, torch.Tensor] = [], points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"], **kwargs ): r""" Plot conditional distribution given all other parameters. The conditionals can be interpreted as slices through the density at a location given by condition. For example: Say we have a 3D density with parameters $\theta_0$, $\theta_1$, $\theta_2$ and a condition $c$ passed by the user in the condition argument. For the plot of $\theta_0$ on the diagonal, this will plot the conditional $p(\theta_0 | \theta_1=c[1], \theta_2=c[2])$. For the upper diagonal of $\theta_1$ and $\theta_2$, it will plot $p(\theta_1, \theta_2 | \theta_0=c[0])$. All other diagonals and upper-diagonals are built in the corresponding way. Args: density: Probability density with a log_prob() method. condition: Condition that all but the one/two regarded parameters are fixed to. The condition should be of shape (1, dim_theta), i.e. it could e.g. be a sample from the posterior distribution. limits: Limits in between which each parameter will be evaluated. points: Additional points to scatter. subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on) resolution: Resolution of the grid at which we evaluate the pdf. figsize: Size of the entire figure. labels: List of strings specifying the names of the parameters. ticks: Position of the ticks. points_colors: Colors of the points. **kwargs: Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details. Returns: figure and axis of posterior distribution plot """ return utils_conditional_pairplot( density=density, condition=condition, limits=limits, points=points, subset=subset, resolution=resolution, figsize=figsize, labels=labels, ticks=ticks, points_colors=points_colors, warn_about_deprecation=False, **kwargs, )

### sbi.analysis.conditional_density.conditional_corrcoeff(density, limits, condition, subset=None, resolution=50) ¶

Returns the conditional correlation matrix of a distribution.

To compute the conditional distribution, we condition all but two parameters to values from condition, and then compute the Pearson correlation coefficient $\rho$ between the remaining two parameters under the distribution density. We do so for any pair of parameters specified in subset, thus creating a matrix containing conditional correlations between any pair of parameters.

If condition is a batch of conditions, this function computes the conditional correlation matrix for each one of them and returns the mean.

Parameters:

Name Type Description Default
density Any

Probability density function with .log_prob() function.

required
limits Tensor

Limits within which to evaluate the density.

required
condition Tensor

Values to condition the density on. If a batch of conditions is passed, we compute the conditional correlation matrix for each of them and return the average conditional correlation matrix.

required
subset Optional[List[int]]

Evaluate the conditional distribution only on a subset of dimensions. If None this function uses all dimensions.

None
resolution int

Number of grid points on which the conditional distribution is evaluated. A higher value increases the accuracy of the estimated correlation but also increases the computational cost.

50

Returns: Average conditional correlation matrix of shape either (num_dim, num_dim) or (len(subset), len(subset)) if subset was specified.

Source code in sbi/analysis/conditional_density.py
 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 def conditional_corrcoeff( density: Any, limits: Tensor, condition: Tensor, subset: Optional[List[int]] = None, resolution: int = 50, ) -> Tensor: r""" Returns the conditional correlation matrix of a distribution. To compute the conditional distribution, we condition all but two parameters to values from condition, and then compute the Pearson correlation coefficient $\rho$ between the remaining two parameters under the distribution density. We do so for any pair of parameters specified in subset, thus creating a matrix containing conditional correlations between any pair of parameters. If condition is a batch of conditions, this function computes the conditional correlation matrix for each one of them and returns the mean. Args: density: Probability density function with .log_prob() function. limits: Limits within which to evaluate the density. condition: Values to condition the density on. If a batch of conditions is passed, we compute the conditional correlation matrix for each of them and return the average conditional correlation matrix. subset: Evaluate the conditional distribution only on a subset of dimensions. If None this function uses all dimensions. resolution: Number of grid points on which the conditional distribution is evaluated. A higher value increases the accuracy of the estimated correlation but also increases the computational cost. Returns: Average conditional correlation matrix of shape either (num_dim, num_dim) or (len(subset), len(subset)) if subset was specified. """ return utils_conditional_corrcoeff( density=density, limits=limits, condition=condition, subset=subset, resolution=resolution, warn_about_deprecation=False, )