# API Reference¶

## Inference¶

### sbi.inference.base.infer(simulator, prior, method, num_simulations, num_workers=1)¶

Return posterior distribution by running simulation-based inference.

This function provides a simple interface to run sbi. Inference is run for a single round and hence the returned posterior $$p(\theta|x)$$ can be sampled and evaluated for any $$x$$ (i.e. it is amortized).

The scope of this function is limited to the most essential features of sbi. For more flexibility (e.g. multi-round inference, different density estimators) please use the flexible interface described here: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $$\theta$$ and maps them to simulations, or observations, x, $$\mathrm{sim}(\theta)\to x$$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
method str

What inference method to use. Either of SNPE, SNLE or SNRE.

required
num_simulations int

Number of simulation calls. More simulations means a longer runtime, but a better posterior estimate.

required
num_workers int

Number of parallel workers to use for simulations.

1

Returns: Posterior over parameters conditional on observations (amortized).

Source code in sbi/inference/base.py
def infer(
simulator: Callable, prior, method: str, num_simulations: int, num_workers: int = 1
) -> NeuralPosterior:
r"""
Return posterior distribution by running simulation-based inference.

This function provides a simple interface to run sbi. Inference is run for a single
round and hence the returned posterior $p(\theta|x)$ can be sampled and evaluated
for any $x$ (i.e. it is amortized).

The scope of this function is limited to the most essential features of sbi. For
more flexibility (e.g. multi-round inference, different density estimators) please
use the flexible interface described here:
https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Args:
simulator: A function that takes parameters $\theta$ and maps them to
simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any
regular Python callable (i.e. function or class with __call__ method)
can be used.
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
method: What inference method to use. Either of SNPE, SNLE or SNRE.
num_simulations: Number of simulation calls. More simulations means a longer
runtime, but a better posterior estimate.
num_workers: Number of parallel workers to use for simulations.

Returns: Posterior over parameters conditional on observations (amortized).
"""

try:
method_fun: Callable = getattr(sbi.inference, method.upper())
except AttributeError:
raise NameError(
"Method not available. method must be one of 'SNPE', 'SNLE', 'SNRE'."
)

simulator, prior = prepare_for_sbi(simulator, prior)

inference = method_fun(prior)
theta, x = simulate_for_sbi(
simulator=simulator,
proposal=prior,
num_simulations=num_simulations,
num_workers=num_workers,
)
_ = inference.append_simulations(theta, x).train()
posterior = inference.build_posterior()

return posterior


### sbi.utils.user_input_checks.prepare_for_sbi(simulator, prior)¶

Prepare simulator, prior and for usage in sbi.

One of the goals is to allow you to use sbi with inputs computed in numpy.

Attempts to meet the following requirements by reshaping and type-casting:

• the simulator function receives as input and returns a Tensor.
• the simulator can simulate batches of parameters and return batches of data.
• the prior does not produce batches and samples and evaluates to Tensor.
• the output shape is a torch.Size((1,N)) (i.e, has a leading batch dimension 1).

If this is not possible, a suitable exception will be raised.

Parameters:

Name Type Description Default
simulator Callable

Simulator as provided by the user.

required
prior

Prior as provided by the user.

required

Returns:

Type Description
Tuple[Callable, torch.distributions.distribution.Distribution]

Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.

Source code in sbi/utils/user_input_checks.py
def prepare_for_sbi(simulator: Callable, prior) -> Tuple[Callable, Distribution]:
"""Prepare simulator, prior and for usage in sbi.

One of the goals is to allow you to use sbi with inputs computed in numpy.

Attempts to meet the following requirements by reshaping and type-casting:

- the simulator function receives as input and returns a Tensor.<br/>
- the simulator can simulate batches of parameters and return batches of data.<br/>
- the prior does not produce batches and samples and evaluates to Tensor.<br/>
- the output shape is a torch.Size((1,N)) (i.e, has a leading batch dimension 1).

If this is not possible, a suitable exception will be raised.

Args:
simulator: Simulator as provided by the user.
prior: Prior as provided by the user.

Returns:
Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.
"""

# Check prior, return PyTorch prior.
prior, _, prior_returns_numpy = process_prior(prior)

# Check simulator, returns PyTorch simulator able to simulate batches.
simulator = process_simulator(simulator, prior, prior_returns_numpy)

# Consistency check after making ready for sbi.
check_sbi_inputs(simulator, prior)

return simulator, prior


### sbi.inference.base.simulate_for_sbi(simulator, proposal, num_simulations, num_workers=1, simulation_batch_size=1, show_progress_bar=True)¶

Returns ($$\theta, x$$) pairs obtained from sampling the proposal and simulating.

This function performs two steps:

• Sample parameters $$\theta$$ from the proposal.
• Simulate these parameters to obtain $$x$$.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $$\theta$$ and maps them to simulations, or observations, x, $$\text{sim}(\theta)\to x$$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
proposal Any

Probability distribution that the parameters $$\theta$$ are sampled from.

required
num_simulations int

Number of simulations that are run.

required
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bar bool

Whether to show a progress bar for simulating. This will not affect whether there will be a progressbar while drawing samples from the proposal.

True

Returns: Sampled parameters $$\theta$$ and simulation-outputs $$x$$.

Source code in sbi/inference/base.py
def simulate_for_sbi(
simulator: Callable,
proposal: Any,
num_simulations: int,
num_workers: int = 1,
simulation_batch_size: int = 1,
show_progress_bar: bool = True,
) -> Tuple[Tensor, Tensor]:
r"""
Returns ($\theta, x$) pairs obtained from sampling the proposal and simulating.

This function performs two steps:

- Sample parameters $\theta$ from the proposal.
- Simulate these parameters to obtain $x$.

Args:
simulator: A function that takes parameters $\theta$ and maps them to
simulations, or observations, x, $\text{sim}(\theta)\to x$. Any
regular Python callable (i.e. function or class with __call__ method)
can be used.
proposal: Probability distribution that the parameters $\theta$ are sampled
from.
num_simulations: Number of simulations that are run.
num_workers: Number of parallel workers to use for simulations.
simulation_batch_size: Number of parameter sets that the simulator
maps to data x at once. If None, we simulate all parameter sets at the
same time. If >= 1, the simulator has to process data of shape
(simulation_batch_size, parameter_dimension).
show_progress_bar: Whether to show a progress bar for simulating. This will not
affect whether there will be a progressbar while drawing samples from the
proposal.

Returns: Sampled parameters $\theta$ and simulation-outputs $x$.
"""

check_if_proposal_has_default_x(proposal)

theta = proposal.sample((num_simulations,))

x = simulate_in_batches(
simulator, theta, simulation_batch_size, num_workers, show_progress_bar
)

return theta, x


###  sbi.inference.snpe.snpe_c.SNPE_C (PosteriorEstimator) ¶

#### __init__(self, prior=None, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special ¶

SNPE-C / APT .

 Automatic Posterior Transformation for Likelihood-free Inference, Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

This class implements two loss variants of SNPE-C: the non-atomic and the atomic version. The atomic loss of SNPE-C can be used for any density estimator, i.e. also for normalizing flows. However, it suffers from leakage issues. On the other hand, the non-atomic loss can only be used only if the proposal distribution is a mixture of Gaussians, the density estimator is a mixture of Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from leakage issues. At the beginning of each round, we print whether the non-atomic or the atomic version is used.

In this codebase, we will automatically switch to the non-atomic loss if the following criteria are fulfilled:
- proposal is a DirectPosterior with density_estimator mdn, as built with utils.sbi.posterior_nn().
- the density estimator is a mdn, as built with utils.sbi.posterior_nn().
- isinstance(prior, MultivariateNormal) (from torch.distributions) or isinstance(prior, sbi.utils.BoxUniform)

Note that custom implementations of any of these densities (or estimators) will not trigger the non-atomic loss, and the algorithm will fall back onto using the atomic loss.

Parameters:

Name Type Description Default
prior Optional[Any]

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

None
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during training.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snpe/snpe_c.py
def __init__(
self,
prior: Optional[Any] = None,
density_estimator: Union[str, Callable] = "maf",
device: str = "cpu",
logging_level: Union[int, str] = "WARNING",
summary_writer: Optional[TensorboardSummaryWriter] = None,
show_progress_bars: bool = True,
**unused_args,
):
r"""SNPE-C / APT .

 _Automatic Posterior Transformation for Likelihood-free Inference_,
Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

This class implements two loss variants of SNPE-C: the non-atomic and the atomic
version. The atomic loss of SNPE-C can be used for any density estimator,
i.e. also for normalizing flows. However, it suffers from leakage issues. On
the other hand, the non-atomic loss can only be used only if the proposal
distribution is a mixture of Gaussians, the density estimator is a mixture of
Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from
leakage issues. At the beginning of each round, we print whether the non-atomic
or the atomic version is used.

In this codebase, we will automatically switch to the non-atomic loss if the
following criteria are fulfilled:<br/>
- proposal is a DirectPosterior with density_estimator mdn, as built
with utils.sbi.posterior_nn().<br/>
- the density estimator is a mdn, as built with
utils.sbi.posterior_nn().<br/>
- isinstance(prior, MultivariateNormal) (from torch.distributions) or
isinstance(prior, sbi.utils.BoxUniform)

Note that custom implementations of any of these densities (or estimators) will
not trigger the non-atomic loss, and the algorithm will fall back onto using
the atomic loss.

Args:
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
density_estimator: If it is a string, use a pre-configured network of the
provided type (one of nsf, maf, mdn, made). Alternatively, a function
that builds a custom neural network can be provided. The function will
be called with the first batch of simulations (theta, x), which can
thus be used for shape inference and potentially for z-scoring. It
needs to return a PyTorch nn.Module implementing the density
estimator. The density estimator needs to provide the methods
.log_prob and .sample().
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
logging_level: Minimum severity of messages to log. One of the strings
INFO, WARNING, DEBUG, ERROR and CRITICAL.
summary_writer: A tensorboard SummaryWriter to control, among others, log
file location (default is <current working directory>/logs.)
show_progress_bars: Whether to show a progressbar during training.
unused_args: Absorbs additional arguments. No entries will be used. If it
is not empty, we warn. In future versions, when the new interface of
0.14.0 is more mature, we will remove this argument.
"""

kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
super().__init__(**kwargs, **unused_args)


#### append_simulations(self, theta, x, proposal=None) inherited ¶

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $$\theta$$, $$x$$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
proposal Optional[Any]

The distribution that the parameters $$\theta$$ were sampled from. Pass None if the parameters were sampled from the prior. If not None, it will trigger a different loss-function.

None

Returns:

Type Description
PosteriorEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snpe/snpe_c.py
def append_simulations(
self, theta: Tensor, x: Tensor, proposal: Optional[Any] = None
) -> "PosteriorEstimator":
r"""
Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
prior or not) and an index indicating which round the batch of simulations came
from.

Args:
theta: Parameter sets.
x: Simulation outputs.
proposal: The distribution that the parameters $\theta$ were sampled from.
Pass None if the parameters were sampled from the prior. If not
None, it will trigger a different loss-function.

Returns:
NeuralInference object (returned so that this function is chainable).
"""

theta, x = validate_theta_and_x(theta, x, training_device=self._device)
self._check_proposal(proposal)

if (
proposal is None
or proposal is self._prior
or (
isinstance(proposal, RestrictedPrior) and proposal._prior is self._prior
)
):
# The _data_round_index will later be used to infer if one should train
# with MLE loss or with atomic loss (see, in train():
# self._round = max(self._data_round_index))
self._data_round_index.append(0)
else:
if not self._data_round_index:
# This catches a pretty specific case: if, in the first round, one
# passes data that does not come from the prior.
self._data_round_index.append(1)
else:
self._data_round_index.append(max(self._data_round_index) + 1)

self._theta_roundwise.append(theta)
self._x_roundwise.append(x)
self._proposal_roundwise.append(proposal)

if self._prior is None or isinstance(self._prior, ImproperEmpirical):
if proposal is not None:
raise ValueError(
"You had not passed a prior at initialization, but now you "
"passed a proposal. If you want to run multi-round SNPE, you have "
"to specify a prior (set the .prior argument or re-initialize "
"the object with a prior distribution). If the samples you passed "
"to append_simulations() were sampled from the prior, you can "
"run single-round inference with "
"append_simulations(..., proposal=None)."
)
theta_prior = self.get_simulations()
self._prior = ImproperEmpirical(theta_prior, ones(theta_prior.shape))

return self


#### build_posterior(self, density_estimator=None, sample_with='rejection', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, sample_with_mcmc=None) inherited ¶

Build posterior from the neural density estimator.

For SNPE, the posterior distribution that is returned here implements the following functionality over the raw neural density estimator:

• correct the calculation of the log probability such that it compensates for the leakage.
• reject samples that lie outside of the prior bounds.
• alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [rejection | mcmc]. With default parameters, rejection samples from the posterior estimated by the neural net and rejects only if the samples are outside of the prior support.

'rejection'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the trained neural net). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
sample_with_mcmc Optional[bool]

Deprecated since sbi v0.16.0. Use sample_with=mcmc instead.

None

Returns:

Type Description
DirectPosterior

Posterior $$p(\theta|x)$$ with .sample() and .log_prob() methods.

Source code in sbi/inference/snpe/snpe_c.py
def build_posterior(
self,
density_estimator: Optional[TorchModule] = None,
sample_with: str = "rejection",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
sample_with_mcmc: Optional[bool] = None,
) -> DirectPosterior:
r"""
Build posterior from the neural density estimator.

For SNPE, the posterior distribution that is returned here implements the
following functionality over the raw neural density estimator:

- correct the calculation of the log probability such that it compensates for
the leakage.
- reject samples that lie outside of the prior bounds.
- alternatively, if leakage is very high (which can happen for multi-round
SNPE), sample from the posterior with MCMC.

Args:
density_estimator: The density estimator that the posterior is based on.
If None, use the latest neural density estimator that was trained.
sample_with: Method to use for sampling from the posterior. Must be one of
[rejection | mcmc]. With default parameters, rejection samples
from the posterior estimated by the neural net and rejects only if the
samples are outside of the prior support.
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior will
draw init locations from prior, whereas sir will use
Sequential-Importance-Resampling using init_strategy_num_candidates
to find init locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the trained
neural net). max_sampling_batch_size as the batchsize of samples
being drawn from the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio. m as multiplier to that ratio.
sample_with_mcmc: Deprecated since sbi v0.16.0. Use sample_with=mcmc

Returns:
Posterior $p(\theta|x)$  with .sample() and .log_prob() methods.
"""
if sample_with_mcmc is not None:
warn(
f"You set sample_with_mcmc={sample_with_mcmc}. This is deprecated "
"since sbi v0.17.0 and will lead to an error in future versions. "
"Please use sample_with='mcmc' instead."
)
if sample_with_mcmc:
sample_with = "mcmc"

if density_estimator is None:
density_estimator = self._neural_net
# If internal net is used device is defined.
device = self._device
else:
# Otherwise, infer it from the device of the net parameters.
device = next(density_estimator.parameters()).device.type

self._posterior = DirectPosterior(
method_family="snpe",
neural_net=density_estimator,
prior=self._prior,
x_shape=self._x_shape,
sample_with=sample_with,
mcmc_method=mcmc_method,
mcmc_parameters=mcmc_parameters,
rejection_sampling_parameters=rejection_sampling_parameters,
device=device,
)

self._posterior._num_trained_rounds = self._round + 1

# Store models at end of each round.
self._model_bank.append(deepcopy(self._posterior))
self._model_bank[-1].net.eval()

return deepcopy(self._posterior)


#### get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited ¶

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snpe/snpe_c.py
def get_dataloaders(
self,
dataset: data.TensorDataset,
training_batch_size: int = 50,
validation_fraction: float = 0.1,
resume_training: bool = False,
"""Return dataloaders for training and validation.

Args:
dataset: holding all theta and x, optionally masks.
training_batch_size: training arg of inference methods.
resume_training: Whether the current call is resuming training so that no
new training and validation indices into the dataset have to be created.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Tuple of dataloaders for training and validation.

"""

# Get total number of training examples.
num_examples = len(dataset)

# Select random train and validation splits from (theta, x) pairs.
num_training_examples = int((1 - validation_fraction) * num_examples)
num_validation_examples = num_examples - num_training_examples

if not resume_training:
permuted_indices = torch.randperm(num_examples)
self.train_indices, self.val_indices = (
permuted_indices[:num_training_examples],
permuted_indices[num_training_examples:],
)

# Create training and validation loaders using a subset sampler.
# Intentionally use dicts to define the default dataloader args
# Then, use dataloader_kwargs to override (or add to) any of these defaults
# https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
"batch_size": min(training_batch_size, num_training_examples),
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.train_indices),
}
)
"batch_size": min(training_batch_size, num_validation_examples),
"shuffle": False,
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.val_indices),
}
)



#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited ¶

Returns all $$\theta$$, $$x$$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snpe/snpe_c.py
def get_simulations(
self,
starting_round: int = 0,
exclude_invalid_x: bool = True,
warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
r"""
Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Args:
starting_round: The earliest round to return samples from (we start counting
from zero).
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training.
warn_on_invalid: Whether to give out a warning if invalid simulations were
found.

Returns: Parameters, simulation outputs, prior masks.
"""

theta = get_simulations_since_round(
self._theta_roundwise, self._data_round_index, starting_round
)
x = get_simulations_since_round(
self._x_roundwise, self._data_round_index, starting_round
)
)

# Check for NaNs in simulations.
is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
# Check for problematic z-scoring
warn_if_zscoring_changes_data(x)
if warn_on_invalid:
warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
warn_on_invalid_x_for_snpec_leakage(
num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
)



#### provide_presimulated(self, theta, x, from_round=0) inherited ¶

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $$\theta$$ and $$x$$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snpe/snpe_c.py
def provide_presimulated(
self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
r"""
Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult
release notes to see how you can update your code:
https://github.com/mackelab/sbi/releases/tag/v0.14.0
More information can be found under the corresponding pull request on github:
https://github.com/mackelab/sbi/pull/378
and tutorials:
https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Args:
theta: Parameter sets used to generate presimulated data.
x: Simulation outputs of presimulated data.
from_round: Which round the data was simulated from. from_round=0 means
that the data came from the first round, i.e. the prior.
"""
raise NameError(
f"Deprecated since sbi 0.14.0. "
f"Instead of using this, please use .append_simulations(). Please "
f"consult release notes to see how you can update your code: "
f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
f"More information can be found under the corresponding pull request on "
f"github: "
f"https://github.com/mackelab/sbi/pull/378"
f"and tutorials: "
f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
)


#### train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, calibration_kernel=None, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, use_combined_loss=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)¶

Return density estimator that approximates the distribution $$p(\theta|x)$$.

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
calibration_kernel Optional[Callable]

A function to calibrate the loss with respect to the simulations x. See Lueckmann, Gonçalves et al., NeurIPS 2017.

None
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
use_combined_loss bool

Whether to train the neural net also on prior samples using maximum likelihood in addition to training it on all samples using atomic loss. The extra MLE loss helps prevent density leaking with bounded priors.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
DirectPosterior

Density estimator that approximates the distribution $$p(\theta|x)$$.

Source code in sbi/inference/snpe/snpe_c.py
def train(
self,
num_atoms: int = 10,
training_batch_size: int = 50,
learning_rate: float = 5e-4,
validation_fraction: float = 0.1,
stop_after_epochs: int = 20,
max_num_epochs: Optional[int] = None,
clip_max_norm: Optional[float] = 5.0,
calibration_kernel: Optional[Callable] = None,
exclude_invalid_x: bool = True,
resume_training: bool = False,
use_combined_loss: bool = False,
retrain_from_scratch_each_round: bool = False,
show_train_summary: bool = False,
) -> DirectPosterior:
r"""
Return density estimator that approximates the distribution $p(\theta|x)$.

Args:
num_atoms: Number of atoms to use for classification.
training_batch_size: Training batch size.
learning_rate: Learning rate for Adam optimizer.
validation_fraction: The fraction of data to use for validation.
stop_after_epochs: The number of epochs to wait for improvement on the
validation set before terminating training.
max_num_epochs: Maximum number of epochs to run. If reached, we stop
training even when the validation loss is still decreasing. If None, we
train until validation loss increases (see also stop_after_epochs).
clip_max_norm: Value at which to clip the total gradient norm in order to
prevent exploding gradients. Use None for no clipping.
calibration_kernel: A function to calibrate the loss with respect to the
simulations x. See Lueckmann, Gonçalves et al., NeurIPS 2017.
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training. Expect errors, silent or explicit, when False.
resume_training: Can be used in case training time is limited, e.g. on a
cluster. If True, the split between train and validation set, the
optimizer, the number of epochs, and the best validation log-prob will
be restored from the last time .train() was called.
from the prior. Training may be sped up by ignoring such less targeted
samples.
use_combined_loss: Whether to train the neural net also on prior samples
using maximum likelihood in addition to training it on all samples using
atomic loss. The extra MLE loss helps prevent density leaking with
bounded priors.
retrain_from_scratch_each_round: Whether to retrain the conditional density
estimator for the posterior from scratch each round.
show_train_summary: Whether to print the number of epochs and validation
loss and leakage after the training.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Density estimator that approximates the distribution $p(\theta|x)$.
"""

# WARNING: sneaky trick ahead. We proxy the parent's train here,
# requiring the signature to have num_atoms, save it for use below, and
# continue. It's sneaky because we are using the object (self) as a namespace
# to pass arguments between functions, and that's implicit state management.
self._num_atoms = num_atoms
self._use_combined_loss = use_combined_loss
kwargs = del_entries(
locals(), entries=("self", "__class__", "num_atoms", "use_combined_loss")
)

self._round = max(self._data_round_index)

if self._round > 0:
# Set the proposal to the last proposal that was passed by the user. For
# atomic SNPE, it does not matter what the proposal is. For non-atomic
# SNPE, we only use the latest data that was passed, i.e. the one from the
# last proposal.
proposal = self._proposal_roundwise[-1]
if hasattr(proposal, "net"):
self.use_non_atomic_loss = (
isinstance(proposal.net._distribution, mdn)
and isinstance(self._neural_net._distribution, mdn)
and check_dist_class(
self._prior, class_to_check=(Uniform, MultivariateNormal)
)
)
else:
self.use_non_atomic_loss = False

algorithm = "non-atomic" if self.use_non_atomic_loss else "atomic"
print(f"Using SNPE-C with {algorithm} loss")

if self.use_non_atomic_loss:
# Take care of z-scoring, pre-compute and store prior terms.
self._set_state_for_mog_proposal()

return super().train(**kwargs)


###  sbi.inference.snle.snle_a.SNLE_A (LikelihoodEstimator) ¶

#### __init__(self, prior, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special ¶

Sequential Neural Likelihood .

 Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows_, Papamakarios et al., AISTATS 2019, https://arxiv.org/abs/1805.07226

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snle/snle_a.py
def __init__(
self,
prior,
density_estimator: Union[str, Callable] = "maf",
device: str = "cpu",
logging_level: Union[int, str] = "WARNING",
summary_writer: Optional[TensorboardSummaryWriter] = None,
show_progress_bars: bool = True,
**unused_args,
):
r"""Sequential Neural Likelihood .

 Sequential Neural Likelihood: Fast Likelihood-free Inference with
Autoregressive Flows_, Papamakarios et al., AISTATS 2019,
https://arxiv.org/abs/1805.07226

Args:
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
density_estimator: If it is a string, use a pre-configured network of the
provided type (one of nsf, maf, mdn, made). Alternatively, a function
that builds a custom neural network can be provided. The function will
be called with the first batch of simulations (theta, x), which can
thus be used for shape inference and potentially for z-scoring. It
needs to return a PyTorch nn.Module implementing the density
estimator. The density estimator needs to provide the methods
.log_prob and .sample().
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
logging_level: Minimum severity of messages to log. One of the strings
INFO, WARNING, DEBUG, ERROR and CRITICAL.
summary_writer: A tensorboard SummaryWriter to control, among others, log
file location (default is <current working directory>/logs.)
show_progress_bars: Whether to show a progressbar during simulation and
sampling.
unused_args: Absorbs additional arguments. No entries will be used. If it
is not empty, we warn. In future versions, when the new interface of
0.14.0 is more mature, we will remove this argument.
"""

kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
super().__init__(**kwargs, **unused_args)


#### append_simulations(self, theta, x, from_round=0) inherited ¶

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $$\theta$$, $$x$$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNLE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
LikelihoodEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snle/snle_a.py
def append_simulations(
self,
theta: Tensor,
x: Tensor,
from_round: int = 0,
) -> "LikelihoodEstimator":
r"""
Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
prior or not) and an index indicating which round the batch of simulations came
from.

Args:
theta: Parameter sets.
x: Simulation outputs.
from_round: Which round the data stemmed from. Round 0 means from the prior.
With default settings, this is not used at all for SNLE. Only when
the user later on requests .train(discard_prior_samples=True), we
use these indices to find which training data stemmed from the prior.

Returns:
NeuralInference object (returned so that this function is chainable).
"""

theta, x = validate_theta_and_x(theta, x, training_device=self._device)

self._theta_roundwise.append(theta)
self._x_roundwise.append(x)
self._data_round_index.append(int(from_round))

return self


#### build_posterior(self, density_estimator=None, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None) inherited ¶

Build posterior from the neural density estimator.

SNLE trains a neural network to approximate the likelihood $$p(x|\theta)$$. The LikelihoodBasedPosterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$ and draw samples from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
LikelihoodBasedPosterior

Posterior $$p(\theta|x)$$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snle/snle_a.py
def build_posterior(
self,
density_estimator: Optional[TorchModule] = None,
sample_with: str = "mcmc",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> LikelihoodBasedPosterior:
r"""
Build posterior from the neural density estimator.

SNLE trains a neural network to approximate the likelihood $p(x|\theta)$. The
LikelihoodBasedPosterior class wraps the trained network such that one can
directly evaluate the unnormalized posterior log probability
$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the
posterior with MCMC.

Args:
density_estimator: The density estimator that the posterior is based on.
If None, use the latest neural density estimator that was trained.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior will
draw init locations from prior, whereas sir will use
Sequential-Importance-Resampling using init_strategy_num_candidates
to find init locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration. num_samples_to_find_max as the
number of samples that are used to find the maximum of the
potential_fn / proposal ratio. num_iter_to_find_max as the number
of gradient ascent iterations to find the maximum of that ratio. m as
multiplier to that ratio.

Returns:
Posterior $p(\theta|x)$  with .sample() and .log_prob() methods
(the returned log-probability is unnormalized).
"""

if density_estimator is None:
density_estimator = self._neural_net
# If internal net is used device is defined.
device = self._device
else:
# Otherwise, infer it from the device of the net parameters.
device = next(density_estimator.parameters()).device.type

self._posterior = LikelihoodBasedPosterior(
method_family="snle",
neural_net=density_estimator,
prior=self._prior,
x_shape=self._x_shape,
sample_with=sample_with,
mcmc_method=mcmc_method,
mcmc_parameters=mcmc_parameters,
rejection_sampling_parameters=rejection_sampling_parameters,
device=device,
)

self._posterior._num_trained_rounds = self._round + 1

# Store models at end of each round.
self._model_bank.append(deepcopy(self._posterior))
self._model_bank[-1].net.eval()

return deepcopy(self._posterior)


#### get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited ¶

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snle/snle_a.py
def get_dataloaders(
self,
dataset: data.TensorDataset,
training_batch_size: int = 50,
validation_fraction: float = 0.1,
resume_training: bool = False,
"""Return dataloaders for training and validation.

Args:
dataset: holding all theta and x, optionally masks.
training_batch_size: training arg of inference methods.
resume_training: Whether the current call is resuming training so that no
new training and validation indices into the dataset have to be created.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Tuple of dataloaders for training and validation.

"""

# Get total number of training examples.
num_examples = len(dataset)

# Select random train and validation splits from (theta, x) pairs.
num_training_examples = int((1 - validation_fraction) * num_examples)
num_validation_examples = num_examples - num_training_examples

if not resume_training:
permuted_indices = torch.randperm(num_examples)
self.train_indices, self.val_indices = (
permuted_indices[:num_training_examples],
permuted_indices[num_training_examples:],
)

# Create training and validation loaders using a subset sampler.
# Intentionally use dicts to define the default dataloader args
# Then, use dataloader_kwargs to override (or add to) any of these defaults
# https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
"batch_size": min(training_batch_size, num_training_examples),
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.train_indices),
}
)
"batch_size": min(training_batch_size, num_validation_examples),
"shuffle": False,
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.val_indices),
}
)



#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited ¶

Returns all $$\theta$$, $$x$$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snle/snle_a.py
def get_simulations(
self,
starting_round: int = 0,
exclude_invalid_x: bool = True,
warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
r"""
Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Args:
starting_round: The earliest round to return samples from (we start counting
from zero).
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training.
warn_on_invalid: Whether to give out a warning if invalid simulations were
found.

Returns: Parameters, simulation outputs, prior masks.
"""

theta = get_simulations_since_round(
self._theta_roundwise, self._data_round_index, starting_round
)
x = get_simulations_since_round(
self._x_roundwise, self._data_round_index, starting_round
)
)

# Check for NaNs in simulations.
is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
# Check for problematic z-scoring
warn_if_zscoring_changes_data(x)
if warn_on_invalid:
warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
warn_on_invalid_x_for_snpec_leakage(
num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
)



#### provide_presimulated(self, theta, x, from_round=0) inherited ¶

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $$\theta$$ and $$x$$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snle/snle_a.py
def provide_presimulated(
self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
r"""
Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult
release notes to see how you can update your code:
https://github.com/mackelab/sbi/releases/tag/v0.14.0
More information can be found under the corresponding pull request on github:
https://github.com/mackelab/sbi/pull/378
and tutorials:
https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Args:
theta: Parameter sets used to generate presimulated data.
x: Simulation outputs of presimulated data.
from_round: Which round the data was simulated from. from_round=0 means
that the data came from the first round, i.e. the prior.
"""
raise NameError(
f"Deprecated since sbi 0.14.0. "
f"Instead of using this, please use .append_simulations(). Please "
f"consult release notes to see how you can update your code: "
f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
f"More information can be found under the corresponding pull request on "
f"github: "
f"https://github.com/mackelab/sbi/pull/378"
f"and tutorials: "
f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
)


#### train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)¶

Return density estimator that approximates the distribution $$p(x|\theta)$$.

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Density estimator that approximates the distribution $$p(x|\theta)$$.

Source code in sbi/inference/snle/snle_a.py
def train(
self,
training_batch_size: int = 50,
learning_rate: float = 5e-4,
validation_fraction: float = 0.1,
stop_after_epochs: int = 20,
max_num_epochs: Optional[int] = None,
clip_max_norm: Optional[float] = 5.0,
exclude_invalid_x: bool = True,
resume_training: bool = False,
retrain_from_scratch_each_round: bool = False,
show_train_summary: bool = False,
) -> NeuralPosterior:
r"""
Return density estimator that approximates the distribution $p(x|\theta)$.

Args:
training_batch_size: Training batch size.
learning_rate: Learning rate for Adam optimizer.
validation_fraction: The fraction of data to use for validation.
stop_after_epochs: The number of epochs to wait for improvement on the
validation set before terminating training.
max_num_epochs: Maximum number of epochs to run. If reached, we stop
training even when the validation loss is still decreasing. If None, we
train until validation loss increases (see also stop_after_epochs).
clip_max_norm: Value at which to clip the total gradient norm in order to
prevent exploding gradients. Use None for no clipping.
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training. Expect errors, silent or explicit, when False.
resume_training: Can be used in case training time is limited, e.g. on a
cluster. If True, the split between train and validation set, the
optimizer, the number of epochs, and the best validation log-prob will
be restored from the last time .train() was called.
from the prior. Training may be sped up by ignoring such less targeted
samples.
retrain_from_scratch_each_round: Whether to retrain the conditional density
estimator for the posterior from scratch each round.
show_train_summary: Whether to print the number of epochs and validation
loss and leakage after the training.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Density estimator that approximates the distribution $p(x|\theta)$.
"""
kwargs = del_entries(locals(), entries=("self", "__class__"))
return super().train(**kwargs)


###  sbi.inference.snre.snre_a.SNRE_A (RatioEstimator) ¶

#### __init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special ¶

AALR, here known as SNRE_A.

 Likelihood-free MCMC with Amortized Approximate Likelihood Ratios, Hermans et al., ICML 2020, https://arxiv.org/abs/1903.04057

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_a.py
def __init__(
self,
prior,
classifier: Union[str, Callable] = "resnet",
device: str = "cpu",
logging_level: Union[int, str] = "warning",
summary_writer: Optional[TensorboardSummaryWriter] = None,
show_progress_bars: bool = True,
**unused_args
):
r"""AALR, here known as SNRE_A.

 _Likelihood-free MCMC with Amortized Approximate Likelihood Ratios_, Hermans
et al., ICML 2020, https://arxiv.org/abs/1903.04057

Args:
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
classifier: Classifier trained to approximate likelihood ratios. If it is
a string, use a pre-configured network of the provided type (one of
linear, mlp, resnet). Alternatively, a function that builds a custom
neural network can be provided. The function will be called with the
first batch of simulations (theta, x), which can thus be used for shape
inference and potentially for z-scoring. It needs to return a PyTorch
nn.Module implementing the classifier.
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
logging_level: Minimum severity of messages to log. One of the strings
INFO, WARNING, DEBUG, ERROR and CRITICAL.
summary_writer: A tensorboard SummaryWriter to control, among others, log
file location (default is <current working directory>/logs.)
show_progress_bars: Whether to show a progressbar during simulation and
sampling.
unused_args: Absorbs additional arguments. No entries will be used. If it
is not empty, we warn. In future versions, when the new interface of
0.14.0 is more mature, we will remove this argument.
"""

kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
super().__init__(**kwargs, **unused_args)


#### append_simulations(self, theta, x, from_round=0) inherited ¶

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $$\theta$$, $$x$$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_a.py
def append_simulations(
self,
theta: Tensor,
x: Tensor,
from_round: int = 0,
) -> "RatioEstimator":
r"""
Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
prior or not) and an index indicating which round the batch of simulations came
from.

Args:
theta: Parameter sets.
x: Simulation outputs.
from_round: Which round the data stemmed from. Round 0 means from the prior.
With default settings, this is not used at all for SNRE. Only when
the user later on requests .train(discard_prior_samples=True), we
use these indices to find which training data stemmed from the prior.
Returns:
NeuralInference object (returned so that this function is chainable).
"""

theta, x = validate_theta_and_x(theta, x, training_device=self._device)

self._theta_roundwise.append(theta)
self._x_roundwise.append(x)
self._data_round_index.append(int(from_round))

return self


#### build_posterior(self, density_estimator=None, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None) inherited ¶

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$. The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
RatioBasedPosterior

Posterior $$p(\theta|x)$$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_a.py
def build_posterior(
self,
density_estimator: Optional[TorchModule] = None,
sample_with: str = "mcmc",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> RatioBasedPosterior:
r"""
Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn
can be used obtain an unnormalized posterior
$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here
wraps the trained network such that one can directly evaluate the unnormalized
posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and
draw samples from the posterior with MCMC. Note that, in the case of
single-round SNRE_A / AALR, it is possible to evaluate the log-probability of
the **normalized** posterior, but sampling still requires MCMC.

Args:
density_estimator: The density estimator that the posterior is based on.
If None, use the latest neural density estimator that was trained.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior will
draw init locations from prior, whereas sir will use
Sequential-Importance-Resampling using init_strategy_num_candidates
to find init locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration. num_samples_to_find_max as the
number of samples that are used to find the maximum of the
potential_fn / proposal ratio. num_iter_to_find_max as the number
of gradient ascent iterations to find the maximum of that ratio. m as
multiplier to that ratio.

Returns:
Posterior $p(\theta|x)$  with .sample() and .log_prob() methods
(the returned log-probability is unnormalized).
"""

if density_estimator is None:
density_estimator = self._neural_net
# If internal net is used device is defined.
device = self._device
else:
# Otherwise, infer it from the device of the net parameters.
device = next(density_estimator.parameters()).device.type

self._posterior = RatioBasedPosterior(
method_family=self.__class__.__name__.lower(),
neural_net=density_estimator,
prior=self._prior,
x_shape=self._x_shape,
sample_with=sample_with,
mcmc_method=mcmc_method,
mcmc_parameters=mcmc_parameters,
rejection_sampling_parameters=rejection_sampling_parameters,
device=device,
)

self._posterior._num_trained_rounds = self._round + 1

# Store models at end of each round.
self._model_bank.append(deepcopy(self._posterior))
self._model_bank[-1].net.eval()

return deepcopy(self._posterior)


#### get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited ¶

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_a.py
def get_dataloaders(
self,
dataset: data.TensorDataset,
training_batch_size: int = 50,
validation_fraction: float = 0.1,
resume_training: bool = False,
"""Return dataloaders for training and validation.

Args:
dataset: holding all theta and x, optionally masks.
training_batch_size: training arg of inference methods.
resume_training: Whether the current call is resuming training so that no
new training and validation indices into the dataset have to be created.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Tuple of dataloaders for training and validation.

"""

# Get total number of training examples.
num_examples = len(dataset)

# Select random train and validation splits from (theta, x) pairs.
num_training_examples = int((1 - validation_fraction) * num_examples)
num_validation_examples = num_examples - num_training_examples

if not resume_training:
permuted_indices = torch.randperm(num_examples)
self.train_indices, self.val_indices = (
permuted_indices[:num_training_examples],
permuted_indices[num_training_examples:],
)

# Create training and validation loaders using a subset sampler.
# Intentionally use dicts to define the default dataloader args
# Then, use dataloader_kwargs to override (or add to) any of these defaults
# https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
"batch_size": min(training_batch_size, num_training_examples),
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.train_indices),
}
)
"batch_size": min(training_batch_size, num_validation_examples),
"shuffle": False,
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.val_indices),
}
)



#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited ¶

Returns all $$\theta$$, $$x$$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_a.py
def get_simulations(
self,
starting_round: int = 0,
exclude_invalid_x: bool = True,
warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
r"""
Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Args:
starting_round: The earliest round to return samples from (we start counting
from zero).
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training.
warn_on_invalid: Whether to give out a warning if invalid simulations were
found.

Returns: Parameters, simulation outputs, prior masks.
"""

theta = get_simulations_since_round(
self._theta_roundwise, self._data_round_index, starting_round
)
x = get_simulations_since_round(
self._x_roundwise, self._data_round_index, starting_round
)
)

# Check for NaNs in simulations.
is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
# Check for problematic z-scoring
warn_if_zscoring_changes_data(x)
if warn_on_invalid:
warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
warn_on_invalid_x_for_snpec_leakage(
num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
)



#### provide_presimulated(self, theta, x, from_round=0) inherited ¶

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $$\theta$$ and $$x$$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_a.py
def provide_presimulated(
self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
r"""
Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult
release notes to see how you can update your code:
https://github.com/mackelab/sbi/releases/tag/v0.14.0
More information can be found under the corresponding pull request on github:
https://github.com/mackelab/sbi/pull/378
and tutorials:
https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Args:
theta: Parameter sets used to generate presimulated data.
x: Simulation outputs of presimulated data.
from_round: Which round the data was simulated from. from_round=0 means
that the data came from the first round, i.e. the prior.
"""
raise NameError(
f"Deprecated since sbi 0.14.0. "
f"Instead of using this, please use .append_simulations(). Please "
f"consult release notes to see how you can update your code: "
f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
f"More information can be found under the corresponding pull request on "
f"github: "
f"https://github.com/mackelab/sbi/pull/378"
f"and tutorials: "
f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
)


#### train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)¶

Return classifier that approximates the ratio $$p(\theta,x)/p(\theta)p(x)$$.

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio $$p(\theta,x)/p(\theta)p(x)$$.

Source code in sbi/inference/snre/snre_a.py
def train(
self,
training_batch_size: int = 50,
learning_rate: float = 5e-4,
validation_fraction: float = 0.1,
stop_after_epochs: int = 20,
max_num_epochs: Optional[int] = None,
clip_max_norm: Optional[float] = 5.0,
exclude_invalid_x: bool = True,
resume_training: bool = False,
retrain_from_scratch_each_round: bool = False,
show_train_summary: bool = False,
) -> NeuralPosterior:
r"""
Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

Args:
training_batch_size: Training batch size.
learning_rate: Learning rate for Adam optimizer.
validation_fraction: The fraction of data to use for validation.
stop_after_epochs: The number of epochs to wait for improvement on the
validation set before terminating training.
max_num_epochs: Maximum number of epochs to run. If reached, we stop
training even when the validation loss is still decreasing. If None, we
train until validation loss increases (see also stop_after_epochs).
clip_max_norm: Value at which to clip the total gradient norm in order to
prevent exploding gradients. Use None for no clipping.
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training. Expect errors, silent or explicit, when False.
resume_training: Can be used in case training time is limited, e.g. on a
cluster. If True, the split between train and validation set, the
optimizer, the number of epochs, and the best validation log-prob will
be restored from the last time .train() was called.
from the prior. Training may be sped up by ignoring such less targeted
samples.
retrain_from_scratch_each_round: Whether to retrain the conditional density
estimator for the posterior from scratch each round.
show_train_summary: Whether to print the number of epochs and validation
loss and leakage after the training.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.
"""

# AALR is defined for num_atoms=2.
# Proxy to super().__call__ to ensure right parameter.
kwargs = del_entries(locals(), entries=("self", "__class__"))
return super().train(**kwargs, num_atoms=2)


###  sbi.inference.snre.snre_b.SNRE_B (RatioEstimator) ¶

#### __init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special ¶

SRE, here known as SNRE_B.

 On Contrastive Learning for Likelihood-free Inference, Durkan et al., ICML 2020, https://arxiv.org/pdf/2002.03712

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_b.py
def __init__(
self,
prior,
classifier: Union[str, Callable] = "resnet",
device: str = "cpu",
logging_level: Union[int, str] = "warning",
summary_writer: Optional[TensorboardSummaryWriter] = None,
show_progress_bars: bool = True,
**unused_args
):
r"""SRE, here known as SNRE_B.

 _On Contrastive Learning for Likelihood-free Inference_, Durkan et al.,
ICML 2020, https://arxiv.org/pdf/2002.03712

Args:
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
classifier: Classifier trained to approximate likelihood ratios. If it is
a string, use a pre-configured network of the provided type (one of
linear, mlp, resnet). Alternatively, a function that builds a custom
neural network can be provided. The function will be called with the
first batch of simulations (theta, x), which can thus be used for shape
inference and potentially for z-scoring. It needs to return a PyTorch
nn.Module implementing the classifier.
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
logging_level: Minimum severity of messages to log. One of the strings
INFO, WARNING, DEBUG, ERROR and CRITICAL.
summary_writer: A tensorboard SummaryWriter to control, among others, log
file location (default is <current working directory>/logs.)
show_progress_bars: Whether to show a progressbar during simulation and
sampling.
unused_args: Absorbs additional arguments. No entries will be used. If it
is not empty, we warn. In future versions, when the new interface of
0.14.0 is more mature, we will remove this argument.
"""

kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
super().__init__(**kwargs, **unused_args)


#### append_simulations(self, theta, x, from_round=0) inherited ¶

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $$\theta$$, $$x$$, prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_b.py
def append_simulations(
self,
theta: Tensor,
x: Tensor,
from_round: int = 0,
) -> "RatioEstimator":
r"""
Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
prior or not) and an index indicating which round the batch of simulations came
from.

Args:
theta: Parameter sets.
x: Simulation outputs.
from_round: Which round the data stemmed from. Round 0 means from the prior.
With default settings, this is not used at all for SNRE. Only when
the user later on requests .train(discard_prior_samples=True), we
use these indices to find which training data stemmed from the prior.
Returns:
NeuralInference object (returned so that this function is chainable).
"""

theta, x = validate_theta_and_x(theta, x, training_device=self._device)

self._theta_roundwise.append(theta)
self._x_roundwise.append(x)
self._data_round_index.append(int(from_round))

return self


#### build_posterior(self, density_estimator=None, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None) inherited ¶

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$. The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
RatioBasedPosterior

Posterior $$p(\theta|x)$$ with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_b.py
def build_posterior(
self,
density_estimator: Optional[TorchModule] = None,
sample_with: str = "mcmc",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> RatioBasedPosterior:
r"""
Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn
can be used obtain an unnormalized posterior
$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here
wraps the trained network such that one can directly evaluate the unnormalized
posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and
draw samples from the posterior with MCMC. Note that, in the case of
single-round SNRE_A / AALR, it is possible to evaluate the log-probability of
the **normalized** posterior, but sampling still requires MCMC.

Args:
density_estimator: The density estimator that the posterior is based on.
If None, use the latest neural density estimator that was trained.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior will
draw init locations from prior, whereas sir will use
Sequential-Importance-Resampling using init_strategy_num_candidates
to find init locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration. num_samples_to_find_max as the
number of samples that are used to find the maximum of the
potential_fn / proposal ratio. num_iter_to_find_max as the number
of gradient ascent iterations to find the maximum of that ratio. m as
multiplier to that ratio.

Returns:
Posterior $p(\theta|x)$  with .sample() and .log_prob() methods
(the returned log-probability is unnormalized).
"""

if density_estimator is None:
density_estimator = self._neural_net
# If internal net is used device is defined.
device = self._device
else:
# Otherwise, infer it from the device of the net parameters.
device = next(density_estimator.parameters()).device.type

self._posterior = RatioBasedPosterior(
method_family=self.__class__.__name__.lower(),
neural_net=density_estimator,
prior=self._prior,
x_shape=self._x_shape,
sample_with=sample_with,
mcmc_method=mcmc_method,
mcmc_parameters=mcmc_parameters,
rejection_sampling_parameters=rejection_sampling_parameters,
device=device,
)

self._posterior._num_trained_rounds = self._round + 1

# Store models at end of each round.
self._model_bank.append(deepcopy(self._posterior))
self._model_bank[-1].net.eval()

return deepcopy(self._posterior)


#### get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited ¶

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_b.py
def get_dataloaders(
self,
dataset: data.TensorDataset,
training_batch_size: int = 50,
validation_fraction: float = 0.1,
resume_training: bool = False,
"""Return dataloaders for training and validation.

Args:
dataset: holding all theta and x, optionally masks.
training_batch_size: training arg of inference methods.
resume_training: Whether the current call is resuming training so that no
new training and validation indices into the dataset have to be created.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Tuple of dataloaders for training and validation.

"""

# Get total number of training examples.
num_examples = len(dataset)

# Select random train and validation splits from (theta, x) pairs.
num_training_examples = int((1 - validation_fraction) * num_examples)
num_validation_examples = num_examples - num_training_examples

if not resume_training:
permuted_indices = torch.randperm(num_examples)
self.train_indices, self.val_indices = (
permuted_indices[:num_training_examples],
permuted_indices[num_training_examples:],
)

# Create training and validation loaders using a subset sampler.
# Intentionally use dicts to define the default dataloader args
# Then, use dataloader_kwargs to override (or add to) any of these defaults
# https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
"batch_size": min(training_batch_size, num_training_examples),
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.train_indices),
}
)
"batch_size": min(training_batch_size, num_validation_examples),
"shuffle": False,
"drop_last": True,
"sampler": data.sampler.SubsetRandomSampler(self.val_indices),
}
)



#### get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited ¶

Returns all $$\theta$$, $$x$$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_b.py
def get_simulations(
self,
starting_round: int = 0,
exclude_invalid_x: bool = True,
warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
r"""
Returns all $\theta$, $x$, and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Args:
starting_round: The earliest round to return samples from (we start counting
from zero).
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training.
warn_on_invalid: Whether to give out a warning if invalid simulations were
found.

Returns: Parameters, simulation outputs, prior masks.
"""

theta = get_simulations_since_round(
self._theta_roundwise, self._data_round_index, starting_round
)
x = get_simulations_since_round(
self._x_roundwise, self._data_round_index, starting_round
)
)

# Check for NaNs in simulations.
is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
# Check for problematic z-scoring
warn_if_zscoring_changes_data(x)
if warn_on_invalid:
warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
warn_on_invalid_x_for_snpec_leakage(
num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
)



#### provide_presimulated(self, theta, x, from_round=0) inherited ¶

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $$\theta$$ and $$x$$ to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_b.py
def provide_presimulated(
self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
r"""
Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult
release notes to see how you can update your code:
https://github.com/mackelab/sbi/releases/tag/v0.14.0
More information can be found under the corresponding pull request on github:
https://github.com/mackelab/sbi/pull/378
and tutorials:
https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external $\theta$ and $x$ to be used for training later on.

Args:
theta: Parameter sets used to generate presimulated data.
x: Simulation outputs of presimulated data.
from_round: Which round the data was simulated from. from_round=0 means
that the data came from the first round, i.e. the prior.
"""
raise NameError(
f"Deprecated since sbi 0.14.0. "
f"Instead of using this, please use .append_simulations(). Please "
f"consult release notes to see how you can update your code: "
f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
f"More information can be found under the corresponding pull request on "
f"github: "
f"https://github.com/mackelab/sbi/pull/378"
f"and tutorials: "
f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
)


#### train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)¶

Return classifier that approximates the ratio $$p(\theta,x)/p(\theta)p(x)$$.

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio $$p(\theta,x)/p(\theta)p(x)$$.

Source code in sbi/inference/snre/snre_b.py
def train(
self,
num_atoms: int = 10,
training_batch_size: int = 50,
learning_rate: float = 5e-4,
validation_fraction: float = 0.1,
stop_after_epochs: int = 20,
max_num_epochs: Optional[int] = None,
clip_max_norm: Optional[float] = 5.0,
exclude_invalid_x: bool = True,
resume_training: bool = False,
retrain_from_scratch_each_round: bool = False,
show_train_summary: bool = False,
) -> NeuralPosterior:
r"""
Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

Args:
num_atoms: Number of atoms to use for classification.
training_batch_size: Training batch size.
learning_rate: Learning rate for Adam optimizer.
validation_fraction: The fraction of data to use for validation.
stop_after_epochs: The number of epochs to wait for improvement on the
validation set before terminating training.
max_num_epochs: Maximum number of epochs to run. If reached, we stop
training even when the validation loss is still decreasing. If None, we
train until validation loss increases (see also stop_after_epochs).
clip_max_norm: Value at which to clip the total gradient norm in order to
prevent exploding gradients. Use None for no clipping.
exclude_invalid_x: Whether to exclude simulation outputs x=NaN or x=±∞
during training. Expect errors, silent or explicit, when False.
resume_training: Can be used in case training time is limited, e.g. on a
cluster. If True, the split between train and validation set, the
optimizer, the number of epochs, and the best validation log-prob will
be restored from the last time .train() was called.
from the prior. Training may be sped up by ignoring such less targeted
samples.
retrain_from_scratch_each_round: Whether to retrain the conditional density
estimator for the posterior from scratch each round.
show_train_summary: Whether to print the number of epochs and validation
loss and leakage after the training.
and validation dataloaders (like, e.g., a collate_fn)

Returns:
Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.
"""
kwargs = del_entries(locals(), entries=("self", "__class__"))
return super().train(**kwargs)


###  sbi.inference.abc.mcabc.MCABC (ABCBASE) ¶

#### __call__(self, x_o, num_simulations, eps=None, quantile=None, lra=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1, kde=False, kde_kwargs={}, return_summary=False) special ¶

Run MCABC and return accepted parameters or KDE object fitted on them.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_simulations int

Number of simulations to run.

required
eps Optional[float]

Acceptance threshold $$\epsilon$$ for distance between observed and simulated data.

None
quantile Optional[float]

Upper quantile of smallest distances for which the corresponding parameters are returned, e.g, q=0.01 will return the top 1%. Exactly one of quantile or eps have to be passed.

None
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1
kde bool

Whether to run KDE on the accepted parameters to return a KDE object from which one can sample.

False
kde_kwargs Optional[Dict[str, Any]]

kwargs for performing KDE: ‘bandwidth=’; either a float, or a string naming a bandwidth heuristics, e.g., ‘cv’ (cross validation), ‘silvermann’ or ‘scott’, default ‘cv’. ‘transform’: transform applied to the parameters before doing KDE. ‘sample_weights’: weights associated with samples. See ‘get_kde’ for more details

{}
return_summary bool

Whether to return the distances and data corresponding to the accepted parameters.

False

Returns:

Type Description
theta (if kde False)

accepted parameters kde (if kde True): KDE object based on accepted parameters from which one can .sample() and .log_prob(). summary (if summary True): dictionary containing the accepted paramters (if kde True), distances and simulated data x.

Source code in sbi/inference/abc/mcabc.py
def __call__(
self,
x_o: Union[Tensor, ndarray],
num_simulations: int,
eps: Optional[float] = None,
quantile: Optional[float] = None,
lra: bool = False,
sass: bool = False,
sass_fraction: float = 0.25,
sass_expansion_degree: int = 1,
kde: bool = False,
kde_kwargs: Optional[Dict[str, Any]] = {},
return_summary: bool = False,
) -> Union[Tuple[Tensor, dict], Tuple[KDEWrapper, dict], Tensor, KDEWrapper]:
r"""Run MCABC and return accepted parameters or KDE object fitted on them.

Args:
x_o: Observed data.
num_simulations: Number of simulations to run.
eps: Acceptance threshold $\epsilon$ for distance between observed and
simulated data.
quantile: Upper quantile of smallest distances for which the corresponding
parameters are returned, e.g, q=0.01 will return the top 1%. Exactly
one of quantile or eps have to be passed.
lra: Whether to run linear regression adjustment as in Beaumont et al. 2002
sass: Whether to determine semi-automatic summary statistics as in
sass_fraction: Fraction of simulation budget used for the initial sass run.
sass_expansion_degree: Degree of the polynomial feature expansion for the
sass regression, default 1 - no expansion.
kde: Whether to run KDE on the accepted parameters to return a KDE
object from which one can sample.
kde_kwargs: kwargs for performing KDE:
'bandwidth='; either a float, or a string naming a bandwidth
heuristics, e.g., 'cv' (cross validation), 'silvermann' or 'scott',
default 'cv'.
'transform': transform applied to the parameters before doing KDE.
'sample_weights': weights associated with samples. See 'get_kde' for
more details
return_summary: Whether to return the distances and data corresponding to
the accepted parameters.

Returns:
theta (if kde False): accepted parameters
kde (if kde True): KDE object based on accepted parameters from which one
can .sample() and .log_prob().
summary (if summary True): dictionary containing the accepted paramters (if
kde True), distances and simulated data x.
"""

# Exactly one of eps or quantile need to be passed.
assert (eps is not None) ^ (
quantile is not None
), "Eps or quantile must be passed, but not both."

# Run SASS and change the simulator and x_o accordingly.
if sass:
num_pilot_simulations = int(sass_fraction * num_simulations)
self.logger.info(
f"Running SASS with {num_pilot_simulations} pilot samples."
)
num_simulations -= num_pilot_simulations

pilot_theta = self.prior.sample((num_pilot_simulations,))
pilot_x = self._batched_simulator(pilot_theta)

sass_transform = self.get_sass_transform(
pilot_theta, pilot_x, sass_expansion_degree
)

simulator = lambda theta: sass_transform(self._batched_simulator(theta))
x_o = sass_transform(x_o)
else:
simulator = self._batched_simulator

# Simulate and calculate distances.
theta = self.prior.sample((num_simulations,))
x = simulator(theta)

# Infer shape of x to test and set x_o.
self.x_shape = x.unsqueeze(0).shape
self.x_o = process_x(x_o, self.x_shape)

distances = self.distance(self.x_o, x)

# Select based on acceptance threshold epsilon.
if eps is not None:
is_accepted = distances < eps
num_accepted = is_accepted.sum().item()
assert num_accepted > 0, f"No parameters accepted, eps={eps} too small"

theta_accepted = theta[is_accepted]
distances_accepted = distances[is_accepted]
x_accepted = x[is_accepted]

# Select based on quantile on sorted distances.
elif quantile is not None:
num_top_samples = int(num_simulations * quantile)
sort_idx = torch.argsort(distances)
theta_accepted = theta[sort_idx][:num_top_samples]
distances_accepted = distances[sort_idx][:num_top_samples]
x_accepted = x[sort_idx][:num_top_samples]

else:
raise ValueError("One of epsilon or quantile has to be passed.")

# Maybe adjust theta with LRA.
if lra:
final_theta = self.run_lra(theta_accepted, x_accepted, observation=self.x_o)
else:
final_theta = theta_accepted

if kde:
self.logger.info(
f"""KDE on {final_theta.shape} samples with bandwidth option
{kde_kwargs["bandwidth"] if "bandwidth" in kde_kwargs else "cv"}.
Beware that KDE can give unreliable results when used with too few
samples and in high dimensions."""
)

kde_dist = get_kde(final_theta, **kde_kwargs)

if return_summary:
return kde_dist, dict(
theta=final_theta, distances=distances_accepted, x=x_accepted
)
else:
return kde_dist
elif return_summary:
return final_theta, dict(distances=distances_accepted, x=x_accepted)
else:
return final_theta


#### __init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True) special ¶

Monte-Carlo Approximate Bayesian Computation (Rejection ABC) .

 Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W. (1999). Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $heta$ and maps them to simulations, or observations, x, $$\mathrm{sim}( heta) o x$$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
Source code in sbi/inference/abc/mcabc.py
def __init__(
self,
simulator: Callable,
prior,
distance: Union[str, Callable] = "l2",
num_workers: int = 1,
simulation_batch_size: int = 1,
show_progress_bars: bool = True,
):
"""Monte-Carlo Approximate Bayesian Computation (Rejection ABC) .

 Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W.
(1999). Population growth of human Y chromosomes: a study of Y chromosome
microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

Args:
simulator: A function that takes parameters $\theta$ and maps them to
simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any
regular Python callable (i.e. function or class with __call__ method)
can be used.
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
distance: Distance function to compare observed and simulated data. Can be
a custom function or one of l1, l2, mse.
num_workers: Number of parallel workers to use for simulations.
simulation_batch_size: Number of parameter sets that the simulator
maps to data x at once. If None, we simulate all parameter sets at the
same time. If >= 1, the simulator has to process data of shape
(simulation_batch_size, parameter_dimension).
show_progress_bars: Whether to show a progressbar during simulation and
sampling.
"""

super().__init__(
simulator=simulator,
prior=prior,
distance=distance,
num_workers=num_workers,
simulation_batch_size=simulation_batch_size,
show_progress_bars=show_progress_bars,
)


#### choose_distance_function(distance_type='l2') inherited ¶

Return distance function for given distance type.

Source code in sbi/inference/abc/mcabc.py
@staticmethod
def choose_distance_function(distance_type: str = "l2") -> Callable:
"""Return distance function for given distance type."""

if distance_type == "mse":
distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1)
elif distance_type == "l2":
distance = lambda xo, x: torch.norm((xo - x), dim=-1)
elif distance_type == "l1":
distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1)
else:
raise ValueError(r"Distance {distance_type} not supported.")

def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor:
"""Return distance over batch dimension.

Args:
observed_data: Observed data, could be 1D.
simulated_data: Batch of simulated data, has batch dimension.

Returns:
Torch tensor with batch of distances.
"""
assert simulated_data.ndim == 2, "simulated data needs batch dimension"

return distance(observed_data, simulated_data)

return distance_fun


#### get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited ¶

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Source code in sbi/inference/abc/mcabc.py
@staticmethod
def get_sass_transform(
theta: torch.Tensor,
x: torch.Tensor,
expansion_degree: int = 1,
sample_weight=None,
) -> Callable:
"""Return semi-automatic summary statitics function.

Running weighted linear regressin as in

Following implementation in
and
https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic
"""
expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False)
# Transform x, remove intercept.
x_expanded = expansion.fit_transform(x)
sumstats_map = np.zeros((x_expanded.shape, theta.shape))

for parameter_idx in range(theta.shape):
regression_model = LinearRegression(fit_intercept=True)
regression_model.fit(
X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight
)
sumstats_map[:, parameter_idx] = regression_model.coef_

sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32)

def sumstats_transform(x):
x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32)
return x_expanded.mm(sumstats_map)

return sumstats_transform


#### run_lra(theta, x, observation, sample_weight=None) inherited ¶

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/mcabc.py
@staticmethod
def run_lra(
theta: torch.Tensor,
x: torch.Tensor,
observation: torch.Tensor,
sample_weight=None,
) -> torch.Tensor:

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254
"""

for parameter_idx in range(theta.shape):
regression_model = LinearRegression(fit_intercept=True)
regression_model.fit(
X=x,
y=theta[:, parameter_idx],
sample_weight=sample_weight,
)
observation.reshape(1, -1)
)



###  sbi.inference.abc.smcabc.SMCABC (ABCBASE) ¶

#### __call__(self, x_o, num_particles, num_initial_pop, num_simulations, epsilon_decay, distance_based_decay=False, ess_min=None, kernel_variance_scale=1.0, use_last_pop_samples=True, return_summary=False, kde=False, kde_kwargs={}, kde_sample_weights=False, lra=False, lra_with_weights=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1) special ¶

Run SMCABC and return accepted parameters or KDE object fitted on them.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_particles int

Number of particles in each population.

required
num_initial_pop int

Number of simulations used for initial population.

required
num_simulations int

Total number of possible simulations.

required
epsilon_decay float

Factor with which the acceptance threshold $$\epsilon$$ decays.

required
distance_based_decay bool

Whether the $$\epsilon$$ decay is constant over populations or calculated from the previous populations distribution of distances.

False
ess_min Optional[float]

Threshold of effective sampling size for resampling weights. Not used when None (default).

None
kernel_variance_scale float

Factor for scaling the perturbation kernel variance.

1.0
use_last_pop_samples bool

Whether to fill up the current population with samples from the previous population when the budget is used up. If False, the current population is discarded and the previous population is returned.

True
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
lra_with_weights bool

Whether to run lra as weighted linear regression with SMC weights

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1
kde bool

Whether to run KDE on the accepted parameters to return a KDE object from which one can sample.

False
kde_kwargs Optional[Dict[str, Any]]

kwargs for performing KDE: ‘bandwidth=’; either a float, or a string naming a bandwidth heuristics, e.g., ‘cv’ (cross validation), ‘silvermann’ or ‘scott’, default ‘cv’. ‘transform’: transform applied to the parameters before doing KDE. ‘sample_weights’: weights associated with samples. See ‘get_kde’ for more details

{}
kde_sample_weights bool

Whether perform weighted KDE with SMC weights or on raw particles.

False
return_summary bool

Whether to return a dictionary with all accepted particles, weights, etc. at the end.

False

Returns:

Type Description
theta (if kde False)

accepted parameters of the last population. kde (if kde True): KDE object fitted on accepted parameters, from which one can .sample() and .log_prob(). summary (if return_summary True): dictionary containing the accepted paramters (if kde True), distances and simulated data x of all populations.

Source code in sbi/inference/abc/smcabc.py
def __call__(
self,
x_o: Union[Tensor, ndarray],
num_particles: int,
num_initial_pop: int,
num_simulations: int,
epsilon_decay: float,
distance_based_decay: bool = False,
ess_min: Optional[float] = None,
kernel_variance_scale: float = 1.0,
use_last_pop_samples: bool = True,
return_summary: bool = False,
kde: bool = False,
kde_kwargs: Optional[Dict[str, Any]] = {},
kde_sample_weights: bool = False,
lra: bool = False,
lra_with_weights: bool = False,
sass: bool = False,
sass_fraction: float = 0.25,
sass_expansion_degree: int = 1,
) -> Union[Tensor, KDEWrapper, Tuple[Tensor, dict], Tuple[KDEWrapper, dict]]:
r"""Run SMCABC and return accepted parameters or KDE object fitted on them.

Args:
x_o: Observed data.
num_particles: Number of particles in each population.
num_initial_pop: Number of simulations used for initial population.
num_simulations: Total number of possible simulations.
epsilon_decay: Factor with which the acceptance threshold $\epsilon$ decays.
distance_based_decay: Whether the $\epsilon$ decay is constant over
populations or calculated from the previous populations distribution of
distances.
ess_min: Threshold of effective sampling size for resampling weights. Not
used when None (default).
kernel_variance_scale: Factor for scaling the perturbation kernel variance.
use_last_pop_samples: Whether to fill up the current population with
samples from the previous population when the budget is used up. If
False, the current population is discarded and the previous population
is returned.
lra: Whether to run linear regression adjustment as in Beaumont et al. 2002
lra_with_weights: Whether to run lra as weighted linear regression with SMC
weights
sass: Whether to determine semi-automatic summary statistics as in
sass_fraction: Fraction of simulation budget used for the initial sass run.
sass_expansion_degree: Degree of the polynomial feature expansion for the
sass regression, default 1 - no expansion.
kde: Whether to run KDE on the accepted parameters to return a KDE
object from which one can sample.
kde_kwargs: kwargs for performing KDE:
'bandwidth='; either a float, or a string naming a bandwidth
heuristics, e.g., 'cv' (cross validation), 'silvermann' or 'scott',
default 'cv'.
'transform': transform applied to the parameters before doing KDE.
'sample_weights': weights associated with samples. See 'get_kde' for
more details
kde_sample_weights: Whether perform weighted KDE with SMC weights or on raw
particles.
return_summary: Whether to return a dictionary with all accepted particles,
weights, etc. at the end.

Returns:
theta (if kde False): accepted parameters of the last population.
kde (if kde True): KDE object fitted on accepted parameters, from which one
can .sample() and .log_prob().
summary (if return_summary True): dictionary containing the accepted
paramters (if kde True), distances and simulated data x of all
populations.
"""

pop_idx = 0
self.num_simulations = num_simulations

# Pilot run for SASS.
if sass:
num_pilot_simulations = int(sass_fraction * num_simulations)
self.logger.info(
f"Running SASS with {num_pilot_simulations} pilot samples."
)
sass_transform = self.run_sass_set_xo(
num_particles, num_pilot_simulations, x_o, lra, sass_expansion_degree
)
# Udpate simulator and xo
x_o = sass_transform(self.x_o)

def sass_simulator(theta):
self.simulation_counter += theta.shape
return sass_transform(self._batched_simulator(theta))

self._simulate_with_budget = sass_simulator

# run initial population
particles, epsilon, distances, x = self._set_xo_and_sample_initial_population(
x_o, num_particles, num_initial_pop
)
log_weights = torch.log(1 / num_particles * ones(num_particles))

self.logger.info(
(
f"population={pop_idx}, eps={epsilon}, ess={1.0}, "
f"num_sims={num_initial_pop}"
)
)

all_particles = [particles]
all_log_weights = [log_weights]
all_distances = [distances]
all_epsilons = [epsilon]
all_x = [x]

while self.simulation_counter < self.num_simulations:

pop_idx += 1
# Decay based on quantile of distances from previous pop.
if distance_based_decay:
epsilon = self._get_next_epsilon(
all_distances[pop_idx - 1], epsilon_decay
)
# Constant decay.
else:
epsilon *= epsilon_decay

# Get kernel variance from previous pop.
self.kernel_variance = self.get_kernel_variance(
all_particles[pop_idx - 1],
torch.exp(all_log_weights[pop_idx - 1]),
samples_per_dim=500,
kernel_variance_scale=kernel_variance_scale,
)
particles, log_weights, distances, x = self._sample_next_population(
particles=all_particles[pop_idx - 1],
log_weights=all_log_weights[pop_idx - 1],
distances=all_distances[pop_idx - 1],
epsilon=epsilon,
x=all_x[pop_idx - 1],
use_last_pop_samples=use_last_pop_samples,
)

# Resample population if effective sampling size is too small.
if ess_min is not None:
particles, log_weights = self.resample_if_ess_too_small(
particles, log_weights, ess_min, pop_idx
)

self.logger.info(
(
f"population={pop_idx} done: eps={epsilon:.6f},"
f" num_sims={self.simulation_counter}."
)
)

# collect results
all_particles.append(particles)
all_log_weights.append(log_weights)
all_distances.append(distances)
all_epsilons.append(epsilon)
all_x.append(x)

# Maybe run LRA and adjust weights.
if lra:
particles=all_particles[-1],
xs=all_x[-1],
observation=x_o,
log_weights=all_log_weights[-1],
lra_with_weights=lra_with_weights,
)
else:
final_particles = all_particles[-1]

if kde:
self.logger.info(
f"""KDE on {final_particles.shape} samples with bandwidth option
{kde_kwargs["bandwidth"] if "bandwidth" in kde_kwargs else "cv"}.
Beware that KDE can give unreliable results when used with too few
samples and in high dimensions."""
)
# Maybe get particles weights from last population for weighted KDE.
if kde_sample_weights:
kde_kwargs["sample_weights"] = all_log_weights[-1].exp()

kde_dist = get_kde(final_particles, **kde_kwargs)

if return_summary:
return (
kde_dist,
dict(
particles=all_particles,
weights=all_log_weights,
epsilons=all_epsilons,
distances=all_distances,
xs=all_x,
),
)
else:
return kde_dist

if return_summary:
return (
final_particles,
dict(
particles=all_particles,
weights=all_log_weights,
epsilons=all_epsilons,
distances=all_distances,
xs=all_x,
),
)
else:
return final_particles


#### __init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True, kernel='gaussian', algorithm_variant='C') special ¶

Sequential Monte Carlo Approximate Bayesian Computation.

We distinguish between three different SMC methods here: - A: Toni et al. 2010 (Phd Thesis) - B: Sisson et al. 2007 (with correction from 2009) - C: Beaumont et al. 2009

In Toni et al. 2010 we find an overview of the differences on page 34: - B: same as A except for resampling of weights if the effective sampling size is too small. - C: same as A except for calculation of the covariance of the perturbation kernel: the kernel covariance is a scaled version of the covariance of the previous population.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $$\theta$$ and maps them to simulations, or observations, x, $$\mathrm{sim}(\theta)\to x$$. Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior Distribution

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
kernel Optional[str]

Perturbation kernel.

'gaussian'
algorithm_variant str

Indicating the choice of algorithm variant, A, B, or C.

'C'
Source code in sbi/inference/abc/smcabc.py
def __init__(
self,
simulator: Callable,
prior: Distribution,
distance: Union[str, Callable] = "l2",
num_workers: int = 1,
simulation_batch_size: int = 1,
show_progress_bars: bool = True,
kernel: Optional[str] = "gaussian",
algorithm_variant: str = "C",
):
r"""Sequential Monte Carlo Approximate Bayesian Computation.

We distinguish between three different SMC methods here:
- A: Toni et al. 2010 (Phd Thesis)
- B: Sisson et al. 2007 (with correction from 2009)
- C: Beaumont et al. 2009

In Toni et al. 2010 we find an overview of the differences on page 34:
- B: same as A except for resampling of weights if the effective sampling
size is too small.
- C: same as A except for calculation of the covariance of the perturbation
kernel: the kernel covariance is a scaled version of the covariance of
the previous population.

Args:
simulator: A function that takes parameters $\theta$ and maps them to
simulations, or observations, x, $\mathrm{sim}(\theta)\to x$. Any
regular Python callable (i.e. function or class with __call__ method)
can be used.
prior: A probability distribution that expresses prior knowledge about the
parameters, e.g. which ranges are meaningful for them. Any
object with .log_prob()and .sample() (for example, a PyTorch
distribution) can be used.
distance: Distance function to compare observed and simulated data. Can be
a custom function or one of l1, l2, mse.
num_workers: Number of parallel workers to use for simulations.
simulation_batch_size: Number of parameter sets that the simulator
maps to data x at once. If None, we simulate all parameter sets at the
same time. If >= 1, the simulator has to process data of shape
(simulation_batch_size, parameter_dimension).
show_progress_bars: Whether to show a progressbar during simulation and
sampling.
kernel: Perturbation kernel.
algorithm_variant: Indicating the choice of algorithm variant, A, B, or C.

"""

super().__init__(
simulator=simulator,
prior=prior,
distance=distance,
num_workers=num_workers,
simulation_batch_size=simulation_batch_size,
show_progress_bars=show_progress_bars,
)

kernels = ("gaussian", "uniform")
assert (
kernel in kernels
), f"Kernel '{kernel}' not supported. Choose one from {kernels}."
self.kernel = kernel

algorithm_variants = ("A", "B", "C")
assert algorithm_variant in algorithm_variants, (
f"SMCABC variant '{algorithm_variant}' not supported, choose one from"
" {algorithm_variants}."
)
self.algorithm_variant = algorithm_variant
self.distance_to_x0 = None
self.simulation_counter = 0
self.num_simulations = 0

# Define simulator that keeps track of budget.
def simulate_with_budget(theta):
self.simulation_counter += theta.shape
return self._batched_simulator(theta)

self._simulate_with_budget = simulate_with_budget


#### choose_distance_function(distance_type='l2') inherited ¶

Return distance function for given distance type.

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def choose_distance_function(distance_type: str = "l2") -> Callable:
"""Return distance function for given distance type."""

if distance_type == "mse":
distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1)
elif distance_type == "l2":
distance = lambda xo, x: torch.norm((xo - x), dim=-1)
elif distance_type == "l1":
distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1)
else:
raise ValueError(r"Distance {distance_type} not supported.")

def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor:
"""Return distance over batch dimension.

Args:
observed_data: Observed data, could be 1D.
simulated_data: Batch of simulated data, has batch dimension.

Returns:
Torch tensor with batch of distances.
"""
assert simulated_data.ndim == 2, "simulated data needs batch dimension"

return distance(observed_data, simulated_data)

return distance_fun


#### get_new_kernel(self, thetas)¶

Return new kernel distribution for a given set of paramters.

Source code in sbi/inference/abc/smcabc.py
def get_new_kernel(self, thetas: Tensor) -> Distribution:
"""Return new kernel distribution for a given set of paramters."""

if self.kernel == "gaussian":
assert self.kernel_variance.ndim == 2
return MultivariateNormal(
loc=thetas, covariance_matrix=self.kernel_variance
)

elif self.kernel == "uniform":
low = thetas - self.kernel_variance
high = thetas + self.kernel_variance
# Move batch shape to event shape to get Uniform that is multivariate in
# parameter dimension.
return Uniform(low=low, high=high).to_event(1)
else:
raise ValueError(f"Kernel, '{self.kernel}' not supported.")


#### get_particle_ranges(self, particles, weights, samples_per_dim=100)¶

Return range of particles in each parameter dimension.

Source code in sbi/inference/abc/smcabc.py
def get_particle_ranges(
self, particles: Tensor, weights: Tensor, samples_per_dim: int = 100
) -> Tensor:
"""Return range of particles in each parameter dimension."""

# get weighted samples
samples = self.sample_from_population_with_weights(
particles,
weights,
num_samples=samples_per_dim * particles.shape,
)

# Variance spans the range of particles for every dimension.
particle_ranges = tensor([max(column) - min(column) for column in samples.T])
assert particle_ranges.ndim < 2
return particle_ranges


#### get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited ¶

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def get_sass_transform(
theta: torch.Tensor,
x: torch.Tensor,
expansion_degree: int = 1,
sample_weight=None,
) -> Callable:
"""Return semi-automatic summary statitics function.

Running weighted linear regressin as in

Following implementation in
and
https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic
"""
expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False)
# Transform x, remove intercept.
x_expanded = expansion.fit_transform(x)
sumstats_map = np.zeros((x_expanded.shape, theta.shape))

for parameter_idx in range(theta.shape):
regression_model = LinearRegression(fit_intercept=True)
regression_model.fit(
X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight
)
sumstats_map[:, parameter_idx] = regression_model.coef_

sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32)

def sumstats_transform(x):
x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32)
return x_expanded.mm(sumstats_map)

return sumstats_transform


#### resample_if_ess_too_small(self, particles, log_weights, ess_min, pop_idx)¶

Return resampled particles and uniform weights if effectice sampling size is too small.

Source code in sbi/inference/abc/smcabc.py
def resample_if_ess_too_small(
self,
particles: Tensor,
log_weights: Tensor,
ess_min: float,
pop_idx: int,
) -> Tuple[Tensor, Tensor]:
"""Return resampled particles and uniform weights if effectice sampling size is
too small.
"""

num_particles = particles.shape
ess = (1 / torch.sum(torch.exp(2.0 * log_weights), dim=0)) / num_particles
# Resampling of weights for low ESS only for Sisson et al. 2007.
if ess < ess_min:
self.logger.info(f"ESS={ess:.2f} too low, resampling pop {pop_idx}...")
# First resample, then set to uniform weights as in Sisson et al. 2007.
particles = self.sample_from_population_with_weights(
particles, torch.exp(log_weights), num_samples=num_particles
)
log_weights = torch.log(1 / num_particles * ones(num_particles))

return particles, log_weights


#### run_lra(theta, x, observation, sample_weight=None) inherited ¶

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def run_lra(
theta: torch.Tensor,
x: torch.Tensor,
observation: torch.Tensor,
sample_weight=None,
) -> torch.Tensor:

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254
"""

for parameter_idx in range(theta.shape):
regression_model = LinearRegression(fit_intercept=True)
regression_model.fit(
X=x,
y=theta[:, parameter_idx],
sample_weight=sample_weight,
)
observation.reshape(1, -1)
)



#### run_lra_update_weights(self, particles, xs, observation, log_weights, lra_with_weights)¶

Return particles and weights adjusted with LRA.

Runs (weighted) linear regression from xs onto particles to adjust the particles.

Updates the SMC weights according to the new particles.

Source code in sbi/inference/abc/smcabc.py
def run_lra_update_weights(
self,
particles: Tensor,
xs: Tensor,
observation: Tensor,
log_weights: Tensor,
lra_with_weights: bool,
) -> Tuple[Tensor, Tensor]:
"""Return particles and weights adjusted with LRA.

Runs (weighted) linear regression from xs onto particles to adjust the
particles.

Updates the SMC weights according to the new particles.
"""

theta=particles,
x=xs,
observation=observation,
sample_weight=log_weights.exp() if lra_with_weights else None,
)

# Update SMC weights with LRA adjusted weights
old_particles=particles,
old_log_weights=log_weights,
)



#### run_sass_set_xo(self, num_particles, num_pilot_simulations, x_o, lra=False, sass_expansion_degree=1)¶

Return transform for semi-automatic summary statistics.

Runs an single round of rejection abc with fixed budget and accepts num_particles simulations to run the regression for sass.

Sets self.x_o once the x_shape can be derived from simulations.

Source code in sbi/inference/abc/smcabc.py
def run_sass_set_xo(
self,
num_particles: int,
num_pilot_simulations: int,
x_o,
lra: bool = False,
sass_expansion_degree: int = 1,
) -> Callable:
"""Return transform for semi-automatic summary statistics.

Runs an single round of rejection abc with fixed budget and accepts
num_particles simulations to run the regression for sass.

Sets self.x_o once the x_shape can be derived from simulations.
"""
(pilot_particles, _, _, pilot_xs,) = self._set_xo_and_sample_initial_population(
x_o, num_particles, num_pilot_simulations
)
if lra:
pilot_particles = self.run_lra(pilot_particles, pilot_xs, self.x_o)
sass_transform = self.get_sass_transform(
pilot_particles,
pilot_xs,
expansion_degree=sass_expansion_degree,
sample_weight=None,
)
return sass_transform


#### sample_from_population_with_weights(particles, weights, num_samples=1) staticmethod ¶

Return samples from particles sampled with weights.

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def sample_from_population_with_weights(
particles: Tensor, weights: Tensor, num_samples: int = 1
) -> Tensor:
"""Return samples from particles sampled with weights."""

# define multinomial with weights as probs
multi = Multinomial(probs=weights)
# sample num samples, with replacement
samples = multi.sample(sample_shape=(num_samples,))
# get indices of success trials
indices = torch.where(samples)
# return those indices from trace
return particles[indices]


## Posteriors¶

###  sbi.inference.posteriors.direct_posterior.DirectPosterior (NeuralPosterior) ¶

Posterior $$p(\theta|x)$$ with log_prob() and sample() methods, obtained with SNPE.

SNPE trains a neural network to directly approximate the posterior distribution. However, for bounded priors, the neural network can have leakage: it puts non-zero mass in regions where the prior is zero. The DirectPosterior class wraps the trained network to deal with these cases.

Specifically, this class offers the following functionality:
- correct the calculation of the log probability such that it compensates for the leakage.
- reject samples that lie outside of the prior bounds.
- alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

#### default_x: Optional[torch.Tensor] inherited property writable ¶

Return default x used by .sample(), .log_prob as conditioning context.

#### mcmc_method: str inherited property writable ¶

Returns MCMC method.

#### mcmc_parameters: dict inherited property writable ¶

Returns MCMC parameters.

#### rejection_sampling_parameters: dict inherited property writable ¶

Returns rejection sampling parameters.

#### sample_with: str inherited property writable ¶

Return True if NeuralPosterior instance should use MCMC in .sample().

#### __init__(self, method_family, neural_net, prior, x_shape, sample_with='rejection', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, device='cpu') special ¶

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of a single simulator output.

required
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. With default parameters, rejection samples from the posterior estimated by the neural net and rejects only if the samples are outside of the prior support.

'rejection'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the trained neural net). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
Source code in sbi/inference/posteriors/direct_posterior.py
def __init__(
self,
method_family: str,
neural_net: nn.Module,
prior,
x_shape: torch.Size,
sample_with: str = "rejection",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
device: str = "cpu",
):
"""
Args:
method_family: One of snpe, snl, snre_a or snre_b.
neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
prior: Prior distribution with .log_prob() and .sample().
x_shape: Shape of a single simulator output.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection]. With default parameters, rejection samples
from the posterior estimated by the neural net and rejects only if the
samples are outside of the prior support.
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the trained
neural net). max_sampling_batch_size as the batchsize of samples
being drawn from the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio. m as multiplier to that ratio.
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
"""

kwargs = del_entries(locals(), entries=("self", "__class__"))
super().__init__(**kwargs)

self._purpose = (
"It allows to .sample() and .log_prob() the posterior and wraps the "
"output of the .net to avoid leakage into regions with 0 prior probability."
)


#### copy_hyperparameters_from(self, posterior) inherited ¶

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

• Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
• default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable

posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/direct_posterior.py
def copy_hyperparameters_from(
self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
"""
Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

- Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
- default_x at which to evaluate the posterior.

Args:
posterior: Posterior that the hyperparameters are copied from.

Returns:
Posterior object with the same hyperparameters as the passed posterior.
This makes the call chainable:
posterior = infer.build_posterior().copy_hyperparameters_from(proposal)
"""

assert isinstance(
posterior, NeuralPosterior
), "copy_state_from must be a NeuralPosterior."

self.set_mcmc_method(posterior._mcmc_method)
self.set_mcmc_parameters(posterior._mcmc_parameters)
self.set_default_x(posterior.default_x)
self._mcmc_init_params = posterior._mcmc_init_params
if hasattr(self, "_sample_with_mcmc"):
self.set_sample_with_mcmc(posterior._sample_with_mcmc)
if hasattr(self, "_rejection_sampling_parameters"):
self.set_rejection_sampling_parameters(
posterior._rejection_sampling_parameters
)

return self


#### leakage_correction(self, x, num_rejection_samples=10000, force_update=False, show_progress_bars=False, rejection_sampling_batch_size=10000)¶

Return leakage correction factor for a leaky posterior density estimate.

The factor is estimated from the acceptance probability during rejection sampling from the posterior.

This is to avoid re-estimating the acceptance probability from scratch whenever log_prob is called and norm_posterior=True. Here, it is estimated only once for self.default_x and saved for later. We re-evaluate only whenever a new x is passed.

Parameters:

Name Type Description Default
x Tensor

Conditioning context for posterior $$p(\theta|x)$$.

required
num_rejection_samples int

Number of samples used to estimate correction factor.

10000
force_update bool

Whether to force a reevaluation of the leakage correction even if the context x is the same as self.default_x. This is useful to enforce a new leakage estimate for rounds after the first (2, 3,..).

False
show_progress_bars bool

Whether to show a progress bar during sampling.

False
rejection_sampling_batch_size int

Batch size for rejection sampling.

10000

Returns:

Type Description
Tensor

Saved or newly-estimated correction factor (as a scalar Tensor).

Source code in sbi/inference/posteriors/direct_posterior.py
@torch.no_grad()
def leakage_correction(
self,
x: Tensor,
num_rejection_samples: int = 10_000,
force_update: bool = False,
show_progress_bars: bool = False,
rejection_sampling_batch_size: int = 10_000,
) -> Tensor:
r"""Return leakage correction factor for a leaky posterior density estimate.

The factor is estimated from the acceptance probability during rejection
sampling from the posterior.

This is to avoid re-estimating the acceptance probability from scratch
whenever log_prob is called and norm_posterior=True. Here, it
is estimated only once for self.default_x and saved for later. We
re-evaluate only whenever a new x is passed.

Arguments:
x: Conditioning context for posterior $p(\theta|x)$.
num_rejection_samples: Number of samples used to estimate correction factor.
force_update: Whether to force a reevaluation of the leakage correction even
if the context x is the same as self.default_x. This is useful to
enforce a new leakage estimate for rounds after the first (2, 3,..).
show_progress_bars: Whether to show a progress bar during sampling.
rejection_sampling_batch_size: Batch size for rejection sampling.

Returns:
Saved or newly-estimated correction factor (as a scalar Tensor).
"""

def acceptance_at(x: Tensor) -> Tensor:

return utils.rejection_sample_posterior_within_prior(
posterior_nn=self.net,
prior=self._prior,
x=x.to(self._device),
num_samples=num_rejection_samples,
show_progress_bars=show_progress_bars,
sample_for_correction_factor=True,
max_sampling_batch_size=rejection_sampling_batch_size,
)

# Check if the provided x matches the default x (short-circuit on identity).
is_new_x = self.default_x is None or (
x is not self.default_x and (x != self.default_x).any()
)

not_saved_at_default_x = self._leakage_density_correction_factor is None

if is_new_x:  # Calculate at x; don't save.
return acceptance_at(x)
elif not_saved_at_default_x or force_update:  # Calculate at default_x; save.
self._leakage_density_correction_factor = acceptance_at(self.default_x)

return self._leakage_density_correction_factor  # type:ignore


#### log_prob(self, theta, x=None, norm_posterior=True, track_gradients=False, leakage_correction_params=None)¶

Returns the log-probability of the posterior $$p(\theta|x).$$

Parameters:

Name Type Description Default
theta Tensor

Parameters $$\theta$$.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
norm_posterior bool

Whether to enforce a normalized posterior density. Renormalization of the posterior is useful when some probability falls out or leaks out of the prescribed prior support. The normalizing factor is calculated via rejection sampling, so if you need speedier but unnormalized log posterior estimates set here norm_posterior=False. The returned log posterior is set to -∞ outside of the prior support regardless of this setting.

True
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False
leakage_correction_params Optional[dict]

A dict of keyword arguments to override the default values of leakage_correction(). Possible options are: num_rejection_samples, force_update, show_progress_bars, and rejection_sampling_batch_size. These parameters only have an effect if norm_posterior=True.

None

Returns:

Type Description
Tensor

(len(θ),)-shaped log posterior probability $$\log p(\theta|x)$$ for θ in the support of the prior, -∞ (corresponding to 0 probability) outside.

Source code in sbi/inference/posteriors/direct_posterior.py
def log_prob(
self,
theta: Tensor,
x: Optional[Tensor] = None,
norm_posterior: bool = True,
leakage_correction_params: Optional[dict] = None,
) -> Tensor:
r"""
Returns the log-probability of the posterior $p(\theta|x).$

Args:
theta: Parameters $\theta$.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
norm_posterior: Whether to enforce a normalized posterior density.
Renormalization of the posterior is useful when some
probability falls out or leaks out of the prescribed prior support.
The normalizing factor is calculated via rejection sampling, so if you
need speedier but unnormalized log posterior estimates set here
norm_posterior=False. The returned log posterior is set to
-∞ outside of the prior support regardless of this setting.
This can be helpful for e.g. sensitivity analysis, but increases memory
consumption.
leakage_correction_params: A dict of keyword arguments to override the
default values of leakage_correction(). Possible options are:
num_rejection_samples, force_update, show_progress_bars, and
rejection_sampling_batch_size.
These parameters only have an effect if norm_posterior=True.

Returns:
(len(θ),)-shaped log posterior probability $\log p(\theta|x)$ for θ in the
support of the prior, -∞ (corresponding to 0 probability) outside.
"""

# TODO Train exited here, entered after sampling?
self.net.eval()

theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)
theta_repeated, x_repeated = self._match_theta_and_x_batch_shapes(theta, x)

# Evaluate on device, move back to cpu for comparison with prior.
unnorm_log_prob = self.net.log_prob(theta_repeated, x_repeated)

# Force probability to be zero outside prior support.
in_prior_support = within_support(self._prior, theta)

in_prior_support,
unnorm_log_prob,
torch.tensor(float("-inf"), dtype=torch.float32, device=self._device),
)

if leakage_correction_params is None:
leakage_correction_params = dict()  # use defaults
log_factor = (
log(
self.leakage_correction(
x=batched_first_of_batch(x), **leakage_correction_params
)
)
if norm_posterior
else 0
)



#### log_prob_conditional(self, theta, condition, dims_to_evaluate, x=None)¶

Evaluates the conditional posterior probability of a MDN at a context x for a value theta given a condition.

This function only works for MDN based posteriors, becuase evaluation is done analytically. For all other density estimators a NotImplementedError will be raised!

Parameters:

Name Type Description Default
theta Tensor

Parameters $heta$.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_evaluate List[int]

Which dimensions to evaluate the sample for. The dimensions not specified in dims_to_evaluate will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p( heta|x)$$. If not provided, fall back onto x passed to set_default_x().

None

Returns:

Type Description
log_prob

(len(θ),)-shaped normalized (!) log posterior probability $\log p( heta|x) for θ in the support of the prior, -∞ (corresponding to 0 probability) outside. Source code in sbi/inference/posteriors/direct_posterior.py def log_prob_conditional( self, theta: Tensor, condition: Tensor, dims_to_evaluate: List[int], x: Optional[Tensor] = None, ) -> Tensor: """Evaluates the conditional posterior probability of a MDN at a context x for a value theta given a condition. This function only works for MDN based posteriors, becuase evaluation is done analytically. For all other density estimators a NotImplementedError will be raised! Args: theta: Parameters$\theta$. condition: Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored. dims_to_evaluate: Which dimensions to evaluate the sample for. The dimensions not specified in dims_to_evaluate will be fixed to values given in condition. x: Conditioning context for posterior$p(\theta|x)$. If not provided, fall back onto x passed to set_default_x(). Returns: log_prob: (len(θ),)-shaped normalized (!) log posterior probability$\log p(\theta|x) for θ in the support of the prior, -∞ (corresponding
to 0 probability) outside.
"""

if type(self.net._distribution) == mdn:
logits, means, precfs, sumlogdiag = extract_and_transform_mog(self, x)
logits, means, precfs, sumlogdiag = condition_mog(
self._prior, condition, dims_to_evaluate, logits, means, precfs
)

batch_size, dim = theta.shape
prec = precfs.transpose(3, 2) @ precfs

self.net.eval()  # leakage correction requires eval mode

if dim != len(dims_to_evaluate):
X = X[:, dims_to_evaluate]

# Implementing leakage correction is difficult for conditioned MDNs,
# because samples from self i.e. the full posterior are used rather
# then from the new, conditioned posterior.
warn("Probabilities are not adjusted for leakage.")

log_prob = mdn.log_prob_mog(
theta,
logits.repeat(batch_size, 1),
means.repeat(batch_size, 1, 1),
prec.repeat(batch_size, 1, 1, 1),
sumlogdiag.repeat(batch_size, 1),
)

self.net.train(True)
return log_prob.detach()

else:
raise NotImplementedError(
"This functionality is only available for MDN based posteriors."
)


#### map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=1000, num_to_optimize=100, save_best_every=10, show_progress_bars=True)¶

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior $$p( heta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

1000
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/direct_posterior.py
def map(
self,
x: Optional[Tensor] = None,
num_iter: int = 1000,
learning_rate: float = 1e-2,
init_method: Union[str, Tensor] = "posterior",
num_init_samples: int = 1_000,
num_to_optimize: int = 100,
save_best_every: int = 10,
show_progress_bars: bool = True,
) -> Tensor:
"""
Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the
log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting
positions (samples from the posterior with the highest log-probability). After
the optimization is done, we select the parameter set that has the highest
log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They
might require hand-tuning for the problem at hand.

Args:
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
num_iter: Number of optimization steps that the algorithm takes
to find the MAP.
learning_rate: Learning rate of the optimizer.
init_method: How to select the starting parameters for the optimization. If
it is a string, it can be either [posterior, prior], which samples
the respective distribution num_init_samples times. If it is a,
the tensor will be used as init locations.
num_init_samples: Draw this number of samples from the posterior and
evaluate the log-probability of all of them.
num_to_optimize: From the drawn num_init_samples, use the
num_to_optimize with highest log-probability as the initial points
for the optimization.
save_best_every: The best log-probability is computed, saved in the
map-attribute, and printed every print_best_every-th iteration.
Computing the best log-probability creates a significant overhead
(thus, the default is 10.)
show_progress_bars: Whether or not to show a progressbar for sampling from
the posterior.

Returns:
The MAP estimate.
"""
return super().map(
x=x,
num_iter=num_iter,
learning_rate=learning_rate,
init_method=init_method,
num_init_samples=num_init_samples,
num_to_optimize=num_to_optimize,
save_best_every=save_best_every,
show_progress_bars=show_progress_bars,
log_prob_kwargs={"norm_posterior": False},
)


#### sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None, sample_with_mcmc=None)¶

Return samples from posterior distribution $$p(\theta|x)$$.

Samples are obtained either with rejection sampling or MCMC. Rejection sampling will be a lot faster if leakage is rather low. If leakage is high (e.g. over 99%, which can happen in multi-round SNPE), MCMC can be faster than rejection sampling.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with Optional[str]

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. With default parameters, rejection samples from the posterior estimated by the neural net and rejects only if the samples are outside of the prior support.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the trained neural net). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
sample_with_mcmc Optional[bool]

Deprecated since sbi v0.17.0. Use sample_with=mcmc instead.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
def sample(
self,
sample_shape: Shape = torch.Size(),
x: Optional[Tensor] = None,
show_progress_bars: bool = True,
sample_with: Optional[str] = None,
mcmc_method: Optional[str] = None,
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
sample_with_mcmc: Optional[bool] = None,
) -> Tensor:
r"""
Return samples from posterior distribution $p(\theta|x)$.

Samples are obtained either with rejection sampling or MCMC. Rejection sampling
will be a lot faster if leakage is rather low. If leakage is high (e.g. over
99%, which can happen in multi-round SNPE), MCMC can be faster than rejection
sampling.

Args:
sample_shape: Desired shape of samples that are drawn from posterior. If
sample_shape is multidimensional we simply draw sample_shape.numel()
samples and then reshape into the desired shape.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
show_progress_bars: Whether to show sampling progress monitor.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection]. With default parameters, rejection samples
from the posterior estimated by the neural net and rejects only if the
samples are outside of the prior support.
mcmc_method: Optional parameter to override self.mcmc_method.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported:
thin to set the thinning factor for the chain.
warmup_steps to set the initial number of samples to discard.
num_chains for the number of chains.
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
enable_transform a bool indicating whether MCMC is performed in
z-scored (and unconstrained) space.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the trained
neural net).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio.
m as multiplier to that ratio.
sample_with_mcmc: Deprecated since sbi v0.17.0. Use sample_with=mcmc

Returns:
Samples from posterior.
"""

if sample_with_mcmc is not None:
warn(
f"You set sample_with_mcmc={sample_with_mcmc}. This is deprecated "
"since sbi v0.17.0 and will lead to an error in future versions. "
"Please use sample_with='mcmc' instead."
)
if sample_with_mcmc:
sample_with = "mcmc"

self.net.eval()

sample_with = sample_with if sample_with is not None else self._sample_with

x, num_samples = self._prepare_for_sample(x, sample_shape)

potential_fn_provider = PotentialFunctionProvider()
if sample_with == "mcmc":
mcmc_method, mcmc_parameters = self._potentially_replace_mcmc_parameters(
mcmc_method, mcmc_parameters
)
transform = mcmc_transform(
self._prior, device=self._device, **mcmc_parameters
)
transformed_samples = self._sample_posterior_mcmc(
num_samples=num_samples,
potential_fn=potential_fn_provider(
self._prior, self.net, x, mcmc_method, transform
),
init_fn=self._build_mcmc_init_fn(
self._prior,
potential_fn_provider(
self._prior, self.net, x, "slice_np", transform
),
transform=transform,
**mcmc_parameters,
),
mcmc_method=mcmc_method,
show_progress_bars=show_progress_bars,
**mcmc_parameters,
)
samples = transform.inv(transformed_samples)
elif sample_with == "rejection":
rejection_sampling_parameters = (
self._potentially_replace_rejection_parameters(
rejection_sampling_parameters
)
)
if "proposal" not in rejection_sampling_parameters:
assert (
not self.net.training
), "Posterior nn must be in eval mode for sampling."

# If the user does not explictly pass a proposal, we sample from the
# neural net estimating the posterior and reject only those samples
# that are outside of the prior support. This can be considered as
# rejection sampling with a very good proposal.
samples = utils.rejection_sample_posterior_within_prior(
posterior_nn=self.net,
prior=self._prior,
x=x.to(self._device),
num_samples=num_samples,
show_progress_bars=show_progress_bars,
sample_for_correction_factor=True,
**rejection_sampling_parameters,
)
else:
samples, _ = rejection_sample(
potential_fn=potential_fn_provider(
self._prior, self.net, x, "rejection"
),
num_samples=num_samples,
show_progress_bars=show_progress_bars,
**rejection_sampling_parameters,
)
else:
raise NameError(
"The only implemented sampling methods are mcmc and rejection."
)

self.net.train(True)

return samples.reshape((*sample_shape, -1))


#### sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, sample_with='mcmc', show_progress_bars=True, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)¶

Return samples from conditional posterior $$p(\theta_i|\theta_j, x)$$.

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. In this method, the value of self.sample_with will be ignored.

'mcmc'
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
def sample_conditional(
self,
sample_shape: Shape,
condition: Tensor,
dims_to_sample: List[int],
x: Optional[Tensor] = None,
sample_with: str = "mcmc",
show_progress_bars: bool = True,
mcmc_method: Optional[str] = None,
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
r"""
Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

In this function, we do not sample from the full posterior, but instead only
from a few parameter dimensions while the other parameter dimensions are kept
fixed at values specified in condition.

Samples are obtained with MCMC.

Args:
sample_shape: Desired shape of samples that are drawn from posterior. If
sample_shape is multidimensional we simply draw sample_shape.numel()
samples and then reshape into the desired shape.
condition: Parameter set that all dimensions not specified in
dims_to_sample will be fixed to. Should contain dim_theta elements,
i.e. it could e.g. be a sample from the posterior distribution.
The entries at all dims_to_sample will be ignored.
dims_to_sample: Which dimensions to sample from. The dimensions not
specified in dims_to_sample will be fixed to values given in
condition.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection]. In this method, the value of
self.sample_with will be ignored.
show_progress_bars: Whether to show sampling progress monitor.
mcmc_method: Optional parameter to override self.mcmc_method.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported:
thin to set the thinning factor for the chain.
warmup_steps to set the initial number of samples to discard.
num_chains for the number of chains.
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
enable_transform a bool indicating whether MCMC is performed in
z-scored (and unconstrained) space.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio.
m as multiplier to that ratio.

Returns:
Samples from conditional posterior.
"""
if not hasattr(self.net, "_distribution"):
raise NotImplementedError(
"sample_conditional is not implemented for SNPE-A."
)

net = self.net
x = atleast_2d_float32_tensor(self._x_else_default_x(x))

if type(net._distribution) is mdn:
condition = atleast_2d_float32_tensor(condition)
num_samples = torch.Size(sample_shape).numel()

logits, means, precfs, _ = extract_and_transform_mog(nn=net, context=x)
logits, means, precfs, _ = condition_mog(
self._prior, condition, dims_to_sample, logits, means, precfs
)

# Currently difficult to integrate sample_posterior_within_prior.
warn(
"Sampling MoG analytically. "
"Some of the samples might not be within the prior support!"
)
samples = mdn.sample_mog(num_samples, logits, means, precfs)
return samples.detach().reshape((*sample_shape, -1))

else:
return super().sample_conditional(
PotentialFunctionProvider(),
sample_shape,
condition,
dims_to_sample,
x,
sample_with,
show_progress_bars,
mcmc_method,
mcmc_parameters,
rejection_sampling_parameters,
)


#### set_default_x(self, x) inherited ¶

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior $$p(theta|x)$$.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
"""Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to
.sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e.
has been trained over multiple rounds to be accurate in the vicinity of a
particular x=x_o (you can check if your posterior object is focused by
printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Args:
x: The default observation to set for the posterior $p(theta|x)$.

Returns:
NeuralPosterior that will use a default x when not explicitly passed.
"""
self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x).to(
self._device
)
self._num_iid_trials = self._x.shape

return self


#### set_mcmc_method(self, method) inherited ¶

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
"""Sets sampling method to for MCMC and returns NeuralPosterior.

Args:
method: Method to use.

Returns:
NeuralPosterior for chainable calls.
"""
self._mcmc_method = method
return self


#### set_mcmc_parameters(self, parameters) inherited ¶

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
"""Sets parameters for MCMC and returns NeuralPosterior.

Args:
parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.

Returns:
NeuralPosterior for chainable calls.
"""
self._mcmc_parameters = parameters
return self


#### set_rejection_sampling_parameters(self, parameters) inherited ¶

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. m as multiplier to that ratio. sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_rejection_sampling_parameters(
self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
"""Sets parameters for rejection sampling and returns NeuralPosterior.

Args:
parameters: Dictonary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution. num_samples_to_find_max
as the number of samples that are used to find the maximum of the
potential_fn / proposal ratio. m as multiplier to that ratio.
sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.

Returns:
NeuralPosterior for chainable calls.
"""
self._rejection_sampling_parameters = parameters
return self


#### set_sample_with(self, sample_with) inherited ¶

Set the sampling method for the NeuralPosterior.

Parameters:

Name Type Description Default
sample_with str

The method to sample with.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_sample_with(self, sample_with: str) -> "NeuralPosterior":
"""Set the sampling method for the NeuralPosterior.

Args:
sample_with: The method to sample with.

Returns:
NeuralPosterior for chainable calls.

Raises:
ValueError: on attempt to turn off MCMC sampling for family of methods that
do not support rejection sampling.
"""
if sample_with not in ("mcmc", "rejection"):
raise NameError(
"The only implemented sampling methods are mcmc and rejection."
)
self._sample_with = sample_with
return self


###  sbi.inference.posteriors.likelihood_based_posterior.LikelihoodBasedPosterior (NeuralPosterior) ¶

Posterior $$p(\theta|x)$$ with log_prob() and sample() methods, obtained with SNLE.

SNLE trains a neural network to approximate the likelihood $$p(x|\theta)$$. The SNLE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$ and draw samples from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

#### default_x: Optional[torch.Tensor] inherited property writable ¶

Return default x used by .sample(), .log_prob as conditioning context.

#### mcmc_method: str inherited property writable ¶

Returns MCMC method.

#### mcmc_parameters: dict inherited property writable ¶

Returns MCMC parameters.

#### rejection_sampling_parameters: dict inherited property writable ¶

Returns rejection sampling parameters.

#### sample_with: str inherited property writable ¶

Return True if NeuralPosterior instance should use MCMC in .sample().

#### __init__(self, method_family, neural_net, prior, x_shape, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, device='cpu') special ¶

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations.

required
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def __init__(
self,
method_family: str,
neural_net: nn.Module,
prior,
x_shape: torch.Size,
sample_with: str = "mcmc",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
device: str = "cpu",
):
"""
Args:
method_family: One of snpe, snl, snre_a or snre_b.
neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
prior: Prior distribution with .log_prob() and .sample().
x_shape: Shape of the simulated data. It can differ from the
observed data the posterior is conditioned on later in the batch
dimension. If it differs, the additional entries are interpreted as
independent and identically distributed data / trials. I.e., the data is
assumed to be generated based on the same (unknown) model parameters or
experimental condations.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration. num_samples_to_find_max as the
number of samples that are used to find the maximum of the
potential_fn / proposal ratio. num_iter_to_find_max as the number
of gradient ascent iterations to find the maximum of that ratio. m as
multiplier to that ratio.
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
"""

kwargs = del_entries(locals(), entries=("self", "__class__"))
super().__init__(**kwargs)

self._purpose = (
"It provides MCMC to .sample() from the posterior and "
"can evaluate the _unnormalized_ posterior density with .log_prob()."
)


#### copy_hyperparameters_from(self, posterior) inherited ¶

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

• Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
• default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable

posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def copy_hyperparameters_from(
self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
"""
Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

- Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
- default_x at which to evaluate the posterior.

Args:
posterior: Posterior that the hyperparameters are copied from.

Returns:
Posterior object with the same hyperparameters as the passed posterior.
This makes the call chainable:
posterior = infer.build_posterior().copy_hyperparameters_from(proposal)
"""

assert isinstance(
posterior, NeuralPosterior
), "copy_state_from must be a NeuralPosterior."

self.set_mcmc_method(posterior._mcmc_method)
self.set_mcmc_parameters(posterior._mcmc_parameters)
self.set_default_x(posterior.default_x)
self._mcmc_init_params = posterior._mcmc_init_params
if hasattr(self, "_sample_with_mcmc"):
self.set_sample_with_mcmc(posterior._sample_with_mcmc)
if hasattr(self, "_rejection_sampling_parameters"):
self.set_rejection_sampling_parameters(
posterior._rejection_sampling_parameters
)

return self


#### log_prob(self, theta, x=None, track_gradients=False)¶

Returns the log-probability of $$p(x|\theta) \cdot p(\theta).$$

This corresponds to an unnormalized posterior log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters $$\theta$$.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability $$\log(p(x|\theta) \cdot p(\theta))$$.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def log_prob(
self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False
) -> Tensor:
r"""
Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

This corresponds to an **unnormalized** posterior log-probability.

Args:
theta: Parameters $\theta$.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
This can be helpful for e.g. sensitivity analysis, but increases memory
consumption.

Returns:
(len(θ),)-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

"""

# TODO Train exited here, entered after sampling?
self.net.eval()

theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)

# Calculate likelihood over trials and in one batch.
warn(
"The log probability from SNL is only correct up to a normalizing constant."
)
log_likelihood_trial_sum = self._log_likelihoods_over_trials(
x=x.to(self._device),
theta=theta.to(self._device),
net=self.net,
)

# Move to cpu for comparison with prior.
return log_likelihood_trial_sum + self._prior.log_prob(theta)


#### map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)¶

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior $$p( heta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def map(
self,
x: Optional[Tensor] = None,
num_iter: int = 1000,
learning_rate: float = 1e-2,
init_method: Union[str, Tensor] = "posterior",
num_init_samples: int = 500,
num_to_optimize: int = 100,
save_best_every: int = 10,
show_progress_bars: bool = True,
) -> Tensor:
"""
Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the
log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting
positions (samples from the posterior with the highest log-probability). After
the optimization is done, we select the parameter set that has the highest
log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They
might require hand-tuning for the problem at hand.

Args:
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
num_iter: Number of optimization steps that the algorithm takes
to find the MAP.
learning_rate: Learning rate of the optimizer.
init_method: How to select the starting parameters for the optimization. If
it is a string, it can be either [posterior, prior], which samples
the respective distribution num_init_samples times. If it is a,
the tensor will be used as init locations.
num_init_samples: Draw this number of samples from the posterior and
evaluate the log-probability of all of them.
num_to_optimize: From the drawn num_init_samples, use the
num_to_optimize with highest log-probability as the initial points
for the optimization.
save_best_every: The best log-probability is computed, saved in the
map-attribute, and printed every print_best_every-th iteration.
Computing the best log-probability creates a significant overhead
(thus, the default is 10.)
show_progress_bars: Whether or not to show a progressbar for sampling from
the posterior.

Returns:
The MAP estimate.
"""
return super().map(
x=x,
num_iter=num_iter,
learning_rate=learning_rate,
init_method=init_method,
num_init_samples=num_init_samples,
num_to_optimize=num_to_optimize,
save_best_every=save_best_every,
show_progress_bars=show_progress_bars,
)


#### sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)¶

Return samples from posterior distribution $$p(\theta|x)$$ with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with Optional[str]

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def sample(
self,
sample_shape: Shape = torch.Size(),
x: Optional[Tensor] = None,
show_progress_bars: bool = True,
sample_with: Optional[str] = None,
mcmc_method: Optional[str] = None,
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
r"""
Return samples from posterior distribution $p(\theta|x)$ with MCMC.

Args:
sample_shape: Desired shape of samples that are drawn from posterior. If
sample_shape is multidimensional we simply draw sample_shape.numel()
samples and then reshape into the desired shape.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
show_progress_bars: Whether to show sampling progress monitor.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Optional parameter to override self.mcmc_method.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported:
thin to set the thinning factor for the chain.
warmup_steps to set the initial number of samples to discard.
num_chains for the number of chains.
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
enable_transform a bool indicating whether MCMC is performed in
z-scored (and unconstrained) space.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio.
m as multiplier to that ratio.

Returns:
Samples from posterior.
"""

self.net.eval()

sample_with = sample_with if sample_with is not None else self._sample_with
x, num_samples = self._prepare_for_sample(x, sample_shape)

potential_fn_provider = PotentialFunctionProvider()
if sample_with == "mcmc":

mcmc_method, mcmc_parameters = self._potentially_replace_mcmc_parameters(
mcmc_method, mcmc_parameters
)

transform = mcmc_transform(
self._prior, device=self._device, **mcmc_parameters
)

transformed_samples = self._sample_posterior_mcmc(
num_samples=num_samples,
potential_fn=potential_fn_provider(
self._prior, self.net, x, mcmc_method, transform
),
init_fn=self._build_mcmc_init_fn(
self._prior,
potential_fn_provider(
self._prior, self.net, x, "slice_np", transform
),
transform=transform,
**mcmc_parameters,
),
mcmc_method=mcmc_method,
show_progress_bars=show_progress_bars,
**mcmc_parameters,
)
samples = transform.inv(transformed_samples)
elif sample_with == "rejection":
rejection_sampling_parameters = (
self._potentially_replace_rejection_parameters(
rejection_sampling_parameters
)
)
if "proposal" not in rejection_sampling_parameters:
rejection_sampling_parameters["proposal"] = self._prior

samples, _ = rejection_sample(
potential_fn=potential_fn_provider(
self._prior,
self.net,
x,
"rejection",
),
num_samples=num_samples,
**rejection_sampling_parameters,
)
else:
raise NameError(
"The only implemented sampling methods are mcmc and rejection."
)

self.net.train(True)

return samples.reshape((*sample_shape, -1))


#### sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, sample_with='mcmc', show_progress_bars=True, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)¶

Return samples from conditional posterior $$p(\theta_i|\theta_j, x)$$.

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. In this method, the value of self._sample_with will be ignored.

'mcmc'
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def sample_conditional(
self,
sample_shape: Shape,
condition: Tensor,
dims_to_sample: List[int],
x: Optional[Tensor] = None,
sample_with: str = "mcmc",
show_progress_bars: bool = True,
mcmc_method: Optional[str] = None,
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
r"""
Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

In this function, we do not sample from the full posterior, but instead only
from a few parameter dimensions while the other parameter dimensions are kept
fixed at values specified in condition.

Samples are obtained with MCMC.

Args:
sample_shape: Desired shape of samples that are drawn from posterior. If
sample_shape is multidimensional we simply draw sample_shape.numel()
samples and then reshape into the desired shape.
condition: Parameter set that all dimensions not specified in
dims_to_sample will be fixed to. Should contain dim_theta elements,
i.e. it could e.g. be a sample from the posterior distribution.
The entries at all dims_to_sample will be ignored.
dims_to_sample: Which dimensions to sample from. The dimensions not
specified in dims_to_sample will be fixed to values given in
condition.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection]. In this method, the value of
self._sample_with will be ignored.
show_progress_bars: Whether to show sampling progress monitor.
mcmc_method: Optional parameter to override self.mcmc_method.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported:
thin to set the thinning factor for the chain.
warmup_steps to set the initial number of samples to discard.
num_chains for the number of chains.
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
enable_transform a bool indicating whether MCMC is performed in
z-scored (and unconstrained) space.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio.
m as multiplier to that ratio.

Returns:
Samples from conditional posterior.
"""

return super().sample_conditional(
PotentialFunctionProvider(),
sample_shape,
condition,
dims_to_sample,
x,
sample_with,
show_progress_bars,
mcmc_method,
mcmc_parameters,
rejection_sampling_parameters,
)


#### set_default_x(self, x) inherited ¶

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior $$p(theta|x)$$.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
"""Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to
.sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e.
has been trained over multiple rounds to be accurate in the vicinity of a
particular x=x_o (you can check if your posterior object is focused by
printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Args:
x: The default observation to set for the posterior $p(theta|x)$.

Returns:
NeuralPosterior that will use a default x when not explicitly passed.
"""
self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x).to(
self._device
)
self._num_iid_trials = self._x.shape

return self


#### set_mcmc_method(self, method) inherited ¶

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
"""Sets sampling method to for MCMC and returns NeuralPosterior.

Args:
method: Method to use.

Returns:
NeuralPosterior for chainable calls.
"""
self._mcmc_method = method
return self


#### set_mcmc_parameters(self, parameters) inherited ¶

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
"""Sets parameters for MCMC and returns NeuralPosterior.

Args:
parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.

Returns:
NeuralPosterior for chainable calls.
"""
self._mcmc_parameters = parameters
return self


#### set_rejection_sampling_parameters(self, parameters) inherited ¶

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. m as multiplier to that ratio. sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_rejection_sampling_parameters(
self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
"""Sets parameters for rejection sampling and returns NeuralPosterior.

Args:
parameters: Dictonary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution. num_samples_to_find_max
as the number of samples that are used to find the maximum of the
potential_fn / proposal ratio. m as multiplier to that ratio.
sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.

Returns:
NeuralPosterior for chainable calls.
"""
self._rejection_sampling_parameters = parameters
return self


#### set_sample_with(self, sample_with) inherited ¶

Set the sampling method for the NeuralPosterior.

Parameters:

Name Type Description Default
sample_with str

The method to sample with.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_sample_with(self, sample_with: str) -> "NeuralPosterior":
"""Set the sampling method for the NeuralPosterior.

Args:
sample_with: The method to sample with.

Returns:
NeuralPosterior for chainable calls.

Raises:
ValueError: on attempt to turn off MCMC sampling for family of methods that
do not support rejection sampling.
"""
if sample_with not in ("mcmc", "rejection"):
raise NameError(
"The only implemented sampling methods are mcmc and rejection."
)
self._sample_with = sample_with
return self


###  sbi.inference.posteriors.ratio_based_posterior.RatioBasedPosterior (NeuralPosterior) ¶

Posterior $$p(\theta|x)$$ with log_prob() and sample() methods, obtained with SNRE.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$. The SNRE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability $$p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$$ and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

The neural network itself can be accessed via the .net attribute.

#### default_x: Optional[torch.Tensor] inherited property writable ¶

Return default x used by .sample(), .log_prob as conditioning context.

#### mcmc_method: str inherited property writable ¶

Returns MCMC method.

#### mcmc_parameters: dict inherited property writable ¶

Returns MCMC parameters.

#### rejection_sampling_parameters: dict inherited property writable ¶

Returns rejection sampling parameters.

#### sample_with: str inherited property writable ¶

Return True if NeuralPosterior instance should use MCMC in .sample().

#### __init__(self, method_family, neural_net, prior, x_shape, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, device='cpu') special ¶

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations.

required
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
Source code in sbi/inference/posteriors/ratio_based_posterior.py
def __init__(
self,
method_family: str,
neural_net: nn.Module,
prior,
x_shape: torch.Size,
sample_with: str = "mcmc",
mcmc_method: str = "slice_np",
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
device: str = "cpu",
):
"""
Args:
method_family: One of snpe, snl, snre_a or snre_b.
neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
prior: Prior distribution with .log_prob() and .sample().
x_shape: Shape of the simulated data. It can differ from the
observed data the posterior is conditioned on later in the batch
dimension. If it differs, the additional entries are interpreted as
independent and identically distributed data / trials. I.e., the data is
assumed to be generated based on the same (unknown) model parameters or
experimental condations.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Method used for MCMC sampling, one of slice_np, slice,
hmc, nuts. Currently defaults to slice_np for a custom numpy
implementation of slice sampling; select hmc, nuts or slice for
Pyro-based sampling.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration. num_samples_to_find_max as the
number of samples that are used to find the maximum of the
potential_fn / proposal ratio. num_iter_to_find_max as the number
of gradient ascent iterations to find the maximum of that ratio. m as
multiplier to that ratio.
device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
"""
kwargs = del_entries(locals(), entries=("self", "__class__"))
super().__init__(**kwargs)


#### copy_hyperparameters_from(self, posterior) inherited ¶

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

• Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
• default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable

posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def copy_hyperparameters_from(
self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
"""
Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

- Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
- default_x at which to evaluate the posterior.

Args:
posterior: Posterior that the hyperparameters are copied from.

Returns:
Posterior object with the same hyperparameters as the passed posterior.
This makes the call chainable:
posterior = infer.build_posterior().copy_hyperparameters_from(proposal)
"""

assert isinstance(
posterior, NeuralPosterior
), "copy_state_from must be a NeuralPosterior."

self.set_mcmc_method(posterior._mcmc_method)
self.set_mcmc_parameters(posterior._mcmc_parameters)
self.set_default_x(posterior.default_x)
self._mcmc_init_params = posterior._mcmc_init_params
if hasattr(self, "_sample_with_mcmc"):
self.set_sample_with_mcmc(posterior._sample_with_mcmc)
if hasattr(self, "_rejection_sampling_parameters"):
self.set_rejection_sampling_parameters(
posterior._rejection_sampling_parameters
)

return self


#### log_prob(self, theta, x=None, track_gradients=False)¶

Returns the log-probability of $$p(x|\theta) \cdot p(\theta).$$

This corresponds to an unnormalized posterior log-probability. Only for single-round SNRE_A / AALR, the returned log-probability will correspond to the normalized log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters $$\theta$$.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability $$\log(p(x|\theta) \cdot p(\theta))$$.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def log_prob(
self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False
) -> Tensor:
r"""Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

This corresponds to an **unnormalized** posterior log-probability. Only for
single-round SNRE_A / AALR, the returned log-probability will correspond to the
**normalized** log-probability.

Args:
theta: Parameters $\theta$.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
This can be helpful for e.g. sensitivity analysis, but increases memory
consumption.

Returns:
(len(θ),)-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

"""

# TODO Train exited here, entered after sampling?
self.net.eval()

theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)

self._warn_log_prob_snre()

# Sum log ratios over x batch of iid trials.
log_ratio = self._log_ratios_over_trials(
x.to(self._device),
theta.to(self._device),
self.net,
)

return log_ratio + self._prior.log_prob(theta)


#### map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)¶

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior $$p( heta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Maximum Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def map(
self,
x: Optional[Tensor] = None,
num_iter: int = 1000,
learning_rate: float = 1e-2,
init_method: Union[str, Tensor] = "posterior",
num_init_samples: int = 500,
num_to_optimize: int = 100,
save_best_every: int = 10,
show_progress_bars: bool = True,
) -> Tensor:
"""
Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the
log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting
positions (samples from the posterior with the highest log-probability). After
the optimization is done, we select the parameter set that has the highest
log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They
might require hand-tuning for the problem at hand.

Args:
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
num_iter: Maximum Number of optimization steps that the algorithm takes
to find the MAP.
learning_rate: Learning rate of the optimizer.
init_method: How to select the starting parameters for the optimization. If
it is a string, it can be either [posterior, prior], which samples
the respective distribution num_init_samples times. If it is a,
the tensor will be used as init locations.
num_init_samples: Draw this number of samples from the posterior and
evaluate the log-probability of all of them.
num_to_optimize: From the drawn num_init_samples, use the
num_to_optimize with highest log-probability as the initial points
for the optimization.
save_best_every: The best log-probability is computed, saved in the
map-attribute, and printed every print_best_every-th iteration.
Computing the best log-probability creates a significant overhead
(thus, the default is 10.)
show_progress_bars: Whether or not to show a progressbar for sampling from
the posterior.

Returns:
The MAP estimate.
"""
return super().map(
x=x,
num_iter=num_iter,
learning_rate=learning_rate,
init_method=init_method,
num_init_samples=num_init_samples,
num_to_optimize=num_to_optimize,
save_best_every=save_best_every,
show_progress_bars=show_progress_bars,
)


#### sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)¶

Return samples from posterior distribution $$p(\theta|x)$$ with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with Optional[str]

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def sample(
self,
sample_shape: Shape = torch.Size(),
x: Optional[Tensor] = None,
show_progress_bars: bool = True,
sample_with: Optional[str] = None,
mcmc_method: Optional[str] = None,
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
r"""
Return samples from posterior distribution $p(\theta|x)$ with MCMC.

Args:
sample_shape: Desired shape of samples that are drawn from posterior. If
sample_shape is multidimensional we simply draw sample_shape.numel()
samples and then reshape into the desired shape.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
show_progress_bars: Whether to show sampling progress monitor.
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection].
mcmc_method: Optional parameter to override self.mcmc_method.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported:
thin to set the thinning factor for the chain.
warmup_steps to set the initial number of samples to discard.
num_chains for the number of chains.
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
enable_transform a bool indicating whether MCMC is performed in
z-scored (and unconstrained) space.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio.
m as multiplier to that ratio.

Returns:
Samples from posterior.
"""

self.net.eval()

sample_with = sample_with if sample_with is not None else self._sample_with
x, num_samples = self._prepare_for_sample(x, sample_shape)

potential_fn_provider = PotentialFunctionProvider()
if sample_with == "mcmc":
mcmc_method, mcmc_parameters = self._potentially_replace_mcmc_parameters(
mcmc_method, mcmc_parameters
)

transform = mcmc_transform(
self._prior, device=self._device, **mcmc_parameters
)

transformed_samples = self._sample_posterior_mcmc(
num_samples=num_samples,
potential_fn=potential_fn_provider(
self._prior, self.net, x, mcmc_method, transform
),
init_fn=self._build_mcmc_init_fn(
self._prior,
potential_fn_provider(
self._prior, self.net, x, "slice_np", transform
),
transform=transform,
**mcmc_parameters,
),
mcmc_method=mcmc_method,
show_progress_bars=show_progress_bars,
**mcmc_parameters,
)
samples = transform.inv(transformed_samples)
elif sample_with == "rejection":
rejection_sampling_parameters = (
self._potentially_replace_rejection_parameters(
rejection_sampling_parameters
)
)
if "proposal" not in rejection_sampling_parameters:
rejection_sampling_parameters["proposal"] = self._prior

samples, _ = rejection_sample(
potential_fn=potential_fn_provider(
self._prior, self.net, x, "rejection"
),
num_samples=num_samples,
**rejection_sampling_parameters,
)
else:
raise NameError(
"The only implemented sampling methods are mcmc and rejection."
)

self.net.train(True)

return samples.reshape((*sample_shape, -1))


#### sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, sample_with='mcmc', show_progress_bars=True, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)¶

Return samples from conditional posterior $$p(\theta_i|\theta_j, x)$$.

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior $$p(\theta|x)$$. If not provided, fall back onto x passed to set_default_x().

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. In this method, the value of self._sample_with will be ignored.

'mcmc'
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def sample_conditional(
self,
sample_shape: Shape,
condition: Tensor,
dims_to_sample: List[int],
x: Optional[Tensor] = None,
sample_with: str = "mcmc",
show_progress_bars: bool = True,
mcmc_method: Optional[str] = None,
mcmc_parameters: Optional[Dict[str, Any]] = None,
rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
r"""
Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

In this function, we do not sample from the full posterior, but instead only
from a few parameter dimensions while the other parameter dimensions are kept
fixed at values specified in condition.

Samples are obtained with MCMC.

Args:
sample_shape: Desired shape of samples that are drawn from posterior. If
sample_shape is multidimensional we simply draw sample_shape.numel()
samples and then reshape into the desired shape.
condition: Parameter set that all dimensions not specified in
dims_to_sample will be fixed to. Should contain dim_theta elements,
i.e. it could e.g. be a sample from the posterior distribution.
The entries at all dims_to_sample will be ignored.
dims_to_sample: Which dimensions to sample from. The dimensions not
specified in dims_to_sample will be fixed to values given in
condition.
x: Conditioning context for posterior $p(\theta|x)$. If not provided,
fall back onto x passed to set_default_x().
sample_with: Method to use for sampling from the posterior. Must be one of
[mcmc | rejection]. In this method, the value of
self._sample_with will be ignored.
show_progress_bars: Whether to show sampling progress monitor.
mcmc_method: Optional parameter to override self.mcmc_method.
mcmc_parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported:
thin to set the thinning factor for the chain.
warmup_steps to set the initial number of samples to discard.
num_chains for the number of chains.
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.
enable_transform a bool indicating whether MCMC is performed in
z-scored (and unconstrained) space.
rejection_sampling_parameters: Dictionary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution (default is the prior).
max_sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.
num_samples_to_find_max as the number of samples that are used to
find the maximum of the potential_fn / proposal ratio.
num_iter_to_find_max as the number of gradient ascent iterations to
find the maximum of that ratio.
m as multiplier to that ratio.

Returns:
Samples from conditional posterior.
"""

return super().sample_conditional(
PotentialFunctionProvider(),
sample_shape,
condition,
dims_to_sample,
x,
sample_with,
show_progress_bars,
mcmc_method,
mcmc_parameters,
rejection_sampling_parameters,
)


#### set_default_x(self, x) inherited ¶

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior $$p(theta|x)$$.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
"""Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to
.sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e.
has been trained over multiple rounds to be accurate in the vicinity of a
particular x=x_o (you can check if your posterior object is focused by
printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Args:
x: The default observation to set for the posterior $p(theta|x)$.

Returns:
NeuralPosterior that will use a default x when not explicitly passed.
"""
self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x).to(
self._device
)
self._num_iid_trials = self._x.shape

return self


#### set_mcmc_method(self, method) inherited ¶

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
"""Sets sampling method to for MCMC and returns NeuralPosterior.

Args:
method: Method to use.

Returns:
NeuralPosterior for chainable calls.
"""
self._mcmc_method = method
return self


#### set_mcmc_parameters(self, parameters) inherited ¶

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
"""Sets parameters for MCMC and returns NeuralPosterior.

Args:
parameters: Dictionary overriding the default parameters for MCMC.
The following parameters are supported: thin to set the thinning
factor for the chain, warmup_steps to set the initial number of
samples to discard, num_chains for the number of chains,
init_strategy for the initialisation strategy for chains; prior
will draw init locations from prior, whereas sir will use Sequential-
Importance-Resampling using init_strategy_num_candidates to find init
locations.

Returns:
NeuralPosterior for chainable calls.
"""
self._mcmc_parameters = parameters
return self


#### set_rejection_sampling_parameters(self, parameters) inherited ¶

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. m as multiplier to that ratio. sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_rejection_sampling_parameters(
self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
"""Sets parameters for rejection sampling and returns NeuralPosterior.

Args:
parameters: Dictonary overriding the default parameters
for rejection sampling. The following parameters are supported:
proposal as the proposal distribtution. num_samples_to_find_max
as the number of samples that are used to find the maximum of the
potential_fn / proposal ratio. m as multiplier to that ratio.
sampling_batch_size as the batchsize of samples being drawn from
the proposal at every iteration.

Returns:
NeuralPosterior for chainable calls.
"""
self._rejection_sampling_parameters = parameters
return self


#### set_sample_with(self, sample_with) inherited ¶

Set the sampling method for the NeuralPosterior.

Parameters:

Name Type Description Default
sample_with str

The method to sample with.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_sample_with(self, sample_with: str) -> "NeuralPosterior":
"""Set the sampling method for the NeuralPosterior.

Args:
sample_with: The method to sample with.

Returns:
NeuralPosterior for chainable calls.

Raises:
ValueError: on attempt to turn off MCMC sampling for family of methods that
do not support rejection sampling.
"""
if sample_with not in ("mcmc", "rejection"):
raise NameError(
"The only implemented sampling methods are mcmc and rejection."
)
self._sample_with = sample_with
return self


## Models¶

### sbi.utils.get_nn_models.posterior_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10)¶

Returns a function that builds a density estimator for learning the posterior.

This function will usually be used for SNPE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters $$\theta$$ before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs $$x$$ before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for simulation outputs $$x$$. This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
def posterior_nn(
model: str,
z_score_theta: bool = True,
z_score_x: bool = True,
hidden_features: int = 50,
num_transforms: int = 5,
num_bins: int = 10,
embedding_net: nn.Module = nn.Identity(),
num_components: int = 10,
) -> Callable:
r"""
Returns a function that builds a density estimator for learning the posterior.

This function will usually be used for SNPE. The returned function is to be passed
to the inference class when using the flexible interface.

Args:
model: The type of density estimator that will be created. One of [mdn,
made, maf, nsf].
z_score_theta: Whether to z-score parameters $\theta$ before passing them into
the network.
z_score_x: Whether to z-score simulation outputs $x$ before passing them into
the network.
hidden_features: Number of hidden features.
num_transforms: Number of transforms when a flow is used. Only relevant if
density estimator is a normalizing flow (i.e. currently either a maf or a
nsf). Ignored if density estimator is a mdn or made.
num_bins: Number of bins used for the splines in nsf. Ignored if density
estimator not nsf.
embedding_net: Optional embedding network for simulation outputs $x$. This
embedding net allows to learn features from potentially high-dimensional
simulation outputs.
num_components: Number of mixture components for a mixture of Gaussians.
Ignored if density estimator is not an mdn.
"""

kwargs = dict(
zip(
(
"z_score_x",
"z_score_y",
"hidden_features",
"num_transforms",
"num_bins",
"embedding_net",
"num_components",
),
(
z_score_theta,
z_score_x,
hidden_features,
num_transforms,
num_bins,
embedding_net,
num_components,
),
)
)

if model == "mdn_snpe_a":
if num_components != 10:
raise ValueError(
"You set num_components. For SNPE-A, this has to be done at "
"instantiation of the inference object, i.e. "
"inference = SNPE_A(..., num_components=20)"
)
kwargs.pop("num_components")

def build_fn(batch_theta, batch_x, num_components):
# Extract the number of components from the kwargs, such that
# they are exposed as a kwargs, offering the possibility to later
# override this kwarg with functools.partial. This is necessary
# in order to make sure that the MDN in SNPE-A only has one
# component when running the Algorithm 1 part.
return build_mdn(
batch_x=batch_theta,
batch_y=batch_x,
num_components=num_components,
**kwargs
)

else:

def build_fn(batch_theta, batch_x):
if model == "mdn":
return build_mdn(batch_x=batch_theta, batch_y=batch_x, **kwargs)
elif model == "maf":
return build_maf(batch_x=batch_theta, batch_y=batch_x, **kwargs)
elif model == "nsf":
return build_nsf(batch_x=batch_theta, batch_y=batch_x, **kwargs)
else:
raise NotImplementedError

return build_fn


### sbi.utils.get_nn_models.likelihood_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10)¶

Returns a function that builds a density estimator for learning the likelihood.

This function will usually be used for SNLE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters $$\theta$$ before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs $$x$$ before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for parameters $$\theta$$.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
def likelihood_nn(
model: str,
z_score_theta: bool = True,
z_score_x: bool = True,
hidden_features: int = 50,
num_transforms: int = 5,
num_bins: int = 10,
embedding_net: nn.Module = nn.Identity(),
num_components: int = 10,
) -> Callable:
r"""
Returns a function that builds a density estimator for learning the likelihood.

This function will usually be used for SNLE. The returned function is to be passed
to the inference class when using the flexible interface.

Args:
model: The type of density estimator that will be created. One of [mdn,
made, maf, nsf].
z_score_theta: Whether to z-score parameters $\theta$ before passing them into
the network.
z_score_x: Whether to z-score simulation outputs $x$ before passing them into
the network.
hidden_features: Number of hidden features.
num_transforms: Number of transforms when a flow is used. Only relevant if
density estimator is a normalizing flow (i.e. currently either a maf or a
nsf). Ignored if density estimator is a mdn or made.
num_bins: Number of bins used for the splines in nsf. Ignored if density
estimator not nsf.
embedding_net: Optional embedding network for parameters $\theta$.
num_components: Number of mixture components for a mixture of Gaussians.
Ignored if density estimator is not an mdn.
"""

kwargs = dict(
zip(
(
"z_score_x",
"z_score_y",
"hidden_features",
"num_transforms",
"num_bins",
"embedding_net",
"num_components",
),
(
z_score_x,
z_score_theta,
hidden_features,
num_transforms,
num_bins,
embedding_net,
num_components,
),
)
)

def build_fn(batch_theta, batch_x):
if model == "mdn":
return build_mdn(batch_x=batch_x, batch_y=batch_theta, **kwargs)
if model == "maf":
return build_maf(batch_x=batch_x, batch_y=batch_theta, **kwargs)
elif model == "nsf":
return build_nsf(batch_x=batch_x, batch_y=batch_theta, **kwargs)
else:
raise NotImplementedError

return build_fn


### sbi.utils.get_nn_models.classifier_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, embedding_net_theta=Identity(), embedding_net_x=Identity())¶

Returns a function that builds a classifier for learning density ratios.

This function will usually be used for SNRE. The returned function is to be passed to the inference class when using the flexible interface.

Note that in the view of the SNRE classifier we build below, x=theta and y=x.

Parameters:

Name Type Description Default
model str

The type of classifier that will be created. One of [linear, mlp, resnet].

required
z_score_theta bool

Whether to z-score parameters $$\theta$$ before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs $$x$$ before passing them into the network.

True
hidden_features int

Number of hidden features.

50
embedding_net_theta Module

Optional embedding network for parameters $$\theta$$.

Identity()
embedding_net_x Module

Optional embedding network for simulation outputs $$x$$. This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
Source code in sbi/utils/get_nn_models.py
def classifier_nn(
model: str,
z_score_theta: bool = True,
z_score_x: bool = True,
hidden_features: int = 50,
embedding_net_theta: nn.Module = nn.Identity(),
embedding_net_x: nn.Module = nn.Identity(),
) -> Callable:
r"""
Returns a function that builds a classifier for learning density ratios.

This function will usually be used for SNRE. The returned function is to be passed
to the inference class when using the flexible interface.

Note that in the view of the SNRE classifier we build below, x=theta and y=x.

Args:
model: The type of classifier that will be created. One of [linear, mlp,
resnet].
z_score_theta: Whether to z-score parameters $\theta$ before passing them into
the network.
z_score_x: Whether to z-score simulation outputs $x$ before passing them into
the network.
hidden_features: Number of hidden features.
embedding_net_theta:  Optional embedding network for parameters $\theta$.
embedding_net_x:  Optional embedding network for simulation outputs $x$. This
embedding net allows to learn features from potentially high-dimensional
simulation outputs.
"""

kwargs = dict(
zip(
(
"z_score_x",
"z_score_y",
"hidden_features",
"embedding_net_x",
"embedding_net_y",
),
(
z_score_theta,
z_score_x,
hidden_features,
embedding_net_theta,
embedding_net_x,
),
)
)

def build_fn(batch_theta, batch_x):
if model == "linear":
return build_linear_classifier(
batch_x=batch_theta, batch_y=batch_x, **kwargs
)
if model == "mlp":
return build_mlp_classifier(batch_x=batch_theta, batch_y=batch_x, **kwargs)
if model == "resnet":
return build_resnet_classifier(
batch_x=batch_theta, batch_y=batch_x, **kwargs
)
else:
raise NotImplementedError

return build_fn


## Analysis¶

### sbi.analysis.plot.pairplot(samples=None, points=None, limits=None, subset=None, upper='hist', diag='hist', figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs)¶

Plot samples in a 2D grid showing marginals and pairwise marginals.

Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution that the samples were drawn from. Each upper-diagonal plot can be interpreted as a 2D-marginal of the distribution.

Parameters:

Name Type Description Default
samples Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

Samples used to build the histogram.

None
points Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

List of additional points to scatter.

None
limits Union[List, torch.Tensor]

Array containing the plot xlim for each parameter dimension. If None, just use the min and max of the passed samples

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on).

None
upper Optional[str]

Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.

'hist'
diag Optional[str]

Plotting style for diagonal, {hist, cond, None}.

'hist'
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
def pairplot(
samples: Union[
List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor
] = None,
points: Optional[
Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor]
] = None,
limits: Optional[Union[List, torch.Tensor]] = None,
subset: List[int] = None,
upper: Optional[str] = "hist",
diag: Optional[str] = "hist",
figsize: Tuple = (10, 10),
labels: Optional[List[str]] = None,
ticks: Union[List, torch.Tensor] = [],
points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"],
**kwargs
):
"""
Plot samples in a 2D grid showing marginals and pairwise marginals.

Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution
that the samples were drawn from. Each upper-diagonal plot can be interpreted as a
2D-marginal of the distribution.

Args:
samples: Samples used to build the histogram.
points: List of additional points to scatter.
limits: Array containing the plot xlim for each parameter dimension. If None,
just use the min and max of the passed samples
subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot
plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and,
if they exist, the 4th, 5th and so on).
upper: Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.
diag: Plotting style for diagonal, {hist, cond, None}.
figsize: Size of the entire figure.
labels: List of strings specifying the names of the parameters.
ticks: Position of the ticks.
points_colors: Colors of the points.
**kwargs: Additional arguments to adjust the plot, see the source code in
_get_default_opts() in sbi.utils.plot for more details.

Returns: figure and axis of posterior distribution plot
"""
return utils_pairplot(
samples=samples,
points=points,
limits=limits,
subset=subset,
upper=upper,
diag=diag,
figsize=figsize,
labels=labels,
ticks=ticks,
points_colors=points_colors,
**kwargs,
)


### sbi.analysis.plot.conditional_pairplot(density, condition, limits, points=None, subset=None, resolution=50, figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs)¶

Plot conditional distribution given all other parameters.

The conditionals can be interpreted as slices through the density at a location given by condition.

For example: Say we have a 3D density with parameters $$\theta_0$$, $$\theta_1$$, $$\theta_2$$ and a condition $$c$$ passed by the user in the condition argument. For the plot of $$\theta_0$$ on the diagonal, this will plot the conditional $$p(\theta_0 | \theta_1=c, \theta_2=c)$$. For the upper diagonal of $$\theta_1$$ and $$\theta_2$$, it will plot $$p(\theta_1, \theta_2 | \theta_0=c)$$. All other diagonals and upper-diagonals are built in the corresponding way.

Parameters:

Name Type Description Default
density Any

Probability density with a log_prob() method.

required
condition Tensor

Condition that all but the one/two regarded parameters are fixed to. The condition should be of shape (1, dim_theta), i.e. it could e.g. be a sample from the posterior distribution.

required
limits Union[List, torch.Tensor]

Limits in between which each parameter will be evaluated.

required
points Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on)

None
resolution int

Resolution of the grid at which we evaluate the pdf.

50
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
def conditional_pairplot(
density: Any,
condition: torch.Tensor,
limits: Union[List, torch.Tensor],
points: Optional[
Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor]
] = None,
subset: List[int] = None,
resolution: int = 50,
figsize: Tuple = (10, 10),
labels: Optional[List[str]] = None,
ticks: Union[List, torch.Tensor] = [],
points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"],
**kwargs
):
r"""
Plot conditional distribution given all other parameters.

The conditionals can be interpreted as slices through the density at a location
given by condition.

For example:
Say we have a 3D density with parameters $\theta_0$, $\theta_1$, $\theta_2$ and
a condition $c$ passed by the user in the condition argument.
For the plot of $\theta_0$ on the diagonal, this will plot the conditional
$p(\theta_0 | \theta_1=c, \theta_2=c)$. For the upper
diagonal of $\theta_1$ and $\theta_2$, it will plot
$p(\theta_1, \theta_2 | \theta_0=c)$. All other diagonals and upper-diagonals
are built in the corresponding way.

Args:
density: Probability density with a log_prob() method.
condition: Condition that all but the one/two regarded parameters are fixed to.
The condition should be of shape (1, dim_theta), i.e. it could e.g. be
a sample from the posterior distribution.
limits: Limits in between which each parameter will be evaluated.
subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot
plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and,
if they exist, the 4th, 5th and so on)
resolution: Resolution of the grid at which we evaluate the pdf.
figsize: Size of the entire figure.
labels: List of strings specifying the names of the parameters.
ticks: Position of the ticks.
points_colors: Colors of the points.
**kwargs: Additional arguments to adjust the plot, see the source code in
_get_default_opts() in sbi.utils.plot for more details.

Returns: figure and axis of posterior distribution plot
"""
return utils_conditional_pairplot(
density=density,
condition=condition,
limits=limits,
points=points,
subset=subset,
resolution=resolution,
figsize=figsize,
labels=labels,
ticks=ticks,
points_colors=points_colors,
**kwargs,
)


### sbi.analysis.conditional_density.conditional_corrcoeff(density, limits, condition, subset=None, resolution=50)¶

Returns the conditional correlation matrix of a distribution.

To compute the conditional distribution, we condition all but two parameters to values from condition, and then compute the Pearson correlation coefficient $$\rho$$ between the remaining two parameters under the distribution density. We do so for any pair of parameters specified in subset, thus creating a matrix containing conditional correlations between any pair of parameters.

If condition is a batch of conditions, this function computes the conditional correlation matrix for each one of them and returns the mean.

Parameters:

Name Type Description Default
density Any

Probability density function with .log_prob() function.

required
limits Tensor

Limits within which to evaluate the density.

required
condition Tensor

Values to condition the density on. If a batch of conditions is passed, we compute the conditional correlation matrix for each of them and return the average conditional correlation matrix.

required
subset Optional[List[int]]

Evaluate the conditional distribution only on a subset of dimensions. If None this function uses all dimensions.

None
resolution int

Number of grid points on which the conditional distribution is evaluated. A higher value increases the accuracy of the estimated correlation but also increases the computational cost.

50

Returns: Average conditional correlation matrix of shape either (num_dim, num_dim) or (len(subset), len(subset)) if subset was specified.

Source code in sbi/analysis/conditional_density.py
def conditional_corrcoeff(
density: Any,
limits: Tensor,
condition: Tensor,
subset: Optional[List[int]] = None,
resolution: int = 50,
) -> Tensor:
r"""
Returns the conditional correlation matrix of a distribution.

To compute the conditional distribution, we condition all but two parameters to
values from condition, and then compute the Pearson correlation
coefficient $\rho$ between the remaining two parameters under the distribution
density. We do so for any pair of parameters specified in subset, thus
creating a matrix containing conditional correlations between any pair of
parameters.

If condition is a batch of conditions, this function computes the conditional
correlation matrix for each one of them and returns the mean.

Args:
density: Probability density function with .log_prob() function.
limits: Limits within which to evaluate the density.
condition: Values to condition the density on. If a batch of conditions is
passed, we compute the conditional correlation matrix for each of them and
return the average conditional correlation matrix.
subset: Evaluate the conditional distribution only on a subset of dimensions.
If None this function uses all dimensions.
resolution: Number of grid points on which the conditional distribution is
evaluated. A higher value increases the accuracy of the estimated
correlation but also increases the computational cost.

Returns: Average conditional correlation matrix of shape either (num_dim, num_dim)
or (len(subset), len(subset)) if subset was specified.
"""
return utils_conditional_corrcoeff(
density=density,
limits=limits,
condition=condition,
subset=subset,
resolution=resolution,