Skip to content

API Reference

Inference

sbi.inference.base.infer(simulator, prior, method, num_simulations, num_workers=1)

Return posterior distribution by running simulation-based inference.

This function provides a simple interface to run sbi. Inference is run for a single round and hence the returned posterior \(p(\theta|x)\) can be sampled and evaluated for any \(x\) (i.e. it is amortized).

The scope of this function is limited to the most essential features of sbi. For more flexibility (e.g. multi-round inference, different density estimators) please use the flexible interface described here: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters \(\theta\) and maps them to simulations, or observations, x, \(\mathrm{sim}(\theta)\to x\). Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
method str

What inference method to use. Either of SNPE, SNLE or SNRE.

required
num_simulations int

Number of simulation calls. More simulations means a longer runtime, but a better posterior estimate.

required
num_workers int

Number of parallel workers to use for simulations.

1

Returns: Posterior over parameters conditional on observations (amortized).

Source code in sbi/inference/base.py
def infer(
    simulator: Callable, prior, method: str, num_simulations: int, num_workers: int = 1
) -> NeuralPosterior:
    r"""
    Return posterior distribution by running simulation-based inference.

    This function provides a simple interface to run sbi. Inference is run for a single
    round and hence the returned posterior $p(\theta|x)$ can be sampled and evaluated
    for any $x$ (i.e. it is amortized).

    The scope of this function is limited to the most essential features of sbi. For
    more flexibility (e.g. multi-round inference, different density estimators) please
    use the flexible interface described here:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\mathrm{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        method: What inference method to use. Either of SNPE, SNLE or SNRE.
        num_simulations: Number of simulation calls. More simulations means a longer
            runtime, but a better posterior estimate.
        num_workers: Number of parallel workers to use for simulations.

    Returns: Posterior over parameters conditional on observations (amortized).
    """

    try:
        method_fun: Callable = getattr(sbi.inference, method.upper())
    except AttributeError:
        raise NameError(
            "Method not available. `method` must be one of 'SNPE', 'SNLE', 'SNRE'."
        )

    simulator, prior = prepare_for_sbi(simulator, prior)

    inference = method_fun(prior)
    theta, x = simulate_for_sbi(
        simulator=simulator,
        proposal=prior,
        num_simulations=num_simulations,
        num_workers=num_workers,
    )
    _ = inference.append_simulations(theta, x).train()
    posterior = inference.build_posterior()

    return posterior

sbi.utils.user_input_checks.prepare_for_sbi(simulator, prior)

Prepare simulator, prior and for usage in sbi.

One of the goals is to allow you to use sbi with inputs computed in numpy.

Attempts to meet the following requirements by reshaping and type-casting:

  • the simulator function receives as input and returns a Tensor.
  • the simulator can simulate batches of parameters and return batches of data.
  • the prior does not produce batches and samples and evaluates to Tensor.
  • the output shape is a torch.Size((1,N)) (i.e, has a leading batch dimension 1).

If this is not possible, a suitable exception will be raised.

Parameters:

Name Type Description Default
simulator Callable

Simulator as provided by the user.

required
prior

Prior as provided by the user.

required

Returns:

Type Description
Tuple[Callable, torch.distributions.distribution.Distribution]

Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.

Source code in sbi/utils/user_input_checks.py
def prepare_for_sbi(simulator: Callable, prior) -> Tuple[Callable, Distribution]:
    """Prepare simulator, prior and for usage in sbi.

    One of the goals is to allow you to use sbi with inputs computed in numpy.

    Attempts to meet the following requirements by reshaping and type-casting:

    - the simulator function receives as input and returns a Tensor.<br/>
    - the simulator can simulate batches of parameters and return batches of data.<br/>
    - the prior does not produce batches and samples and evaluates to Tensor.<br/>
    - the output shape is a `torch.Size((1,N))` (i.e, has a leading batch dimension 1).

    If this is not possible, a suitable exception will be raised.

    Args:
        simulator: Simulator as provided by the user.
        prior: Prior as provided by the user.

    Returns:
        Tuple (simulator, prior, x_shape) checked and matching the requirements of sbi.
    """

    # Check prior, return PyTorch prior.
    prior, _, prior_returns_numpy = process_prior(prior)

    # Check simulator, returns PyTorch simulator able to simulate batches.
    simulator = process_simulator(simulator, prior, prior_returns_numpy)

    # Consistency check after making ready for sbi.
    check_sbi_inputs(simulator, prior)

    return simulator, prior

sbi.inference.base.simulate_for_sbi(simulator, proposal, num_simulations, num_workers=1, simulation_batch_size=1, show_progress_bar=True)

Returns (\(\theta, x\)) pairs obtained from sampling the proposal and simulating.

This function performs two steps:

  • Sample parameters \(\theta\) from the proposal.
  • Simulate these parameters to obtain \(x\).

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters \(\theta\) and maps them to simulations, or observations, x, \(\text{sim}(\theta)\to x\). Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
proposal Any

Probability distribution that the parameters \(\theta\) are sampled from.

required
num_simulations int

Number of simulations that are run.

required
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bar bool

Whether to show a progress bar for simulating. This will not affect whether there will be a progressbar while drawing samples from the proposal.

True

Returns: Sampled parameters \(\theta\) and simulation-outputs \(x\).

Source code in sbi/inference/base.py
def simulate_for_sbi(
    simulator: Callable,
    proposal: Any,
    num_simulations: int,
    num_workers: int = 1,
    simulation_batch_size: int = 1,
    show_progress_bar: bool = True,
) -> Tuple[Tensor, Tensor]:
    r"""
    Returns ($\theta, x$) pairs obtained from sampling the proposal and simulating.

    This function performs two steps:

    - Sample parameters $\theta$ from the `proposal`.
    - Simulate these parameters to obtain $x$.

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\text{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        proposal: Probability distribution that the parameters $\theta$ are sampled
            from.
        num_simulations: Number of simulations that are run.
        num_workers: Number of parallel workers to use for simulations.
        simulation_batch_size: Number of parameter sets that the simulator
            maps to data x at once. If None, we simulate all parameter sets at the
            same time. If >= 1, the simulator has to process data of shape
            (simulation_batch_size, parameter_dimension).
        show_progress_bar: Whether to show a progress bar for simulating. This will not
            affect whether there will be a progressbar while drawing samples from the
            proposal.

    Returns: Sampled parameters $\theta$ and simulation-outputs $x$.
    """

    check_if_proposal_has_default_x(proposal)

    theta = proposal.sample((num_simulations,))

    x = simulate_in_batches(
        simulator, theta, simulation_batch_size, num_workers, show_progress_bar
    )

    return theta, x

sbi.inference.snpe.snpe_c.SNPE_C (PosteriorEstimator)

__init__(self, prior=None, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special

SNPE-C / APT [1].

[1] Automatic Posterior Transformation for Likelihood-free Inference, Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

This class implements two loss variants of SNPE-C: the non-atomic and the atomic version. The atomic loss of SNPE-C can be used for any density estimator, i.e. also for normalizing flows. However, it suffers from leakage issues. On the other hand, the non-atomic loss can only be used only if the proposal distribution is a mixture of Gaussians, the density estimator is a mixture of Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from leakage issues. At the beginning of each round, we print whether the non-atomic or the atomic version is used.

In this codebase, we will automatically switch to the non-atomic loss if the following criteria are fulfilled:
- proposal is a DirectPosterior with density_estimator mdn, as built with utils.sbi.posterior_nn().
- the density estimator is a mdn, as built with utils.sbi.posterior_nn().
- isinstance(prior, MultivariateNormal) (from torch.distributions) or isinstance(prior, sbi.utils.BoxUniform)

Note that custom implementations of any of these densities (or estimators) will not trigger the non-atomic loss, and the algorithm will fall back onto using the atomic loss.

Parameters:

Name Type Description Default
prior Optional[Any]

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

None
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during training.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snpe/snpe_c.py
def __init__(
    self,
    prior: Optional[Any] = None,
    density_estimator: Union[str, Callable] = "maf",
    device: str = "cpu",
    logging_level: Union[int, str] = "WARNING",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args,
):
    r"""SNPE-C / APT [1].

    [1] _Automatic Posterior Transformation for Likelihood-free Inference_,
        Greenberg et al., ICML 2019, https://arxiv.org/abs/1905.07488.

    This class implements two loss variants of SNPE-C: the non-atomic and the atomic
    version. The atomic loss of SNPE-C can be used for any density estimator,
    i.e. also for normalizing flows. However, it suffers from leakage issues. On
    the other hand, the non-atomic loss can only be used only if the proposal
    distribution is a mixture of Gaussians, the density estimator is a mixture of
    Gaussians, and the prior is either Gaussian or Uniform. It does not suffer from
    leakage issues. At the beginning of each round, we print whether the non-atomic
    or the atomic version is used.

    In this codebase, we will automatically switch to the non-atomic loss if the
    following criteria are fulfilled:<br/>
    - proposal is a `DirectPosterior` with density_estimator `mdn`, as built
        with `utils.sbi.posterior_nn()`.<br/>
    - the density estimator is a `mdn`, as built with
        `utils.sbi.posterior_nn()`.<br/>
    - `isinstance(prior, MultivariateNormal)` (from `torch.distributions`) or
        `isinstance(prior, sbi.utils.BoxUniform)`

    Note that custom implementations of any of these densities (or estimators) will
    not trigger the non-atomic loss, and the algorithm will fall back onto using
    the atomic loss.

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        density_estimator: If it is a string, use a pre-configured network of the
            provided type (one of nsf, maf, mdn, made). Alternatively, a function
            that builds a custom neural network can be provided. The function will
            be called with the first batch of simulations (theta, x), which can
            thus be used for shape inference and potentially for z-scoring. It
            needs to return a PyTorch `nn.Module` implementing the density
            estimator. The density estimator needs to provide the methods
            `.log_prob` and `.sample()`.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during training.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, proposal=None) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \(\theta\), \(x\), prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
proposal Optional[Any]

The distribution that the parameters \(\theta\) were sampled from. Pass None if the parameters were sampled from the prior. If not None, it will trigger a different loss-function.

None

Returns:

Type Description
PosteriorEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snpe/snpe_c.py
def append_simulations(
    self, theta: Tensor, x: Tensor, proposal: Optional[Any] = None
) -> "PosteriorEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        proposal: The distribution that the parameters $\theta$ were sampled from.
            Pass `None` if the parameters were sampled from the prior. If not
            `None`, it will trigger a different loss-function.

    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)
    self._check_proposal(proposal)

    if (
        proposal is None
        or proposal is self._prior
        or (
            isinstance(proposal, RestrictedPrior) and proposal._prior is self._prior
        )
    ):
        # The `_data_round_index` will later be used to infer if one should train
        # with MLE loss or with atomic loss (see, in `train()`:
        # self._round = max(self._data_round_index))
        self._data_round_index.append(0)
        self._prior_masks.append(mask_sims_from_prior(0, theta.size(0)))
    else:
        if not self._data_round_index:
            # This catches a pretty specific case: if, in the first round, one
            # passes data that does not come from the prior.
            self._data_round_index.append(1)
        else:
            self._data_round_index.append(max(self._data_round_index) + 1)
        self._prior_masks.append(mask_sims_from_prior(1, theta.size(0)))

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._proposal_roundwise.append(proposal)

    if self._prior is None or isinstance(self._prior, ImproperEmpirical):
        if proposal is not None:
            raise ValueError(
                "You had not passed a prior at initialization, but now you "
                "passed a proposal. If you want to run multi-round SNPE, you have "
                "to specify a prior (set the `.prior` argument or re-initialize "
                "the object with a prior distribution). If the samples you passed "
                "to `append_simulations()` were sampled from the prior, you can "
                "run single-round inference with "
                "`append_simulations(..., proposal=None)`."
            )
        theta_prior = self.get_simulations()[0]
        self._prior = ImproperEmpirical(theta_prior, ones(theta_prior.shape[0]))

    return self

build_posterior(self, density_estimator=None, sample_with='rejection', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, sample_with_mcmc=None) inherited

Build posterior from the neural density estimator.

For SNPE, the posterior distribution that is returned here implements the following functionality over the raw neural density estimator:

  • correct the calculation of the log probability such that it compensates for the leakage.
  • reject samples that lie outside of the prior bounds.
  • alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [rejection | mcmc]. With default parameters, rejection samples from the posterior estimated by the neural net and rejects only if the samples are outside of the prior support.

'rejection'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the trained neural net). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
sample_with_mcmc Optional[bool]

Deprecated since sbi v0.16.0. Use sample_with=mcmc instead.

None

Returns:

Type Description
DirectPosterior

Posterior \(p(\theta|x)\) with .sample() and .log_prob() methods.

Source code in sbi/inference/snpe/snpe_c.py
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    sample_with: str = "rejection",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    sample_with_mcmc: Optional[bool] = None,
) -> DirectPosterior:
    r"""
    Build posterior from the neural density estimator.

    For SNPE, the posterior distribution that is returned here implements the
    following functionality over the raw neural density estimator:

    - correct the calculation of the log probability such that it compensates for
        the leakage.
    - reject samples that lie outside of the prior bounds.
    - alternatively, if leakage is very high (which can happen for multi-round
        SNPE), sample from the posterior with MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`rejection` | `mcmc`]. With default parameters, `rejection` samples
            from the posterior estimated by the neural net and rejects only if the
            samples are outside of the prior support.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the trained
            neural net). `max_sampling_batch_size` as the batchsize of samples
            being drawn from the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio. `m` as multiplier to that ratio.
        sample_with_mcmc: Deprecated since `sbi v0.16.0`. Use `sample_with=mcmc`
            instead.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods.
    """
    if sample_with_mcmc is not None:
        warn(
            f"You set `sample_with_mcmc={sample_with_mcmc}`. This is deprecated "
            "since `sbi v0.17.0` and will lead to an error in future versions. "
            "Please use `sample_with='mcmc'` instead."
        )
        if sample_with_mcmc:
            sample_with = "mcmc"

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device.type

    self._posterior = DirectPosterior(
        method_family="snpe",
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        sample_with=sample_with,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        rejection_sampling_parameters=rejection_sampling_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snpe/snpe_c.py
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \(\theta\), \(x\), and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snpe/snpe_c.py
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \(\theta\) and \(x\) to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snpe/snpe_c.py
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, calibration_kernel=None, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, use_combined_loss=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return density estimator that approximates the distribution \(p(\theta|x)\).

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
calibration_kernel Optional[Callable]

A function to calibrate the loss with respect to the simulations x. See Lueckmann, Gonçalves et al., NeurIPS 2017.

None
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
use_combined_loss bool

Whether to train the neural net also on prior samples using maximum likelihood in addition to training it on all samples using atomic loss. The extra MLE loss helps prevent density leaking with bounded priors.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
DirectPosterior

Density estimator that approximates the distribution \(p(\theta|x)\).

Source code in sbi/inference/snpe/snpe_c.py
def train(
    self,
    num_atoms: int = 10,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    calibration_kernel: Optional[Callable] = None,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    use_combined_loss: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> DirectPosterior:
    r"""
    Return density estimator that approximates the distribution $p(\theta|x)$.

    Args:
        num_atoms: Number of atoms to use for classification.
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        calibration_kernel: A function to calibrate the loss with respect to the
            simulations `x`. See Lueckmann, Gonçalves et al., NeurIPS 2017.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        use_combined_loss: Whether to train the neural net also on prior samples
            using maximum likelihood in addition to training it on all samples using
            atomic loss. The extra MLE loss helps prevent density leaking with
            bounded priors.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Density estimator that approximates the distribution $p(\theta|x)$.
    """

    # WARNING: sneaky trick ahead. We proxy the parent's `train` here,
    # requiring the signature to have `num_atoms`, save it for use below, and
    # continue. It's sneaky because we are using the object (self) as a namespace
    # to pass arguments between functions, and that's implicit state management.
    self._num_atoms = num_atoms
    self._use_combined_loss = use_combined_loss
    kwargs = del_entries(
        locals(), entries=("self", "__class__", "num_atoms", "use_combined_loss")
    )

    self._round = max(self._data_round_index)

    if self._round > 0:
        # Set the proposal to the last proposal that was passed by the user. For
        # atomic SNPE, it does not matter what the proposal is. For non-atomic
        # SNPE, we only use the latest data that was passed, i.e. the one from the
        # last proposal.
        proposal = self._proposal_roundwise[-1]
        if hasattr(proposal, "net"):
            self.use_non_atomic_loss = (
                isinstance(proposal.net._distribution, mdn)
                and isinstance(self._neural_net._distribution, mdn)
                and check_dist_class(
                    self._prior, class_to_check=(Uniform, MultivariateNormal)
                )[0]
            )
        else:
            self.use_non_atomic_loss = False

        algorithm = "non-atomic" if self.use_non_atomic_loss else "atomic"
        print(f"Using SNPE-C with {algorithm} loss")

        if self.use_non_atomic_loss:
            # Take care of z-scoring, pre-compute and store prior terms.
            self._set_state_for_mog_proposal()

    return super().train(**kwargs)

sbi.inference.snle.snle_a.SNLE_A (LikelihoodEstimator)

__init__(self, prior, density_estimator='maf', device='cpu', logging_level='WARNING', summary_writer=None, show_progress_bars=True, **unused_args) special

Sequential Neural Likelihood [1].

[1] Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows_, Papamakarios et al., AISTATS 2019, https://arxiv.org/abs/1805.07226

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
density_estimator Union[str, Callable]

If it is a string, use a pre-configured network of the provided type (one of nsf, maf, mdn, made). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the density estimator. The density estimator needs to provide the methods .log_prob and .sample().

'maf'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'WARNING'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snle/snle_a.py
def __init__(
    self,
    prior,
    density_estimator: Union[str, Callable] = "maf",
    device: str = "cpu",
    logging_level: Union[int, str] = "WARNING",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args,
):
    r"""Sequential Neural Likelihood [1].

    [1] Sequential Neural Likelihood: Fast Likelihood-free Inference with
    Autoregressive Flows_, Papamakarios et al., AISTATS 2019,
    https://arxiv.org/abs/1805.07226

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        density_estimator: If it is a string, use a pre-configured network of the
            provided type (one of nsf, maf, mdn, made). Alternatively, a function
            that builds a custom neural network can be provided. The function will
            be called with the first batch of simulations (theta, x), which can
            thus be used for shape inference and potentially for z-scoring. It
            needs to return a PyTorch `nn.Module` implementing the density
            estimator. The density estimator needs to provide the methods
            `.log_prob` and `.sample()`.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \(\theta\), \(x\), prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNLE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
LikelihoodEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snle/snle_a.py
def append_simulations(
    self,
    theta: Tensor,
    x: Tensor,
    from_round: int = 0,
) -> "LikelihoodEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        from_round: Which round the data stemmed from. Round 0 means from the prior.
            With default settings, this is not used at all for `SNLE`. Only when
            the user later on requests `.train(discard_prior_samples=True)`, we
            use these indices to find which training data stemmed from the prior.

    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0)))
    self._data_round_index.append(int(from_round))

    return self

build_posterior(self, density_estimator=None, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None) inherited

Build posterior from the neural density estimator.

SNLE trains a neural network to approximate the likelihood \(p(x|\theta)\). The LikelihoodBasedPosterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\) and draw samples from the posterior with MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
LikelihoodBasedPosterior

Posterior \(p(\theta|x)\) with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snle/snle_a.py
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    sample_with: str = "mcmc",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> LikelihoodBasedPosterior:
    r"""
    Build posterior from the neural density estimator.

    SNLE trains a neural network to approximate the likelihood $p(x|\theta)$. The
    `LikelihoodBasedPosterior` class wraps the trained network such that one can
    directly evaluate the unnormalized posterior log probability
    $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and draw samples from the
    posterior with MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration. `num_samples_to_find_max` as the
            number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `num_iter_to_find_max` as the number
            of gradient ascent iterations to find the maximum of that ratio. `m` as
            multiplier to that ratio.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods
        (the returned log-probability is unnormalized).
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device.type

    self._posterior = LikelihoodBasedPosterior(
        method_family="snle",
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        sample_with=sample_with,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        rejection_sampling_parameters=rejection_sampling_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snle/snle_a.py
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \(\theta\), \(x\), and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snle/snle_a.py
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \(\theta\) and \(x\) to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snle/snle_a.py
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return density estimator that approximates the distribution \(p(x|\theta)\).

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Density estimator that approximates the distribution \(p(x|\theta)\).

Source code in sbi/inference/snle/snle_a.py
def train(
    self,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> NeuralPosterior:
    r"""
    Return density estimator that approximates the distribution $p(x|\theta)$.

    Args:
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Density estimator that approximates the distribution $p(x|\theta)$.
    """
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    return super().train(**kwargs)

sbi.inference.snre.snre_a.SNRE_A (RatioEstimator)

__init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special

AALR[1], here known as SNRE_A.

[1] Likelihood-free MCMC with Amortized Approximate Likelihood Ratios, Hermans et al., ICML 2020, https://arxiv.org/abs/1903.04057

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_a.py
def __init__(
    self,
    prior,
    classifier: Union[str, Callable] = "resnet",
    device: str = "cpu",
    logging_level: Union[int, str] = "warning",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args
):
    r"""AALR[1], here known as SNRE_A.

    [1] _Likelihood-free MCMC with Amortized Approximate Likelihood Ratios_, Hermans
        et al., ICML 2020, https://arxiv.org/abs/1903.04057

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        classifier: Classifier trained to approximate likelihood ratios. If it is
            a string, use a pre-configured network of the provided type (one of
            linear, mlp, resnet). Alternatively, a function that builds a custom
            neural network can be provided. The function will be called with the
            first batch of simulations (theta, x), which can thus be used for shape
            inference and potentially for z-scoring. It needs to return a PyTorch
            `nn.Module` implementing the classifier.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \(\theta\), \(x\), prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_a.py
def append_simulations(
    self,
    theta: Tensor,
    x: Tensor,
    from_round: int = 0,
) -> "RatioEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        from_round: Which round the data stemmed from. Round 0 means from the prior.
            With default settings, this is not used at all for `SNRE`. Only when
            the user later on requests `.train(discard_prior_samples=True)`, we
            use these indices to find which training data stemmed from the prior.
    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0)))
    self._data_round_index.append(int(from_round))

    return self

build_posterior(self, density_estimator=None, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None) inherited

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\). The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\) and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
RatioBasedPosterior

Posterior \(p(\theta|x)\) with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_a.py
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    sample_with: str = "mcmc",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> RatioBasedPosterior:
    r"""
    Build posterior from the neural density estimator.

    SNRE trains a neural network to approximate likelihood ratios, which in turn
    can be used obtain an unnormalized posterior
    $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here
    wraps the trained network such that one can directly evaluate the unnormalized
    posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and
    draw samples from the posterior with MCMC. Note that, in the case of
    single-round SNRE_A / AALR, it is possible to evaluate the log-probability of
    the **normalized** posterior, but sampling still requires MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration. `num_samples_to_find_max` as the
            number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `num_iter_to_find_max` as the number
            of gradient ascent iterations to find the maximum of that ratio. `m` as
            multiplier to that ratio.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods
        (the returned log-probability is unnormalized).
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device.type

    self._posterior = RatioBasedPosterior(
        method_family=self.__class__.__name__.lower(),
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        sample_with=sample_with,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        rejection_sampling_parameters=rejection_sampling_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_a.py
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \(\theta\), \(x\), and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_a.py
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \(\theta\) and \(x\) to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_a.py
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return classifier that approximates the ratio \(p(\theta,x)/p(\theta)p(x)\).

Parameters:

Name Type Description Default
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio \(p(\theta,x)/p(\theta)p(x)\).

Source code in sbi/inference/snre/snre_a.py
def train(
    self,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> NeuralPosterior:
    r"""
    Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

    Args:
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.
    """

    # AALR is defined for `num_atoms=2`.
    # Proxy to `super().__call__` to ensure right parameter.
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    return super().train(**kwargs, num_atoms=2)

sbi.inference.snre.snre_b.SNRE_B (RatioEstimator)

__init__(self, prior, classifier='resnet', device='cpu', logging_level='warning', summary_writer=None, show_progress_bars=True, **unused_args) special

SRE[1], here known as SNRE_B.

[1] On Contrastive Learning for Likelihood-free Inference, Durkan et al., ICML 2020, https://arxiv.org/pdf/2002.03712

Parameters:

Name Type Description Default
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
classifier Union[str, Callable]

Classifier trained to approximate likelihood ratios. If it is a string, use a pre-configured network of the provided type (one of linear, mlp, resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of simulations (theta, x), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.

'resnet'
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
logging_level Union[int, str]

Minimum severity of messages to log. One of the strings INFO, WARNING, DEBUG, ERROR and CRITICAL.

'warning'
summary_writer Optional[Writer]

A tensorboard SummaryWriter to control, among others, log file location (default is <current working directory>/logs.)

None
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
unused_args

Absorbs additional arguments. No entries will be used. If it is not empty, we warn. In future versions, when the new interface of 0.14.0 is more mature, we will remove this argument.

{}
Source code in sbi/inference/snre/snre_b.py
def __init__(
    self,
    prior,
    classifier: Union[str, Callable] = "resnet",
    device: str = "cpu",
    logging_level: Union[int, str] = "warning",
    summary_writer: Optional[TensorboardSummaryWriter] = None,
    show_progress_bars: bool = True,
    **unused_args
):
    r"""SRE[1], here known as SNRE_B.

    [1] _On Contrastive Learning for Likelihood-free Inference_, Durkan et al.,
        ICML 2020, https://arxiv.org/pdf/2002.03712

    Args:
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        classifier: Classifier trained to approximate likelihood ratios. If it is
            a string, use a pre-configured network of the provided type (one of
            linear, mlp, resnet). Alternatively, a function that builds a custom
            neural network can be provided. The function will be called with the
            first batch of simulations (theta, x), which can thus be used for shape
            inference and potentially for z-scoring. It needs to return a PyTorch
            `nn.Module` implementing the classifier.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
        logging_level: Minimum severity of messages to log. One of the strings
            INFO, WARNING, DEBUG, ERROR and CRITICAL.
        summary_writer: A tensorboard `SummaryWriter` to control, among others, log
            file location (default is `<current working directory>/logs`.)
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        unused_args: Absorbs additional arguments. No entries will be used. If it
            is not empty, we warn. In future versions, when the new interface of
            0.14.0 is more mature, we will remove this argument.
    """

    kwargs = del_entries(locals(), entries=("self", "__class__", "unused_args"))
    super().__init__(**kwargs, **unused_args)

append_simulations(self, theta, x, from_round=0) inherited

Store parameters and simulation outputs to use them for later training.

Data are stored as entries in lists for each type of variable (parameter/data).

Stores \(\theta\), \(x\), prior_masks (indicating if simulations are coming from the prior or not) and an index indicating which round the batch of simulations came from.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets.

required
x Tensor

Simulation outputs.

required
from_round int

Which round the data stemmed from. Round 0 means from the prior. With default settings, this is not used at all for SNRE. Only when the user later on requests .train(discard_prior_samples=True), we use these indices to find which training data stemmed from the prior.

0

Returns:

Type Description
RatioEstimator

NeuralInference object (returned so that this function is chainable).

Source code in sbi/inference/snre/snre_b.py
def append_simulations(
    self,
    theta: Tensor,
    x: Tensor,
    from_round: int = 0,
) -> "RatioEstimator":
    r"""
    Store parameters and simulation outputs to use them for later training.

    Data are stored as entries in lists for each type of variable (parameter/data).

    Stores $\theta$, $x$, prior_masks (indicating if simulations are coming from the
    prior or not) and an index indicating which round the batch of simulations came
    from.

    Args:
        theta: Parameter sets.
        x: Simulation outputs.
        from_round: Which round the data stemmed from. Round 0 means from the prior.
            With default settings, this is not used at all for `SNRE`. Only when
            the user later on requests `.train(discard_prior_samples=True)`, we
            use these indices to find which training data stemmed from the prior.
    Returns:
        NeuralInference object (returned so that this function is chainable).
    """

    theta, x = validate_theta_and_x(theta, x, training_device=self._device)

    self._theta_roundwise.append(theta)
    self._x_roundwise.append(x)
    self._prior_masks.append(mask_sims_from_prior(int(from_round), theta.size(0)))
    self._data_round_index.append(int(from_round))

    return self

build_posterior(self, density_estimator=None, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None) inherited

Build posterior from the neural density estimator.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\). The posterior returned here wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\) and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

Parameters:

Name Type Description Default
density_estimator Optional[Module]

The density estimator that the posterior is based on. If None, use the latest neural density estimator that was trained.

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential-Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
RatioBasedPosterior

Posterior \(p(\theta|x)\) with .sample() and .log_prob() methods (the returned log-probability is unnormalized).

Source code in sbi/inference/snre/snre_b.py
def build_posterior(
    self,
    density_estimator: Optional[TorchModule] = None,
    sample_with: str = "mcmc",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> RatioBasedPosterior:
    r"""
    Build posterior from the neural density estimator.

    SNRE trains a neural network to approximate likelihood ratios, which in turn
    can be used obtain an unnormalized posterior
    $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$. The posterior returned here
    wraps the trained network such that one can directly evaluate the unnormalized
    posterior log-probability $p(\theta|x) \propto p(x|\theta) \cdot p(\theta)$ and
    draw samples from the posterior with MCMC. Note that, in the case of
    single-round SNRE_A / AALR, it is possible to evaluate the log-probability of
    the **normalized** posterior, but sampling still requires MCMC.

    Args:
        density_estimator: The density estimator that the posterior is based on.
            If `None`, use the latest neural density estimator that was trained.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior` will
            draw init locations from prior, whereas `sir` will use
            Sequential-Importance-Resampling using `init_strategy_num_candidates`
            to find init locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration. `num_samples_to_find_max` as the
            number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `num_iter_to_find_max` as the number
            of gradient ascent iterations to find the maximum of that ratio. `m` as
            multiplier to that ratio.

    Returns:
        Posterior $p(\theta|x)$  with `.sample()` and `.log_prob()` methods
        (the returned log-probability is unnormalized).
    """

    if density_estimator is None:
        density_estimator = self._neural_net
        # If internal net is used device is defined.
        device = self._device
    else:
        # Otherwise, infer it from the device of the net parameters.
        device = next(density_estimator.parameters()).device.type

    self._posterior = RatioBasedPosterior(
        method_family=self.__class__.__name__.lower(),
        neural_net=density_estimator,
        prior=self._prior,
        x_shape=self._x_shape,
        sample_with=sample_with,
        mcmc_method=mcmc_method,
        mcmc_parameters=mcmc_parameters,
        rejection_sampling_parameters=rejection_sampling_parameters,
        device=device,
    )

    self._posterior._num_trained_rounds = self._round + 1

    # Store models at end of each round.
    self._model_bank.append(deepcopy(self._posterior))
    self._model_bank[-1].net.eval()

    return deepcopy(self._posterior)

get_dataloaders(self, dataset, training_batch_size=50, validation_fraction=0.1, resume_training=False, dataloader_kwargs=None) inherited

Return dataloaders for training and validation.

Parameters:

Name Type Description Default
dataset TensorDataset

holding all theta and x, optionally masks.

required
training_batch_size int

training arg of inference methods.

50
resume_training bool

Whether the current call is resuming training so that no new training and validation indices into the dataset have to be created.

False
dataloader_kwargs Optional[dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
Tuple[torch.utils.data.dataloader.DataLoader, torch.utils.data.dataloader.DataLoader]

Tuple of dataloaders for training and validation.

Source code in sbi/inference/snre/snre_b.py
def get_dataloaders(
    self,
    dataset: data.TensorDataset,
    training_batch_size: int = 50,
    validation_fraction: float = 0.1,
    resume_training: bool = False,
    dataloader_kwargs: Optional[dict] = None,
) -> Tuple[data.DataLoader, data.DataLoader]:
    """Return dataloaders for training and validation.

    Args:
        dataset: holding all theta and x, optionally masks.
        training_batch_size: training arg of inference methods.
        resume_training: Whether the current call is resuming training so that no
            new training and validation indices into the dataset have to be created.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Tuple of dataloaders for training and validation.

    """

    # Get total number of training examples.
    num_examples = len(dataset)

    # Select random train and validation splits from (theta, x) pairs.
    num_training_examples = int((1 - validation_fraction) * num_examples)
    num_validation_examples = num_examples - num_training_examples

    if not resume_training:
        permuted_indices = torch.randperm(num_examples)
        self.train_indices, self.val_indices = (
            permuted_indices[:num_training_examples],
            permuted_indices[num_training_examples:],
        )

    # Create training and validation loaders using a subset sampler.
    # Intentionally use dicts to define the default dataloader args
    # Then, use dataloader_kwargs to override (or add to) any of these defaults
    # https://stackoverflow.com/questions/44784577/in-method-call-args-how-to-override-keyword-argument-of-unpacked-dict
    train_loader_kwargs = {
        "batch_size": min(training_batch_size, num_training_examples),
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.train_indices),
    }
    train_loader_kwargs = (
        dict(train_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else train_loader_kwargs
    )
    val_loader_kwargs = {
        "batch_size": min(training_batch_size, num_validation_examples),
        "shuffle": False,
        "drop_last": True,
        "sampler": data.sampler.SubsetRandomSampler(self.val_indices),
    }
    val_loader_kwargs = (
        dict(val_loader_kwargs, **dataloader_kwargs)
        if dataloader_kwargs is not None
        else val_loader_kwargs
    )
    train_loader = data.DataLoader(dataset, **train_loader_kwargs)
    val_loader = data.DataLoader(dataset, **val_loader_kwargs)

    return train_loader, val_loader

get_simulations(self, starting_round=0, exclude_invalid_x=True, warn_on_invalid=True) inherited

Returns all \(\theta\), \(x\), and prior_masks from rounds >= starting_round.

If requested, do not return invalid data.

Parameters:

Name Type Description Default
starting_round int

The earliest round to return samples from (we start counting from zero).

0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training.

True
warn_on_invalid bool

Whether to give out a warning if invalid simulations were found.

True

Returns: Parameters, simulation outputs, prior masks.

Source code in sbi/inference/snre/snre_b.py
def get_simulations(
    self,
    starting_round: int = 0,
    exclude_invalid_x: bool = True,
    warn_on_invalid: bool = True,
) -> Tuple[Tensor, Tensor, Tensor]:
    r"""
    Returns all $\theta$, $x$, and prior_masks from rounds >= `starting_round`.

    If requested, do not return invalid data.

    Args:
        starting_round: The earliest round to return samples from (we start counting
            from zero).
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training.
        warn_on_invalid: Whether to give out a warning if invalid simulations were
            found.

    Returns: Parameters, simulation outputs, prior masks.
    """

    theta = get_simulations_since_round(
        self._theta_roundwise, self._data_round_index, starting_round
    )
    x = get_simulations_since_round(
        self._x_roundwise, self._data_round_index, starting_round
    )
    prior_masks = get_simulations_since_round(
        self._prior_masks, self._data_round_index, starting_round
    )

    # Check for NaNs in simulations.
    is_valid_x, num_nans, num_infs = handle_invalid_x(x, exclude_invalid_x)
    # Check for problematic z-scoring
    warn_if_zscoring_changes_data(x)
    if warn_on_invalid:
        warn_on_invalid_x(num_nans, num_infs, exclude_invalid_x)
        warn_on_invalid_x_for_snpec_leakage(
            num_nans, num_infs, exclude_invalid_x, type(self).__name__, self._round
        )

    return theta[is_valid_x], x[is_valid_x], prior_masks[is_valid_x]

provide_presimulated(self, theta, x, from_round=0) inherited

Deprecated since sbi 0.14.0.

Instead of using this, please use .append_simulations(). Please consult release notes to see how you can update your code: https://github.com/mackelab/sbi/releases/tag/v0.14.0 More information can be found under the corresponding pull request on github: https://github.com/mackelab/sbi/pull/378 and tutorials: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

Provide external \(\theta\) and \(x\) to be used for training later on.

Parameters:

Name Type Description Default
theta Tensor

Parameter sets used to generate presimulated data.

required
x Tensor

Simulation outputs of presimulated data.

required
from_round int

Which round the data was simulated from. from_round=0 means that the data came from the first round, i.e. the prior.

0
Source code in sbi/inference/snre/snre_b.py
def provide_presimulated(
    self, theta: Tensor, x: Tensor, from_round: int = 0
) -> None:
    r"""
    Deprecated since sbi 0.14.0.

    Instead of using this, please use `.append_simulations()`. Please consult
    release notes to see how you can update your code:
    https://github.com/mackelab/sbi/releases/tag/v0.14.0
    More information can be found under the corresponding pull request on github:
    https://github.com/mackelab/sbi/pull/378
    and tutorials:
    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

    Provide external $\theta$ and $x$ to be used for training later on.

    Args:
        theta: Parameter sets used to generate presimulated data.
        x: Simulation outputs of presimulated data.
        from_round: Which round the data was simulated from. `from_round=0` means
            that the data came from the first round, i.e. the prior.
    """
    raise NameError(
        f"Deprecated since sbi 0.14.0. "
        f"Instead of using this, please use `.append_simulations()`. Please "
        f"consult release notes to see how you can update your code: "
        f"https://github.com/mackelab/sbi/releases/tag/v0.14.0"
        f"More information can be found under the corresponding pull request on "
        f"github: "
        f"https://github.com/mackelab/sbi/pull/378"
        f"and tutorials: "
        f"https://www.mackelab.org/sbi/tutorial/02_flexible_interface/",
    )

train(self, num_atoms=10, training_batch_size=50, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=None, clip_max_norm=5.0, exclude_invalid_x=True, resume_training=False, discard_prior_samples=False, retrain_from_scratch_each_round=False, show_train_summary=False, dataloader_kwargs=None)

Return classifier that approximates the ratio \(p(\theta,x)/p(\theta)p(x)\).

Parameters:

Name Type Description Default
num_atoms int

Number of atoms to use for classification.

10
training_batch_size int

Training batch size.

50
learning_rate float

Learning rate for Adam optimizer.

0.0005
validation_fraction float

The fraction of data to use for validation.

0.1
stop_after_epochs int

The number of epochs to wait for improvement on the validation set before terminating training.

20
max_num_epochs Optional[int]

Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. If None, we train until validation loss increases (see also stop_after_epochs).

None
clip_max_norm Optional[float]

Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.

5.0
exclude_invalid_x bool

Whether to exclude simulation outputs x=NaN or x=±∞ during training. Expect errors, silent or explicit, when False.

True
resume_training bool

Can be used in case training time is limited, e.g. on a cluster. If True, the split between train and validation set, the optimizer, the number of epochs, and the best validation log-prob will be restored from the last time .train() was called.

False
discard_prior_samples bool

Whether to discard samples simulated in round 1, i.e. from the prior. Training may be sped up by ignoring such less targeted samples.

False
retrain_from_scratch_each_round bool

Whether to retrain the conditional density estimator for the posterior from scratch each round.

False
show_train_summary bool

Whether to print the number of epochs and validation loss and leakage after the training.

False
dataloader_kwargs Optional[Dict]

Additional or updated kwargs to be passed to the training and validation dataloaders (like, e.g., a collate_fn)

None

Returns:

Type Description
NeuralPosterior

Classifier that approximates the ratio \(p(\theta,x)/p(\theta)p(x)\).

Source code in sbi/inference/snre/snre_b.py
def train(
    self,
    num_atoms: int = 10,
    training_batch_size: int = 50,
    learning_rate: float = 5e-4,
    validation_fraction: float = 0.1,
    stop_after_epochs: int = 20,
    max_num_epochs: Optional[int] = None,
    clip_max_norm: Optional[float] = 5.0,
    exclude_invalid_x: bool = True,
    resume_training: bool = False,
    discard_prior_samples: bool = False,
    retrain_from_scratch_each_round: bool = False,
    show_train_summary: bool = False,
    dataloader_kwargs: Optional[Dict] = None,
) -> NeuralPosterior:
    r"""
    Return classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.

    Args:
        num_atoms: Number of atoms to use for classification.
        training_batch_size: Training batch size.
        learning_rate: Learning rate for Adam optimizer.
        validation_fraction: The fraction of data to use for validation.
        stop_after_epochs: The number of epochs to wait for improvement on the
            validation set before terminating training.
        max_num_epochs: Maximum number of epochs to run. If reached, we stop
            training even when the validation loss is still decreasing. If None, we
            train until validation loss increases (see also `stop_after_epochs`).
        clip_max_norm: Value at which to clip the total gradient norm in order to
            prevent exploding gradients. Use None for no clipping.
        exclude_invalid_x: Whether to exclude simulation outputs `x=NaN` or `x=±∞`
            during training. Expect errors, silent or explicit, when `False`.
        resume_training: Can be used in case training time is limited, e.g. on a
            cluster. If `True`, the split between train and validation set, the
            optimizer, the number of epochs, and the best validation log-prob will
            be restored from the last time `.train()` was called.
        discard_prior_samples: Whether to discard samples simulated in round 1, i.e.
            from the prior. Training may be sped up by ignoring such less targeted
            samples.
        retrain_from_scratch_each_round: Whether to retrain the conditional density
            estimator for the posterior from scratch each round.
        show_train_summary: Whether to print the number of epochs and validation
            loss and leakage after the training.
        dataloader_kwargs: Additional or updated kwargs to be passed to the training
            and validation dataloaders (like, e.g., a collate_fn)

    Returns:
        Classifier that approximates the ratio $p(\theta,x)/p(\theta)p(x)$.
    """
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    return super().train(**kwargs)

sbi.inference.abc.mcabc.MCABC (ABCBASE)

__call__(self, x_o, num_simulations, eps=None, quantile=None, lra=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1, kde=False, kde_kwargs={}, return_summary=False) special

Run MCABC and return accepted parameters or KDE object fitted on them.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_simulations int

Number of simulations to run.

required
eps Optional[float]

Acceptance threshold \(\epsilon\) for distance between observed and simulated data.

None
quantile Optional[float]

Upper quantile of smallest distances for which the corresponding parameters are returned, e.g, q=0.01 will return the top 1%. Exactly one of quantile or eps have to be passed.

None
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1
kde bool

Whether to run KDE on the accepted parameters to return a KDE object from which one can sample.

False
kde_kwargs Optional[Dict[str, Any]]

kwargs for performing KDE: ‘bandwidth=’; either a float, or a string naming a bandwidth heuristics, e.g., ‘cv’ (cross validation), ‘silvermann’ or ‘scott’, default ‘cv’. ‘transform’: transform applied to the parameters before doing KDE. ‘sample_weights’: weights associated with samples. See ‘get_kde’ for more details

{}
return_summary bool

Whether to return the distances and data corresponding to the accepted parameters.

False

Returns:

Type Description
theta (if kde False)

accepted parameters kde (if kde True): KDE object based on accepted parameters from which one can .sample() and .log_prob(). summary (if summary True): dictionary containing the accepted paramters (if kde True), distances and simulated data x.

Source code in sbi/inference/abc/mcabc.py
def __call__(
    self,
    x_o: Union[Tensor, ndarray],
    num_simulations: int,
    eps: Optional[float] = None,
    quantile: Optional[float] = None,
    lra: bool = False,
    sass: bool = False,
    sass_fraction: float = 0.25,
    sass_expansion_degree: int = 1,
    kde: bool = False,
    kde_kwargs: Optional[Dict[str, Any]] = {},
    return_summary: bool = False,
) -> Union[Tuple[Tensor, dict], Tuple[KDEWrapper, dict], Tensor, KDEWrapper]:
    r"""Run MCABC and return accepted parameters or KDE object fitted on them.

    Args:
        x_o: Observed data.
        num_simulations: Number of simulations to run.
        eps: Acceptance threshold $\epsilon$ for distance between observed and
            simulated data.
        quantile: Upper quantile of smallest distances for which the corresponding
            parameters are returned, e.g, q=0.01 will return the top 1%. Exactly
            one of quantile or `eps` have to be passed.
        lra: Whether to run linear regression adjustment as in Beaumont et al. 2002
        sass: Whether to determine semi-automatic summary statistics as in
            Fearnhead & Prangle 2012.
        sass_fraction: Fraction of simulation budget used for the initial sass run.
        sass_expansion_degree: Degree of the polynomial feature expansion for the
            sass regression, default 1 - no expansion.
        kde: Whether to run KDE on the accepted parameters to return a KDE
            object from which one can sample.
        kde_kwargs: kwargs for performing KDE:
            'bandwidth='; either a float, or a string naming a bandwidth
            heuristics, e.g., 'cv' (cross validation), 'silvermann' or 'scott',
            default 'cv'.
            'transform': transform applied to the parameters before doing KDE.
            'sample_weights': weights associated with samples. See 'get_kde' for
            more details
        return_summary: Whether to return the distances and data corresponding to
            the accepted parameters.

    Returns:
        theta (if kde False): accepted parameters
        kde (if kde True): KDE object based on accepted parameters from which one
            can .sample() and .log_prob().
        summary (if summary True): dictionary containing the accepted paramters (if
            kde True), distances and simulated data x.
    """

    # Exactly one of eps or quantile need to be passed.
    assert (eps is not None) ^ (
        quantile is not None
    ), "Eps or quantile must be passed, but not both."

    # Run SASS and change the simulator and x_o accordingly.
    if sass:
        num_pilot_simulations = int(sass_fraction * num_simulations)
        self.logger.info(
            f"Running SASS with {num_pilot_simulations} pilot samples."
        )
        num_simulations -= num_pilot_simulations

        pilot_theta = self.prior.sample((num_pilot_simulations,))
        pilot_x = self._batched_simulator(pilot_theta)

        sass_transform = self.get_sass_transform(
            pilot_theta, pilot_x, sass_expansion_degree
        )

        simulator = lambda theta: sass_transform(self._batched_simulator(theta))
        x_o = sass_transform(x_o)
    else:
        simulator = self._batched_simulator

    # Simulate and calculate distances.
    theta = self.prior.sample((num_simulations,))
    x = simulator(theta)

    # Infer shape of x to test and set x_o.
    self.x_shape = x[0].unsqueeze(0).shape
    self.x_o = process_x(x_o, self.x_shape)

    distances = self.distance(self.x_o, x)

    # Select based on acceptance threshold epsilon.
    if eps is not None:
        is_accepted = distances < eps
        num_accepted = is_accepted.sum().item()
        assert num_accepted > 0, f"No parameters accepted, eps={eps} too small"

        theta_accepted = theta[is_accepted]
        distances_accepted = distances[is_accepted]
        x_accepted = x[is_accepted]

    # Select based on quantile on sorted distances.
    elif quantile is not None:
        num_top_samples = int(num_simulations * quantile)
        sort_idx = torch.argsort(distances)
        theta_accepted = theta[sort_idx][:num_top_samples]
        distances_accepted = distances[sort_idx][:num_top_samples]
        x_accepted = x[sort_idx][:num_top_samples]

    else:
        raise ValueError("One of epsilon or quantile has to be passed.")

    # Maybe adjust theta with LRA.
    if lra:
        self.logger.info("Running Linear regression adjustment.")
        final_theta = self.run_lra(theta_accepted, x_accepted, observation=self.x_o)
    else:
        final_theta = theta_accepted

    if kde:
        self.logger.info(
            f"""KDE on {final_theta.shape[0]} samples with bandwidth option
            {kde_kwargs["bandwidth"] if "bandwidth" in kde_kwargs else "cv"}.
            Beware that KDE can give unreliable results when used with too few
            samples and in high dimensions."""
        )

        kde_dist = get_kde(final_theta, **kde_kwargs)

        if return_summary:
            return kde_dist, dict(
                theta=final_theta, distances=distances_accepted, x=x_accepted
            )
        else:
            return kde_dist
    elif return_summary:
        return final_theta, dict(distances=distances_accepted, x=x_accepted)
    else:
        return final_theta

__init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True) special

Monte-Carlo Approximate Bayesian Computation (Rejection ABC) [1].

[1] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W. (1999). Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters $ heta$ and maps them to simulations, or observations, x, \(\mathrm{sim}( heta) o x\). Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
Source code in sbi/inference/abc/mcabc.py
def __init__(
    self,
    simulator: Callable,
    prior,
    distance: Union[str, Callable] = "l2",
    num_workers: int = 1,
    simulation_batch_size: int = 1,
    show_progress_bars: bool = True,
):
    """Monte-Carlo Approximate Bayesian Computation (Rejection ABC) [1].

    [1] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W.
    (1999). Population growth of human Y chromosomes: a study of Y chromosome
    microsatellites. Molecular biology and evolution, 16(12), 1791-1798.

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\mathrm{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        distance: Distance function to compare observed and simulated data. Can be
            a custom function or one of `l1`, `l2`, `mse`.
        num_workers: Number of parallel workers to use for simulations.
        simulation_batch_size: Number of parameter sets that the simulator
            maps to data x at once. If None, we simulate all parameter sets at the
            same time. If >= 1, the simulator has to process data of shape
            (simulation_batch_size, parameter_dimension).
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
    """

    super().__init__(
        simulator=simulator,
        prior=prior,
        distance=distance,
        num_workers=num_workers,
        simulation_batch_size=simulation_batch_size,
        show_progress_bars=show_progress_bars,
    )

choose_distance_function(distance_type='l2') inherited

Return distance function for given distance type.

Source code in sbi/inference/abc/mcabc.py
@staticmethod
def choose_distance_function(distance_type: str = "l2") -> Callable:
    """Return distance function for given distance type."""

    if distance_type == "mse":
        distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1)
    elif distance_type == "l2":
        distance = lambda xo, x: torch.norm((xo - x), dim=-1)
    elif distance_type == "l1":
        distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1)
    else:
        raise ValueError(r"Distance {distance_type} not supported.")

    def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor:
        """Return distance over batch dimension.

        Args:
            observed_data: Observed data, could be 1D.
            simulated_data: Batch of simulated data, has batch dimension.

        Returns:
            Torch tensor with batch of distances.
        """
        assert simulated_data.ndim == 2, "simulated data needs batch dimension"

        return distance(observed_data, simulated_data)

    return distance_fun

get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Following implementation in https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity and https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic

Source code in sbi/inference/abc/mcabc.py
@staticmethod
def get_sass_transform(
    theta: torch.Tensor,
    x: torch.Tensor,
    expansion_degree: int = 1,
    sample_weight=None,
) -> Callable:
    """Return semi-automatic summary statitics function.

    Running weighted linear regressin as in
    Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

    Following implementation in
    https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity
    and
    https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic
    """
    expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False)
    # Transform x, remove intercept.
    x_expanded = expansion.fit_transform(x)
    sumstats_map = np.zeros((x_expanded.shape[1], theta.shape[1]))

    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight
        )
        sumstats_map[:, parameter_idx] = regression_model.coef_

    sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32)

    def sumstats_transform(x):
        x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32)
        return x_expanded.mm(sumstats_map)

    return sumstats_transform

run_lra(theta, x, observation, sample_weight=None) inherited

Return parameters adjusted with linear regression adjustment.

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/mcabc.py
@staticmethod
def run_lra(
    theta: torch.Tensor,
    x: torch.Tensor,
    observation: torch.Tensor,
    sample_weight=None,
) -> torch.Tensor:
    """Return parameters adjusted with linear regression adjustment.

    Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254
    """

    theta_adjusted = theta
    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x,
            y=theta[:, parameter_idx],
            sample_weight=sample_weight,
        )
        theta_adjusted[:, parameter_idx] += regression_model.predict(
            observation.reshape(1, -1)
        )
        theta_adjusted[:, parameter_idx] -= regression_model.predict(x)

    return theta_adjusted

sbi.inference.abc.smcabc.SMCABC (ABCBASE)

__call__(self, x_o, num_particles, num_initial_pop, num_simulations, epsilon_decay, distance_based_decay=False, ess_min=None, kernel_variance_scale=1.0, use_last_pop_samples=True, return_summary=False, kde=False, kde_kwargs={}, kde_sample_weights=False, lra=False, lra_with_weights=False, sass=False, sass_fraction=0.25, sass_expansion_degree=1) special

Run SMCABC and return accepted parameters or KDE object fitted on them.

Parameters:

Name Type Description Default
x_o Union[torch.Tensor, numpy.ndarray]

Observed data.

required
num_particles int

Number of particles in each population.

required
num_initial_pop int

Number of simulations used for initial population.

required
num_simulations int

Total number of possible simulations.

required
epsilon_decay float

Factor with which the acceptance threshold \(\epsilon\) decays.

required
distance_based_decay bool

Whether the \(\epsilon\) decay is constant over populations or calculated from the previous populations distribution of distances.

False
ess_min Optional[float]

Threshold of effective sampling size for resampling weights. Not used when None (default).

None
kernel_variance_scale float

Factor for scaling the perturbation kernel variance.

1.0
use_last_pop_samples bool

Whether to fill up the current population with samples from the previous population when the budget is used up. If False, the current population is discarded and the previous population is returned.

True
lra bool

Whether to run linear regression adjustment as in Beaumont et al. 2002

False
lra_with_weights bool

Whether to run lra as weighted linear regression with SMC weights

False
sass bool

Whether to determine semi-automatic summary statistics as in Fearnhead & Prangle 2012.

False
sass_fraction float

Fraction of simulation budget used for the initial sass run.

0.25
sass_expansion_degree int

Degree of the polynomial feature expansion for the sass regression, default 1 - no expansion.

1
kde bool

Whether to run KDE on the accepted parameters to return a KDE object from which one can sample.

False
kde_kwargs Optional[Dict[str, Any]]

kwargs for performing KDE: ‘bandwidth=’; either a float, or a string naming a bandwidth heuristics, e.g., ‘cv’ (cross validation), ‘silvermann’ or ‘scott’, default ‘cv’. ‘transform’: transform applied to the parameters before doing KDE. ‘sample_weights’: weights associated with samples. See ‘get_kde’ for more details

{}
kde_sample_weights bool

Whether perform weighted KDE with SMC weights or on raw particles.

False
return_summary bool

Whether to return a dictionary with all accepted particles, weights, etc. at the end.

False

Returns:

Type Description
theta (if kde False)

accepted parameters of the last population. kde (if kde True): KDE object fitted on accepted parameters, from which one can .sample() and .log_prob(). summary (if return_summary True): dictionary containing the accepted paramters (if kde True), distances and simulated data x of all populations.

Source code in sbi/inference/abc/smcabc.py
def __call__(
    self,
    x_o: Union[Tensor, ndarray],
    num_particles: int,
    num_initial_pop: int,
    num_simulations: int,
    epsilon_decay: float,
    distance_based_decay: bool = False,
    ess_min: Optional[float] = None,
    kernel_variance_scale: float = 1.0,
    use_last_pop_samples: bool = True,
    return_summary: bool = False,
    kde: bool = False,
    kde_kwargs: Optional[Dict[str, Any]] = {},
    kde_sample_weights: bool = False,
    lra: bool = False,
    lra_with_weights: bool = False,
    sass: bool = False,
    sass_fraction: float = 0.25,
    sass_expansion_degree: int = 1,
) -> Union[Tensor, KDEWrapper, Tuple[Tensor, dict], Tuple[KDEWrapper, dict]]:
    r"""Run SMCABC and return accepted parameters or KDE object fitted on them.

    Args:
        x_o: Observed data.
        num_particles: Number of particles in each population.
        num_initial_pop: Number of simulations used for initial population.
        num_simulations: Total number of possible simulations.
        epsilon_decay: Factor with which the acceptance threshold $\epsilon$ decays.
        distance_based_decay: Whether the $\epsilon$ decay is constant over
            populations or calculated from the previous populations distribution of
            distances.
        ess_min: Threshold of effective sampling size for resampling weights. Not
            used when None (default).
        kernel_variance_scale: Factor for scaling the perturbation kernel variance.
        use_last_pop_samples: Whether to fill up the current population with
            samples from the previous population when the budget is used up. If
            False, the current population is discarded and the previous population
            is returned.
        lra: Whether to run linear regression adjustment as in Beaumont et al. 2002
        lra_with_weights: Whether to run lra as weighted linear regression with SMC
            weights
        sass: Whether to determine semi-automatic summary statistics as in
            Fearnhead & Prangle 2012.
        sass_fraction: Fraction of simulation budget used for the initial sass run.
        sass_expansion_degree: Degree of the polynomial feature expansion for the
            sass regression, default 1 - no expansion.
        kde: Whether to run KDE on the accepted parameters to return a KDE
            object from which one can sample.
        kde_kwargs: kwargs for performing KDE:
            'bandwidth='; either a float, or a string naming a bandwidth
            heuristics, e.g., 'cv' (cross validation), 'silvermann' or 'scott',
            default 'cv'.
            'transform': transform applied to the parameters before doing KDE.
            'sample_weights': weights associated with samples. See 'get_kde' for
            more details
        kde_sample_weights: Whether perform weighted KDE with SMC weights or on raw
            particles.
        return_summary: Whether to return a dictionary with all accepted particles,
            weights, etc. at the end.

    Returns:
        theta (if kde False): accepted parameters of the last population.
        kde (if kde True): KDE object fitted on accepted parameters, from which one
            can .sample() and .log_prob().
        summary (if return_summary True): dictionary containing the accepted
            paramters (if kde True), distances and simulated data x of all
            populations.
    """

    pop_idx = 0
    self.num_simulations = num_simulations

    # Pilot run for SASS.
    if sass:
        num_pilot_simulations = int(sass_fraction * num_simulations)
        self.logger.info(
            f"Running SASS with {num_pilot_simulations} pilot samples."
        )
        sass_transform = self.run_sass_set_xo(
            num_particles, num_pilot_simulations, x_o, lra, sass_expansion_degree
        )
        # Udpate simulator and xo
        x_o = sass_transform(self.x_o)

        def sass_simulator(theta):
            self.simulation_counter += theta.shape[0]
            return sass_transform(self._batched_simulator(theta))

        self._simulate_with_budget = sass_simulator

    # run initial population
    particles, epsilon, distances, x = self._set_xo_and_sample_initial_population(
        x_o, num_particles, num_initial_pop
    )
    log_weights = torch.log(1 / num_particles * ones(num_particles))

    self.logger.info(
        (
            f"population={pop_idx}, eps={epsilon}, ess={1.0}, "
            f"num_sims={num_initial_pop}"
        )
    )

    all_particles = [particles]
    all_log_weights = [log_weights]
    all_distances = [distances]
    all_epsilons = [epsilon]
    all_x = [x]

    while self.simulation_counter < self.num_simulations:

        pop_idx += 1
        # Decay based on quantile of distances from previous pop.
        if distance_based_decay:
            epsilon = self._get_next_epsilon(
                all_distances[pop_idx - 1], epsilon_decay
            )
        # Constant decay.
        else:
            epsilon *= epsilon_decay

        # Get kernel variance from previous pop.
        self.kernel_variance = self.get_kernel_variance(
            all_particles[pop_idx - 1],
            torch.exp(all_log_weights[pop_idx - 1]),
            samples_per_dim=500,
            kernel_variance_scale=kernel_variance_scale,
        )
        particles, log_weights, distances, x = self._sample_next_population(
            particles=all_particles[pop_idx - 1],
            log_weights=all_log_weights[pop_idx - 1],
            distances=all_distances[pop_idx - 1],
            epsilon=epsilon,
            x=all_x[pop_idx - 1],
            use_last_pop_samples=use_last_pop_samples,
        )

        # Resample population if effective sampling size is too small.
        if ess_min is not None:
            particles, log_weights = self.resample_if_ess_too_small(
                particles, log_weights, ess_min, pop_idx
            )

        self.logger.info(
            (
                f"population={pop_idx} done: eps={epsilon:.6f},"
                f" num_sims={self.simulation_counter}."
            )
        )

        # collect results
        all_particles.append(particles)
        all_log_weights.append(log_weights)
        all_distances.append(distances)
        all_epsilons.append(epsilon)
        all_x.append(x)

    # Maybe run LRA and adjust weights.
    if lra:
        self.logger.info("Running Linear regression adjustment.")
        adjusted_particles, adjusted_weights = self.run_lra_update_weights(
            particles=all_particles[-1],
            xs=all_x[-1],
            observation=x_o,
            log_weights=all_log_weights[-1],
            lra_with_weights=lra_with_weights,
        )
        final_particles = adjusted_particles
    else:
        final_particles = all_particles[-1]

    if kde:
        self.logger.info(
            f"""KDE on {final_particles.shape[0]} samples with bandwidth option
            {kde_kwargs["bandwidth"] if "bandwidth" in kde_kwargs else "cv"}.
            Beware that KDE can give unreliable results when used with too few
            samples and in high dimensions."""
        )
        # Maybe get particles weights from last population for weighted KDE.
        if kde_sample_weights:
            kde_kwargs["sample_weights"] = all_log_weights[-1].exp()

        kde_dist = get_kde(final_particles, **kde_kwargs)

        if return_summary:
            return (
                kde_dist,
                dict(
                    particles=all_particles,
                    weights=all_log_weights,
                    epsilons=all_epsilons,
                    distances=all_distances,
                    xs=all_x,
                ),
            )
        else:
            return kde_dist

    if return_summary:
        return (
            final_particles,
            dict(
                particles=all_particles,
                weights=all_log_weights,
                epsilons=all_epsilons,
                distances=all_distances,
                xs=all_x,
            ),
        )
    else:
        return final_particles

__init__(self, simulator, prior, distance='l2', num_workers=1, simulation_batch_size=1, show_progress_bars=True, kernel='gaussian', algorithm_variant='C') special

Sequential Monte Carlo Approximate Bayesian Computation.

We distinguish between three different SMC methods here: - A: Toni et al. 2010 (Phd Thesis) - B: Sisson et al. 2007 (with correction from 2009) - C: Beaumont et al. 2009

In Toni et al. 2010 we find an overview of the differences on page 34: - B: same as A except for resampling of weights if the effective sampling size is too small. - C: same as A except for calculation of the covariance of the perturbation kernel: the kernel covariance is a scaled version of the covariance of the previous population.

Parameters:

Name Type Description Default
simulator Callable

A function that takes parameters \(\theta\) and maps them to simulations, or observations, x, \(\mathrm{sim}(\theta)\to x\). Any regular Python callable (i.e. function or class with __call__ method) can be used.

required
prior Distribution

A probability distribution that expresses prior knowledge about the parameters, e.g. which ranges are meaningful for them. Any object with .log_prob()and .sample() (for example, a PyTorch distribution) can be used.

required
distance Union[str, Callable]

Distance function to compare observed and simulated data. Can be a custom function or one of l1, l2, mse.

'l2'
num_workers int

Number of parallel workers to use for simulations.

1
simulation_batch_size int

Number of parameter sets that the simulator maps to data x at once. If None, we simulate all parameter sets at the same time. If >= 1, the simulator has to process data of shape (simulation_batch_size, parameter_dimension).

1
show_progress_bars bool

Whether to show a progressbar during simulation and sampling.

True
kernel Optional[str]

Perturbation kernel.

'gaussian'
algorithm_variant str

Indicating the choice of algorithm variant, A, B, or C.

'C'
Source code in sbi/inference/abc/smcabc.py
def __init__(
    self,
    simulator: Callable,
    prior: Distribution,
    distance: Union[str, Callable] = "l2",
    num_workers: int = 1,
    simulation_batch_size: int = 1,
    show_progress_bars: bool = True,
    kernel: Optional[str] = "gaussian",
    algorithm_variant: str = "C",
):
    r"""Sequential Monte Carlo Approximate Bayesian Computation.

    We distinguish between three different SMC methods here:
        - A: Toni et al. 2010 (Phd Thesis)
        - B: Sisson et al. 2007 (with correction from 2009)
        - C: Beaumont et al. 2009

    In Toni et al. 2010 we find an overview of the differences on page 34:
        - B: same as A except for resampling of weights if the effective sampling
            size is too small.
        - C: same as A except for calculation of the covariance of the perturbation
            kernel: the kernel covariance is a scaled version of the covariance of
            the previous population.

    Args:
        simulator: A function that takes parameters $\theta$ and maps them to
            simulations, or observations, `x`, $\mathrm{sim}(\theta)\to x$. Any
            regular Python callable (i.e. function or class with `__call__` method)
            can be used.
        prior: A probability distribution that expresses prior knowledge about the
            parameters, e.g. which ranges are meaningful for them. Any
            object with `.log_prob()`and `.sample()` (for example, a PyTorch
            distribution) can be used.
        distance: Distance function to compare observed and simulated data. Can be
            a custom function or one of `l1`, `l2`, `mse`.
        num_workers: Number of parallel workers to use for simulations.
        simulation_batch_size: Number of parameter sets that the simulator
            maps to data x at once. If None, we simulate all parameter sets at the
            same time. If >= 1, the simulator has to process data of shape
            (simulation_batch_size, parameter_dimension).
        show_progress_bars: Whether to show a progressbar during simulation and
            sampling.
        kernel: Perturbation kernel.
        algorithm_variant: Indicating the choice of algorithm variant, A, B, or C.

    """

    super().__init__(
        simulator=simulator,
        prior=prior,
        distance=distance,
        num_workers=num_workers,
        simulation_batch_size=simulation_batch_size,
        show_progress_bars=show_progress_bars,
    )

    kernels = ("gaussian", "uniform")
    assert (
        kernel in kernels
    ), f"Kernel '{kernel}' not supported. Choose one from {kernels}."
    self.kernel = kernel

    algorithm_variants = ("A", "B", "C")
    assert algorithm_variant in algorithm_variants, (
        f"SMCABC variant '{algorithm_variant}' not supported, choose one from"
        " {algorithm_variants}."
    )
    self.algorithm_variant = algorithm_variant
    self.distance_to_x0 = None
    self.simulation_counter = 0
    self.num_simulations = 0

    # Define simulator that keeps track of budget.
    def simulate_with_budget(theta):
        self.simulation_counter += theta.shape[0]
        return self._batched_simulator(theta)

    self._simulate_with_budget = simulate_with_budget

choose_distance_function(distance_type='l2') inherited

Return distance function for given distance type.

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def choose_distance_function(distance_type: str = "l2") -> Callable:
    """Return distance function for given distance type."""

    if distance_type == "mse":
        distance = lambda xo, x: torch.mean((xo - x) ** 2, dim=-1)
    elif distance_type == "l2":
        distance = lambda xo, x: torch.norm((xo - x), dim=-1)
    elif distance_type == "l1":
        distance = lambda xo, x: torch.mean(abs(xo - x), dim=-1)
    else:
        raise ValueError(r"Distance {distance_type} not supported.")

    def distance_fun(observed_data: Tensor, simulated_data: Tensor) -> Tensor:
        """Return distance over batch dimension.

        Args:
            observed_data: Observed data, could be 1D.
            simulated_data: Batch of simulated data, has batch dimension.

        Returns:
            Torch tensor with batch of distances.
        """
        assert simulated_data.ndim == 2, "simulated data needs batch dimension"

        return distance(observed_data, simulated_data)

    return distance_fun

get_new_kernel(self, thetas)

Return new kernel distribution for a given set of paramters.

Source code in sbi/inference/abc/smcabc.py
def get_new_kernel(self, thetas: Tensor) -> Distribution:
    """Return new kernel distribution for a given set of paramters."""

    if self.kernel == "gaussian":
        assert self.kernel_variance.ndim == 2
        return MultivariateNormal(
            loc=thetas, covariance_matrix=self.kernel_variance
        )

    elif self.kernel == "uniform":
        low = thetas - self.kernel_variance
        high = thetas + self.kernel_variance
        # Move batch shape to event shape to get Uniform that is multivariate in
        # parameter dimension.
        return Uniform(low=low, high=high).to_event(1)
    else:
        raise ValueError(f"Kernel, '{self.kernel}' not supported.")

get_particle_ranges(self, particles, weights, samples_per_dim=100)

Return range of particles in each parameter dimension.

Source code in sbi/inference/abc/smcabc.py
def get_particle_ranges(
    self, particles: Tensor, weights: Tensor, samples_per_dim: int = 100
) -> Tensor:
    """Return range of particles in each parameter dimension."""

    # get weighted samples
    samples = self.sample_from_population_with_weights(
        particles,
        weights,
        num_samples=samples_per_dim * particles.shape[1],
    )

    # Variance spans the range of particles for every dimension.
    particle_ranges = tensor([max(column) - min(column) for column in samples.T])
    assert particle_ranges.ndim < 2
    return particle_ranges

get_sass_transform(theta, x, expansion_degree=1, sample_weight=None) inherited

Return semi-automatic summary statitics function.

Running weighted linear regressin as in Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

Following implementation in https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity and https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def get_sass_transform(
    theta: torch.Tensor,
    x: torch.Tensor,
    expansion_degree: int = 1,
    sample_weight=None,
) -> Callable:
    """Return semi-automatic summary statitics function.

    Running weighted linear regressin as in
    Fearnhead & Prandle 2012: https://arxiv.org/abs/1004.1112

    Following implementation in
    https://abcpy.readthedocs.io/en/latest/_modules/abcpy/statistics.html#Identity
    and
    https://pythonhosted.org/abcpy/_modules/abcpy/summaryselections.html#Semiautomatic
    """
    expansion = PolynomialFeatures(degree=expansion_degree, include_bias=False)
    # Transform x, remove intercept.
    x_expanded = expansion.fit_transform(x)
    sumstats_map = np.zeros((x_expanded.shape[1], theta.shape[1]))

    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x_expanded, y=theta[:, parameter_idx], sample_weight=sample_weight
        )
        sumstats_map[:, parameter_idx] = regression_model.coef_

    sumstats_map = torch.tensor(sumstats_map, dtype=torch.float32)

    def sumstats_transform(x):
        x_expanded = torch.tensor(expansion.fit_transform(x), dtype=torch.float32)
        return x_expanded.mm(sumstats_map)

    return sumstats_transform

resample_if_ess_too_small(self, particles, log_weights, ess_min, pop_idx)

Return resampled particles and uniform weights if effectice sampling size is too small.

Source code in sbi/inference/abc/smcabc.py
def resample_if_ess_too_small(
    self,
    particles: Tensor,
    log_weights: Tensor,
    ess_min: float,
    pop_idx: int,
) -> Tuple[Tensor, Tensor]:
    """Return resampled particles and uniform weights if effectice sampling size is
    too small.
    """

    num_particles = particles.shape[0]
    ess = (1 / torch.sum(torch.exp(2.0 * log_weights), dim=0)) / num_particles
    # Resampling of weights for low ESS only for Sisson et al. 2007.
    if ess < ess_min:
        self.logger.info(f"ESS={ess:.2f} too low, resampling pop {pop_idx}...")
        # First resample, then set to uniform weights as in Sisson et al. 2007.
        particles = self.sample_from_population_with_weights(
            particles, torch.exp(log_weights), num_samples=num_particles
        )
        log_weights = torch.log(1 / num_particles * ones(num_particles))

    return particles, log_weights

run_lra(theta, x, observation, sample_weight=None) inherited

Return parameters adjusted with linear regression adjustment.

Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def run_lra(
    theta: torch.Tensor,
    x: torch.Tensor,
    observation: torch.Tensor,
    sample_weight=None,
) -> torch.Tensor:
    """Return parameters adjusted with linear regression adjustment.

    Implementation as in Beaumont et al. 2002: https://arxiv.org/abs/1707.01254
    """

    theta_adjusted = theta
    for parameter_idx in range(theta.shape[1]):
        regression_model = LinearRegression(fit_intercept=True)
        regression_model.fit(
            X=x,
            y=theta[:, parameter_idx],
            sample_weight=sample_weight,
        )
        theta_adjusted[:, parameter_idx] += regression_model.predict(
            observation.reshape(1, -1)
        )
        theta_adjusted[:, parameter_idx] -= regression_model.predict(x)

    return theta_adjusted

run_lra_update_weights(self, particles, xs, observation, log_weights, lra_with_weights)

Return particles and weights adjusted with LRA.

Runs (weighted) linear regression from xs onto particles to adjust the particles.

Updates the SMC weights according to the new particles.

Source code in sbi/inference/abc/smcabc.py
def run_lra_update_weights(
    self,
    particles: Tensor,
    xs: Tensor,
    observation: Tensor,
    log_weights: Tensor,
    lra_with_weights: bool,
) -> Tuple[Tensor, Tensor]:
    """Return particles and weights adjusted with LRA.

    Runs (weighted) linear regression from xs onto particles to adjust the
    particles.

    Updates the SMC weights according to the new particles.
    """

    adjusted_particels = self.run_lra(
        theta=particles,
        x=xs,
        observation=observation,
        sample_weight=log_weights.exp() if lra_with_weights else None,
    )

    # Update SMC weights with LRA adjusted weights
    adjusted_log_weights = self._calculate_new_log_weights(
        new_particles=adjusted_particels,
        old_particles=particles,
        old_log_weights=log_weights,
    )

    return adjusted_particels, adjusted_log_weights

run_sass_set_xo(self, num_particles, num_pilot_simulations, x_o, lra=False, sass_expansion_degree=1)

Return transform for semi-automatic summary statistics.

Runs an single round of rejection abc with fixed budget and accepts num_particles simulations to run the regression for sass.

Sets self.x_o once the x_shape can be derived from simulations.

Source code in sbi/inference/abc/smcabc.py
def run_sass_set_xo(
    self,
    num_particles: int,
    num_pilot_simulations: int,
    x_o,
    lra: bool = False,
    sass_expansion_degree: int = 1,
) -> Callable:
    """Return transform for semi-automatic summary statistics.

    Runs an single round of rejection abc with fixed budget and accepts
    num_particles simulations to run the regression for sass.

    Sets self.x_o once the x_shape can be derived from simulations.
    """
    (pilot_particles, _, _, pilot_xs,) = self._set_xo_and_sample_initial_population(
        x_o, num_particles, num_pilot_simulations
    )
    # Adjust with LRA.
    if lra:
        pilot_particles = self.run_lra(pilot_particles, pilot_xs, self.x_o)
    sass_transform = self.get_sass_transform(
        pilot_particles,
        pilot_xs,
        expansion_degree=sass_expansion_degree,
        sample_weight=None,
    )
    return sass_transform

sample_from_population_with_weights(particles, weights, num_samples=1) staticmethod

Return samples from particles sampled with weights.

Source code in sbi/inference/abc/smcabc.py
@staticmethod
def sample_from_population_with_weights(
    particles: Tensor, weights: Tensor, num_samples: int = 1
) -> Tensor:
    """Return samples from particles sampled with weights."""

    # define multinomial with weights as probs
    multi = Multinomial(probs=weights)
    # sample num samples, with replacement
    samples = multi.sample(sample_shape=(num_samples,))
    # get indices of success trials
    indices = torch.where(samples)[1]
    # return those indices from trace
    return particles[indices]

Posteriors

sbi.inference.posteriors.direct_posterior.DirectPosterior (NeuralPosterior)

Posterior \(p(\theta|x)\) with log_prob() and sample() methods, obtained with SNPE.

SNPE trains a neural network to directly approximate the posterior distribution. However, for bounded priors, the neural network can have leakage: it puts non-zero mass in regions where the prior is zero. The DirectPosterior class wraps the trained network to deal with these cases.

Specifically, this class offers the following functionality:
- correct the calculation of the log probability such that it compensates for the leakage.
- reject samples that lie outside of the prior bounds.
- alternatively, if leakage is very high (which can happen for multi-round SNPE), sample from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

mcmc_method: str inherited property writable

Returns MCMC method.

mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

rejection_sampling_parameters: dict inherited property writable

Returns rejection sampling parameters.

sample_with: str inherited property writable

Return True if NeuralPosterior instance should use MCMC in .sample().

__init__(self, method_family, neural_net, prior, x_shape, sample_with='rejection', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of a single simulator output.

required
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. With default parameters, rejection samples from the posterior estimated by the neural net and rejects only if the samples are outside of the prior support.

'rejection'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the trained neural net). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
Source code in sbi/inference/posteriors/direct_posterior.py
def __init__(
    self,
    method_family: str,
    neural_net: nn.Module,
    prior,
    x_shape: torch.Size,
    sample_with: str = "rejection",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    device: str = "cpu",
):
    """
    Args:
        method_family: One of snpe, snl, snre_a or snre_b.
        neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
        prior: Prior distribution with `.log_prob()` and `.sample()`.
        x_shape: Shape of a single simulator output.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`]. With default parameters, `rejection` samples
            from the posterior estimated by the neural net and rejects only if the
            samples are outside of the prior support.
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the trained
            neural net). `max_sampling_batch_size` as the batchsize of samples
            being drawn from the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio. `m` as multiplier to that ratio.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
    """

    kwargs = del_entries(locals(), entries=("self", "__class__"))
    super().__init__(**kwargs)

    self._purpose = (
        "It allows to .sample() and .log_prob() the posterior and wraps the "
        "output of the .net to avoid leakage into regions with 0 prior probability."
    )

copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

  • Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
  • default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable

posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/direct_posterior.py
def copy_hyperparameters_from(
    self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
    """
    Copies the hyperparameters from a given posterior to `self`.

    The hyperparameters that are copied are:

    - Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
    - `default_x` at which to evaluate the posterior.

    Args:
        posterior: Posterior that the hyperparameters are copied from.

    Returns:
        Posterior object with the same hyperparameters as the passed posterior.
        This makes the call chainable:
        `posterior = infer.build_posterior().copy_hyperparameters_from(proposal)`
    """

    assert isinstance(
        posterior, NeuralPosterior
    ), "`copy_state_from` must be a `NeuralPosterior`."

    self.set_mcmc_method(posterior._mcmc_method)
    self.set_mcmc_parameters(posterior._mcmc_parameters)
    self.set_default_x(posterior.default_x)
    self._mcmc_init_params = posterior._mcmc_init_params
    if hasattr(self, "_sample_with_mcmc"):
        self.set_sample_with_mcmc(posterior._sample_with_mcmc)
    if hasattr(self, "_rejection_sampling_parameters"):
        self.set_rejection_sampling_parameters(
            posterior._rejection_sampling_parameters
        )

    return self

leakage_correction(self, x, num_rejection_samples=10000, force_update=False, show_progress_bars=False, rejection_sampling_batch_size=10000)

Return leakage correction factor for a leaky posterior density estimate.

The factor is estimated from the acceptance probability during rejection sampling from the posterior.

This is to avoid re-estimating the acceptance probability from scratch whenever log_prob is called and norm_posterior=True. Here, it is estimated only once for self.default_x and saved for later. We re-evaluate only whenever a new x is passed.

Parameters:

Name Type Description Default
x Tensor

Conditioning context for posterior \(p(\theta|x)\).

required
num_rejection_samples int

Number of samples used to estimate correction factor.

10000
force_update bool

Whether to force a reevaluation of the leakage correction even if the context x is the same as self.default_x. This is useful to enforce a new leakage estimate for rounds after the first (2, 3,..).

False
show_progress_bars bool

Whether to show a progress bar during sampling.

False
rejection_sampling_batch_size int

Batch size for rejection sampling.

10000

Returns:

Type Description
Tensor

Saved or newly-estimated correction factor (as a scalar Tensor).

Source code in sbi/inference/posteriors/direct_posterior.py
@torch.no_grad()
def leakage_correction(
    self,
    x: Tensor,
    num_rejection_samples: int = 10_000,
    force_update: bool = False,
    show_progress_bars: bool = False,
    rejection_sampling_batch_size: int = 10_000,
) -> Tensor:
    r"""Return leakage correction factor for a leaky posterior density estimate.

    The factor is estimated from the acceptance probability during rejection
    sampling from the posterior.

    This is to avoid re-estimating the acceptance probability from scratch
    whenever `log_prob` is called and `norm_posterior=True`. Here, it
    is estimated only once for `self.default_x` and saved for later. We
    re-evaluate only whenever a new `x` is passed.

    Arguments:
        x: Conditioning context for posterior $p(\theta|x)$.
        num_rejection_samples: Number of samples used to estimate correction factor.
        force_update: Whether to force a reevaluation of the leakage correction even
            if the context `x` is the same as `self.default_x`. This is useful to
            enforce a new leakage estimate for rounds after the first (2, 3,..).
        show_progress_bars: Whether to show a progress bar during sampling.
        rejection_sampling_batch_size: Batch size for rejection sampling.

    Returns:
        Saved or newly-estimated correction factor (as a scalar `Tensor`).
    """

    def acceptance_at(x: Tensor) -> Tensor:

        return utils.rejection_sample_posterior_within_prior(
            posterior_nn=self.net,
            prior=self._prior,
            x=x.to(self._device),
            num_samples=num_rejection_samples,
            show_progress_bars=show_progress_bars,
            sample_for_correction_factor=True,
            max_sampling_batch_size=rejection_sampling_batch_size,
        )[1]

    # Check if the provided x matches the default x (short-circuit on identity).
    is_new_x = self.default_x is None or (
        x is not self.default_x and (x != self.default_x).any()
    )

    not_saved_at_default_x = self._leakage_density_correction_factor is None

    if is_new_x:  # Calculate at x; don't save.
        return acceptance_at(x)
    elif not_saved_at_default_x or force_update:  # Calculate at default_x; save.
        self._leakage_density_correction_factor = acceptance_at(self.default_x)

    return self._leakage_density_correction_factor  # type:ignore

log_prob(self, theta, x=None, norm_posterior=True, track_gradients=False, leakage_correction_params=None)

Returns the log-probability of the posterior \(p(\theta|x).\)

Parameters:

Name Type Description Default
theta Tensor

Parameters \(\theta\).

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
norm_posterior bool

Whether to enforce a normalized posterior density. Renormalization of the posterior is useful when some probability falls out or leaks out of the prescribed prior support. The normalizing factor is calculated via rejection sampling, so if you need speedier but unnormalized log posterior estimates set here norm_posterior=False. The returned log posterior is set to -∞ outside of the prior support regardless of this setting.

True
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False
leakage_correction_params Optional[dict]

A dict of keyword arguments to override the default values of leakage_correction(). Possible options are: num_rejection_samples, force_update, show_progress_bars, and rejection_sampling_batch_size. These parameters only have an effect if norm_posterior=True.

None

Returns:

Type Description
Tensor

(len(θ),)-shaped log posterior probability \(\log p(\theta|x)\) for θ in the support of the prior, -∞ (corresponding to 0 probability) outside.

Source code in sbi/inference/posteriors/direct_posterior.py
def log_prob(
    self,
    theta: Tensor,
    x: Optional[Tensor] = None,
    norm_posterior: bool = True,
    track_gradients: bool = False,
    leakage_correction_params: Optional[dict] = None,
) -> Tensor:
    r"""
    Returns the log-probability of the posterior $p(\theta|x).$

    Args:
        theta: Parameters $\theta$.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        norm_posterior: Whether to enforce a normalized posterior density.
            Renormalization of the posterior is useful when some
            probability falls out or leaks out of the prescribed prior support.
            The normalizing factor is calculated via rejection sampling, so if you
            need speedier but unnormalized log posterior estimates set here
            `norm_posterior=False`. The returned log posterior is set to
            -∞ outside of the prior support regardless of this setting.
        track_gradients: Whether the returned tensor supports tracking gradients.
            This can be helpful for e.g. sensitivity analysis, but increases memory
            consumption.
        leakage_correction_params: A `dict` of keyword arguments to override the
            default values of `leakage_correction()`. Possible options are:
            `num_rejection_samples`, `force_update`, `show_progress_bars`, and
            `rejection_sampling_batch_size`.
            These parameters only have an effect if `norm_posterior=True`.

    Returns:
        `(len(θ),)`-shaped log posterior probability $\log p(\theta|x)$ for θ in the
        support of the prior, -∞ (corresponding to 0 probability) outside.
    """

    # TODO Train exited here, entered after sampling?
    self.net.eval()

    theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)
    theta_repeated, x_repeated = self._match_theta_and_x_batch_shapes(theta, x)

    with torch.set_grad_enabled(track_gradients):

        # Evaluate on device, move back to cpu for comparison with prior.
        unnorm_log_prob = self.net.log_prob(theta_repeated, x_repeated)

        # Force probability to be zero outside prior support.
        in_prior_support = within_support(self._prior, theta)

        masked_log_prob = torch.where(
            in_prior_support,
            unnorm_log_prob,
            torch.tensor(float("-inf"), dtype=torch.float32, device=self._device),
        )

        if leakage_correction_params is None:
            leakage_correction_params = dict()  # use defaults
        log_factor = (
            log(
                self.leakage_correction(
                    x=batched_first_of_batch(x), **leakage_correction_params
                )
            )
            if norm_posterior
            else 0
        )

        return masked_log_prob - log_factor

log_prob_conditional(self, theta, condition, dims_to_evaluate, x=None)

Evaluates the conditional posterior probability of a MDN at a context x for a value theta given a condition.

This function only works for MDN based posteriors, becuase evaluation is done analytically. For all other density estimators a NotImplementedError will be raised!

Parameters:

Name Type Description Default
theta Tensor

Parameters $ heta$.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_evaluate List[int]

Which dimensions to evaluate the sample for. The dimensions not specified in dims_to_evaluate will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p( heta|x)\). If not provided, fall back onto x passed to set_default_x().

None

Returns:

Type Description
log_prob

(len(θ),)-shaped normalized (!) log posterior probability $\log p( heta|x) for θ in the support of the prior, -∞ (corresponding to 0 probability) outside.

Source code in sbi/inference/posteriors/direct_posterior.py
def log_prob_conditional(
    self,
    theta: Tensor,
    condition: Tensor,
    dims_to_evaluate: List[int],
    x: Optional[Tensor] = None,
) -> Tensor:
    """Evaluates the conditional posterior probability of a MDN at a context x for
    a value theta given a condition.

    This function only works for MDN based posteriors, becuase evaluation is done
    analytically. For all other density estimators a `NotImplementedError` will be
    raised!

    Args:
        theta: Parameters $\theta$.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_evaluate: Which dimensions to evaluate the sample for.
            The dimensions not specified in `dims_to_evaluate` will be fixed to values given in `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.

    Returns:
        log_prob: `(len(θ),)`-shaped normalized (!) log posterior probability
            $\log p(\theta|x) for θ in the support of the prior, -∞ (corresponding
            to 0 probability) outside.
    """

    if type(self.net._distribution) == mdn:
        logits, means, precfs, sumlogdiag = extract_and_transform_mog(self, x)
        logits, means, precfs, sumlogdiag = condition_mog(
            self._prior, condition, dims_to_evaluate, logits, means, precfs
        )

        batch_size, dim = theta.shape
        prec = precfs.transpose(3, 2) @ precfs

        self.net.eval()  # leakage correction requires eval mode

        if dim != len(dims_to_evaluate):
            X = X[:, dims_to_evaluate]

        # Implementing leakage correction is difficult for conditioned MDNs,
        # because samples from self i.e. the full posterior are used rather
        # then from the new, conditioned posterior.
        warn("Probabilities are not adjusted for leakage.")

        log_prob = mdn.log_prob_mog(
            theta,
            logits.repeat(batch_size, 1),
            means.repeat(batch_size, 1, 1),
            prec.repeat(batch_size, 1, 1, 1),
            sumlogdiag.repeat(batch_size, 1),
        )

        self.net.train(True)
        return log_prob.detach()

    else:
        raise NotImplementedError(
            "This functionality is only available for MDN based posteriors."
        )

map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=1000, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior \(p( heta|x)\). If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

1000
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/direct_posterior.py
def map(
    self,
    x: Optional[Tensor] = None,
    num_iter: int = 1000,
    learning_rate: float = 1e-2,
    init_method: Union[str, Tensor] = "posterior",
    num_init_samples: int = 1_000,
    num_to_optimize: int = 100,
    save_best_every: int = 10,
    show_progress_bars: bool = True,
) -> Tensor:
    """
    Returns the maximum-a-posteriori estimate (MAP).

    The method can be interrupted (Ctrl-C) when the user sees that the
    log-probability converges. The best estimate will be saved in `self.map_`.

    The MAP is obtained by running gradient ascent from a given number of starting
    positions (samples from the posterior with the highest log-probability). After
    the optimization is done, we select the parameter set that has the highest
    log-probability after the optimization.

    Warning: The default values used by this function are not well-tested. They
    might require hand-tuning for the problem at hand.

    Args:
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        num_iter: Number of optimization steps that the algorithm takes
            to find the MAP.
        learning_rate: Learning rate of the optimizer.
        init_method: How to select the starting parameters for the optimization. If
            it is a string, it can be either [`posterior`, `prior`], which samples
            the respective distribution `num_init_samples` times. If it is a,
            the tensor will be used as init locations.
        num_init_samples: Draw this number of samples from the posterior and
            evaluate the log-probability of all of them.
        num_to_optimize: From the drawn `num_init_samples`, use the
            `num_to_optimize` with highest log-probability as the initial points
            for the optimization.
        save_best_every: The best log-probability is computed, saved in the
            `map`-attribute, and printed every `print_best_every`-th iteration.
            Computing the best log-probability creates a significant overhead
            (thus, the default is `10`.)
        show_progress_bars: Whether or not to show a progressbar for sampling from
            the posterior.

    Returns:
        The MAP estimate.
    """
    return super().map(
        x=x,
        num_iter=num_iter,
        learning_rate=learning_rate,
        init_method=init_method,
        num_init_samples=num_init_samples,
        num_to_optimize=num_to_optimize,
        save_best_every=save_best_every,
        show_progress_bars=show_progress_bars,
        log_prob_kwargs={"norm_posterior": False},
    )

sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None, sample_with_mcmc=None)

Return samples from posterior distribution \(p(\theta|x)\).

Samples are obtained either with rejection sampling or MCMC. Rejection sampling will be a lot faster if leakage is rather low. If leakage is high (e.g. over 99%, which can happen in multi-round SNPE), MCMC can be faster than rejection sampling.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with Optional[str]

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. With default parameters, rejection samples from the posterior estimated by the neural net and rejects only if the samples are outside of the prior support.

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the trained neural net). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
sample_with_mcmc Optional[bool]

Deprecated since sbi v0.17.0. Use sample_with=mcmc instead.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
def sample(
    self,
    sample_shape: Shape = torch.Size(),
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    sample_with: Optional[str] = None,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    sample_with_mcmc: Optional[bool] = None,
) -> Tensor:
    r"""
    Return samples from posterior distribution $p(\theta|x)$.

    Samples are obtained either with rejection sampling or MCMC. Rejection sampling
    will be a lot faster if leakage is rather low. If leakage is high (e.g. over
    99%, which can happen in multi-round SNPE), MCMC can be faster than rejection
    sampling.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`]. With default parameters, `rejection` samples
            from the posterior estimated by the neural net and rejects only if the
            samples are outside of the prior support.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported:
            `thin` to set the thinning factor for the chain.
            `warmup_steps` to set the initial number of samples to discard.
            `num_chains` for the number of chains.
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
            `enable_transform` a bool indicating whether MCMC is performed in
            z-scored (and unconstrained) space.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the trained
            neural net).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio.
            `m` as multiplier to that ratio.
        sample_with_mcmc: Deprecated since `sbi v0.17.0`. Use `sample_with=mcmc`
            instead.

    Returns:
        Samples from posterior.
    """

    if sample_with_mcmc is not None:
        warn(
            f"You set `sample_with_mcmc={sample_with_mcmc}`. This is deprecated "
            "since `sbi v0.17.0` and will lead to an error in future versions. "
            "Please use `sample_with='mcmc'` instead."
        )
        if sample_with_mcmc:
            sample_with = "mcmc"

    self.net.eval()

    sample_with = sample_with if sample_with is not None else self._sample_with

    x, num_samples = self._prepare_for_sample(x, sample_shape)

    potential_fn_provider = PotentialFunctionProvider()
    if sample_with == "mcmc":
        mcmc_method, mcmc_parameters = self._potentially_replace_mcmc_parameters(
            mcmc_method, mcmc_parameters
        )
        transform = mcmc_transform(
            self._prior, device=self._device, **mcmc_parameters
        )
        transformed_samples = self._sample_posterior_mcmc(
            num_samples=num_samples,
            potential_fn=potential_fn_provider(
                self._prior, self.net, x, mcmc_method, transform
            ),
            init_fn=self._build_mcmc_init_fn(
                self._prior,
                potential_fn_provider(
                    self._prior, self.net, x, "slice_np", transform
                ),
                transform=transform,
                **mcmc_parameters,
            ),
            mcmc_method=mcmc_method,
            show_progress_bars=show_progress_bars,
            **mcmc_parameters,
        )
        samples = transform.inv(transformed_samples)
    elif sample_with == "rejection":
        rejection_sampling_parameters = (
            self._potentially_replace_rejection_parameters(
                rejection_sampling_parameters
            )
        )
        if "proposal" not in rejection_sampling_parameters:
            assert (
                not self.net.training
            ), "Posterior nn must be in eval mode for sampling."

            # If the user does not explictly pass a `proposal`, we sample from the
            # neural net estimating the posterior and reject only those samples
            # that are outside of the prior support. This can be considered as
            # rejection sampling with a very good proposal.
            samples = utils.rejection_sample_posterior_within_prior(
                posterior_nn=self.net,
                prior=self._prior,
                x=x.to(self._device),
                num_samples=num_samples,
                show_progress_bars=show_progress_bars,
                sample_for_correction_factor=True,
                **rejection_sampling_parameters,
            )[0]
        else:
            samples, _ = rejection_sample(
                potential_fn=potential_fn_provider(
                    self._prior, self.net, x, "rejection"
                ),
                num_samples=num_samples,
                show_progress_bars=show_progress_bars,
                **rejection_sampling_parameters,
            )
    else:
        raise NameError(
            "The only implemented sampling methods are `mcmc` and `rejection`."
        )

    self.net.train(True)

    return samples.reshape((*sample_shape, -1))

sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, sample_with='mcmc', show_progress_bars=True, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from conditional posterior \(p(\theta_i|\theta_j, x)\).

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. In this method, the value of self.sample_with will be ignored.

'mcmc'
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/direct_posterior.py
def sample_conditional(
    self,
    sample_shape: Shape,
    condition: Tensor,
    dims_to_sample: List[int],
    x: Optional[Tensor] = None,
    sample_with: str = "mcmc",
    show_progress_bars: bool = True,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

    In this function, we do not sample from the full posterior, but instead only
    from a few parameter dimensions while the other parameter dimensions are kept
    fixed at values specified in `condition`.

    Samples are obtained with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_sample: Which dimensions to sample from. The dimensions not
            specified in `dims_to_sample` will be fixed to values given in
            `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`]. In this method, the value of
            `self.sample_with` will be ignored.
        show_progress_bars: Whether to show sampling progress monitor.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported:
            `thin` to set the thinning factor for the chain.
            `warmup_steps` to set the initial number of samples to discard.
            `num_chains` for the number of chains.
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
            `enable_transform` a bool indicating whether MCMC is performed in
            z-scored (and unconstrained) space.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio.
            `m` as multiplier to that ratio.

    Returns:
        Samples from conditional posterior.
    """
    if not hasattr(self.net, "_distribution"):
        raise NotImplementedError(
            "`sample_conditional` is not implemented for SNPE-A."
        )

    net = self.net
    x = atleast_2d_float32_tensor(self._x_else_default_x(x))

    if type(net._distribution) is mdn:
        condition = atleast_2d_float32_tensor(condition)
        num_samples = torch.Size(sample_shape).numel()

        logits, means, precfs, _ = extract_and_transform_mog(nn=net, context=x)
        logits, means, precfs, _ = condition_mog(
            self._prior, condition, dims_to_sample, logits, means, precfs
        )

        # Currently difficult to integrate `sample_posterior_within_prior`.
        warn(
            "Sampling MoG analytically. "
            "Some of the samples might not be within the prior support!"
        )
        samples = mdn.sample_mog(num_samples, logits, means, precfs)
        return samples.detach().reshape((*sample_shape, -1))

    else:
        return super().sample_conditional(
            PotentialFunctionProvider(),
            sample_shape,
            condition,
            dims_to_sample,
            x,
            sample_with,
            show_progress_bars,
            mcmc_method,
            mcmc_parameters,
            rejection_sampling_parameters,
        )

set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior \(p(theta|x)\).

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
    """Set new default x for `.sample(), .log_prob` to use as conditioning context.

    This is a pure convenience to avoid having to repeatedly specify `x` in calls to
    `.sample()` and `.log_prob()` - only θ needs to be passed.

    This convenience is particularly useful when the posterior is focused, i.e.
    has been trained over multiple rounds to be accurate in the vicinity of a
    particular `x=x_o` (you can check if your posterior object is focused by
    printing it).

    NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
    that calls like `posterior.set_default_x(my_x).sample(mytheta)` are possible.

    Args:
        x: The default observation to set for the posterior $p(theta|x)$.

    Returns:
        `NeuralPosterior` that will use a default `x` when not explicitly passed.
    """
    self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x).to(
        self._device
    )
    self._num_iid_trials = self._x.shape[0]

    return self

set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
    """Sets sampling method to for MCMC and returns `NeuralPosterior`.

    Args:
        method: Method to use.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_method = method
    return self

set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
    """Sets parameters for MCMC and returns `NeuralPosterior`.

    Args:
        parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_parameters = parameters
    return self

set_rejection_sampling_parameters(self, parameters) inherited

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. m as multiplier to that ratio. sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration.

required

Returns:

Type Description
NeuralPosterior

`NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_rejection_sampling_parameters(
    self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
    """Sets parameters for rejection sampling and returns `NeuralPosterior`.

    Args:
        parameters: Dictonary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution. `num_samples_to_find_max`
            as the number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `m` as multiplier to that ratio.
            `sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.

    Returns:
        `NeuralPosterior for chainable calls.
    """
    self._rejection_sampling_parameters = parameters
    return self

set_sample_with(self, sample_with) inherited

Set the sampling method for the NeuralPosterior.

Parameters:

Name Type Description Default
sample_with str

The method to sample with.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/direct_posterior.py
def set_sample_with(self, sample_with: str) -> "NeuralPosterior":
    """Set the sampling method for the `NeuralPosterior`.

    Args:
        sample_with: The method to sample with.

    Returns:
        `NeuralPosterior` for chainable calls.

    Raises:
        ValueError: on attempt to turn off MCMC sampling for family of methods that
            do not support rejection sampling.
    """
    if sample_with not in ("mcmc", "rejection"):
        raise NameError(
            "The only implemented sampling methods are `mcmc` and `rejection`."
        )
    self._sample_with = sample_with
    return self

sbi.inference.posteriors.likelihood_based_posterior.LikelihoodBasedPosterior (NeuralPosterior)

Posterior \(p(\theta|x)\) with log_prob() and sample() methods, obtained with SNLE.

SNLE trains a neural network to approximate the likelihood \(p(x|\theta)\). The SNLE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log probability \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\) and draw samples from the posterior with MCMC.

The neural network itself can be accessed via the .net attribute.

default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

mcmc_method: str inherited property writable

Returns MCMC method.

mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

rejection_sampling_parameters: dict inherited property writable

Returns rejection sampling parameters.

sample_with: str inherited property writable

Return True if NeuralPosterior instance should use MCMC in .sample().

__init__(self, method_family, neural_net, prior, x_shape, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations.

required
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def __init__(
    self,
    method_family: str,
    neural_net: nn.Module,
    prior,
    x_shape: torch.Size,
    sample_with: str = "mcmc",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    device: str = "cpu",
):
    """
    Args:
        method_family: One of snpe, snl, snre_a or snre_b.
        neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
        prior: Prior distribution with `.log_prob()` and `.sample()`.
        x_shape: Shape of the simulated data. It can differ from the
            observed data the posterior is conditioned on later in the batch
            dimension. If it differs, the additional entries are interpreted as
            independent and identically distributed data / trials. I.e., the data is
            assumed to be generated based on the same (unknown) model parameters or
            experimental condations.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration. `num_samples_to_find_max` as the
            number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `num_iter_to_find_max` as the number
            of gradient ascent iterations to find the maximum of that ratio. `m` as
            multiplier to that ratio.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
    """

    kwargs = del_entries(locals(), entries=("self", "__class__"))
    super().__init__(**kwargs)

    self._purpose = (
        "It provides MCMC to .sample() from the posterior and "
        "can evaluate the _unnormalized_ posterior density with .log_prob()."
    )

copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

  • Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
  • default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable

posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def copy_hyperparameters_from(
    self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
    """
    Copies the hyperparameters from a given posterior to `self`.

    The hyperparameters that are copied are:

    - Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
    - `default_x` at which to evaluate the posterior.

    Args:
        posterior: Posterior that the hyperparameters are copied from.

    Returns:
        Posterior object with the same hyperparameters as the passed posterior.
        This makes the call chainable:
        `posterior = infer.build_posterior().copy_hyperparameters_from(proposal)`
    """

    assert isinstance(
        posterior, NeuralPosterior
    ), "`copy_state_from` must be a `NeuralPosterior`."

    self.set_mcmc_method(posterior._mcmc_method)
    self.set_mcmc_parameters(posterior._mcmc_parameters)
    self.set_default_x(posterior.default_x)
    self._mcmc_init_params = posterior._mcmc_init_params
    if hasattr(self, "_sample_with_mcmc"):
        self.set_sample_with_mcmc(posterior._sample_with_mcmc)
    if hasattr(self, "_rejection_sampling_parameters"):
        self.set_rejection_sampling_parameters(
            posterior._rejection_sampling_parameters
        )

    return self

log_prob(self, theta, x=None, track_gradients=False)

Returns the log-probability of \(p(x|\theta) \cdot p(\theta).\)

This corresponds to an unnormalized posterior log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters \(\theta\).

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability \(\log(p(x|\theta) \cdot p(\theta))\).

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def log_prob(
    self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False
) -> Tensor:
    r"""
    Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

    This corresponds to an **unnormalized** posterior log-probability.

    Args:
        theta: Parameters $\theta$.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        track_gradients: Whether the returned tensor supports tracking gradients.
            This can be helpful for e.g. sensitivity analysis, but increases memory
            consumption.

    Returns:
        `(len(θ),)`-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

    """

    # TODO Train exited here, entered after sampling?
    self.net.eval()

    theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)

    # Calculate likelihood over trials and in one batch.
    warn(
        "The log probability from SNL is only correct up to a normalizing constant."
    )
    log_likelihood_trial_sum = self._log_likelihoods_over_trials(
        x=x.to(self._device),
        theta=theta.to(self._device),
        net=self.net,
        track_gradients=track_gradients,
    )

    # Move to cpu for comparison with prior.
    return log_likelihood_trial_sum + self._prior.log_prob(theta)

map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior \(p( heta|x)\). If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def map(
    self,
    x: Optional[Tensor] = None,
    num_iter: int = 1000,
    learning_rate: float = 1e-2,
    init_method: Union[str, Tensor] = "posterior",
    num_init_samples: int = 500,
    num_to_optimize: int = 100,
    save_best_every: int = 10,
    show_progress_bars: bool = True,
) -> Tensor:
    """
    Returns the maximum-a-posteriori estimate (MAP).

    The method can be interrupted (Ctrl-C) when the user sees that the
    log-probability converges. The best estimate will be saved in `self.map_`.

    The MAP is obtained by running gradient ascent from a given number of starting
    positions (samples from the posterior with the highest log-probability). After
    the optimization is done, we select the parameter set that has the highest
    log-probability after the optimization.

    Warning: The default values used by this function are not well-tested. They
    might require hand-tuning for the problem at hand.

    Args:
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        num_iter: Number of optimization steps that the algorithm takes
            to find the MAP.
        learning_rate: Learning rate of the optimizer.
        init_method: How to select the starting parameters for the optimization. If
            it is a string, it can be either [`posterior`, `prior`], which samples
            the respective distribution `num_init_samples` times. If it is a,
            the tensor will be used as init locations.
        num_init_samples: Draw this number of samples from the posterior and
            evaluate the log-probability of all of them.
        num_to_optimize: From the drawn `num_init_samples`, use the
            `num_to_optimize` with highest log-probability as the initial points
            for the optimization.
        save_best_every: The best log-probability is computed, saved in the
            `map`-attribute, and printed every `print_best_every`-th iteration.
            Computing the best log-probability creates a significant overhead
            (thus, the default is `10`.)
        show_progress_bars: Whether or not to show a progressbar for sampling from
            the posterior.

    Returns:
        The MAP estimate.
    """
    return super().map(
        x=x,
        num_iter=num_iter,
        learning_rate=learning_rate,
        init_method=init_method,
        num_init_samples=num_init_samples,
        num_to_optimize=num_to_optimize,
        save_best_every=save_best_every,
        show_progress_bars=show_progress_bars,
    )

sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from posterior distribution \(p(\theta|x)\) with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with Optional[str]

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def sample(
    self,
    sample_shape: Shape = torch.Size(),
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    sample_with: Optional[str] = None,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from posterior distribution $p(\theta|x)$ with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported:
            `thin` to set the thinning factor for the chain.
            `warmup_steps` to set the initial number of samples to discard.
            `num_chains` for the number of chains.
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
            `enable_transform` a bool indicating whether MCMC is performed in
            z-scored (and unconstrained) space.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio.
            `m` as multiplier to that ratio.

    Returns:
        Samples from posterior.
    """

    self.net.eval()

    sample_with = sample_with if sample_with is not None else self._sample_with
    x, num_samples = self._prepare_for_sample(x, sample_shape)

    potential_fn_provider = PotentialFunctionProvider()
    if sample_with == "mcmc":

        mcmc_method, mcmc_parameters = self._potentially_replace_mcmc_parameters(
            mcmc_method, mcmc_parameters
        )

        transform = mcmc_transform(
            self._prior, device=self._device, **mcmc_parameters
        )

        transformed_samples = self._sample_posterior_mcmc(
            num_samples=num_samples,
            potential_fn=potential_fn_provider(
                self._prior, self.net, x, mcmc_method, transform
            ),
            init_fn=self._build_mcmc_init_fn(
                self._prior,
                potential_fn_provider(
                    self._prior, self.net, x, "slice_np", transform
                ),
                transform=transform,
                **mcmc_parameters,
            ),
            mcmc_method=mcmc_method,
            show_progress_bars=show_progress_bars,
            **mcmc_parameters,
        )
        samples = transform.inv(transformed_samples)
    elif sample_with == "rejection":
        rejection_sampling_parameters = (
            self._potentially_replace_rejection_parameters(
                rejection_sampling_parameters
            )
        )
        if "proposal" not in rejection_sampling_parameters:
            rejection_sampling_parameters["proposal"] = self._prior

        samples, _ = rejection_sample(
            potential_fn=potential_fn_provider(
                self._prior,
                self.net,
                x,
                "rejection",
            ),
            num_samples=num_samples,
            **rejection_sampling_parameters,
        )
    else:
        raise NameError(
            "The only implemented sampling methods are `mcmc` and `rejection`."
        )

    self.net.train(True)

    return samples.reshape((*sample_shape, -1))

sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, sample_with='mcmc', show_progress_bars=True, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from conditional posterior \(p(\theta_i|\theta_j, x)\).

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. In this method, the value of self._sample_with will be ignored.

'mcmc'
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def sample_conditional(
    self,
    sample_shape: Shape,
    condition: Tensor,
    dims_to_sample: List[int],
    x: Optional[Tensor] = None,
    sample_with: str = "mcmc",
    show_progress_bars: bool = True,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

    In this function, we do not sample from the full posterior, but instead only
    from a few parameter dimensions while the other parameter dimensions are kept
    fixed at values specified in `condition`.

    Samples are obtained with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_sample: Which dimensions to sample from. The dimensions not
            specified in `dims_to_sample` will be fixed to values given in
            `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`]. In this method, the value of
            `self._sample_with` will be ignored.
        show_progress_bars: Whether to show sampling progress monitor.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported:
            `thin` to set the thinning factor for the chain.
            `warmup_steps` to set the initial number of samples to discard.
            `num_chains` for the number of chains.
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
            `enable_transform` a bool indicating whether MCMC is performed in
            z-scored (and unconstrained) space.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio.
            `m` as multiplier to that ratio.

    Returns:
        Samples from conditional posterior.
    """

    return super().sample_conditional(
        PotentialFunctionProvider(),
        sample_shape,
        condition,
        dims_to_sample,
        x,
        sample_with,
        show_progress_bars,
        mcmc_method,
        mcmc_parameters,
        rejection_sampling_parameters,
    )

set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior \(p(theta|x)\).

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
    """Set new default x for `.sample(), .log_prob` to use as conditioning context.

    This is a pure convenience to avoid having to repeatedly specify `x` in calls to
    `.sample()` and `.log_prob()` - only θ needs to be passed.

    This convenience is particularly useful when the posterior is focused, i.e.
    has been trained over multiple rounds to be accurate in the vicinity of a
    particular `x=x_o` (you can check if your posterior object is focused by
    printing it).

    NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
    that calls like `posterior.set_default_x(my_x).sample(mytheta)` are possible.

    Args:
        x: The default observation to set for the posterior $p(theta|x)$.

    Returns:
        `NeuralPosterior` that will use a default `x` when not explicitly passed.
    """
    self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x).to(
        self._device
    )
    self._num_iid_trials = self._x.shape[0]

    return self

set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
    """Sets sampling method to for MCMC and returns `NeuralPosterior`.

    Args:
        method: Method to use.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_method = method
    return self

set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
    """Sets parameters for MCMC and returns `NeuralPosterior`.

    Args:
        parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_parameters = parameters
    return self

set_rejection_sampling_parameters(self, parameters) inherited

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. m as multiplier to that ratio. sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration.

required

Returns:

Type Description
NeuralPosterior

`NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_rejection_sampling_parameters(
    self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
    """Sets parameters for rejection sampling and returns `NeuralPosterior`.

    Args:
        parameters: Dictonary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution. `num_samples_to_find_max`
            as the number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `m` as multiplier to that ratio.
            `sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.

    Returns:
        `NeuralPosterior for chainable calls.
    """
    self._rejection_sampling_parameters = parameters
    return self

set_sample_with(self, sample_with) inherited

Set the sampling method for the NeuralPosterior.

Parameters:

Name Type Description Default
sample_with str

The method to sample with.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/likelihood_based_posterior.py
def set_sample_with(self, sample_with: str) -> "NeuralPosterior":
    """Set the sampling method for the `NeuralPosterior`.

    Args:
        sample_with: The method to sample with.

    Returns:
        `NeuralPosterior` for chainable calls.

    Raises:
        ValueError: on attempt to turn off MCMC sampling for family of methods that
            do not support rejection sampling.
    """
    if sample_with not in ("mcmc", "rejection"):
        raise NameError(
            "The only implemented sampling methods are `mcmc` and `rejection`."
        )
    self._sample_with = sample_with
    return self

sbi.inference.posteriors.ratio_based_posterior.RatioBasedPosterior (NeuralPosterior)

Posterior \(p(\theta|x)\) with log_prob() and sample() methods, obtained with SNRE.

SNRE trains a neural network to approximate likelihood ratios, which in turn can be used obtain an unnormalized posterior \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\). The SNRE_Posterior class wraps the trained network such that one can directly evaluate the unnormalized posterior log-probability \(p(\theta|x) \propto p(x|\theta) \cdot p(\theta)\) and draw samples from the posterior with MCMC. Note that, in the case of single-round SNRE_A / AALR, it is possible to evaluate the log-probability of the normalized posterior, but sampling still requires MCMC.

The neural network itself can be accessed via the .net attribute.

default_x: Optional[torch.Tensor] inherited property writable

Return default x used by .sample(), .log_prob as conditioning context.

mcmc_method: str inherited property writable

Returns MCMC method.

mcmc_parameters: dict inherited property writable

Returns MCMC parameters.

rejection_sampling_parameters: dict inherited property writable

Returns rejection sampling parameters.

sample_with: str inherited property writable

Return True if NeuralPosterior instance should use MCMC in .sample().

__init__(self, method_family, neural_net, prior, x_shape, sample_with='mcmc', mcmc_method='slice_np', mcmc_parameters=None, rejection_sampling_parameters=None, device='cpu') special

Parameters:

Name Type Description Default
method_family str

One of snpe, snl, snre_a or snre_b.

required
neural_net Module

A classifier for SNRE, a density estimator for SNPE and SNL.

required
prior

Prior distribution with .log_prob() and .sample().

required
x_shape Size

Shape of the simulated data. It can differ from the observed data the posterior is conditioned on later in the batch dimension. If it differs, the additional entries are interpreted as independent and identically distributed data / trials. I.e., the data is assumed to be generated based on the same (unknown) model parameters or experimental condations.

required
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

'mcmc'
mcmc_method str

Method used for MCMC sampling, one of slice_np, slice, hmc, nuts. Currently defaults to slice_np for a custom numpy implementation of slice sampling; select hmc, nuts or slice for Pyro-based sampling.

'slice_np'
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None
device str

Training device, e.g., “cpu”, “cuda” or “cuda:{0, 1, …}”.

'cpu'
Source code in sbi/inference/posteriors/ratio_based_posterior.py
def __init__(
    self,
    method_family: str,
    neural_net: nn.Module,
    prior,
    x_shape: torch.Size,
    sample_with: str = "mcmc",
    mcmc_method: str = "slice_np",
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
    device: str = "cpu",
):
    """
    Args:
        method_family: One of snpe, snl, snre_a or snre_b.
        neural_net: A classifier for SNRE, a density estimator for SNPE and SNL.
        prior: Prior distribution with `.log_prob()` and `.sample()`.
        x_shape: Shape of the simulated data. It can differ from the
            observed data the posterior is conditioned on later in the batch
            dimension. If it differs, the additional entries are interpreted as
            independent and identically distributed data / trials. I.e., the data is
            assumed to be generated based on the same (unknown) model parameters or
            experimental condations.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Method used for MCMC sampling, one of `slice_np`, `slice`,
            `hmc`, `nuts`. Currently defaults to `slice_np` for a custom numpy
            implementation of slice sampling; select `hmc`, `nuts` or `slice` for
            Pyro-based sampling.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration. `num_samples_to_find_max` as the
            number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `num_iter_to_find_max` as the number
            of gradient ascent iterations to find the maximum of that ratio. `m` as
            multiplier to that ratio.
        device: Training device, e.g., "cpu", "cuda" or "cuda:{0, 1, ...}".
    """
    kwargs = del_entries(locals(), entries=("self", "__class__"))
    super().__init__(**kwargs)

copy_hyperparameters_from(self, posterior) inherited

Copies the hyperparameters from a given posterior to self.

The hyperparameters that are copied are:

  • Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
  • default_x at which to evaluate the posterior.

Parameters:

Name Type Description Default
posterior NeuralPosterior

Posterior that the hyperparameters are copied from.

required

Returns:

Type Description
Posterior object with the same hyperparameters as the passed posterior. This makes the call chainable

posterior = infer.build_posterior().copy_hyperparameters_from(proposal)

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def copy_hyperparameters_from(
    self, posterior: "NeuralPosterior"
) -> "NeuralPosterior":
    """
    Copies the hyperparameters from a given posterior to `self`.

    The hyperparameters that are copied are:

    - Sampling parameters (MCMC for all methods, rejection sampling for SNPE).
    - `default_x` at which to evaluate the posterior.

    Args:
        posterior: Posterior that the hyperparameters are copied from.

    Returns:
        Posterior object with the same hyperparameters as the passed posterior.
        This makes the call chainable:
        `posterior = infer.build_posterior().copy_hyperparameters_from(proposal)`
    """

    assert isinstance(
        posterior, NeuralPosterior
    ), "`copy_state_from` must be a `NeuralPosterior`."

    self.set_mcmc_method(posterior._mcmc_method)
    self.set_mcmc_parameters(posterior._mcmc_parameters)
    self.set_default_x(posterior.default_x)
    self._mcmc_init_params = posterior._mcmc_init_params
    if hasattr(self, "_sample_with_mcmc"):
        self.set_sample_with_mcmc(posterior._sample_with_mcmc)
    if hasattr(self, "_rejection_sampling_parameters"):
        self.set_rejection_sampling_parameters(
            posterior._rejection_sampling_parameters
        )

    return self

log_prob(self, theta, x=None, track_gradients=False)

Returns the log-probability of \(p(x|\theta) \cdot p(\theta).\)

This corresponds to an unnormalized posterior log-probability. Only for single-round SNRE_A / AALR, the returned log-probability will correspond to the normalized log-probability.

Parameters:

Name Type Description Default
theta Tensor

Parameters \(\theta\).

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
track_gradients bool

Whether the returned tensor supports tracking gradients. This can be helpful for e.g. sensitivity analysis, but increases memory consumption.

False

Returns:

Type Description
Tensor

(len(θ),)-shaped log-probability \(\log(p(x|\theta) \cdot p(\theta))\).

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def log_prob(
    self, theta: Tensor, x: Optional[Tensor] = None, track_gradients: bool = False
) -> Tensor:
    r"""Returns the log-probability of $p(x|\theta) \cdot p(\theta).$

    This corresponds to an **unnormalized** posterior log-probability. Only for
    single-round SNRE_A / AALR, the returned log-probability will correspond to the
    **normalized** log-probability.

    Args:
        theta: Parameters $\theta$.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        track_gradients: Whether the returned tensor supports tracking gradients.
            This can be helpful for e.g. sensitivity analysis, but increases memory
            consumption.

    Returns:
        `(len(θ),)`-shaped log-probability $\log(p(x|\theta) \cdot p(\theta))$.

    """

    # TODO Train exited here, entered after sampling?
    self.net.eval()

    theta, x = self._prepare_theta_and_x_for_log_prob_(theta, x)

    self._warn_log_prob_snre()

    # Sum log ratios over x batch of iid trials.
    log_ratio = self._log_ratios_over_trials(
        x.to(self._device),
        theta.to(self._device),
        self.net,
        track_gradients=track_gradients,
    )

    return log_ratio + self._prior.log_prob(theta)

map(self, x=None, num_iter=1000, learning_rate=0.01, init_method='posterior', num_init_samples=500, num_to_optimize=100, save_best_every=10, show_progress_bars=True)

Returns the maximum-a-posteriori estimate (MAP).

The method can be interrupted (Ctrl-C) when the user sees that the log-probability converges. The best estimate will be saved in self.map_.

The MAP is obtained by running gradient ascent from a given number of starting positions (samples from the posterior with the highest log-probability). After the optimization is done, we select the parameter set that has the highest log-probability after the optimization.

Warning: The default values used by this function are not well-tested. They might require hand-tuning for the problem at hand.

Parameters:

Name Type Description Default
x Optional[torch.Tensor]

Conditioning context for posterior \(p( heta|x)\). If not provided, fall back onto x passed to set_default_x().

None
num_iter int

Maximum Number of optimization steps that the algorithm takes to find the MAP.

1000
learning_rate float

Learning rate of the optimizer.

0.01
init_method Union[str, torch.Tensor]

How to select the starting parameters for the optimization. If it is a string, it can be either [posterior, prior], which samples the respective distribution num_init_samples times. If it is a, the tensor will be used as init locations.

'posterior'
num_init_samples int

Draw this number of samples from the posterior and evaluate the log-probability of all of them.

500
num_to_optimize int

From the drawn num_init_samples, use the num_to_optimize with highest log-probability as the initial points for the optimization.

100
save_best_every int

The best log-probability is computed, saved in the map-attribute, and printed every print_best_every-th iteration. Computing the best log-probability creates a significant overhead (thus, the default is 10.)

10
show_progress_bars bool

Whether or not to show a progressbar for sampling from the posterior.

True

Returns:

Type Description
Tensor

The MAP estimate.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def map(
    self,
    x: Optional[Tensor] = None,
    num_iter: int = 1000,
    learning_rate: float = 1e-2,
    init_method: Union[str, Tensor] = "posterior",
    num_init_samples: int = 500,
    num_to_optimize: int = 100,
    save_best_every: int = 10,
    show_progress_bars: bool = True,
) -> Tensor:
    """
    Returns the maximum-a-posteriori estimate (MAP).

    The method can be interrupted (Ctrl-C) when the user sees that the
    log-probability converges. The best estimate will be saved in `self.map_`.

    The MAP is obtained by running gradient ascent from a given number of starting
    positions (samples from the posterior with the highest log-probability). After
    the optimization is done, we select the parameter set that has the highest
    log-probability after the optimization.

    Warning: The default values used by this function are not well-tested. They
    might require hand-tuning for the problem at hand.

    Args:
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        num_iter: Maximum Number of optimization steps that the algorithm takes
            to find the MAP.
        learning_rate: Learning rate of the optimizer.
        init_method: How to select the starting parameters for the optimization. If
            it is a string, it can be either [`posterior`, `prior`], which samples
            the respective distribution `num_init_samples` times. If it is a,
            the tensor will be used as init locations.
        num_init_samples: Draw this number of samples from the posterior and
            evaluate the log-probability of all of them.
        num_to_optimize: From the drawn `num_init_samples`, use the
            `num_to_optimize` with highest log-probability as the initial points
            for the optimization.
        save_best_every: The best log-probability is computed, saved in the
            `map`-attribute, and printed every `print_best_every`-th iteration.
            Computing the best log-probability creates a significant overhead
            (thus, the default is `10`.)
        show_progress_bars: Whether or not to show a progressbar for sampling from
            the posterior.

    Returns:
        The MAP estimate.
    """
    return super().map(
        x=x,
        num_iter=num_iter,
        learning_rate=learning_rate,
        init_method=init_method,
        num_init_samples=num_init_samples,
        num_to_optimize=num_to_optimize,
        save_best_every=save_best_every,
        show_progress_bars=show_progress_bars,
    )

sample(self, sample_shape=torch.Size([]), x=None, show_progress_bars=True, sample_with=None, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from posterior distribution \(p(\theta|x)\) with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

torch.Size([])
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
show_progress_bars bool

Whether to show sampling progress monitor.

True
sample_with Optional[str]

Method to use for sampling from the posterior. Must be one of [mcmc | rejection].

None
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def sample(
    self,
    sample_shape: Shape = torch.Size(),
    x: Optional[Tensor] = None,
    show_progress_bars: bool = True,
    sample_with: Optional[str] = None,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from posterior distribution $p(\theta|x)$ with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        show_progress_bars: Whether to show sampling progress monitor.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`].
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported:
            `thin` to set the thinning factor for the chain.
            `warmup_steps` to set the initial number of samples to discard.
            `num_chains` for the number of chains.
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
            `enable_transform` a bool indicating whether MCMC is performed in
            z-scored (and unconstrained) space.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio.
            `m` as multiplier to that ratio.

    Returns:
        Samples from posterior.
    """

    self.net.eval()

    sample_with = sample_with if sample_with is not None else self._sample_with
    x, num_samples = self._prepare_for_sample(x, sample_shape)

    potential_fn_provider = PotentialFunctionProvider()
    if sample_with == "mcmc":
        mcmc_method, mcmc_parameters = self._potentially_replace_mcmc_parameters(
            mcmc_method, mcmc_parameters
        )

        transform = mcmc_transform(
            self._prior, device=self._device, **mcmc_parameters
        )

        transformed_samples = self._sample_posterior_mcmc(
            num_samples=num_samples,
            potential_fn=potential_fn_provider(
                self._prior, self.net, x, mcmc_method, transform
            ),
            init_fn=self._build_mcmc_init_fn(
                self._prior,
                potential_fn_provider(
                    self._prior, self.net, x, "slice_np", transform
                ),
                transform=transform,
                **mcmc_parameters,
            ),
            mcmc_method=mcmc_method,
            show_progress_bars=show_progress_bars,
            **mcmc_parameters,
        )
        samples = transform.inv(transformed_samples)
    elif sample_with == "rejection":
        rejection_sampling_parameters = (
            self._potentially_replace_rejection_parameters(
                rejection_sampling_parameters
            )
        )
        if "proposal" not in rejection_sampling_parameters:
            rejection_sampling_parameters["proposal"] = self._prior

        samples, _ = rejection_sample(
            potential_fn=potential_fn_provider(
                self._prior, self.net, x, "rejection"
            ),
            num_samples=num_samples,
            **rejection_sampling_parameters,
        )
    else:
        raise NameError(
            "The only implemented sampling methods are `mcmc` and `rejection`."
        )

    self.net.train(True)

    return samples.reshape((*sample_shape, -1))

sample_conditional(self, sample_shape, condition, dims_to_sample, x=None, sample_with='mcmc', show_progress_bars=True, mcmc_method=None, mcmc_parameters=None, rejection_sampling_parameters=None)

Return samples from conditional posterior \(p(\theta_i|\theta_j, x)\).

In this function, we do not sample from the full posterior, but instead only from a few parameter dimensions while the other parameter dimensions are kept fixed at values specified in condition.

Samples are obtained with MCMC.

Parameters:

Name Type Description Default
sample_shape Union[torch.Size, Tuple[int, ...]]

Desired shape of samples that are drawn from posterior. If sample_shape is multidimensional we simply draw sample_shape.numel() samples and then reshape into the desired shape.

required
condition Tensor

Parameter set that all dimensions not specified in dims_to_sample will be fixed to. Should contain dim_theta elements, i.e. it could e.g. be a sample from the posterior distribution. The entries at all dims_to_sample will be ignored.

required
dims_to_sample List[int]

Which dimensions to sample from. The dimensions not specified in dims_to_sample will be fixed to values given in condition.

required
x Optional[torch.Tensor]

Conditioning context for posterior \(p(\theta|x)\). If not provided, fall back onto x passed to set_default_x().

None
sample_with str

Method to use for sampling from the posterior. Must be one of [mcmc | rejection]. In this method, the value of self._sample_with will be ignored.

'mcmc'
show_progress_bars bool

Whether to show sampling progress monitor.

True
mcmc_method Optional[str]

Optional parameter to override self.mcmc_method.

None
mcmc_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain. warmup_steps to set the initial number of samples to discard. num_chains for the number of chains. init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations. enable_transform a bool indicating whether MCMC is performed in z-scored (and unconstrained) space.

None
rejection_sampling_parameters Optional[Dict[str, Any]]

Dictionary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution (default is the prior). max_sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. num_iter_to_find_max as the number of gradient ascent iterations to find the maximum of that ratio. m as multiplier to that ratio.

None

Returns:

Type Description
Tensor

Samples from conditional posterior.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def sample_conditional(
    self,
    sample_shape: Shape,
    condition: Tensor,
    dims_to_sample: List[int],
    x: Optional[Tensor] = None,
    sample_with: str = "mcmc",
    show_progress_bars: bool = True,
    mcmc_method: Optional[str] = None,
    mcmc_parameters: Optional[Dict[str, Any]] = None,
    rejection_sampling_parameters: Optional[Dict[str, Any]] = None,
) -> Tensor:
    r"""
    Return samples from conditional posterior $p(\theta_i|\theta_j, x)$.

    In this function, we do not sample from the full posterior, but instead only
    from a few parameter dimensions while the other parameter dimensions are kept
    fixed at values specified in `condition`.

    Samples are obtained with MCMC.

    Args:
        sample_shape: Desired shape of samples that are drawn from posterior. If
            sample_shape is multidimensional we simply draw `sample_shape.numel()`
            samples and then reshape into the desired shape.
        condition: Parameter set that all dimensions not specified in
            `dims_to_sample` will be fixed to. Should contain dim_theta elements,
            i.e. it could e.g. be a sample from the posterior distribution.
            The entries at all `dims_to_sample` will be ignored.
        dims_to_sample: Which dimensions to sample from. The dimensions not
            specified in `dims_to_sample` will be fixed to values given in
            `condition`.
        x: Conditioning context for posterior $p(\theta|x)$. If not provided,
            fall back onto `x` passed to `set_default_x()`.
        sample_with: Method to use for sampling from the posterior. Must be one of
            [`mcmc` | `rejection`]. In this method, the value of
            `self._sample_with` will be ignored.
        show_progress_bars: Whether to show sampling progress monitor.
        mcmc_method: Optional parameter to override `self.mcmc_method`.
        mcmc_parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported:
            `thin` to set the thinning factor for the chain.
            `warmup_steps` to set the initial number of samples to discard.
            `num_chains` for the number of chains.
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.
            `enable_transform` a bool indicating whether MCMC is performed in
            z-scored (and unconstrained) space.
        rejection_sampling_parameters: Dictionary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution (default is the prior).
            `max_sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.
            `num_samples_to_find_max` as the number of samples that are used to
            find the maximum of the `potential_fn / proposal` ratio.
            `num_iter_to_find_max` as the number of gradient ascent iterations to
            find the maximum of that ratio.
            `m` as multiplier to that ratio.

    Returns:
        Samples from conditional posterior.
    """

    return super().sample_conditional(
        PotentialFunctionProvider(),
        sample_shape,
        condition,
        dims_to_sample,
        x,
        sample_with,
        show_progress_bars,
        mcmc_method,
        mcmc_parameters,
        rejection_sampling_parameters,
    )

set_default_x(self, x) inherited

Set new default x for .sample(), .log_prob to use as conditioning context.

This is a pure convenience to avoid having to repeatedly specify x in calls to .sample() and .log_prob() - only θ needs to be passed.

This convenience is particularly useful when the posterior is focused, i.e. has been trained over multiple rounds to be accurate in the vicinity of a particular x=x_o (you can check if your posterior object is focused by printing it).

NOTE: this method is chainable, i.e. will return the NeuralPosterior object so that calls like posterior.set_default_x(my_x).sample(mytheta) are possible.

Parameters:

Name Type Description Default
x Tensor

The default observation to set for the posterior \(p(theta|x)\).

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior that will use a default x when not explicitly passed.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_default_x(self, x: Tensor) -> "NeuralPosterior":
    """Set new default x for `.sample(), .log_prob` to use as conditioning context.

    This is a pure convenience to avoid having to repeatedly specify `x` in calls to
    `.sample()` and `.log_prob()` - only θ needs to be passed.

    This convenience is particularly useful when the posterior is focused, i.e.
    has been trained over multiple rounds to be accurate in the vicinity of a
    particular `x=x_o` (you can check if your posterior object is focused by
    printing it).

    NOTE: this method is chainable, i.e. will return the NeuralPosterior object so
    that calls like `posterior.set_default_x(my_x).sample(mytheta)` are possible.

    Args:
        x: The default observation to set for the posterior $p(theta|x)$.

    Returns:
        `NeuralPosterior` that will use a default `x` when not explicitly passed.
    """
    self._x = process_x(x, self._x_shape, allow_iid_x=self._allow_iid_x).to(
        self._device
    )
    self._num_iid_trials = self._x.shape[0]

    return self

set_mcmc_method(self, method) inherited

Sets sampling method to for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
method str

Method to use.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_mcmc_method(self, method: str) -> "NeuralPosterior":
    """Sets sampling method to for MCMC and returns `NeuralPosterior`.

    Args:
        method: Method to use.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_method = method
    return self

set_mcmc_parameters(self, parameters) inherited

Sets parameters for MCMC and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictionary overriding the default parameters for MCMC. The following parameters are supported: thin to set the thinning factor for the chain, warmup_steps to set the initial number of samples to discard, num_chains for the number of chains, init_strategy for the initialisation strategy for chains; prior will draw init locations from prior, whereas sir will use Sequential- Importance-Resampling using init_strategy_num_candidates to find init locations.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_mcmc_parameters(self, parameters: Dict[str, Any]) -> "NeuralPosterior":
    """Sets parameters for MCMC and returns `NeuralPosterior`.

    Args:
        parameters: Dictionary overriding the default parameters for MCMC.
            The following parameters are supported: `thin` to set the thinning
            factor for the chain, `warmup_steps` to set the initial number of
            samples to discard, `num_chains` for the number of chains,
            `init_strategy` for the initialisation strategy for chains; `prior`
            will draw init locations from prior, whereas `sir` will use Sequential-
            Importance-Resampling using `init_strategy_num_candidates` to find init
            locations.

    Returns:
        `NeuralPosterior` for chainable calls.
    """
    self._mcmc_parameters = parameters
    return self

set_rejection_sampling_parameters(self, parameters) inherited

Sets parameters for rejection sampling and returns NeuralPosterior.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

Dictonary overriding the default parameters for rejection sampling. The following parameters are supported: proposal as the proposal distribtution. num_samples_to_find_max as the number of samples that are used to find the maximum of the potential_fn / proposal ratio. m as multiplier to that ratio. sampling_batch_size as the batchsize of samples being drawn from the proposal at every iteration.

required

Returns:

Type Description
NeuralPosterior

`NeuralPosterior for chainable calls.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_rejection_sampling_parameters(
    self, parameters: Dict[str, Any]
) -> "NeuralPosterior":
    """Sets parameters for rejection sampling and returns `NeuralPosterior`.

    Args:
        parameters: Dictonary overriding the default parameters
            for rejection sampling. The following parameters are supported:
            `proposal` as the proposal distribtution. `num_samples_to_find_max`
            as the number of samples that are used to find the maximum of the
            `potential_fn / proposal` ratio. `m` as multiplier to that ratio.
            `sampling_batch_size` as the batchsize of samples being drawn from
            the proposal at every iteration.

    Returns:
        `NeuralPosterior for chainable calls.
    """
    self._rejection_sampling_parameters = parameters
    return self

set_sample_with(self, sample_with) inherited

Set the sampling method for the NeuralPosterior.

Parameters:

Name Type Description Default
sample_with str

The method to sample with.

required

Returns:

Type Description
NeuralPosterior

NeuralPosterior for chainable calls.

Exceptions:

Type Description
ValueError

on attempt to turn off MCMC sampling for family of methods that do not support rejection sampling.

Source code in sbi/inference/posteriors/ratio_based_posterior.py
def set_sample_with(self, sample_with: str) -> "NeuralPosterior":
    """Set the sampling method for the `NeuralPosterior`.

    Args:
        sample_with: The method to sample with.

    Returns:
        `NeuralPosterior` for chainable calls.

    Raises:
        ValueError: on attempt to turn off MCMC sampling for family of methods that
            do not support rejection sampling.
    """
    if sample_with not in ("mcmc", "rejection"):
        raise NameError(
            "The only implemented sampling methods are `mcmc` and `rejection`."
        )
    self._sample_with = sample_with
    return self

Models

sbi.utils.get_nn_models.posterior_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10)

Returns a function that builds a density estimator for learning the posterior.

This function will usually be used for SNPE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters \(\theta\) before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs \(x\) before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for simulation outputs \(x\). This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
def posterior_nn(
    model: str,
    z_score_theta: bool = True,
    z_score_x: bool = True,
    hidden_features: int = 50,
    num_transforms: int = 5,
    num_bins: int = 10,
    embedding_net: nn.Module = nn.Identity(),
    num_components: int = 10,
) -> Callable:
    r"""
    Returns a function that builds a density estimator for learning the posterior.

    This function will usually be used for SNPE. The returned function is to be passed
    to the inference class when using the flexible interface.

    Args:
        model: The type of density estimator that will be created. One of [`mdn`,
            `made`, `maf`, `nsf`].
        z_score_theta: Whether to z-score parameters $\theta$ before passing them into
            the network.
        z_score_x: Whether to z-score simulation outputs $x$ before passing them into
            the network.
        hidden_features: Number of hidden features.
        num_transforms: Number of transforms when a flow is used. Only relevant if
            density estimator is a normalizing flow (i.e. currently either a `maf` or a
            `nsf`). Ignored if density estimator is a `mdn` or `made`.
        num_bins: Number of bins used for the splines in `nsf`. Ignored if density
            estimator not `nsf`.
        embedding_net: Optional embedding network for simulation outputs $x$. This
            embedding net allows to learn features from potentially high-dimensional
            simulation outputs.
        num_components: Number of mixture components for a mixture of Gaussians.
            Ignored if density estimator is not an mdn.
    """

    kwargs = dict(
        zip(
            (
                "z_score_x",
                "z_score_y",
                "hidden_features",
                "num_transforms",
                "num_bins",
                "embedding_net",
                "num_components",
            ),
            (
                z_score_theta,
                z_score_x,
                hidden_features,
                num_transforms,
                num_bins,
                embedding_net,
                num_components,
            ),
        )
    )

    if model == "mdn_snpe_a":
        if num_components != 10:
            raise ValueError(
                "You set `num_components`. For SNPE-A, this has to be done at "
                "instantiation of the inference object, i.e. "
                "`inference = SNPE_A(..., num_components=20)`"
            )
        kwargs.pop("num_components")

        def build_fn(batch_theta, batch_x, num_components):
            # Extract the number of components from the kwargs, such that
            # they are exposed as a kwargs, offering the possibility to later
            # override this kwarg with functools.partial. This is necessary
            # in order to make sure that the MDN in SNPE-A only has one
            # component when running the Algorithm 1 part.
            return build_mdn(
                batch_x=batch_theta,
                batch_y=batch_x,
                num_components=num_components,
                **kwargs
            )

    else:

        def build_fn(batch_theta, batch_x):
            if model == "mdn":
                return build_mdn(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            elif model == "made":
                return build_made(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            elif model == "maf":
                return build_maf(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            elif model == "nsf":
                return build_nsf(batch_x=batch_theta, batch_y=batch_x, **kwargs)
            else:
                raise NotImplementedError

    return build_fn

sbi.utils.get_nn_models.likelihood_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, num_transforms=5, num_bins=10, embedding_net=Identity(), num_components=10)

Returns a function that builds a density estimator for learning the likelihood.

This function will usually be used for SNLE. The returned function is to be passed to the inference class when using the flexible interface.

Parameters:

Name Type Description Default
model str

The type of density estimator that will be created. One of [mdn, made, maf, nsf].

required
z_score_theta bool

Whether to z-score parameters \(\theta\) before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs \(x\) before passing them into the network.

True
hidden_features int

Number of hidden features.

50
num_transforms int

Number of transforms when a flow is used. Only relevant if density estimator is a normalizing flow (i.e. currently either a maf or a nsf). Ignored if density estimator is a mdn or made.

5
num_bins int

Number of bins used for the splines in nsf. Ignored if density estimator not nsf.

10
embedding_net Module

Optional embedding network for parameters \(\theta\).

Identity()
num_components int

Number of mixture components for a mixture of Gaussians. Ignored if density estimator is not an mdn.

10
Source code in sbi/utils/get_nn_models.py
def likelihood_nn(
    model: str,
    z_score_theta: bool = True,
    z_score_x: bool = True,
    hidden_features: int = 50,
    num_transforms: int = 5,
    num_bins: int = 10,
    embedding_net: nn.Module = nn.Identity(),
    num_components: int = 10,
) -> Callable:
    r"""
    Returns a function that builds a density estimator for learning the likelihood.

    This function will usually be used for SNLE. The returned function is to be passed
    to the inference class when using the flexible interface.

    Args:
        model: The type of density estimator that will be created. One of [`mdn`,
            `made`, `maf`, `nsf`].
        z_score_theta: Whether to z-score parameters $\theta$ before passing them into
            the network.
        z_score_x: Whether to z-score simulation outputs $x$ before passing them into
            the network.
        hidden_features: Number of hidden features.
        num_transforms: Number of transforms when a flow is used. Only relevant if
            density estimator is a normalizing flow (i.e. currently either a `maf` or a
            `nsf`). Ignored if density estimator is a `mdn` or `made`.
        num_bins: Number of bins used for the splines in `nsf`. Ignored if density
            estimator not `nsf`.
        embedding_net: Optional embedding network for parameters $\theta$.
        num_components: Number of mixture components for a mixture of Gaussians.
            Ignored if density estimator is not an mdn.
    """

    kwargs = dict(
        zip(
            (
                "z_score_x",
                "z_score_y",
                "hidden_features",
                "num_transforms",
                "num_bins",
                "embedding_net",
                "num_components",
            ),
            (
                z_score_x,
                z_score_theta,
                hidden_features,
                num_transforms,
                num_bins,
                embedding_net,
                num_components,
            ),
        )
    )

    def build_fn(batch_theta, batch_x):
        if model == "mdn":
            return build_mdn(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        if model == "made":
            return build_made(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        if model == "maf":
            return build_maf(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        elif model == "nsf":
            return build_nsf(batch_x=batch_x, batch_y=batch_theta, **kwargs)
        else:
            raise NotImplementedError

    return build_fn

sbi.utils.get_nn_models.classifier_nn(model, z_score_theta=True, z_score_x=True, hidden_features=50, embedding_net_theta=Identity(), embedding_net_x=Identity())

Returns a function that builds a classifier for learning density ratios.

This function will usually be used for SNRE. The returned function is to be passed to the inference class when using the flexible interface.

Note that in the view of the SNRE classifier we build below, x=theta and y=x.

Parameters:

Name Type Description Default
model str

The type of classifier that will be created. One of [linear, mlp, resnet].

required
z_score_theta bool

Whether to z-score parameters \(\theta\) before passing them into the network.

True
z_score_x bool

Whether to z-score simulation outputs \(x\) before passing them into the network.

True
hidden_features int

Number of hidden features.

50
embedding_net_theta Module

Optional embedding network for parameters \(\theta\).

Identity()
embedding_net_x Module

Optional embedding network for simulation outputs \(x\). This embedding net allows to learn features from potentially high-dimensional simulation outputs.

Identity()
Source code in sbi/utils/get_nn_models.py
def classifier_nn(
    model: str,
    z_score_theta: bool = True,
    z_score_x: bool = True,
    hidden_features: int = 50,
    embedding_net_theta: nn.Module = nn.Identity(),
    embedding_net_x: nn.Module = nn.Identity(),
) -> Callable:
    r"""
    Returns a function that builds a classifier for learning density ratios.

    This function will usually be used for SNRE. The returned function is to be passed
    to the inference class when using the flexible interface.

    Note that in the view of the SNRE classifier we build below, x=theta and y=x.

    Args:
        model: The type of classifier that will be created. One of [`linear`, `mlp`,
            `resnet`].
        z_score_theta: Whether to z-score parameters $\theta$ before passing them into
            the network.
        z_score_x: Whether to z-score simulation outputs $x$ before passing them into
            the network.
        hidden_features: Number of hidden features.
        embedding_net_theta:  Optional embedding network for parameters $\theta$.
        embedding_net_x:  Optional embedding network for simulation outputs $x$. This
            embedding net allows to learn features from potentially high-dimensional
            simulation outputs.
    """

    kwargs = dict(
        zip(
            (
                "z_score_x",
                "z_score_y",
                "hidden_features",
                "embedding_net_x",
                "embedding_net_y",
            ),
            (
                z_score_theta,
                z_score_x,
                hidden_features,
                embedding_net_theta,
                embedding_net_x,
            ),
        )
    )

    def build_fn(batch_theta, batch_x):
        if model == "linear":
            return build_linear_classifier(
                batch_x=batch_theta, batch_y=batch_x, **kwargs
            )
        if model == "mlp":
            return build_mlp_classifier(batch_x=batch_theta, batch_y=batch_x, **kwargs)
        if model == "resnet":
            return build_resnet_classifier(
                batch_x=batch_theta, batch_y=batch_x, **kwargs
            )
        else:
            raise NotImplementedError

    return build_fn

Analysis

sbi.analysis.plot.pairplot(samples=None, points=None, limits=None, subset=None, upper='hist', diag='hist', figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs)

Plot samples in a 2D grid showing marginals and pairwise marginals.

Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution that the samples were drawn from. Each upper-diagonal plot can be interpreted as a 2D-marginal of the distribution.

Parameters:

Name Type Description Default
samples Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

Samples used to build the histogram.

None
points Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

List of additional points to scatter.

None
limits Union[List, torch.Tensor]

Array containing the plot xlim for each parameter dimension. If None, just use the min and max of the passed samples

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on).

None
upper Optional[str]

Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.

'hist'
diag Optional[str]

Plotting style for diagonal, {hist, cond, None}.

'hist'
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
def pairplot(
    samples: Union[
        List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor
    ] = None,
    points: Optional[
        Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor]
    ] = None,
    limits: Optional[Union[List, torch.Tensor]] = None,
    subset: List[int] = None,
    upper: Optional[str] = "hist",
    diag: Optional[str] = "hist",
    figsize: Tuple = (10, 10),
    labels: Optional[List[str]] = None,
    ticks: Union[List, torch.Tensor] = [],
    points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"],
    **kwargs
):
    """
    Plot samples in a 2D grid showing marginals and pairwise marginals.

    Each of the diagonal plots can be interpreted as a 1D-marginal of the distribution
    that the samples were drawn from. Each upper-diagonal plot can be interpreted as a
    2D-marginal of the distribution.

    Args:
        samples: Samples used to build the histogram.
        points: List of additional points to scatter.
        limits: Array containing the plot xlim for each parameter dimension. If None,
            just use the min and max of the passed samples
        subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot
            plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and,
            if they exist, the 4th, 5th and so on).
        upper: Plotting style for upper diagonal, {hist, scatter, contour, cond, None}.
        diag: Plotting style for diagonal, {hist, cond, None}.
        figsize: Size of the entire figure.
        labels: List of strings specifying the names of the parameters.
        ticks: Position of the ticks.
        points_colors: Colors of the `points`.
        **kwargs: Additional arguments to adjust the plot, see the source code in
            `_get_default_opts()` in `sbi.utils.plot` for more details.

    Returns: figure and axis of posterior distribution plot
    """
    return utils_pairplot(
        samples=samples,
        points=points,
        limits=limits,
        subset=subset,
        upper=upper,
        diag=diag,
        figsize=figsize,
        labels=labels,
        ticks=ticks,
        points_colors=points_colors,
        warn_about_deprecation=False,
        **kwargs,
    )

sbi.analysis.plot.conditional_pairplot(density, condition, limits, points=None, subset=None, resolution=50, figsize=(10, 10), labels=None, ticks=[], points_colors=['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf'], **kwargs)

Plot conditional distribution given all other parameters.

The conditionals can be interpreted as slices through the density at a location given by condition.

For example: Say we have a 3D density with parameters \(\theta_0\), \(\theta_1\), \(\theta_2\) and a condition \(c\) passed by the user in the condition argument. For the plot of \(\theta_0\) on the diagonal, this will plot the conditional \(p(\theta_0 | \theta_1=c[1], \theta_2=c[2])\). For the upper diagonal of \(\theta_1\) and \(\theta_2\), it will plot \(p(\theta_1, \theta_2 | \theta_0=c[0])\). All other diagonals and upper-diagonals are built in the corresponding way.

Parameters:

Name Type Description Default
density Any

Probability density with a log_prob() method.

required
condition Tensor

Condition that all but the one/two regarded parameters are fixed to. The condition should be of shape (1, dim_theta), i.e. it could e.g. be a sample from the posterior distribution.

required
limits Union[List, torch.Tensor]

Limits in between which each parameter will be evaluated.

required
points Union[List[numpy.ndarray], List[torch.Tensor], numpy.ndarray, torch.Tensor]

Additional points to scatter.

None
subset List[int]

List containing the dimensions to plot. E.g. subset=[1,3] will plot plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and, if they exist, the 4th, 5th and so on)

None
resolution int

Resolution of the grid at which we evaluate the pdf.

50
figsize Tuple

Size of the entire figure.

(10, 10)
labels Optional[List[str]]

List of strings specifying the names of the parameters.

None
ticks Union[List, torch.Tensor]

Position of the ticks.

[]
points_colors List[str]

Colors of the points.

['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd', '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
**kwargs

Additional arguments to adjust the plot, see the source code in _get_default_opts() in sbi.utils.plot for more details.

{}

Returns: figure and axis of posterior distribution plot

Source code in sbi/analysis/plot.py
def conditional_pairplot(
    density: Any,
    condition: torch.Tensor,
    limits: Union[List, torch.Tensor],
    points: Optional[
        Union[List[np.ndarray], List[torch.Tensor], np.ndarray, torch.Tensor]
    ] = None,
    subset: List[int] = None,
    resolution: int = 50,
    figsize: Tuple = (10, 10),
    labels: Optional[List[str]] = None,
    ticks: Union[List, torch.Tensor] = [],
    points_colors: List[str] = plt.rcParams["axes.prop_cycle"].by_key()["color"],
    **kwargs
):
    r"""
    Plot conditional distribution given all other parameters.

    The conditionals can be interpreted as slices through the `density` at a location
    given by `condition`.

    For example:
    Say we have a 3D density with parameters $\theta_0$, $\theta_1$, $\theta_2$ and
    a condition $c$ passed by the user in the `condition` argument.
    For the plot of $\theta_0$ on the diagonal, this will plot the conditional
    $p(\theta_0 | \theta_1=c[1], \theta_2=c[2])$. For the upper
    diagonal of $\theta_1$ and $\theta_2$, it will plot
    $p(\theta_1, \theta_2 | \theta_0=c[0])$. All other diagonals and upper-diagonals
    are built in the corresponding way.

    Args:
        density: Probability density with a `log_prob()` method.
        condition: Condition that all but the one/two regarded parameters are fixed to.
            The condition should be of shape (1, dim_theta), i.e. it could e.g. be
            a sample from the posterior distribution.
        limits: Limits in between which each parameter will be evaluated.
        points: Additional points to scatter.
        subset: List containing the dimensions to plot. E.g. subset=[1,3] will plot
            plot only the 1st and 3rd dimension but will discard the 0th and 2nd (and,
            if they exist, the 4th, 5th and so on)
        resolution: Resolution of the grid at which we evaluate the `pdf`.
        figsize: Size of the entire figure.
        labels: List of strings specifying the names of the parameters.
        ticks: Position of the ticks.
        points_colors: Colors of the `points`.
        **kwargs: Additional arguments to adjust the plot, see the source code in
            `_get_default_opts()` in `sbi.utils.plot` for more details.

    Returns: figure and axis of posterior distribution plot
    """
    return utils_conditional_pairplot(
        density=density,
        condition=condition,
        limits=limits,
        points=points,
        subset=subset,
        resolution=resolution,
        figsize=figsize,
        labels=labels,
        ticks=ticks,
        points_colors=points_colors,
        warn_about_deprecation=False,
        **kwargs,
    )

sbi.analysis.conditional_density.conditional_corrcoeff(density, limits, condition, subset=None, resolution=50)

Returns the conditional correlation matrix of a distribution.

To compute the conditional distribution, we condition all but two parameters to values from condition, and then compute the Pearson correlation coefficient \(\rho\) between the remaining two parameters under the distribution density. We do so for any pair of parameters specified in subset, thus creating a matrix containing conditional correlations between any pair of parameters.

If condition is a batch of conditions, this function computes the conditional correlation matrix for each one of them and returns the mean.

Parameters:

Name Type Description Default
density Any

Probability density function with .log_prob() function.

required
limits Tensor

Limits within which to evaluate the density.

required
condition Tensor

Values to condition the density on. If a batch of conditions is passed, we compute the conditional correlation matrix for each of them and return the average conditional correlation matrix.

required
subset Optional[List[int]]

Evaluate the conditional distribution only on a subset of dimensions. If None this function uses all dimensions.

None
resolution int

Number of grid points on which the conditional distribution is evaluated. A higher value increases the accuracy of the estimated correlation but also increases the computational cost.

50

Returns: Average conditional correlation matrix of shape either (num_dim, num_dim) or (len(subset), len(subset)) if subset was specified.

Source code in sbi/analysis/conditional_density.py
def conditional_corrcoeff(
    density: Any,
    limits: Tensor,
    condition: Tensor,
    subset: Optional[List[int]] = None,
    resolution: int = 50,
) -> Tensor:
    r"""
    Returns the conditional correlation matrix of a distribution.

    To compute the conditional distribution, we condition all but two parameters to
    values from `condition`, and then compute the Pearson correlation
    coefficient $\rho$ between the remaining two parameters under the distribution
    `density`. We do so for any pair of parameters specified in `subset`, thus
    creating a matrix containing conditional correlations between any pair of
    parameters.

    If `condition` is a batch of conditions, this function computes the conditional
    correlation matrix for each one of them and returns the mean.

    Args:
        density: Probability density function with `.log_prob()` function.
        limits: Limits within which to evaluate the `density`.
        condition: Values to condition the `density` on. If a batch of conditions is
            passed, we compute the conditional correlation matrix for each of them and
            return the average conditional correlation matrix.
        subset: Evaluate the conditional distribution only on a subset of dimensions.
            If `None` this function uses all dimensions.
        resolution: Number of grid points on which the conditional distribution is
            evaluated. A higher value increases the accuracy of the estimated
            correlation but also increases the computational cost.

    Returns: Average conditional correlation matrix of shape either `(num_dim, num_dim)`
    or `(len(subset), len(subset))` if `subset` was specified.
    """
    return utils_conditional_corrcoeff(
        density=density,
        limits=limits,
        condition=condition,
        subset=subset,
        resolution=resolution,
        warn_about_deprecation=False,
    )