API

Module-wide re-exports

Turing.jl directly re-exports the entire public API of the following packages:

Please see the individual packages for their documentation.

Individual exports and re-exports

All of the following symbols are exported unqualified by Turing, even though the documentation suggests that many of them are qualified. That means, for example, you can just write

using Turing

@model function my_model() end

sample(my_model(), Prior(), 100)

instead of

DynamicPPL.@model function my_model() end

sample(my_model(), Turing.Inference.Prior(), 100)

even though Prior() is actually defined in the Turing.Inference module and @model in the DynamicPPL package.

Modelling

Exported symbolDocumentationDescription
@modelDynamicPPL.@modelDefine a probabilistic model
@varnameAbstractPPL.@varnameGenerate a VarName from a Julia expression
@submodelDynamicPPL.@submodelDefine a submodel

Inference

Exported symbolDocumentationDescription
sampleStatsBase.sampleSample from a model

Samplers

Exported symbolDocumentationDescription
PriorTuring.Inference.PriorSample from the prior distribution
MHTuring.Inference.MHMetropolis–Hastings
EmceeTuring.Inference.EmceeAffine-invariant ensemble sampler
ESSTuring.Inference.ESSElliptical slice sampling
GibbsTuring.Inference.GibbsGibbs sampling
GibbsConditionalTuring.Inference.GibbsConditionalA "pseudo-sampler" to provide analytical conditionals to Gibbs
HMCTuring.Inference.HMCHamiltonian Monte Carlo
SGLDTuring.Inference.SGLDStochastic gradient Langevin dynamics
SGHMCTuring.Inference.SGHMCStochastic gradient Hamiltonian Monte Carlo
PolynomialStepsizeTuring.Inference.PolynomialStepsizeReturns a function which generates polynomially decaying step sizes
HMCDATuring.Inference.HMCDAHamiltonian Monte Carlo with dual averaging
NUTSTuring.Inference.NUTSNo-U-Turn Sampler
ISTuring.Inference.ISImportance sampling
SMCTuring.Inference.SMCSequential Monte Carlo
PGTuring.Inference.PGParticle Gibbs
CSMCTuring.Inference.CSMCThe same as PG
externalsamplerTuring.Inference.externalsamplerWrap an external sampler for use in Turing

Variational inference

See the variational inference tutorial for a walkthrough on how to use these.

Exported symbolDocumentationDescription
viAdvancedVI.viPerform variational inference
ADVIAdvancedVI.ADVIConstruct an instance of a VI algorithm

Automatic differentiation types

These are used to specify the automatic differentiation backend to use. See the AD guide for more information.

Exported symbolDocumentationDescription
AutoForwardDiffADTypes.AutoForwardDiffForwardDiff.jl backend
AutoReverseDiffADTypes.AutoReverseDiffReverseDiff.jl backend
AutoZygoteADTypes.AutoZygoteZygote.jl backend
AutoMooncakeADTypes.AutoMooncakeMooncake.jl backend

Debugging

Turing.setprogress!Function
setprogress!(progress::Bool)

Enable progress logging in Turing if progress is true, and disable it otherwise.

source

Distributions

These distributions are defined in Turing.jl, but not in Distributions.jl.

Turing.FlatType
Flat()

The flat distribution is the improper distribution of real numbers that has the improper probability density function

\[f(x) = 1.\]

source
Turing.FlatPosType
FlatPos(l::Real)

The positive flat distribution with real-valued parameter l is the improper distribution of real numbers that has the improper probability density function

\[f(x) = \begin{cases} 0 & \text{if } x \leq l, \\ 1 & \text{otherwise}. \end{cases}\]

source
Turing.BinomialLogitType
BinomialLogit(n, logitp)

The Binomial distribution with logit parameterization characterizes the number of successes in a sequence of independent trials.

It has two parameters: n, the number of trials, and logitp, the logit of the probability of success in an individual trial, with the distribution

\[P(X = k) = {n \choose k}{(\text{logistic}(logitp))}^k (1 - \text{logistic}(logitp))^{n-k}, \quad \text{ for } k = 0,1,2, \ldots, n.\]

See also: Binomial

source
Turing.OrderedLogisticType
OrderedLogistic(η, c::AbstractVector)

The ordered logistic distribution with real-valued parameter η and cutpoints c has the probability mass function

\[P(X = k) = \begin{cases} 1 - \text{logistic}(\eta - c_1) & \text{if } k = 1, \\ \text{logistic}(\eta - c_{k-1}) - \text{logistic}(\eta - c_k) & \text{if } 1 < k < K, \\ \text{logistic}(\eta - c_{K-1}) & \text{if } k = K, \end{cases}\]

where K = length(c) + 1.

source
Turing.LogPoissonType
LogPoisson(logλ)

The Poisson distribution with logarithmic parameterization of the rate parameter describes the number of independent events occurring within a unit time interval, given the average rate of occurrence $\exp(\log\lambda)$.

The distribution has the probability mass function

\[P(X = k) = \frac{e^{k \cdot \log\lambda}}{k!} e^{-e^{\log\lambda}}, \quad \text{ for } k = 0,1,2,\ldots.\]

See also: Poisson

source

BernoulliLogit is part of Distributions.jl since version 0.25.77. If you are using an older version of Distributions where this isn't defined, Turing will export the same distribution.

Distributions.BernoulliLogitType
BernoulliLogit(logitp=0.0)

A Bernoulli distribution that is parameterized by the logit logitp = logit(p) = log(p/(1-p)) of its success rate p.

\[P(X = k) = \begin{cases} \operatorname{logistic}(-logitp) = \frac{1}{1 + \exp{(logitp)}} & \quad \text{for } k = 0, \\ \operatorname{logistic}(logitp) = \frac{1}{1 + \exp{(-logitp)}} & \quad \text{for } k = 1. \end{cases}\]

External links:

See also Bernoulli

source

Tools to work with distributions

Exported symbolDocumentationDescription
filldistDistributionsAD.filldistCreate a product distribution from a distribution and integers
arraydistDistributionsAD.arraydistCreate a product distribution from an array of distributions
NamedDistDynamicPPL.NamedDistA distribution that carries the name of the variable

Predictions

StatsAPI.predictFunction
predict([rng::AbstractRNG,] model::Model, chain::MCMCChains.Chains; include_all=false)

Execute model conditioned on each sample in chain, and return the resulting Chains.

If include_all is false, the returned Chains will contain only those variables sampled/not present in chain.

Details

Internally calls Turing.Inference.transitions_from_chain to obtained the samples and then converts these into a Chains object using AbstractMCMC.bundle_samples.

Example

julia> using Turing; Turing.setprogress!(false);
[ Info: [Turing]: progress logging is disabled globally

julia> @model function linear_reg(x, y, σ = 0.1)
           β ~ Normal(0, 1)

           for i ∈ eachindex(y)
               y[i] ~ Normal(β * x[i], σ)
           end
       end;

julia> σ = 0.1; f(x) = 2 * x + 0.1 * randn();

julia> Δ = 0.1; xs_train = 0:Δ:10; ys_train = f.(xs_train);

julia> xs_test = [10 + Δ, 10 + 2 * Δ]; ys_test = f.(xs_test);

julia> m_train = linear_reg(xs_train, ys_train, σ);

julia> chain_lin_reg = sample(m_train, NUTS(100, 0.65), 200);
┌ Info: Found initial step size
└   ϵ = 0.003125

julia> m_test = linear_reg(xs_test, Vector{Union{Missing, Float64}}(undef, length(ys_test)), σ);

julia> predictions = predict(m_test, chain_lin_reg)
Object of type Chains, with data of type 100×2×1 Array{Float64,3}

Iterations        = 1:100
Thinning interval = 1
Chains            = 1
Samples per chain = 100
parameters        = y[1], y[2]

2-element Array{ChainDataFrame,1}

Summary Statistics
  parameters     mean     std  naive_se     mcse       ess   r_hat
  ──────────  ───────  ──────  ────────  ───────  ────────  ──────
        y[1]  20.1974  0.1007    0.0101  missing  101.0711  0.9922
        y[2]  20.3867  0.1062    0.0106  missing  101.4889  0.9903

Quantiles
  parameters     2.5%    25.0%    50.0%    75.0%    97.5%
  ──────────  ───────  ───────  ───────  ───────  ───────
        y[1]  20.0342  20.1188  20.2135  20.2588  20.4188
        y[2]  20.1870  20.3178  20.3839  20.4466  20.5895

julia> ys_pred = vec(mean(Array(group(predictions, :y)); dims = 1));

julia> sum(abs2, ys_test - ys_pred) ≤ 0.1
true
source

Querying model probabilities and quantities

Please see the generated quantities and probability interface guides for more information.

Exported symbolDocumentationDescription
generated_quantitiesDynamicPPL.generated_quantitiesCalculate additional quantities defined in a model
pointwise_loglikelihoodsDynamicPPL.pointwise_loglikelihoodsCompute log likelihoods for each sample in a chain
logpriorDynamicPPL.logpriorCompute log prior probability
logjointDynamicPPL.logjointCompute log joint probability
LogDensityFunctionDynamicPPL.LogDensityFunctionWrap a Turing model to satisfy LogDensityFunctions.jl interface
conditionAbstractPPL.conditionCondition a model on data
deconditionAbstractPPL.deconditionRemove conditioning on data
conditionedDynamicPPL.conditionedReturn the conditioned values of a model
fixDynamicPPL.fixFix the value of a variable
unfixDynamicPPL.unfixUnfix the value of a variable
OrderedDictOrderedCollections.OrderedDictAn ordered dictionary

Extra re-exports from Bijectors

Note that Bijectors itself does not export ordered.

Bijectors.orderedFunction
ordered(d::Distribution)

Return a Distribution whose support are ordered vectors, i.e., vectors with increasingly ordered elements.

Specifically, d is restricted to the subspace of its domain containing only ordered elements.

Warning

rand is implemented using rejection sampling, which can be slow for high-dimensional distributions. In such cases, consider using MCMC methods to sample from the distribution instead.

Warning

The resulting ordered distribution is un-normalized, which can cause issues in some contexts, e.g. in hierarchical models where the parameters of the ordered distribution are themselves sampled. See the notes below for a more detailed discussion.

Notes on ordered being un-normalized

The resulting ordered distribution is un-normalized. This is not a problem if used in a context where the normalizing factor is irrelevant, but if the value of the normalizing factor impacts the resulting computation, the results may be inaccurate.

For example, if the distribution is used in sampling a posterior distribution with MCMC and the parameters of the ordered distribution are themselves sampled, then the normalizing factor would in general be needed for accurate sampling, and ordered should not be used. However, if the parameters are fixed, then since MCMC does not require distributions be normalized, ordered may be used without problems.

A common case is where the distribution being ordered is a joint distribution of n identical univariate distributions. In this case the normalization factor works out to be the constant n!, and ordered can again be used without problems even if the parameters of the univariate distribution are sampled.

source

Point estimates

See the mode estimation tutorial for more information.

Exported symbolDocumentationDescription
maximum_a_posterioriTuring.Optimisation.maximum_a_posterioriFind a MAP estimate for a model
maximum_likelihoodTuring.Optimisation.maximum_likelihoodFind a MLE estimate for a model
MAPTuring.Optimisation.MAPType to use with Optim.jl for MAP estimation
MLETuring.Optimisation.MLEType to use with Optim.jl for MLE estimation