API
Module-wide re-exports
Turing.jl directly re-exports the entire public API of the following packages:
Please see the individual packages for their documentation.
Individual exports and re-exports
All of the following symbols are exported unqualified by Turing, even though the documentation suggests that many of them are qualified. That means, for example, you can just write
using Turing
@model function my_model() end
sample(my_model(), Prior(), 100)
instead of
DynamicPPL.@model function my_model() end
sample(my_model(), Turing.Inference.Prior(), 100)
even though Prior()
is actually defined in the Turing.Inference
module and @model
in the DynamicPPL
package.
Modelling
Exported symbol | Documentation | Description |
---|---|---|
@model | DynamicPPL.@model | Define a probabilistic model |
@varname | AbstractPPL.@varname | Generate a VarName from a Julia expression |
@submodel | DynamicPPL.@submodel | Define a submodel |
Inference
Exported symbol | Documentation | Description |
---|---|---|
sample | StatsBase.sample | Sample from a model |
Samplers
Exported symbol | Documentation | Description |
---|---|---|
Prior | Turing.Inference.Prior | Sample from the prior distribution |
MH | Turing.Inference.MH | Metropolis–Hastings |
Emcee | Turing.Inference.Emcee | Affine-invariant ensemble sampler |
ESS | Turing.Inference.ESS | Elliptical slice sampling |
Gibbs | Turing.Inference.Gibbs | Gibbs sampling |
GibbsConditional | Turing.Inference.GibbsConditional | A "pseudo-sampler" to provide analytical conditionals to Gibbs |
HMC | Turing.Inference.HMC | Hamiltonian Monte Carlo |
SGLD | Turing.Inference.SGLD | Stochastic gradient Langevin dynamics |
SGHMC | Turing.Inference.SGHMC | Stochastic gradient Hamiltonian Monte Carlo |
PolynomialStepsize | Turing.Inference.PolynomialStepsize | Returns a function which generates polynomially decaying step sizes |
HMCDA | Turing.Inference.HMCDA | Hamiltonian Monte Carlo with dual averaging |
NUTS | Turing.Inference.NUTS | No-U-Turn Sampler |
IS | Turing.Inference.IS | Importance sampling |
SMC | Turing.Inference.SMC | Sequential Monte Carlo |
PG | Turing.Inference.PG | Particle Gibbs |
CSMC | Turing.Inference.CSMC | The same as PG |
externalsampler | Turing.Inference.externalsampler | Wrap an external sampler for use in Turing |
Variational inference
See the variational inference tutorial for a walkthrough on how to use these.
Exported symbol | Documentation | Description |
---|---|---|
vi | AdvancedVI.vi | Perform variational inference |
ADVI | AdvancedVI.ADVI | Construct an instance of a VI algorithm |
Automatic differentiation types
These are used to specify the automatic differentiation backend to use. See the AD guide for more information.
Exported symbol | Documentation | Description |
---|---|---|
AutoForwardDiff | ADTypes.AutoForwardDiff | ForwardDiff.jl backend |
AutoReverseDiff | ADTypes.AutoReverseDiff | ReverseDiff.jl backend |
AutoZygote | ADTypes.AutoZygote | Zygote.jl backend |
AutoMooncake | ADTypes.AutoMooncake | Mooncake.jl backend |
Debugging
Turing.setprogress!
— Functionsetprogress!(progress::Bool)
Enable progress logging in Turing if progress
is true
, and disable it otherwise.
Distributions
These distributions are defined in Turing.jl, but not in Distributions.jl.
Turing.Flat
— TypeFlat()
The flat distribution is the improper distribution of real numbers that has the improper probability density function
\[f(x) = 1.\]
Turing.FlatPos
— TypeFlatPos(l::Real)
The positive flat distribution with real-valued parameter l
is the improper distribution of real numbers that has the improper probability density function
\[f(x) = \begin{cases} 0 & \text{if } x \leq l, \\ 1 & \text{otherwise}. \end{cases}\]
Turing.BinomialLogit
— TypeBinomialLogit(n, logitp)
The Binomial distribution with logit parameterization characterizes the number of successes in a sequence of independent trials.
It has two parameters: n
, the number of trials, and logitp
, the logit of the probability of success in an individual trial, with the distribution
\[P(X = k) = {n \choose k}{(\text{logistic}(logitp))}^k (1 - \text{logistic}(logitp))^{n-k}, \quad \text{ for } k = 0,1,2, \ldots, n.\]
See also: Binomial
Turing.OrderedLogistic
— TypeOrderedLogistic(η, c::AbstractVector)
The ordered logistic distribution with real-valued parameter η
and cutpoints c
has the probability mass function
\[P(X = k) = \begin{cases} 1 - \text{logistic}(\eta - c_1) & \text{if } k = 1, \\ \text{logistic}(\eta - c_{k-1}) - \text{logistic}(\eta - c_k) & \text{if } 1 < k < K, \\ \text{logistic}(\eta - c_{K-1}) & \text{if } k = K, \end{cases}\]
where K = length(c) + 1
.
Turing.LogPoisson
— TypeLogPoisson(logλ)
The Poisson distribution with logarithmic parameterization of the rate parameter describes the number of independent events occurring within a unit time interval, given the average rate of occurrence $\exp(\log\lambda)$.
The distribution has the probability mass function
\[P(X = k) = \frac{e^{k \cdot \log\lambda}}{k!} e^{-e^{\log\lambda}}, \quad \text{ for } k = 0,1,2,\ldots.\]
See also: Poisson
BernoulliLogit
is part of Distributions.jl since version 0.25.77. If you are using an older version of Distributions where this isn't defined, Turing will export the same distribution.
Distributions.BernoulliLogit
— TypeBernoulliLogit(logitp=0.0)
A Bernoulli distribution that is parameterized by the logit logitp = logit(p) = log(p/(1-p))
of its success rate p
.
\[P(X = k) = \begin{cases} \operatorname{logistic}(-logitp) = \frac{1}{1 + \exp{(logitp)}} & \quad \text{for } k = 0, \\ \operatorname{logistic}(logitp) = \frac{1}{1 + \exp{(-logitp)}} & \quad \text{for } k = 1. \end{cases}\]
External links:
See also Bernoulli
Tools to work with distributions
Exported symbol | Documentation | Description |
---|---|---|
filldist | DistributionsAD.filldist | Create a product distribution from a distribution and integers |
arraydist | DistributionsAD.arraydist | Create a product distribution from an array of distributions |
NamedDist | DynamicPPL.NamedDist | A distribution that carries the name of the variable |
Predictions
StatsAPI.predict
— Functionpredict([rng::AbstractRNG,] model::Model, chain::MCMCChains.Chains; include_all=false)
Execute model
conditioned on each sample in chain
, and return the resulting Chains
.
If include_all
is false
, the returned Chains
will contain only those variables sampled/not present in chain
.
Details
Internally calls Turing.Inference.transitions_from_chain
to obtained the samples and then converts these into a Chains
object using AbstractMCMC.bundle_samples
.
Example
julia> using Turing; Turing.setprogress!(false);
[ Info: [Turing]: progress logging is disabled globally
julia> @model function linear_reg(x, y, σ = 0.1)
β ~ Normal(0, 1)
for i ∈ eachindex(y)
y[i] ~ Normal(β * x[i], σ)
end
end;
julia> σ = 0.1; f(x) = 2 * x + 0.1 * randn();
julia> Δ = 0.1; xs_train = 0:Δ:10; ys_train = f.(xs_train);
julia> xs_test = [10 + Δ, 10 + 2 * Δ]; ys_test = f.(xs_test);
julia> m_train = linear_reg(xs_train, ys_train, σ);
julia> chain_lin_reg = sample(m_train, NUTS(100, 0.65), 200);
┌ Info: Found initial step size
└ ϵ = 0.003125
julia> m_test = linear_reg(xs_test, Vector{Union{Missing, Float64}}(undef, length(ys_test)), σ);
julia> predictions = predict(m_test, chain_lin_reg)
Object of type Chains, with data of type 100×2×1 Array{Float64,3}
Iterations = 1:100
Thinning interval = 1
Chains = 1
Samples per chain = 100
parameters = y[1], y[2]
2-element Array{ChainDataFrame,1}
Summary Statistics
parameters mean std naive_se mcse ess r_hat
────────── ─────── ────── ──────── ─────── ──────── ──────
y[1] 20.1974 0.1007 0.0101 missing 101.0711 0.9922
y[2] 20.3867 0.1062 0.0106 missing 101.4889 0.9903
Quantiles
parameters 2.5% 25.0% 50.0% 75.0% 97.5%
────────── ─────── ─────── ─────── ─────── ───────
y[1] 20.0342 20.1188 20.2135 20.2588 20.4188
y[2] 20.1870 20.3178 20.3839 20.4466 20.5895
julia> ys_pred = vec(mean(Array(group(predictions, :y)); dims = 1));
julia> sum(abs2, ys_test - ys_pred) ≤ 0.1
true
Querying model probabilities and quantities
Please see the generated quantities and probability interface guides for more information.
Exported symbol | Documentation | Description |
---|---|---|
generated_quantities | DynamicPPL.generated_quantities | Calculate additional quantities defined in a model |
pointwise_loglikelihoods | DynamicPPL.pointwise_loglikelihoods | Compute log likelihoods for each sample in a chain |
logprior | DynamicPPL.logprior | Compute log prior probability |
logjoint | DynamicPPL.logjoint | Compute log joint probability |
LogDensityFunction | DynamicPPL.LogDensityFunction | Wrap a Turing model to satisfy LogDensityFunctions.jl interface |
condition | AbstractPPL.condition | Condition a model on data |
decondition | AbstractPPL.decondition | Remove conditioning on data |
conditioned | DynamicPPL.conditioned | Return the conditioned values of a model |
fix | DynamicPPL.fix | Fix the value of a variable |
unfix | DynamicPPL.unfix | Unfix the value of a variable |
OrderedDict | OrderedCollections.OrderedDict | An ordered dictionary |
Extra re-exports from Bijectors
Note that Bijectors itself does not export ordered
.
Bijectors.ordered
— Functionordered(d::Distribution)
Return a Distribution
whose support are ordered vectors, i.e., vectors with increasingly ordered elements.
Specifically, d
is restricted to the subspace of its domain containing only ordered elements.
rand
is implemented using rejection sampling, which can be slow for high-dimensional distributions. In such cases, consider using MCMC methods to sample from the distribution instead.
The resulting ordered distribution is un-normalized, which can cause issues in some contexts, e.g. in hierarchical models where the parameters of the ordered distribution are themselves sampled. See the notes below for a more detailed discussion.
Notes on ordered
being un-normalized
The resulting ordered distribution is un-normalized. This is not a problem if used in a context where the normalizing factor is irrelevant, but if the value of the normalizing factor impacts the resulting computation, the results may be inaccurate.
For example, if the distribution is used in sampling a posterior distribution with MCMC and the parameters of the ordered distribution are themselves sampled, then the normalizing factor would in general be needed for accurate sampling, and ordered
should not be used. However, if the parameters are fixed, then since MCMC does not require distributions be normalized, ordered
may be used without problems.
A common case is where the distribution being ordered is a joint distribution of n
identical univariate distributions. In this case the normalization factor works out to be the constant n!
, and ordered
can again be used without problems even if the parameters of the univariate distribution are sampled.
Point estimates
See the mode estimation tutorial for more information.
Exported symbol | Documentation | Description |
---|---|---|
maximum_a_posteriori | Turing.Optimisation.maximum_a_posteriori | Find a MAP estimate for a model |
maximum_likelihood | Turing.Optimisation.maximum_likelihood | Find a MLE estimate for a model |
MAP | Turing.Optimisation.MAP | Type to use with Optim.jl for MAP estimation |
MLE | Turing.Optimisation.MLE | Type to use with Optim.jl for MLE estimation |