using AbstractMCMC: AbstractMCMC
using Distributions
using LinearAlgebra: I
using LogDensityProblems
using Random
using Statistics: mean, stdInterface Guide
The sampling interface
Turing’s samplers are built on the interface defined in AbstractMCMC.jl, a small set of types and functions that any Markov chain Monte Carlo sampler can implement. A sampler written against this interface gets multiple-chain sampling, warmup and thinning, progress logging and callbacks from sample without writing any of that itself, and can be used on Turing models through externalsampler.
This guide implements the interface without Turing, using a Metropolis-Hastings sampler as the example. The Implementing Samplers page works through a second, more involved example, and the External Samplers page lists what Turing additionally needs from a sampler.
Interface overview
A sampler implements:
- A subtype of
AbstractMCMC.AbstractSampler. The sampler struct holds settings only. Anything that changes between iterations belongs in the state, not in the sampler. - Two methods of
AbstractMCMC.step. The first,step(rng, model, sampler; kwargs...), produces the initial draw. The second,step(rng, model, sampler, state; kwargs...), produces the next draw from the current state. Both return a tuple(transition, state): the transition is what the user sees in the output, and the state carries whatever the next step needs.
Optionally, it can also implement:
AbstractMCMC.getparams(state)andAbstractMCMC.getstats(state), which Turing’sexternalsampleruses to read parameters and statistics from the state.AbstractMCMC.bundle_samples(samples, model, sampler, state, chain_type; kwargs...), to convert the vector of transitions into a chain format of your choice whensampleis called with thatchain_type.
Everything else comes from sample: MCMCThreads(), MCMCDistributed() and MCMCSerial() for several chains, num_warmup, discard_initial and thinning, progress logging, callback, and initial_params and initial_state for choosing where to start.
Why do you have an interface?
Markov chain Monte Carlo methods share most of their machinery: run a kernel repeatedly, keep the draws, report progress, run several chains at once. An interface lets each sampler package implement only its kernel and share the rest, and lets one sampler serve every model implementation that speaks the interface, Turing models included.
Implementing Metropolis-Hastings without Turing
Metropolis-Hastings is often the first sampling method that people meet, and it is short enough to implement completely here. A full implementation with several proposal types lives in AdvancedMH.jl.
Imports
Model
The sampler needs one thing from the model: the log density at a parameter vector. AbstractMCMC wraps any object that implements the LogDensityProblems.jl interface in AbstractMCMC.LogDensityModel, and samplers define their step methods on that wrapper. Turing models are turned into such objects for you, so a sampler written this way runs on them unchanged.
For this guide the target is the posterior of the mean and standard deviation of a Normal distribution given 30 observations, with a flat prior.
struct NormalDensity{T}
data::T
end
function LogDensityProblems.logdensity(d::NormalDensity, θ)
μ, σ = θ
return σ > 0 ? sum(logpdf.(Normal(μ, σ), d.data)) : -Inf
end
LogDensityProblems.dimension(::NormalDensity) = 2
function LogDensityProblems.capabilities(::Type{<:NormalDensity})
return LogDensityProblems.LogDensityOrder{0}()
end
data = rand(Xoshiro(1), Normal(5, 3), 30)
model = AbstractMCMC.LogDensityModel(NormalDensity(data))AbstractMCMC.LogDensityModel{NormalDensity{Vector{Float64}}}(NormalDensity{Vector{Float64}}([5.185798220942241, 5.8352174424920005, 3.2125267539078433, 5.139978168720146, 8.257382064629828, 0.2703052322420476, 5.527819973903224, 7.596142416227975, -3.370843016647921, -0.6760466746777385 … 6.2765441528633925, 4.101734010614404, -2.149189201900941, 1.9921594273721348, 7.916073183081872, 9.63922977486613, 3.2474058556742875, 6.403246356193962, 3.825439750868912, 6.753237256970654]))Sampler
The sampler holds the proposal distribution and nothing else.
struct MetropolisHastings{D} <: AbstractMCMC.AbstractSampler
proposal::D
endState
The state holds the current position and its log density. Caching the log density means each iteration evaluates the model once, for the proposal, rather than twice.
struct MHState{T,L}
θ::T
lp::L
endThe transition, the value returned to the user, is the parameter vector itself.
Steps
Metropolis-Hastings proposes \theta' \sim q(\theta' \mid \theta) and accepts it with probability
\alpha = \min\left[1, \frac{\pi(\theta')}{\pi(\theta)} \frac{q(\theta \mid \theta')}{q(\theta' \mid \theta)}\right].A symmetric random-walk proposal has q(\theta \mid \theta') = q(\theta' \mid \theta), so the ratio of proposal densities cancels and the log acceptance probability is \min[0, \log \pi(\theta') - \log \pi(\theta)].
Supply a starting point with finite log density through initial_params. sample passes that keyword argument through to the first step, which refuses to start without it or outside the support, because the acceptance test below cannot move a chain whose current log density is -Inf.
function AbstractMCMC.step(
rng::Random.AbstractRNG,
model::AbstractMCMC.LogDensityModel,
sampler::MetropolisHastings;
initial_params=nothing,
kwargs...,
)
initial_params === nothing && throw(ArgumentError("initial_params is required"))
θ = initial_params
lp = LogDensityProblems.logdensity(model.logdensity, θ)
isfinite(lp) || throw(ArgumentError("initial_params must have finite log density"))
return θ, MHState(θ, lp)
endEvery later step proposes a move from the current state and accepts or rejects it. On rejection it returns the current position again, together with the unchanged state.
function AbstractMCMC.step(
rng::Random.AbstractRNG,
model::AbstractMCMC.LogDensityModel,
sampler::MetropolisHastings,
state::MHState;
kwargs...,
)
θ_proposed = state.θ + rand(rng, sampler.proposal)
lp_proposed = LogDensityProblems.logdensity(model.logdensity, θ_proposed)
if log(rand(rng)) < lp_proposed - state.lp
return θ_proposed, MHState(θ_proposed, lp_proposed)
else
return state.θ, state
end
endSampling
That is the whole sampler. sample runs the two methods, throws away the first 2000 draws, and then collects the 20000 requested transitions and returns them as a vector. discard_initial adds steps in front of the requested number, it does not take them out of it.
sampler = MetropolisHastings(MvNormal(zeros(2), 0.25 * I))
draws = AbstractMCMC.sample(
Xoshiro(2), model, sampler, 20_000; initial_params=[0.0, 1.0], discard_initial=2_000
)
θs = reduce(hcat, draws)
mean(θs; dims=2)Sampling 0%| | ETA: N/A Sampling 0%|▎ | ETA: 0:05:30 Sampling 1%|▍ | ETA: 0:02:46 Sampling 2%|▋ | ETA: 0:01:50 Sampling 2%|▉ | ETA: 0:01:22 Sampling 2%|█ | ETA: 0:01:05 Sampling 3%|█▎ | ETA: 0:00:54 Sampling 4%|█▌ | ETA: 0:00:46 Sampling 4%|█▋ | ETA: 0:00:40 Sampling 4%|█▉ | ETA: 0:00:36 Sampling 5%|██▏ | ETA: 0:00:32 Sampling 6%|██▎ | ETA: 0:00:29 Sampling 6%|██▌ | ETA: 0:00:26 Sampling 6%|██▊ | ETA: 0:00:24 Sampling 7%|███ | ETA: 0:00:22 Sampling 8%|███▏ | ETA: 0:00:21 Sampling 8%|███▍ | ETA: 0:00:19 Sampling 8%|███▋ | ETA: 0:00:18 Sampling 9%|███▊ | ETA: 0:00:17 Sampling 10%|████ | ETA: 0:00:17 Sampling 10%|████▎ | ETA: 0:00:16 Sampling 10%|████▍ | ETA: 0:00:15 Sampling 11%|████▋ | ETA: 0:00:14 Sampling 12%|████▉ | ETA: 0:00:14 Sampling 12%|█████ | ETA: 0:00:13 Sampling 12%|█████▎ | ETA: 0:00:12 Sampling 13%|█████▌ | ETA: 0:00:12 Sampling 14%|█████▋ | ETA: 0:00:11 Sampling 14%|█████▉ | ETA: 0:00:11 Sampling 14%|██████▏ | ETA: 0:00:10 Sampling 15%|██████▎ | ETA: 0:00:10 Sampling 16%|██████▌ | ETA: 0:00:10 Sampling 16%|██████▊ | ETA: 0:00:09 Sampling 16%|██████▉ | ETA: 0:00:09 Sampling 17%|███████▏ | ETA: 0:00:09 Sampling 18%|███████▍ | ETA: 0:00:08 Sampling 18%|███████▌ | ETA: 0:00:08 Sampling 18%|███████▊ | ETA: 0:00:08 Sampling 19%|████████ | ETA: 0:00:08 Sampling 20%|████████▎ | ETA: 0:00:07 Sampling 20%|████████▍ | ETA: 0:00:07 Sampling 20%|████████▋ | ETA: 0:00:07 Sampling 21%|████████▉ | ETA: 0:00:07 Sampling 22%|█████████ | ETA: 0:00:06 Sampling 22%|█████████▎ | ETA: 0:00:06 Sampling 22%|█████████▌ | ETA: 0:00:06 Sampling 23%|█████████▋ | ETA: 0:00:06 Sampling 24%|█████████▉ | ETA: 0:00:06 Sampling 24%|██████████▏ | ETA: 0:00:06 Sampling 24%|██████████▎ | ETA: 0:00:05 Sampling 25%|██████████▌ | ETA: 0:00:05 Sampling 26%|██████████▊ | ETA: 0:00:05 Sampling 26%|██████████▉ | ETA: 0:00:05 Sampling 26%|███████████▏ | ETA: 0:00:05 Sampling 27%|███████████▍ | ETA: 0:00:05 Sampling 28%|███████████▌ | ETA: 0:00:05 Sampling 28%|███████████▊ | ETA: 0:00:05 Sampling 28%|████████████ | ETA: 0:00:04 Sampling 29%|████████████▏ | ETA: 0:00:04 Sampling 30%|████████████▍ | ETA: 0:00:04 Sampling 30%|████████████▋ | ETA: 0:00:04 Sampling 30%|████████████▊ | ETA: 0:00:04 Sampling 31%|█████████████ | ETA: 0:00:04 Sampling 32%|█████████████▎ | ETA: 0:00:04 Sampling 32%|█████████████▌ | ETA: 0:00:04 Sampling 32%|█████████████▋ | ETA: 0:00:04 Sampling 33%|█████████████▉ | ETA: 0:00:04 Sampling 34%|██████████████▏ | ETA: 0:00:04 Sampling 34%|██████████████▎ | ETA: 0:00:03 Sampling 34%|██████████████▌ | ETA: 0:00:03 Sampling 35%|██████████████▊ | ETA: 0:00:03 Sampling 36%|██████████████▉ | ETA: 0:00:03 Sampling 36%|███████████████▏ | ETA: 0:00:03 Sampling 36%|███████████████▍ | ETA: 0:00:03 Sampling 37%|███████████████▌ | ETA: 0:00:03 Sampling 38%|███████████████▊ | ETA: 0:00:03 Sampling 38%|████████████████ | ETA: 0:00:03 Sampling 38%|████████████████▏ | ETA: 0:00:03 Sampling 39%|████████████████▍ | ETA: 0:00:03 Sampling 40%|████████████████▋ | ETA: 0:00:03 Sampling 40%|████████████████▊ | ETA: 0:00:03 Sampling 40%|█████████████████ | ETA: 0:00:03 Sampling 41%|█████████████████▎ | ETA: 0:00:03 Sampling 42%|█████████████████▍ | ETA: 0:00:03 Sampling 42%|█████████████████▋ | ETA: 0:00:02 Sampling 42%|█████████████████▉ | ETA: 0:00:02 Sampling 43%|██████████████████ | ETA: 0:00:02 Sampling 44%|██████████████████▎ | ETA: 0:00:02 Sampling 44%|██████████████████▌ | ETA: 0:00:02 Sampling 44%|██████████████████▊ | ETA: 0:00:02 Sampling 45%|██████████████████▉ | ETA: 0:00:02 Sampling 46%|███████████████████▏ | ETA: 0:00:02 Sampling 46%|███████████████████▍ | ETA: 0:00:02 Sampling 46%|███████████████████▌ | ETA: 0:00:02 Sampling 47%|███████████████████▊ | ETA: 0:00:02 Sampling 48%|████████████████████ | ETA: 0:00:02 Sampling 48%|████████████████████▏ | ETA: 0:00:02 Sampling 48%|████████████████████▍ | ETA: 0:00:02 Sampling 49%|████████████████████▋ | ETA: 0:00:02 Sampling 50%|████████████████████▊ | ETA: 0:00:02 Sampling 50%|█████████████████████ | ETA: 0:00:02 Sampling 50%|█████████████████████▎ | ETA: 0:00:02 Sampling 51%|█████████████████████▍ | ETA: 0:00:02 Sampling 52%|█████████████████████▋ | ETA: 0:00:02 Sampling 52%|█████████████████████▉ | ETA: 0:00:02 Sampling 52%|██████████████████████ | ETA: 0:00:02 Sampling 53%|██████████████████████▎ | ETA: 0:00:02 Sampling 54%|██████████████████████▌ | ETA: 0:00:02 Sampling 54%|██████████████████████▋ | ETA: 0:00:02 Sampling 55%|██████████████████████▉ | ETA: 0:00:01 Sampling 55%|███████████████████████▏ | ETA: 0:00:01 Sampling 56%|███████████████████████▎ | ETA: 0:00:01 Sampling 56%|███████████████████████▌ | ETA: 0:00:01 Sampling 56%|███████████████████████▊ | ETA: 0:00:01 Sampling 57%|████████████████████████ | ETA: 0:00:01 Sampling 57%|████████████████████████▏ | ETA: 0:00:01 Sampling 58%|████████████████████████▍ | ETA: 0:00:01 Sampling 58%|████████████████████████▋ | ETA: 0:00:01 Sampling 59%|████████████████████████▊ | ETA: 0:00:01 Sampling 60%|█████████████████████████ | ETA: 0:00:01 Sampling 60%|█████████████████████████▎ | ETA: 0:00:01 Sampling 60%|█████████████████████████▍ | ETA: 0:00:01 Sampling 61%|█████████████████████████▋ | ETA: 0:00:01 Sampling 62%|█████████████████████████▉ | ETA: 0:00:01 Sampling 62%|██████████████████████████ | ETA: 0:00:01 Sampling 62%|██████████████████████████▎ | ETA: 0:00:01 Sampling 63%|██████████████████████████▌ | ETA: 0:00:01 Sampling 64%|██████████████████████████▋ | ETA: 0:00:01 Sampling 64%|██████████████████████████▉ | ETA: 0:00:01 Sampling 64%|███████████████████████████▏ | ETA: 0:00:01 Sampling 65%|███████████████████████████▎ | ETA: 0:00:01 Sampling 66%|███████████████████████████▌ | ETA: 0:00:01 Sampling 66%|███████████████████████████▊ | ETA: 0:00:01 Sampling 66%|███████████████████████████▉ | ETA: 0:00:01 Sampling 67%|████████████████████████████▏ | ETA: 0:00:01 Sampling 68%|████████████████████████████▍ | ETA: 0:00:01 Sampling 68%|████████████████████████████▌ | ETA: 0:00:01 Sampling 68%|████████████████████████████▊ | ETA: 0:00:01 Sampling 69%|█████████████████████████████ | ETA: 0:00:01 Sampling 70%|█████████████████████████████▎ | ETA: 0:00:01 Sampling 70%|█████████████████████████████▍ | ETA: 0:00:01 Sampling 70%|█████████████████████████████▋ | ETA: 0:00:01 Sampling 71%|█████████████████████████████▉ | ETA: 0:00:01 Sampling 72%|██████████████████████████████ | ETA: 0:00:01 Sampling 72%|██████████████████████████████▎ | ETA: 0:00:01 Sampling 72%|██████████████████████████████▌ | ETA: 0:00:01 Sampling 73%|██████████████████████████████▋ | ETA: 0:00:01 Sampling 74%|██████████████████████████████▉ | ETA: 0:00:01 Sampling 74%|███████████████████████████████▏ | ETA: 0:00:01 Sampling 74%|███████████████████████████████▎ | ETA: 0:00:01 Sampling 75%|███████████████████████████████▌ | ETA: 0:00:01 Sampling 76%|███████████████████████████████▊ | ETA: 0:00:01 Sampling 76%|███████████████████████████████▉ | ETA: 0:00:01 Sampling 76%|████████████████████████████████▏ | ETA: 0:00:01 Sampling 77%|████████████████████████████████▍ | ETA: 0:00:01 Sampling 78%|████████████████████████████████▌ | ETA: 0:00:01 Sampling 78%|████████████████████████████████▊ | ETA: 0:00:01 Sampling 78%|█████████████████████████████████ | ETA: 0:00:00 Sampling 79%|█████████████████████████████████▏ | ETA: 0:00:00 Sampling 80%|█████████████████████████████████▍ | ETA: 0:00:00 Sampling 80%|█████████████████████████████████▋ | ETA: 0:00:00 Sampling 80%|█████████████████████████████████▊ | ETA: 0:00:00 Sampling 81%|██████████████████████████████████ | ETA: 0:00:00 Sampling 82%|██████████████████████████████████▎ | ETA: 0:00:00 Sampling 82%|██████████████████████████████████▌ | ETA: 0:00:00 Sampling 82%|██████████████████████████████████▋ | ETA: 0:00:00 Sampling 83%|██████████████████████████████████▉ | ETA: 0:00:00 Sampling 84%|███████████████████████████████████▏ | ETA: 0:00:00 Sampling 84%|███████████████████████████████████▎ | ETA: 0:00:00 Sampling 84%|███████████████████████████████████▌ | ETA: 0:00:00 Sampling 85%|███████████████████████████████████▊ | ETA: 0:00:00 Sampling 86%|███████████████████████████████████▉ | ETA: 0:00:00 Sampling 86%|████████████████████████████████████▏ | ETA: 0:00:00 Sampling 86%|████████████████████████████████████▍ | ETA: 0:00:00 Sampling 87%|████████████████████████████████████▌ | ETA: 0:00:00 Sampling 88%|████████████████████████████████████▊ | ETA: 0:00:00 Sampling 88%|█████████████████████████████████████ | ETA: 0:00:00 Sampling 88%|█████████████████████████████████████▏ | ETA: 0:00:00 Sampling 89%|█████████████████████████████████████▍ | ETA: 0:00:00 Sampling 90%|█████████████████████████████████████▋ | ETA: 0:00:00 Sampling 90%|█████████████████████████████████████▊ | ETA: 0:00:00 Sampling 90%|██████████████████████████████████████ | ETA: 0:00:00 Sampling 91%|██████████████████████████████████████▎ | ETA: 0:00:00 Sampling 92%|██████████████████████████████████████▍ | ETA: 0:00:00 Sampling 92%|██████████████████████████████████████▋ | ETA: 0:00:00 Sampling 92%|██████████████████████████████████████▉ | ETA: 0:00:00 Sampling 93%|███████████████████████████████████████ | ETA: 0:00:00 Sampling 94%|███████████████████████████████████████▎ | ETA: 0:00:00 Sampling 94%|███████████████████████████████████████▌ | ETA: 0:00:00 Sampling 94%|███████████████████████████████████████▊ | ETA: 0:00:00 Sampling 95%|███████████████████████████████████████▉ | ETA: 0:00:00 Sampling 96%|████████████████████████████████████████▏ | ETA: 0:00:00 Sampling 96%|████████████████████████████████████████▍ | ETA: 0:00:00 Sampling 96%|████████████████████████████████████████▌ | ETA: 0:00:00 Sampling 97%|████████████████████████████████████████▊ | ETA: 0:00:00 Sampling 98%|█████████████████████████████████████████ | ETA: 0:00:00 Sampling 98%|█████████████████████████████████████████▏| ETA: 0:00:00 Sampling 98%|█████████████████████████████████████████▍| ETA: 0:00:00 Sampling 99%|█████████████████████████████████████████▋| ETA: 0:00:00 Sampling 100%|█████████████████████████████████████████▊| ETA: 0:00:00 Sampling 100%|██████████████████████████████████████████| Time: 0:00:01 Sampling 100%|██████████████████████████████████████████| Time: 0:00:02
2×1 Matrix{Float64}:
4.113592187410646
3.4119262240698984The posterior means sit close to the sample mean and standard deviation of the data, as they should under a flat prior.
mean(data), std(data)(4.1202834091735205, 3.2469040202463892)Several chains, in parallel, need no extra code. With MCMCThreads(), initial_params takes one starting point per chain.
chains = AbstractMCMC.sample(
Xoshiro(3),
model,
sampler,
AbstractMCMC.MCMCThreads(),
5_000,
3;
initial_params=fill([0.0, 1.0], 3),
discard_initial=1_000,
)
[mean(reduce(hcat, chain); dims=2) for chain in chains]Sampling (3 threads) 0%| | ETA: N/A Sampling (3 threads) 1%|▏ | ETA: 0:01:59 Sampling (3 threads) 1%|▎ | ETA: 0:00:59 Sampling (3 threads) 2%|▌ | ETA: 0:00:39 Sampling (3 threads) 2%|▋ | ETA: 0:00:29 Sampling (3 threads) 3%|▊ | ETA: 0:00:23 Sampling (3 threads) 3%|▉ | ETA: 0:00:19 Sampling (3 threads) 4%|█ | ETA: 0:00:16 Sampling (3 threads) 4%|█▎ | ETA: 0:00:14 Sampling (3 threads) 5%|█▍ | ETA: 0:00:13 Sampling (3 threads) 5%|█▌ | ETA: 0:00:11 Sampling (3 threads) 6%|█▋ | ETA: 0:00:10 Sampling (3 threads) 6%|█▊ | ETA: 0:00:09 Sampling (3 threads) 7%|██ | ETA: 0:00:09 Sampling (3 threads) 7%|██▏ | ETA: 0:00:08 Sampling (3 threads) 8%|██▎ | ETA: 0:00:07 Sampling (3 threads) 8%|██▍ | ETA: 0:00:07 Sampling (3 threads) 9%|██▌ | ETA: 0:00:06 Sampling (3 threads) 9%|██▊ | ETA: 0:00:06 Sampling (3 threads) 10%|██▉ | ETA: 0:00:06 Sampling (3 threads) 10%|███ | ETA: 0:00:05 Sampling (3 threads) 11%|███▏ | ETA: 0:00:05 Sampling (3 threads) 11%|███▎ | ETA: 0:00:05 Sampling (3 threads) 12%|███▌ | ETA: 0:00:05 Sampling (3 threads) 12%|███▋ | ETA: 0:00:04 Sampling (3 threads) 13%|███▊ | ETA: 0:00:04 Sampling (3 threads) 13%|███▉ | ETA: 0:00:04 Sampling (3 threads) 14%|████ | ETA: 0:00:04 Sampling (3 threads) 14%|████▎ | ETA: 0:00:04 Sampling (3 threads) 15%|████▍ | ETA: 0:00:04 Sampling (3 threads) 15%|████▌ | ETA: 0:00:03 Sampling (3 threads) 16%|████▋ | ETA: 0:00:03 Sampling (3 threads) 16%|████▊ | ETA: 0:00:03 Sampling (3 threads) 17%|█████ | ETA: 0:00:03 Sampling (3 threads) 17%|█████▏ | ETA: 0:00:03 Sampling (3 threads) 18%|█████▎ | ETA: 0:00:03 Sampling (3 threads) 18%|█████▍ | ETA: 0:00:03 Sampling (3 threads) 19%|█████▋ | ETA: 0:00:03 Sampling (3 threads) 19%|█████▊ | ETA: 0:00:03 Sampling (3 threads) 20%|█████▉ | ETA: 0:00:03 Sampling (3 threads) 20%|██████ | ETA: 0:00:02 Sampling (3 threads) 21%|██████▏ | ETA: 0:00:02 Sampling (3 threads) 21%|██████▍ | ETA: 0:00:02 Sampling (3 threads) 22%|██████▌ | ETA: 0:00:02 Sampling (3 threads) 22%|██████▋ | ETA: 0:00:02 Sampling (3 threads) 23%|██████▊ | ETA: 0:00:02 Sampling (3 threads) 23%|██████▉ | ETA: 0:00:02 Sampling (3 threads) 24%|███████▏ | ETA: 0:00:02 Sampling (3 threads) 24%|███████▎ | ETA: 0:00:02 Sampling (3 threads) 25%|███████▍ | ETA: 0:00:02 Sampling (3 threads) 25%|███████▌ | ETA: 0:00:02 Sampling (3 threads) 26%|███████▋ | ETA: 0:00:02 Sampling (3 threads) 26%|███████▉ | ETA: 0:00:02 Sampling (3 threads) 27%|████████ | ETA: 0:00:02 Sampling (3 threads) 27%|████████▏ | ETA: 0:00:02 Sampling (3 threads) 28%|████████▎ | ETA: 0:00:02 Sampling (3 threads) 28%|████████▍ | ETA: 0:00:02 Sampling (3 threads) 29%|████████▋ | ETA: 0:00:02 Sampling (3 threads) 29%|████████▊ | ETA: 0:00:02 Sampling (3 threads) 30%|████████▉ | ETA: 0:00:01 Sampling (3 threads) 30%|█████████ | ETA: 0:00:01 Sampling (3 threads) 31%|█████████▏ | ETA: 0:00:01 Sampling (3 threads) 31%|█████████▍ | ETA: 0:00:01 Sampling (3 threads) 32%|█████████▌ | ETA: 0:00:01 Sampling (3 threads) 32%|█████████▋ | ETA: 0:00:01 Sampling (3 threads) 33%|█████████▊ | ETA: 0:00:01 Sampling (3 threads) 33%|█████████▉ | ETA: 0:00:01 Sampling (3 threads) 34%|██████████▏ | ETA: 0:00:01 Sampling (3 threads) 34%|██████████▎ | ETA: 0:00:01 Sampling (3 threads) 35%|██████████▍ | ETA: 0:00:01 Sampling (3 threads) 35%|██████████▌ | ETA: 0:00:01 Sampling (3 threads) 36%|██████████▋ | ETA: 0:00:01 Sampling (3 threads) 36%|██████████▉ | ETA: 0:00:01 Sampling (3 threads) 37%|███████████ | ETA: 0:00:01 Sampling (3 threads) 37%|███████████▏ | ETA: 0:00:01 Sampling (3 threads) 38%|███████████▎ | ETA: 0:00:01 Sampling (3 threads) 38%|███████████▍ | ETA: 0:00:01 Sampling (3 threads) 39%|███████████▋ | ETA: 0:00:01 Sampling (3 threads) 39%|███████████▊ | ETA: 0:00:01 Sampling (3 threads) 40%|███████████▉ | ETA: 0:00:01 Sampling (3 threads) 40%|████████████ | ETA: 0:00:01 Sampling (3 threads) 41%|████████████▏ | ETA: 0:00:01 Sampling (3 threads) 41%|████████████▍ | ETA: 0:00:01 Sampling (3 threads) 42%|████████████▌ | ETA: 0:00:01 Sampling (3 threads) 42%|████████████▋ | ETA: 0:00:01 Sampling (3 threads) 43%|████████████▊ | ETA: 0:00:01 Sampling (3 threads) 43%|████████████▉ | ETA: 0:00:01 Sampling (3 threads) 44%|█████████████▏ | ETA: 0:00:01 Sampling (3 threads) 44%|█████████████▎ | ETA: 0:00:01 Sampling (3 threads) 45%|█████████████▍ | ETA: 0:00:01 Sampling (3 threads) 45%|█████████████▌ | ETA: 0:00:01 Sampling (3 threads) 46%|█████████████▋ | ETA: 0:00:01 Sampling (3 threads) 46%|█████████████▉ | ETA: 0:00:01 Sampling (3 threads) 47%|██████████████ | ETA: 0:00:01 Sampling (3 threads) 47%|██████████████▏ | ETA: 0:00:01 Sampling (3 threads) 48%|██████████████▎ | ETA: 0:00:01 Sampling (3 threads) 48%|██████████████▍ | ETA: 0:00:01 Sampling (3 threads) 49%|██████████████▋ | ETA: 0:00:01 Sampling (3 threads) 49%|██████████████▊ | ETA: 0:00:01 Sampling (3 threads) 50%|██████████████▉ | ETA: 0:00:01 Sampling (3 threads) 50%|███████████████ | ETA: 0:00:01 Sampling (3 threads) 51%|███████████████▎ | ETA: 0:00:01 Sampling (3 threads) 51%|███████████████▍ | ETA: 0:00:01 Sampling (3 threads) 52%|███████████████▌ | ETA: 0:00:01 Sampling (3 threads) 52%|███████████████▋ | ETA: 0:00:01 Sampling (3 threads) 53%|███████████████▊ | ETA: 0:00:01 Sampling (3 threads) 53%|████████████████ | ETA: 0:00:01 Sampling (3 threads) 54%|████████████████▏ | ETA: 0:00:01 Sampling (3 threads) 54%|████████████████▎ | ETA: 0:00:01 Sampling (3 threads) 55%|████████████████▍ | ETA: 0:00:01 Sampling (3 threads) 55%|████████████████▌ | ETA: 0:00:01 Sampling (3 threads) 56%|████████████████▊ | ETA: 0:00:01 Sampling (3 threads) 56%|████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 57%|█████████████████ | ETA: 0:00:00 Sampling (3 threads) 57%|█████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 58%|█████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 58%|█████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 59%|█████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 59%|█████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 60%|█████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 60%|██████████████████ | ETA: 0:00:00 Sampling (3 threads) 61%|██████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 61%|██████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 62%|██████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 62%|██████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 63%|██████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 63%|███████████████████ | ETA: 0:00:00 Sampling (3 threads) 64%|███████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 64%|███████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 65%|███████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 65%|███████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 66%|███████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 66%|███████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 67%|████████████████████ | ETA: 0:00:00 Sampling (3 threads) 67%|████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 68%|████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 68%|████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 69%|████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 69%|████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 70%|████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 70%|█████████████████████ | ETA: 0:00:00 Sampling (3 threads) 71%|█████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 71%|█████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 72%|█████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 72%|█████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 73%|█████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 73%|██████████████████████ | ETA: 0:00:00 Sampling (3 threads) 74%|██████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 74%|██████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 75%|██████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 75%|██████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 76%|██████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 76%|██████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 77%|███████████████████████ | ETA: 0:00:00 Sampling (3 threads) 77%|███████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 78%|███████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 78%|███████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 79%|███████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 79%|███████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 80%|███████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 80%|████████████████████████ | ETA: 0:00:00 Sampling (3 threads) 81%|████████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 81%|████████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 82%|████████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 82%|████████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 83%|████████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 83%|█████████████████████████ | ETA: 0:00:00 Sampling (3 threads) 84%|█████████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 84%|█████████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 85%|█████████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 85%|█████████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 86%|█████████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 86%|█████████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 87%|██████████████████████████ | ETA: 0:00:00 Sampling (3 threads) 87%|██████████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 88%|██████████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 88%|██████████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 89%|██████████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 89%|██████████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 90%|██████████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 90%|███████████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 91%|███████████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 91%|███████████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 92%|███████████████████████████▌ | ETA: 0:00:00 Sampling (3 threads) 92%|███████████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 93%|███████████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 93%|████████████████████████████ | ETA: 0:00:00 Sampling (3 threads) 94%|████████████████████████████▏ | ETA: 0:00:00 Sampling (3 threads) 94%|████████████████████████████▎ | ETA: 0:00:00 Sampling (3 threads) 95%|████████████████████████████▍ | ETA: 0:00:00 Sampling (3 threads) 95%|████████████████████████████▋ | ETA: 0:00:00 Sampling (3 threads) 96%|████████████████████████████▊ | ETA: 0:00:00 Sampling (3 threads) 96%|████████████████████████████▉ | ETA: 0:00:00 Sampling (3 threads) 97%|█████████████████████████████ | ETA: 0:00:00 Sampling (3 threads) 97%|█████████████████████████████▏| ETA: 0:00:00 Sampling (3 threads) 98%|█████████████████████████████▍| ETA: 0:00:00 Sampling (3 threads) 98%|█████████████████████████████▌| ETA: 0:00:00 Sampling (3 threads) 99%|█████████████████████████████▋| ETA: 0:00:00 Sampling (3 threads) 99%|█████████████████████████████▊| ETA: 0:00:00 Sampling (3 threads) 100%|██████████████████████████████| Time: 0:00:00
3-element Vector{Matrix{Float64}}:
[4.080717546174043; 3.3972664470008973;;]
[4.166681561816429; 3.395930515496544;;]
[4.079812200724063; 3.410402489594473;;]Chain formats
By default sample returns the vector of transitions, which is what chain_type=Any means. To return a richer object, implement AbstractMCMC.bundle_samples(samples, model, sampler, state, chain_type; kwargs...) for the chain_type you want to support, and users select it with the chain_type keyword argument of sample. When the sampler is used on a Turing model through externalsampler, Turing builds its own chain from AbstractMCMC.getparams(state) and AbstractMCMC.getstats(state), so those two methods are what to implement next. The External Samplers page describes that path.
Conclusion
Two step methods and a sampler type are enough to plug a new algorithm into sample, and from there into Turing. The interface keeps evolving, so please open an issue at AbstractMCMC with feature requests or problems.