VarNamedTuple

In many places Turing.jl uses a custom data structure, VarNamedTuple, to represent mappings of VarNames to arbitrary values.

This completely replaces the usage of NamedTuples or OrderedDict{VarName} in previous versions.

NoteA refresher on VarNames

A VarName is an object that represents an expression on the left-hand side of a tilde-statement. For example, x, x[1], and x.a are all valid VarNames.

VarNames can be constructed using the @varname macro:

using AbstractPPL # Reexported by DynamicPPL and Turing too

@varname(x), @varname(x[1]), @varname(x.a)
(x, x[1], x.a)

For a more detailed explanation of VarNames, see the AbstractPPL documentation.

Currently, VarNamedTuple is defined in DynamicPPL.jl; it may be moved to AbstractPPL.jl in the future once its functionality has stabilised.

Using VarNamedTuples

Very often, VarNamedTuples are constructed automatically inside Turing.jl models, and you do not need to create them yourself. Here is a simple example of a VarNamedTuple created automatically by Turing.jl when running mode estimation:

using Turing

@model function demo_model()
    x = Vector{Float64}(undef, 2)
    x[1] ~ Normal()
    x[2] ~ Beta(2, 2)
    y ~ Normal(x[1] + x[2], 1)
end
model = demo_model() | (; y = 1.0)

res = maximum_a_posteriori(model)

# This is a VarNamedTuple.
res.params
VarNamedTuple
└─ x => [0.2353417097732074, 0.5293165803432824]

As far as using VarNamedTuples goes, they behave very similarly to Dict{VarName}s. You can access the stored values using getindex:

res.params[@varname(x[1])]
0.2353417097732074

The nice thing about VarNamedTuples is that they contain knowledge about the structure of the variables inside them (which is stored during the model evaluation). For example, this particular VarNamedTuple knows that x is a length-2 vector, so you can access

res.params[@varname(x)]
2-element Vector{Float64}:
 0.2353417097732074
 0.5293165803432824

even though x itself was never on the left-hand side of a tilde-statement (only x[1] and x[2] were). This is not possible with a Dict{VarName}. You can even do things like:

res.params[@varname(x[end])]
0.5293165803432824

and it will work ‘as expected’.

Put simply, indexing into a variable in a VarNamedTuple mimics indexing into the original variable itself as far as possible.

Creating VarNamedTuples

If you only ever need to read from a VarNamedTuple, then the above section would suffice. However, there are also some cases where we ask users to construct a VarNamedTuple.

Some cases where Turing users may need to construct a VarNamedTuples include the following:

  • Providing initial parameters for MCMC sampling or optimisation;
  • Providing parameters to condition models on, or to fix.
NoteA deeper dive into VarNamedTuples

If you are developing against Turing or DynamicPPL (e.g. if you are writing custom inference algorithms), you will also probably need to create VarNamedTuples. In this case you will likely have to understand their lower-level APIs. We strongly recommend reading the DynamicPPL docs, where we explain the design and implementation of VarNamedTuples in much more detail.

To create a VarNamedTuple, you can use the VarNamedTuple constructor directly:

VarNamedTuple(x = 1, y = "a", z = [1, 2, 3])
VarNamedTuple
├─ x => 1
├─ y => "a"
└─ z => [1, 2, 3]

However, this direct constructor only works for variables that are top-level symbols. If you have VarNames that contain indexing or field access, we recommend using the @vnt macro, which is exported from DynamicPPL and Turing.

using Turing

vnt = @vnt begin
    x := 1
    y.a.b := "a"
    z[1] := 10
end
┌ Warning: Creating a growable `Base.Array` of dimension 1 to store values. This may not match the actual type or size of the actual `AbstractArray` that will be used inside the DynamicPPL model.
│ 
│  If this is not the type or size that you expect, please see: https://turinglang.org/docs/uri/growablearray
└ @ DynamicPPL.VarNamedTuples ~/.julia/packages/DynamicPPL/dm0sE/src/varnamedtuple/partial_array.jl:796

VarNamedTuple
├─ x => 1
├─ y => VarNamedTuple
│       └─ a => VarNamedTuple
│               └─ b => "a"
└─ z => PartialArray size=(1,) data::DynamicPPL.VarNamedTuples.GrowableArray{Int64, 1}
        └─ (1,) => 10

Here, each line with := indicates that we are setting that VarName to the corresponding value. You can have any valid VarName on the left-hand side. (Note that you must use colon-equals; we reserve the syntax x = y for future use.)

GrowableArrays

In the above example, vnt is a VarNamedTuple with three entries. However, you may have noticed the warning issued about a GrowableArray. What does that mean?

The problem with the above call is that when setting z[1], the VarNamedTuple does not yet know what z is supposed to be. It is probably a vector, but in principle it could be a matrix (where z[1] is using linear indexing). Furthermore, we don’t know what type of array it is. It could be Base.Array, or it could be some custom array type, like OffsetArray.

GrowableArray is DynamicPPL’s way of representing an array whose size and type are not yet known. When you set z[1] := 10, DynamicPPL creates a one-dimensional GrowableArray for z, which can then be ‘grown’ as more entries are set. However, this is a heuristic, and may not always be correct; hence the warning.

Templating

To avoid this, we strongly recommend that whenever you have variables that are arrays or structs, you provide a ‘template’ for them. A template is an array that has the same type and shape as the variable that will eventually be used in the model.

For example, if your model looks like this:

@model function demo_template()
    # ...
    z = zeros(2, 2, 2)
    z[1] ~ Normal()
    # ...
end
demo_template (generic function with 2 methods)

then the template for z should be any Base.Array{T,3} of size (2, 2, 2). (The element type does not matter, as it will be inferred from the values you set.)

To specify a template, you can use the @template macro inside the @vnt block. The following example, for example, says that z inside the model will be a 3-dimensional Base.Array of size (2, 2, 2). The fact that it contains zeros is irrelevant, so you can provide any template that is structurally the same.

vnt = @vnt begin
    @template z = zeros(2, 2, 2)
    z[1] := 1.0
end
VarNamedTuple
└─ z => PartialArray size=(2, 2, 2) data::Array{Float64, 3}
        └─ (1, 1, 1) => 1.0

Notice now that the created VarNamedTuple knows that z is a 3-dimensional array, so no warnings are issued. Furthermore, you can now index into it as if it were a 3D array:

vnt[@varname(z[1, 1, 1])]
1.0

(With a GrowableArray, this would have errored.)

When setting a template, you can use any valid Julia expression on the right-hand side (such as variables from the surrounding scope). Any expressions in templates are only evaluated once.

You can also omit the right-hand side, in which case the template will be assumed to be the variable with that name:

# Declare this variable outside.
z = zeros(2, 2, 2)

# The following is equivalent to `@template z = z`.
vnt = @vnt begin
    @template z
    z[1] := 1.0
end
VarNamedTuple
└─ z => PartialArray size=(2, 2, 2) data::Array{Float64, 3}
        └─ (1, 1, 1) => 1.0

Multiple templates can also be set on the same line, using space-separated assignments: @template x = expr1 y = expr2 ....

Nested values

If you have nested structs or arrays, you need to provide templates for the top-level symbol.

vnt = @vnt begin
    @template y = (a = zeros(2), b = zeros(3))
    y.a[1] := 1.0
    y.b[2] := 2.0
end
VarNamedTuple
└─ y => VarNamedTuple
        ├─ a => PartialArray size=(2,) data::Vector{Float64}
        │       └─ (1,) => 1.0
        └─ b => PartialArray size=(3,) data::Vector{Float64}
                └─ (2,) => 2.0

This restriction will probably be lifted in future versions; for example if you are trying to set a value y.a[1], you could provide a template for y.a without providing one for y.

Back to top