Custom Structure Instance¶
Warning
SDMX 3.2 has not been released and is currently in internal review, so the model described on this page is provisional and may change as a result of the review process. See the introduction for details.
Overview¶
A Custom Structure Instance (CSI) is a maintainable artefact whose class is defined by a Custom Structure Definition (CSD) rather than by the SDMX standard. A CSI behaves like every other maintainable structure in the registry: it has an Agency, an identifier and a version, it has names, descriptions and annotations, it has a URN, it is versioned and audited, and it is submitted and queried through the standard structure web services.
Where the meaning is clear this guide calls a Custom Structure Instance simply an instance, and a Custom Structure Definition a definition.
An instance may be maintained by any Agency. There is no requirement for the Agency which maintains an instance to be the Agency which maintains the definition it conforms to - one Agency publishes the schema, and the whole community can create structures against it.
Structure Properties¶
| Structure Type | Instance of a Custom Structure Definition |
| Maintainable | Yes |
| Identifiable | Yes |
| Item Scheme | Only when the definition declares base="ItemScheme" |
| SDMX Information Model Versions | 3.2 (draft, in review) |
| URN namespace | urn:sdmx:org.sdmx.infomodel.custom.{csd-agency}.{ClassName} urn:sdmx:org.sdmx.infomodel.custom.itemscheme.{csd-agency}.{ClassName} |
| REST API resource | {csd-agency}:{classname} - see Web Services |
Conformance and the definition reference¶
Every instance carries a mandatory reference to the Custom Structure Definition it conforms to, under the reserved name CustomStructureDefinition. In SDMX-ML the instance element additionally sits in a namespace which is the URN of that definition, so the conformance declaration is unambiguous and can be seen without parsing the content:
<pivottable:PivotTable
xmlns:pivottable="urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition=IMF:PIVOT_TABLE(1.0.0)"
agencyID="OECD" id="POP" version="1.0.0">
<com:Name xml:lang="en">Population Pivot</com:Name>
<pivottable:CustomStructureDefinition>urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition=IMF:PIVOT_TABLE(1.0.0)</pivottable:CustomStructureDefinition>
...
</pivottable:PivotTable>
Because both the namespace and the reference include the version of the definition, instances conforming to different versions of the same definition can appear in a single message.
The definition must already exist in the registry when an instance is submitted. Without it the instance cannot be validated, and the submission is rejected.
URNs¶
The URN of an instance is built from the definition, not from the instance's own Agency:
- the package is
custom, orcustom.itemschemewhen the definition has anItemSchemebase, followed by the lower case, period separated Agency identifier of the Agency which maintains the definition; - the class is the
sdmxClassNamedeclared by the definition; - the Agency, ID and version are those of the instance.
| Artefact | URN |
|---|---|
| The definition, maintained by IMF | urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition=IMF:PIVOT_TABLE(1.0.0) |
| An instance, maintained by OECD | urn:sdmx:org.sdmx.infomodel.custom.imf.PivotTable=OECD:POP(1.0.0) |
| A nameable object inside that instance | urn:sdmx:org.sdmx.infomodel.custom.imf.PivotTableRowCol=OECD:POP(1.0.0).SEX |
| An item scheme instance, maintained by ECB | urn:sdmx:org.sdmx.infomodel.custom.itemscheme.imf.Glossary=ECB:STAT_GLOSSARY(1.0.0) |
| An item of that scheme | urn:sdmx:org.sdmx.infomodel.custom.itemscheme.imf.GlossaryTerm=ECB:STAT_GLOSSARY(1.0.0).GDP |
| A nested child item | urn:sdmx:org.sdmx.infomodel.custom.itemscheme.imf.GlossaryTerm=ECB:STAT_GLOSSARY(1.0.0).GDP.GDP_PC |
Putting the definition's Agency in the package is what keeps class names from colliding: two Agencies may both define a class called PivotTable, and their instances remain distinguishable because one is custom.imf.PivotTable and the other custom.oecd.PivotTable. For a sub Agency such as IMF.STA the package is custom.imf.sta.
The final token before the = is always the class name, so an instance URN is parseable by position without knowing the definition in advance.
The class is namespaced, but not versioned¶
Two independent version axes
This section is about the version of the Custom Structure Definition, not the version of the instance. The two are completely independent of one another, and it is easy to conflate them.
A Custom Structure Definition plays the same role for its instances that the SDMX Information Model plays for standard structures, and the analogy is exact enough to be worth following through.
Publishing a new version of a definition is analogous to a new release of the SDMX Information Model - the model definition itself has changed, in the same way that SDMX 3.1 changed what a standard structure may contain relative to SDMX 3.0. Reading across:
| SDMX Information Model | Custom structures |
|---|---|
| The SDMX Information Model | A Custom Structure Definition |
| A release of the model - SDMX 3.0, 3.1, 3.2 | A version of the definition - IMF:PIVOT_TABLE(1.0.0), (1.1.0), (2.0.0) |
A class in the model, for example Codelist |
The class a definition declares, for example custom.imf.PivotTable |
The REST resource codelist |
The REST resource imf:pivottable |
A Codelist, for example SDMX:CL_FREQ(2.1) |
An instance, for example OECD:POP(1.0.0) |
The consequences fall straight out of the analogy.
The class is not versioned by the model. Codelist is Codelist whether the Codelist in front of you was authored under SDMX 2.1 or SDMX 3.1 - the class name does not carry the model release. In the same way custom.imf.PivotTable is one single structure type, shared by instances conforming to every version of IMF:PIVOT_TABLE. The Agency qualifies the class; the version of the definition does not.
The resource spans every version of the model. Querying codelist returns Codelists regardless of which release of the SDMX Information Model they were built under. Likewise imf:pivottable returns every Pivot Table, whichever version of the definition it conforms to.
An instance's own version is unrelated to the model's. CL_FREQ(2.1) is version 2.1 of that Codelist; it tells you nothing about which release of the SDMX Information Model it uses. Identically, the (1.0.0) in custom.imf.PivotTable=OECD:POP(1.0.0) is the version of the instance, and says nothing about which version of the definition it conforms to. The OECD versions its Pivot Table on its own schedule; the IMF versions the definition on its own.
Where the analogy improves on the standard model: a standard SDMX artefact does not generally declare which release of the Information Model it conforms to - you infer it from the message format. A custom structure instance is explicit, because it references its definition directly:
This is a direct reference, meaning it resolves to exactly one definition. It is not, however, obliged to name an absolute version. Like any other direct reference in SDMX it may use semantic version wildcard syntax to late bind, and still resolves to a single definition:
| Reference | Resolves to |
|---|---|
IMF:PIVOT_TABLE(1.0.0) |
Exactly that version, for the life of the instance |
IMF:PIVOT_TABLE(1.0+.0) |
The latest minor version within 1.0, so 1.1.0 once it is published |
IMF:PIVOT_TABLE(+) |
The latest stable version |
So the maintainer of the instance chooses how it tracks the model:
- Pin an absolute version to freeze conformance - "this Pivot Table is built to version 1.0.0 of the definition". Nothing the maintainer of the definition publishes later changes what the instance conforms to.
- Late bind to follow the model as it is released - "this Pivot Table is built to the latest 1.x definition". The instance conforms to whichever version currently matches, which is what you want when the definition is expected to grow and instances should pick up the additions without being resubmitted. This relies on the definition being extended compatibly, because content valid under
1.0.0must remain valid under1.1.0.
The class recorded against an instance is derived from the definition it was validated against, rather than being asserted independently, so an instance can never claim a class which its definition does not declare. An instance can only be constructed when its definition is available.
The result as the definition - the model - evolves:
- publishing a new version of a definition does not invalidate, migrate or retro-validate instances which pin an absolute version. Just as a new release of the SDMX Information Model does not rewrite the Codelists already in the registry, a new version of a definition leaves pinned instances exactly as they are;
- instances which late bind pick the new version up, so a new version must remain compatible with the content those instances already report;
- instances conforming to different versions of the same definition coexist in the registry, and may be returned together in one message;
- a query on the resource returns instances of every version of the definition, so a consumer which cares about the distinction should read each instance's
CustomStructureDefinitionreference; - a definition cannot be deleted while instances reference it, because that reference is a real cross reference like any other.
Changing the class name is a breaking change
The one change which is not protected is changing sdmxClassName in a new version of a definition. The class is part of the type identity, so this does not version the class - it introduces a different class alongside the old one. In the analogy, it is not a new release of the model renaming one of its classes; it is a new release introducing a wholly separate class and abandoning the old one.
Existing instances keep the old class in their URNs and remain queryable only under the old resource, while new instances get the new class and a new resource. The Agency also becomes responsible for ensuring the new class name does not collide with another of its own definitions. Treat sdmxClassName as fixed for the life of a definition; if the class genuinely needs to change, publish it as a new definition.
Reporting content¶
Every declared property is reported by its identifier. What the value looks like depends on the representation the definition gave the property.
Values¶
A ValueFormat property is reported as a simple value:
A localisable property (isMultiLingual="true") is reported once per language:
A repeatable property is simply reported more than once. In JSON a property whose maxOccurs allows more than one value is always written as an array, even when a single value is reported, so that consumers can rely on the shape of the response.
References¶
A Reference property is reported as the URN of the referenced artefact:
Wildcarded and late binding version references are permitted, so a reference such as Dataflow=OECD:DF_POP(1.0+.0) can be used to follow the latest minor version of the referenced artefact.
Indirect references¶
An IndirectReference property is reported as a plain identifier. The definition names the property which provides the context:
Here dimension is an indirect reference to a Dimension whose context is the dataflow property. SEX is enough, because the Dataflow is known.
Complex objects¶
A CustomTypeReference property is reported as a nested object whose content is the properties of the custom type:
Objects of an Identifiable custom type must supply an id. Objects of a Nameable custom type must supply an id and at least one Name, and may supply a Description; both are multilingual. Objects of an Annotatable custom type must not supply any of them - they are anonymous. Where a name or description is supplied without a language, en is assumed.
Identifiable and nameable objects are given a generated URN, built from the URN of the instance, the class name of their type, and their identifier.
Item scheme instances¶
When the definition declares base="ItemScheme", the values of the items property are the items of the scheme. Each item requires an id and a Name.
<glossary:items id="GDP">
<com:Name xml:lang="en">Gross Domestic Product</com:Name>
<glossary:definition xml:lang="en">The total monetary value of all final goods and services produced within a country in a given period.</glossary:definition>
<glossary:children id="GDP_PC">
<com:Name xml:lang="en">GDP per capita</com:Name>
<glossary:definition xml:lang="en">Gross domestic product divided by the population of the country.</glossary:definition>
</glossary:children>
</glossary:items>
"items": [
{
"id": "GDP",
"names": {"en": "Gross Domestic Product"},
"definition": [
{"locale": "en", "value": "The total monetary value of all final goods and services produced within a country in a given period."}
],
"children": [
{
"id": "GDP_PC",
"names": {"en": "GDP per capita"},
"definition": [
{"locale": "en", "value": "Gross domestic product divided by the population of the country."}
]
}
]
}
]
Item scheme instances behave like standard item schemes:
- items can be queried individually through the REST API by adding the item identifier to the query path;
- a query for one item returns the scheme marked as partial, containing that item and its ancestors;
- items keep the order in which they are reported.
Hierarchies are modelled explicitly by the definition. In the example above the children property of the item type is what creates the hierarchy; the URN of a nested item reflects its position in that hierarchy (...GlossaryTerm=ECB:STAT_GLOSSARY(1.0.0).GDP.GDP_PC).
Validation of an instance¶
An instance is validated against its definition on submission. The content is checked as a whole - the instance is either accepted in full or rejected.
| Check | Rejected when |
|---|---|
| Known properties | A reported property is not declared by the definition or by the relevant custom type, including through its extension chain. |
| Value shape | A simple value is reported for a complex property, or an object for a simple property. |
| Cardinality | A property is reported more times than its maxOccurs allows, or fewer times than its minOccurs requires. |
| Required properties | A property with a minOccurs of 1 or more is not reported at all - unless it is a member of a mutually exclusive set. |
| Mutual exclusivity | Two or more members of a mutually exclusive set are reported together. |
| Base attributes | An identifiable or nameable object has no id; a nameable object has no Name; an annotatable object declares a name or description. |
| Reference resolution | A Reference property does not hold a URN, or the referenced artefact does not exist. |
| Indirect reference resolution | The context cannot be resolved, or the identified object does not exist in the resolved context. |
Custom type properties are validated recursively, so the rules above apply at every level of nesting.
The text format facets of a value property (pattern, minLength, maxValue and so on) are held by the definition but are not yet applied to reported values.
Referential integrity¶
Instances take part in cross reference resolution exactly like standard structures, which is what makes custom structures useful rather than merely storable: impact analysis, reference queries and delete protection all work against them.
Direct references¶
A Reference property produces a direct cross reference from the instance (or from the enclosing identifiable object) to the named artefact. The artefact must exist at submission time:
Could not resolve reference from structure PivotTable 'SDMX:POP(1.0.0)' to structure
Dataflow 'urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=SDMX:DF_POP(1.0.0)'
Indirect references¶
An indirect reference is resolved by turning the reported identifier into a full URN:
- the context property named by the definition is located. It is looked up in the closest enclosing scope which reports it, so a property declared once at the root of the instance serves every nested object;
- if the context is itself an indirect reference, it is resolved first, recursively, giving a chain such as Code identifier → Dimension → Dataflow;
- the resolved context is used to find the maintainable which parents the target. If the context is already of the target's maintainable type it is used directly; otherwise the context artefact's references are walked until the right type is found - for example a Dataflow references a Data Structure Definition, which is the maintainable parent of a Dimension;
- the target URN is generated from that maintainable, the target class and the reported identifier.
Taking the Pivot Table as an example, where the dataflow property references OECD:DF_POP(1.0.0) and that Dataflow uses the Data Structure Definition OECD:POP(1.0.0), the reported value SEX on a dimension property whose context is dataflow resolves to:
If the Dataflow exists but the Dimension does not, the submission is rejected:
Could not resolve reference from structure PivotTableSlice 'SDMX:POP(1.0.0).TIME' to structure
Dimension 'urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=SDMX:POP(1.0.0).TIME_PERIOD'
Once stored, these resolved references behave like any other cross reference, so:
- a
references=childrenquery on the instance returns the definition it conforms to together with the artefacts it references; - a
references=parentsquery on a Dataflow returns the custom structure instances which reference it.
Versioning¶
Instances are versioned independently of the definition they conform to.
- An instance follows the standard versioning rules, so
1.0.0,1.0.1and1.1.0of the same instance can coexist, and version queries such as1.0+.0and~resolve as they do for any structure. - The version of the definition an instance names is fixed for that instance. Publishing a new version of the definition leaves existing instances conforming to the version they name.
- Instances conforming to different versions of the same definition may be returned together in one message.