SDMX Custom Structure Definitions (beta)¶

SDMX 3.2 allows Agencies to define their own custom metadata, extending the structure model for organisation specific needs. The Custom Structural Metadata framework requires two types of structures:

Custom Structure Definition (CSD) - defines the schema (the template) for a new structure type. This contains the properties and allowable content for custom content of a given type.
Custom Structure Instance (CSI) - an instance of a custom structure, which adheres to the rules defined by the Custom Structure Definition

SDMX 3.2 is not yet released

Custom Structure Definitions are part of SDMX 3.2, which has not been released. At the time of writing SDMX 3.2 is in internal review within the SDMX Technical Working Group; it has not yet reached public review.

Everything described in this guide is therefore provisional. The information model, the SDMX-ML and SDMX-JSON syntax, and the implementation in FMR may all change as a result of the internal and public review processes, and are not guaranteed to be backwards compatible with what is documented here.

Custom structures are provided for evaluation and feedback, and should not yet be relied on for production metadata.

Overview¶

The SDMX Information Model defines a fixed set of maintainable artefacts - Codelists, Concept Schemes, Data Structure Definitions, Dataflows, and so on. Every one of these classes is defined by the standard, which means that an organisation needing to manage a type of structural metadata the standard does not describe has historically had to manage it outside SDMX altogether.

A Custom Structure Definition (CSD) removes that restriction. A CSD is itself a maintainable SDMX artefact, and its purpose is to define the schema of a new class of maintainable artefact which does not exist in the SDMX Information Model. Once an Agency has published a CSD, any Agency can create and maintain Custom Structure Instances (CSIs) of it, and those instances are first class SDMX maintainable artefacts: they have an Agency, an identifier, a version, names, descriptions and annotations; they have URNs; they take part in cross referencing; and they are submitted and queried through the standard SDMX structure web services.

A Custom Structure Definition is therefore best understood as metadata about a structure type, in the same way that a Data Structure Definition is metadata about a dataset.

Note

Custom Structure Definitions are introduced by the draft SDMX 3.2 specification, so they can only be expressed in formats which support SDMX 3.2 - SDMX-ML 3.2, SDMX-JSON 2.2, and Fusion JSON.

The two artefacts¶

Artefact	What it is	Who maintains it
Custom Structure Definition (CSD)	The schema. Declares the class name of the new structure type, the properties instances may report, and any locally defined complex types.	The Agency inventing the new structure type.
Custom Structure Instance (CSI)	A structure which conforms to a CSD, reporting values for the properties the CSD declares.	Any Agency, not necessarily the Agency which maintains the CSD.

For example the IMF might publish a Custom Structure Definition with the identifier PIVOT_TABLE which declares the class name PivotTable. The OECD, the ECB, and any other Agency can then create their own Pivot Tables, each of which is validated against the IMF's definition.

The class those instances belong to is namespaced to the Agency which maintains the definition, not to the Agency which maintains the instance. The class name PivotTable on its own is not the identity of the type; the type is qualified by the definition's Agency, giving custom.imf.PivotTable. So an OECD Pivot Table built against the IMF's definition has the URN:

urn:sdmx:org.sdmx.infomodel.custom.imf.PivotTable=OECD:POP(1.0.0)

This is what allows two Agencies to coin the same class name without colliding - custom.imf.PivotTable and custom.oecd.PivotTable are different types, and are queried through different REST API resources (imf:pivottable and oecd:pivottable).

The class is qualified by the Agency but not by the version of the definition, so a single class covers instances of every version of IMF:PIVOT_TABLE. Conformance is determined instead by each instance's own reference to the definition, which either pins an absolute version - so publishing a new version leaves the instance untouched - or late binds with semantic version wildcard syntax to follow the definition as it evolves. See the class is namespaced, but not versioned for what this does and does not protect.

A first example¶

The following Custom Structure Definition declares a new class called Glossary, whose instances are item schemes of glossary terms. Each term has a name (inherited from the SDMX Nameable base), a multilingual definition, and may itself contain child terms.

SDMX-ML 3.2SDMX-JSON 2.2

<str:CustomStructureDefinition agencyID="IMF" id="GLOSSARY" version="1.0.0"
                               sdmxClassName="Glossary" base="ItemScheme">
    <com:Name xml:lang="en">Glossary</com:Name>
    <str:Property id="items" minOccurs="0">
        <str:CustomTypeReference type="TermType"/>
    </str:Property>
    <str:CustomType id="TermType" extends="Nameable" sdmxClassName="GlossaryTerm">
        <com:Name xml:lang="en">Term</com:Name>
        <str:Property id="definition" minOccurs="0">
            <str:ValueFormat textType="String" isMultiLingual="true"/>
        </str:Property>
        <str:Property id="children" minOccurs="0">
            <str:CustomTypeReference type="TermType"/>
        </str:Property>
    </str:CustomType>
</str:CustomStructureDefinition>

{
    "id": "GLOSSARY",
    "agencyID": "IMF",
    "version": "1.0.0",
    "names": {"en": "Glossary"},
    "sdmxClassName": "Glossary",
    "base": "ItemScheme",
    "properties": [
        {"id": "items", "minOccurs": 0, "customType": "TermType"}
    ],
    "customTypes": [
        {
            "id": "TermType",
            "names": {"en": "Term"},
            "extends": "Nameable",
            "sdmxClassName": "GlossaryTerm",
            "properties": [
                {"id": "definition", "minOccurs": 0,
                 "format": {"dataType": "String", "isMultiLingual": true}},
                {"id": "children", "minOccurs": 0, "customType": "TermType"}
            ]
        }
    ]
}

An instance of that definition, maintained by a different Agency:

SDMX-ML 3.2SDMX-JSON 2.2

<glossary:Glossary xmlns:glossary="urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition=IMF:GLOSSARY(1.0.0)"
                   agencyID="ECB" id="STAT_GLOSSARY" version="1.0.0">
    <com:Name xml:lang="en">Statistical Glossary</com:Name>
    <glossary:CustomStructureDefinition>urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition=IMF:GLOSSARY(1.0.0)</glossary:CustomStructureDefinition>
    <glossary:items id="GDP">
        <com:Name xml:lang="en">Gross Domestic Product</com:Name>
        <glossary:definition xml:lang="en">The total monetary value of all final goods and services produced within a country in a given period.</glossary:definition>
        <glossary:definition xml:lang="fr">La valeur monetaire totale de tous les biens et services finaux produits dans un pays sur une periode donnee.</glossary:definition>
    </glossary:items>
</glossary:Glossary>

{
    "id": "STAT_GLOSSARY",
    "agencyID": "ECB",
    "version": "1.0.0",
    "names": {"en": "Statistical Glossary"},
    "customStructureDefinition": "urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition=IMF:GLOSSARY(1.0.0)",
    "items": [
        {
            "id": "GDP",
            "names": {"en": "Gross Domestic Product"},
            "definition": [
                {"locale": "en", "value": "The total monetary value of all final goods and services produced within a country in a given period."},
                {"locale": "fr", "value": "La valeur monetaire totale de tous les biens et services finaux produits dans un pays sur une periode donnee."}
            ]
        }
    ]
}

Complete examples, including a non item scheme example, are on the Worked Examples page.

The model at a glance¶

CustomStructureDefinition            (maintainable; the schema)
 |- sdmxClassName                    the class name used in the URN of instances, e.g. PivotTable
 |- base                             Maintainable | ItemScheme
 |- Property *                       the properties of the instance itself
 |   |- id, minOccurs, maxOccurs
 |   '- one representation:          ValueFormat | Reference | IndirectReference | CustomTypeReference
 |- MutuallyExclusive *              sets of properties of which at most one may be reported
 '- CustomType *                     locally defined complex types
     |- extends                      Annotatable | Identifiable | Nameable
     |- extendsType                  another custom type in the same definition
     |- sdmxClassName                the class name used in URNs of nested objects
     |- Property *
     '- MutuallyExclusive *

Terminology¶

Term	Meaning
Custom Structure Definition (CSD)	The maintainable artefact which declares a new structure type: its class name, its properties, and its custom types.
Custom Structure Instance (CSI)	A maintainable artefact which conforms to a CSD. Referred to as an instance throughout this guide once the context is clear.
Class name (`sdmxClassName`)	The name of the class the CSD introduces, for example `PivotTable`. It appears in the URN of every CSI, and is decoupled from the identifier of the CSD. The class is always qualified by the Agency maintaining the CSD, so the full class is `custom.imf.PivotTable`.
Base	Whether instances are plain maintainable artefacts (`Maintainable`) or Item Schemes (`ItemScheme`).
Property	A single named piece of content that instances may report. Every property has exactly one representation and a cardinality.
Representation	How a property's value is expressed: a formatted value, a URN reference, an identifier resolved in a context, or a complex object.
Custom type	A complex type declared inside the CSD and used as the representation of one or more properties. Custom types may extend one another and may be recursive.

When to use a Custom Structure Definition¶

Custom Structure Definitions are deliberately open ended, so it is worth being clear about when they are the right tool.

Use a Custom Structure Definition when:

the metadata being managed is a structure in its own right - it has an owner, an identity, a version and a lifecycle - and no SDMX class describes it;
the metadata needs to reference other SDMX artefacts, and to participate in referential integrity and reference queries;
the structure needs to be published and consumed by other systems through the standard SDMX structure API.

Prefer the existing model when:

the requirement is to attach descriptive information to an existing artefact - use Reference Metadata and a Metadata Structure Definition;
the requirement is small and local - an Annotation is usually enough;
an SDMX class already exists for the concept, even if it is an imperfect fit. Instances of standard classes are understood by every SDMX tool, whereas instances of a custom class are only understood by systems which have read your definition.

Support and current limitations¶

Custom structures are read and written in SDMX-ML 3.2, SDMX-JSON 2.2, and Fusion JSON. They cannot be expressed in earlier SDMX formats, which have no place to put them.
Reference properties can be restricted to a class of SDMX artefact. Restricting a reference property to instances of another Custom Structure Definition (the csd attribute of a reference Target) is described by the standard but is not yet supported; a definition which uses it is rejected on read.
The text format facets of a value property (pattern, minLength, maxValue and so on) are held in the definition but are not yet applied to reported values. Cardinality, mutual exclusivity, required properties, value shape and referential integrity are all enforced.
Custom structures are maintained through the structure web services. There is no dedicated maintenance wizard in the user interface.

Where next¶

The Custom Structure Definition - defining a new structure type in detail.
Custom Structure Instances - creating structures which conform to a definition.
Worked Examples - complete definitions and instances in XML and JSON.
Web Services - submitting and querying custom structures.