Custom Structure Definition¶
Warning
SDMX 3.2 has not been released and is currently in internal review, so the model described on this page is provisional and may change as a result of the review process. See the introduction for details.
Overview¶
A Custom Structure Definition (CSD) declares the schema of a new class of maintainable artefact. It is itself a maintainable artefact, so it is owned by an Agency, has an identifier and a version, and is submitted, queried, versioned and audited exactly like any other structure in the registry.
A definition declares:
- the class name of the structure type it introduces;
- the base its instances extend - a plain maintainable artefact, or an item scheme;
- an ordered set of properties which instances may report, each with a cardinality and exactly one representation;
- optional mutually exclusive sets over those properties;
- optional locally defined custom types, which are complex types used as the representation of properties.
A Custom Structure Definition never references another structure. It constrains its properties by class (for example "a reference to a Dataflow"), not by naming a specific artefact, so a definition can be published and maintained independently of the content of the registry.
Structure Properties¶
| Structure Type | Standard SDMX Structural Metadata Artefact |
| Maintainable | Yes |
| Identifiable | Yes |
| Item Scheme | No |
| SDMX Information Model Versions | 3.2 (draft, in review) |
| URN namespace | urn:sdmx:org.sdmx.infomodel.csd.CustomStructureDefinition |
| REST API resource | customstructuredefinition |
The URN of a definition follows the standard maintainable pattern:
Class name¶
Every definition must declare an sdmxClassName. This is the name of the class being introduced, and it is the name which appears in the URN of every instance and as the element name of every instance in SDMX-ML.
The class name is not the identity of the type on its own. The type is always qualified by the Agency which maintains the definition, so the definition above introduces the class custom.imf.PivotTable (custom.itemscheme.imf.… had the base been ItemScheme). This qualification carries through everywhere the type is used - the URN package of instances, the identity of the type within the registry, and the REST API resource - which is why two Agencies may each declare a class named PivotTable without colliding.
| Rule | Detail |
|---|---|
| Required | A definition without an sdmxClassName is rejected. |
| Format | Must start with a letter and may contain only letters and digits. By convention it is upper camel case, for example PivotTable. |
| Independent of the ID | The class name is not derived from, and need not resemble, the identifier of the definition. IMF:PIVOT_TABLE(1.0.0) declaring class PivotTable is typical, but not required. |
| Unique within the definition | The class name of the definition and the class names of all its custom types must be distinct from one another. |
| Unique within the Agency | Because the URN of an instance is built from the Agency of the definition and the class name, an Agency must not use the same class name in two different definitions. This is the responsibility of the maintaining Agency. |
Base type¶
The base attribute determines what kind of maintainable artefact the instances are. It defaults to Maintainable.
| Base | Instances are | Notes |
|---|---|---|
Maintainable |
Plain maintainable artefacts | Content is whatever the declared properties allow. |
ItemScheme |
Item Schemes | The definition must declare a root property with the reserved identifier items, whose representation is a custom type. That property carries the items of the scheme. |
An item scheme based definition gains the behaviour users expect of an item scheme: its items are individually identifiable, they can be queried individually through the REST API, and the returned scheme is marked as partial when only some items are requested.
Hierarchies within an item scheme are explicit, not implicit. There is no built in parent/child relationship; if the items form a hierarchy, the item type declares a recursive property for its children, as the children property does in the Glossary example.
Properties¶
A property is a single named piece of content that an instance may report. Properties are declared by the definition itself (in which case instances report them at the top level) and by each custom type (in which case objects of that type report them).
| Attribute / element | Purpose |
|---|---|
id |
The identifier of the property. It becomes an element name in SDMX-ML and a member name in JSON, so it must be a valid XML NCName. Must be unique within its container. |
minOccurs |
The minimum number of values which must be reported. Defaults to 1, which makes the property mandatory. Set to 0 for an optional property. |
maxOccurs |
The maximum number of values which may be reported. If it is not supplied the property is unbounded; a single valued property must state maxOccurs="1". |
Description |
An optional single human readable description of the property. A property has no name; its identifier serves that purpose. |
| representation | Exactly one of ValueFormat, Reference, IndirectReference or CustomTypeReference. |
Note
A property which is a member of a mutually exclusive set is in practice optional, whatever its minOccurs, because at most one member of the set may be present.
Cardinality¶
An omitted minOccurs or maxOccurs means the same thing in every format:
| Omitted attribute | Meaning |
|---|---|
minOccurs |
1 - the property is mandatory |
maxOccurs |
unbounded - the property has no upper limit |
So the two ends of the cardinality behave differently by default, and it is worth reading that twice: a property declared with no cardinality at all is mandatory and repeatable.
<!-- mandatory, unbounded -->
<str:Property id="definition">
<str:ValueFormat textType="String" isMultiLingual="true"/>
</str:Property>
<!-- optional, unbounded -->
<str:Property id="rows" minOccurs="0">
<str:CustomTypeReference type="RowColType"/>
</str:Property>
<!-- mandatory, single valued -->
<str:Property id="dataflow" maxOccurs="1">
<str:Reference>
<str:Target class="Dataflow"/>
</str:Reference>
</str:Property>
The same three properties in the same order - mandatory unbounded, optional unbounded, mandatory single valued. minOccurs and maxOccurs are plain numbers:
A single valued property must say so
Because an absent maxOccurs means unbounded, a property which may only be reported once must state maxOccurs explicitly. This is deliberately different from the XML Schema convention, where an absent maxOccurs defaults to 1: the JSON formats have no literal for "unbounded", so absence is the only way to express it there, and SDMX-ML follows the same rule so that omitting the attribute never changes meaning between formats.
In SDMX-ML the literal maxOccurs="unbounded" also remains valid and means exactly the same as omitting the attribute. It is not written on output - an unbounded property is serialised by leaving maxOccurs out, in both XML and JSON.
Representations¶
Every property has exactly one representation, which determines what a reported value looks like and how it is validated.
| Representation | Reported value | Typical use |
|---|---|---|
ValueFormat |
A formatted value | A string, number, boolean, date or time period; a multilingual text |
Reference |
The URN of an SDMX artefact | "this table is built over Dataflow X" |
IndirectReference |
The identifier of an object, resolved against a context established by another property | "the Dimension called REF_AREA, in the Dataflow referenced by the dataflow property" |
CustomTypeReference |
A complex object conforming to a custom type of this definition | Rows, columns, terms, headers - any structured sub object |
In SDMX-ML the four representations are a schema level choice, so a schema valid Property element always carries exactly one of them. Supplying two is rejected, although the error reports it as a duplicate property identifier rather than as a representation problem.
In JSON the representation is the keyword which appears alongside the property's id - format, reference, indirectReference or customType. A property with none of them is read as a value property, so an unrestricted value can be written as just an identifier and a cardinality:
A value property always ends up with a text type. Where none is supplied - because the representation was omitted entirely, or because it restricts nothing - the text type defaults to String. The property above is therefore equivalent to, and written back as:
and in SDMX-ML as <str:ValueFormat textType="String"/>. The same applies to an empty <str:ValueFormat/> on input.
ValueFormat¶
ValueFormat gives the property a text type and, optionally, the standard SDMX representation facets. The text type is not mandatory; a value property which does not declare one is a String.
Any SDMX text type may be used, including String, Boolean, Integer, Decimal, Count, URI, DateTime and the time period types such as ObservationalTimePeriod and ReportingYear. The available facets are those of any SDMX representation: minValue, maxValue, minLength, maxLength, decimals, pattern, startTime, endTime, interval, sequence and so on.
Setting isMultiLingual="true" on a String makes the property localisable. A localisable property is reported once per language, so make it unbounded if more than one language is expected:
Reference¶
A reference property holds the URN of an SDMX artefact. The permitted targets are given by zero or more Target elements; where several are supplied, the permitted set is their union. Where none is supplied, the property may reference any SDMX identifiable artefact.
The class attribute accepts any SDMX artefact class - Dataflow, Codelist, Code, Dimension, ConceptScheme, DataStructure, Category and so on - as well as the open values Any, AnyMaintainable and AnyIdentifiable.
A reference property contributes a real cross reference from the instance, so the referenced artefact must exist when the instance is submitted, and the instance is returned by a references=parents query on the referenced artefact.
Note
The standard also allows a Target to be restricted by csd, limiting the property to instances of a named Custom Structure Definition. This is not yet supported; a definition which uses it is rejected on read.
IndirectReference¶
An indirect reference property holds a plain identifier rather than a URN, and names another property which establishes the context in which that identifier is resolved. This is what allows a custom structure to say "the Dimension REF_AREA" without repeating the whole Dataflow, DSD and Dimension URN on every value.
| Attribute | Purpose |
|---|---|
targetClass |
The class of the object being identified, for example Dimension, Code, Measure. |
context |
The identifier of the property which establishes the resolution context. |
The context may name a sibling property, or a property declared at the root of the definition (which is how a property inside a custom type can resolve against the Dataflow declared once by the instance). The context may itself be another indirect reference property, allowing chains - a Code identifier resolved in the context of a Dimension, which is in turn resolved in the context of a Dataflow. A chain must terminate at a Reference property and must not be cyclic.
How the resolution actually happens at submission time, and what happens when it fails, is covered under referential integrity on the instance page.
CustomTypeReference¶
A custom type reference gives the property a complex value conforming to one of the definition's custom types.
A property may reference the type which contains it, which is how recursive structures such as hierarchies and nested table headers are expressed.
Mutually exclusive properties¶
A MutuallyExclusive element declares that at most one of a set of two or more properties may be reported. It may be declared by the definition itself and by any custom type, and it may only reference properties declared in the same container.
<str:Property id="headingText" maxOccurs="1">
<str:ValueFormat textType="String"/>
</str:Property>
<str:Property id="headingCode" maxOccurs="1">
<str:Reference>
<str:Target class="Code"/>
</str:Reference>
</str:Property>
<str:MutuallyExclusive>
<str:Member property="headingText"/>
<str:Member property="headingCode"/>
</str:MutuallyExclusive>
Here a row heading can either be free text or a reference to a Code, but not both. A set with fewer than two members, or a set referencing a property which is not declared in the same container, is rejected.
Custom types¶
A custom type is a complex type defined locally within the definition. Custom types exist to be used as the representation of properties; they are not independently maintainable and cannot be shared between definitions.
| Attribute / element | Purpose |
|---|---|
id |
The identifier of the type, unique within the definition. This is the value used by a CustomTypeReference. |
extends |
The abstract SDMX base type the custom type extends: Annotatable, Identifiable or Nameable. |
extendsType |
The identifier of another custom type in the same definition which this type extends. Mutually exclusive with extends. |
sdmxClassName |
The class name used in the URNs of objects of this type inside instances. Required for a type which is identifiable or nameable, and rejected on an annotatable type, which has no URN. See Base types. |
Name / Description |
Optional multilingual documentation of the type itself. |
Property / MutuallyExclusive |
The content model of the type, exactly as for the definition. |
Base types¶
The base type determines which SDMX base attributes objects of the type carry implicitly, in addition to the declared properties.
| Base | Implicit content | URN | sdmxClassName |
|---|---|---|---|
Annotatable |
Annotations only. The object is an anonymous bag of property values. | None | Must not be supplied |
Identifiable |
Annotations, a mandatory id, and a generated URN |
Yes | Required |
Nameable |
Annotations, a mandatory id, a generated URN, a mandatory multilingual name, and an optional multilingual description |
Yes | Required |
If neither extends nor extendsType is supplied the type extends Annotatable.
Choose Annotatable for a value object which only exists inside its parent - a key/value pair, a coordinate, a formatting option. Choose Identifiable or Nameable when the object needs to be addressable in its own right, because those objects get URNs and can be referenced.
The class name exists only for URN generation
sdmxClassName has exactly one purpose on a custom type: supplying the class token of the URNs generated for objects of that type. It follows that only a type whose objects have URNs can have one.
An annotatable type has no identifier and no URN, so it has no class name. The attribute does not apply, and a definition which supplies one anyway is rejected - this is not a case of the value being quietly ignored. The same applies to a type which reaches Annotatable through its extendsType chain rather than by naming the base directly.
Conversely, a type which is identifiable or nameable, whether directly through extends or transitively through its chain, must declare a class name; a definition which omits it is rejected.
In the Collection Table example, DimensionValueType is annotatable and carries no class name, while HeaderType is identifiable and declares sdmxClassName="Header":
<!-- annotatable: a key/value pair which exists only inside its parent, so no URN and no class name -->
<str:CustomType id="DimensionValueType" extends="Annotatable">
<com:Name xml:lang="en">Key Value Pair</com:Name>
...
</str:CustomType>
<!-- identifiable: objects get an id and a generated URN, so a class name is required -->
<str:CustomType id="HeaderType" extends="Identifiable" sdmxClassName="Header">
<com:Name xml:lang="en">Table Header, Row, or Slice</com:Name>
...
</str:CustomType>
The annotatable type carries no class name; the identifiable type must declare one:
Extension¶
extendsType lets one custom type extend another in the same definition:
<str:CustomType id="RowColType" extends="Nameable" sdmxClassName="PivotTableRowCol">
...
</str:CustomType>
<str:CustomType id="SliceType" extendsType="RowColType" sdmxClassName="PivotTableSlice">
<com:Name xml:lang="en">Slice</com:Name>
<str:Property id="position" minOccurs="0" maxOccurs="1">
<str:ValueFormat textType="Integer" minValue="0"/>
</str:Property>
</str:CustomType>
"customTypes": [
{
"id": "RowColType",
"extends": "Nameable",
"sdmxClassName": "PivotTableRowCol"
},
{
"id": "SliceType",
"names": {"en": "Slice"},
"extendsType": "RowColType",
"sdmxClassName": "PivotTableSlice",
"properties": [
{"id": "position", "minOccurs": 0, "maxOccurs": 1,
"format": {"minValue": 0, "dataType": "Integer"}}
]
}
]
An object of type SliceType may report every property of RowColType plus position, and is bound by the mutually exclusive sets of both types. Extension rules:
- Extension is additive only. A subtype adds properties; it cannot redefine, remove or restrict inherited ones.
- Extension chains must be acyclic, and every
extendsTypemust resolve to a type in the same definition. - The base at the root of the chain determines whether the type is identifiable or nameable.
SliceTypeis nameable becauseRowColTypeis, even thoughSliceTypeitself names no abstract base. - Where the chain is identifiable or nameable, each type in it declares its own unique class name, so that URNs of nested objects remain unambiguous. In the example above, rows are
PivotTableRowColobjects and slices arePivotTableSliceobjects. A chain rooted atAnnotatableproduces no URNs, so no type in it carries a class name.
Implicit content and reserved identifiers¶
Choosing a base does two things at once, and they are two sides of the same coin:
- it supplies content implicitly, some of it mandatory. You do not declare this content, and instances must report it;
- it therefore reserves the identifiers of that content, so you cannot declare a property of the same name.
What each base supplies¶
At the level of the definition, every instance is a maintainable artefact, so it always carries an Agency, an identifier, a version, at least one name, optional descriptions, and annotations. An ItemScheme base additionally requires an item collection.
| Base of the definition | Supplied implicitly by every instance | You must declare |
|---|---|---|
Maintainable |
agencyId, id, version, name, description, annotations, and the CustomStructureDefinition reference |
nothing extra |
ItemScheme |
the same, plus the item collection | a root items property, whose representation is a custom type - see Base type |
At the level of a custom type, the base determines the implicit base attributes of each object:
| Base of the custom type | Supplied implicitly | Mandatory in an instance |
|---|---|---|
Annotatable |
annotations |
nothing - the object is anonymous |
Identifiable |
annotations, id, a generated URN |
id |
Nameable |
annotations, id, a generated URN, name, description |
id and at least one name; description is optional |
An object of an annotatable type must not report an id, name or description, and an object of an identifiable (but not nameable) type must not report a name or description - that content does not exist on those types.
Which identifiers are reserved¶
Reserved identifiers cannot be used as property identifiers in the container which reserves them.
| Container | Reserved identifiers | Reserved because |
|---|---|---|
| The definition (root properties) | agencyId, id, version, name, description, annotations |
Maintainable level attributes which every instance carries |
| The definition (root properties) | CustomStructureDefinition |
The mandatory reference from every instance to its defining CSD |
A Nameable custom type |
annotations, id, name, description |
All are implicit base attributes of the type |
An Identifiable custom type |
annotations, id |
Implicit base attributes of the type |
An Annotatable custom type |
annotations |
The only implicit content an annotatable type has |
The set therefore narrows as the base gets simpler: an annotatable type reserves only annotations, so id, name and description are free for you to use as ordinary properties of your own design, because the type has no implicit ones to clash with.
Avoid name and description on an identifiable type
An Identifiable custom type reserves only annotations and id, so the definition validator will accept a declared name or description property on one. Instances cannot use it: an identifiable object which reports a name or description is rejected, because those attributes belong to Nameable.
If an identifiable type needs a label, either make the type Nameable and use the implicit name, or give the property a different identifier. The Collection Table example takes the second route, giving HeaderType a multilingual label property.
items¶
items is the identifier of the item collection of an item scheme based definition. It is required - a definition declaring base="ItemScheme" with no root items property is rejected - and it should not be used for any other purpose.
Validation of a definition¶
A definition is fully validated when it is read, before it reaches the database. The following are all rejected:
- a missing
sdmxClassNameon the definition itself, or one which is not a valid class name; - a duplicate property identifier within the definition or within any one custom type;
- use of a property identifier reserved by its container, which depends on the base;
- a duplicate custom type identifier, or a duplicate class name across the definition and its custom types;
- an
extendsTypewhich does not resolve, or an extension chain which is cyclic; - an
IdentifiableorNameablecustom type with no class name; - an
Annotatablecustom type, or a type whose extension chain resolves to one, which declares a class name; - a
CustomTypeReferencenaming a custom type which does not exist; - an
IndirectReferencewhosecontextdoes not name a reference or indirect reference property, or whose context chain is cyclic or does not terminate at a reference property; - a
MutuallyExclusiveset with fewer than two members, or naming a property which is not declared in the same container; base="ItemScheme"with no rootitemsproperty;- a property declaring more than one representation, which is reported as a duplicate property identifier.
Maintaining a definition¶
A definition is versioned like any other maintainable artefact, and follows the same versioning rules. Because instances reference the definition by URN, including its version, the version of a definition is significant:
- an instance which pins an absolute version conforms to that version for its lifetime; publishing a new version does not invalidate, migrate or retro-validate it;
- an instance may instead late bind, referencing the definition with semantic version wildcard syntax such as
IMF:PIVOT_TABLE(1.0+.0), in which case it follows the latest matching version; - instances conforming to different versions of the same definition can coexist in the registry, and even in the same message;
- a definition cannot be deleted while instances reference it.
This makes a new minor version the natural way to extend a definition - adding an optional property, adding a custom type. Instances which pin 1.0.0 are untouched, and instances which late bind pick the addition up without being resubmitted. It also means a minor version must stay compatible with content that existing instances already report, since late bound instances will be held against it: add optional properties rather than mandatory ones, and do not tighten cardinality or introduce new mutually exclusive sets over existing properties.
Two things do not follow the version, and are worth keeping in mind:
- The class is not versioned.
custom.imf.PivotTableis one type across every version of the definition, served by one REST API resource. Version separation lives in each instance's reference to the definition, not in the class. sdmxClassNameshould be treated as fixed for the life of a definition. Changing it in a new version does not version the class, it introduces a different one, leaving existing instances under the old class and old resource. If the class must change, publish a new definition instead.
Both are covered in more detail under the class is namespaced, but not versioned.