Progress

Part D · Data Platformoptional

Exercise 9 · Semantics

Define business concepts once and link your contracts to them.

~25 min0 of 5 steps done

Each of your contracts has its own copy of the descriptions of order_id, order_total, and sku. And a description says what a field contains, not which business concept it is. Here you define each concept once, in one ontology file with stable IRIs, upload it in one go, and link the contracts from the previous exercise to it.

You will learn

Write a business ontology as one file: entities, properties, relationships
Identify concepts with IRIs that work on any platform instance
Link data contract fields to concepts with authoritativeDefinitions
Answer "which data products contain X?" with a reverse lookup

Write the ontology

Write the ontology file

Create semantics.yaml in the repository root. It holds the complete namespace ecommerce:

prefixes declares ecom: as a short form of the IRI base, so every concept can write iri: ecom:Order.
EntityType concepts are the business objects (Order, Article). ValueType concepts are their properties (Order ID, Order Total, SKU).
hasProperty relationships attach properties to entities. relatedTo relationships connect entities: an order contains articles.

semantics.yaml

version: 0.2.0.dev0
name: ecommerce
description: Business concepts of the e-commerce platform.
custom_properties:
  display_name: E-Commerce
prefixes:
  ecom: https://learn.datacontract.com/ontology/ecommerce#
ontology:
  - concept: Order
    id: order
    type: EntityType
    description: A customer order in the e-commerce platform.
    iri: ecom:Order
    relationships:
      - id: order_has_order_id
        name: order_id
        type: hasProperty
        roles:
          - concept: Order ID
      - id: order_has_order_total
        name: order_total
        type: hasProperty
        roles:
          - concept: Order Total
      - id: order_contains_article
        name: contains
        type: relatedTo
        description: An order contains one or more articles.
        roles:
          - concept: Article
        verbalizes:
          - "{Order} contains {Article}"
  - concept: Article
    id: article
    type: EntityType
    description: A product that can be bought in the e-commerce platform.
    iri: ecom:Article
    relationships:
      - id: article_has_sku
        name: sku
        type: hasProperty
        roles:
          - concept: SKU
  - concept: Order ID
    id: order_id
    type: ValueType
    description: Unique identifier of an order (UUID).
    iri: ecom:orderId
  - concept: Order Total
    id: order_total
    type: ValueType
    description: Total amount of an order in cents, never negative.
    iri: ecom:orderTotal
  - concept: SKU
    id: sku
    type: ValueType
    description: Stock keeping unit, the unique identifier of an article.
    iri: ecom:sku

Upload it in one go

One PUT replaces the whole namespace with the file: it creates the namespace, all concepts, and all relationships in the right order. The Entropy Data CLI has no command for this yet, so call the API directly. It uses the same API key and host as the CLI:

curl -sS -f -X PUT "$ENTROPY_DATA_HOST/api/semantics/experimental/namespaces/ecommerce/ontology.yaml" \
  -H "x-api-key: $ENTROPY_DATA_API_KEY" \
  -H "Content-Type: application/yaml" \
  --data-binary @semantics.yaml -w "HTTP %{http_code}\n"

HTTP 200

On Windows, a successful call prints nothing.

Community Edition

The setup script wrote ENTROPY_DATA_API_KEY and ENTROPY_DATA_HOST into .env, but curl and Invoke-RestMethod don't read that file. Load the two values into your shell first:

set -a; source .env; set +a

Check what arrived:

entropy-data semantics concepts list ecommerce

               semantic-concepts (page 0)
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┓
┃ ID          ┃ Name        ┃ Kind     ┃ Group ┃ Status ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━┩
│ article     │ Article     │ entity   │       │        │
│ order       │ Order       │ entity   │       │        │
│ order_id    │ Order ID    │ property │       │        │
│ order_total │ Order Total │ property │       │        │
│ sku         │ SKU         │ property │       │        │
└─────────────┴─────────────┴──────────┴───────┴────────┘

Want to change something later? Edit semantics.yaml and upload it again. Concepts you removed from the file are removed from the namespace, too.

Link your data contracts

Link the contract fields by IRI

Link schemas and fields to the concepts with authoritativeDefinitions of type semantics, using the full IRI as url:

YAML

schema:
  - name: orders
    authoritativeDefinitions:
      - type: semantics
        url: https://learn.datacontract.com/ontology/ecommerce#Order
    properties:
      - name: order_id
        authoritativeDefinitions:
          - type: semantics
            url: https://learn.datacontract.com/ontology/ecommerce#orderId

Do the same for every field that has a concept:

Field	IRI
`order_total`	`https://learn.datacontract.com/ontology/ecommerce#orderTotal`
`sku`	`https://learn.datacontract.com/ontology/ecommerce#sku`

Link them in orders_v1, orders_v2, and sku_sales_per_year. Then remove the copied descriptions from these fields. The definition now lives in the concept.

Check that the contracts are still valid:

datacontract lint orders_v1.odcs.yaml
datacontract lint orders_v2.odcs.yaml
datacontract lint sku_sales_per_year.odcs.yaml

╭────────┬────────────────────────────────────────────┬───────┬─────────╮
│ Result │ Check                                      │ Field │ Details │
├────────┼────────────────────────────────────────────┼───────┼─────────┤
│ passed │ Data contract is valid against ODCS v3.2.0 │       │         │
╰────────┴────────────────────────────────────────────┴───────┴─────────╯
🟢 data contract is valid. Run 1 checks. Took 0.17 seconds.
…
🟢 data contract is valid. Run 1 checks. Took 0.17 seconds.
…
🟢 data contract is valid. Run 1 checks. Took 0.14 seconds.

Re-publish the contracts

entropy-data datacontracts put orders_v1 --file orders_v1.odcs.yaml
entropy-data datacontracts put orders_v2 --file orders_v2.odcs.yaml
entropy-data datacontracts put sku_sales_per_year --file sku_sales_per_year.odcs.yaml

Data contract 'orders_v1' saved.
Open https://app.entropy-data.com/tutorial-simon/datacontracts/orders_v1
Data contract 'orders_v2' saved.
Open https://app.entropy-data.com/tutorial-simon/datacontracts/orders_v2
Data contract 'sku_sales_per_year' saved.
Open https://app.entropy-data.com/tutorial-simon/datacontracts/sku_sales_per_year

Explore the ontology

In the web UI, go to Semantics → E-Commerce and open Article → SKU:

The diagram shows the ontology: Order with its properties, contains, Article with SKU.
Metadata shows the concept's IRI, both as ecom:sku and in full.
Data Products is the reverse lookup: every data product whose contracts link to the concept. "Which data products contain SKUs?" is now one click.

The SKU concept: ontology diagram, IRI, and the data products that use it

The data product list shows the linked concepts, too:

Data products with their linked concepts in the Semantics column

Quick check

Why link contract fields by IRI instead of by the concept's URL on the platform?