# Semantics

Define business concepts once and link your contracts to them.

Source: https://learn.datacontract.com/en/semantics/

Each of your contracts has its own copy of the descriptions of `order_id`, `order_total`, and `sku`.
And a description says what a field *contains*, not which business concept it *is*.
Here you define each concept **once**, in one ontology file with stable **IRIs**, upload it in one go, and link the contracts from [the previous exercise](https://learn.datacontract.com/en/publish/) to it.

**You will learn:**
- Write a business ontology as one file: entities, properties, relationships
- Identify concepts with IRIs that work on any platform instance
- Link data contract fields to concepts with `authoritativeDefinitions`
- Answer "which data products contain X?" with a reverse lookup

> **Prerequisite**
>
> Builds on [Publish to Entropy Data](https://learn.datacontract.com/en/publish/): your contracts are published and `entropy-data connection test` works. The Community Edition from the repository has Semantics enabled.

> **Contract vs. ontology**
>
> A data contract describes **one dataset**. An ontology describes the **business language** shared by all datasets: *an Order has an Order Total, an Order contains Articles.*
> When fields point to concepts, the meaning lives in one place, and you can search all datasets by meaning instead of column name.

> **Why IRIs?**
>
> An IRI (Internationalized Resource Identifier) is a globally unique, stable name for a concept, e.g. `https://learn.datacontract.com/ontology/ecommerce#sku`.
> It does not depend on the platform's host or your organization name, so the same contract links correctly on the cloud, on a local Community Edition, and in any tool that understands the ontology.
> The IRI does not need to be a reachable web page. The platform resolves it to the concept that carries it.

## Write the ontology

### Write the ontology file

Create `semantics.yaml` in the repository root. It holds the complete namespace `ecommerce`:

- `prefixes` declares `ecom:` as a short form of the IRI base, so every concept can write `iri: ecom:Order`.
- `EntityType` concepts are the business objects (Order, Article). `ValueType` concepts are their properties (Order ID, Order Total, SKU).
- `hasProperty` relationships attach properties to entities. `relatedTo` relationships connect entities: an order contains articles.

```yaml title=semantics.yaml
version: 0.2.0.dev0
name: ecommerce
description: Business concepts of the e-commerce platform.
custom_properties:
  display_name: E-Commerce
prefixes:
  ecom: https://learn.datacontract.com/ontology/ecommerce#
ontology:
  - concept: Order
    id: order
    type: EntityType
    description: A customer order in the e-commerce platform.
    iri: ecom:Order
    relationships:
      - id: order_has_order_id
        name: order_id
        type: hasProperty
        roles:
          - concept: Order ID
      - id: order_has_order_total
        name: order_total
        type: hasProperty
        roles:
          - concept: Order Total
      - id: order_contains_article
        name: contains
        type: relatedTo
        description: An order contains one or more articles.
        roles:
          - concept: Article
        verbalizes:
          - "{Order} contains {Article}"
  - concept: Article
    id: article
    type: EntityType
    description: A product that can be bought in the e-commerce platform.
    iri: ecom:Article
    relationships:
      - id: article_has_sku
        name: sku
        type: hasProperty
        roles:
          - concept: SKU
  - concept: Order ID
    id: order_id
    type: ValueType
    description: Unique identifier of an order (UUID).
    iri: ecom:orderId
  - concept: Order Total
    id: order_total
    type: ValueType
    description: Total amount of an order in cents, never negative.
    iri: ecom:orderTotal
  - concept: SKU
    id: sku
    type: ValueType
    description: Stock keeping unit, the unique identifier of an article.
    iri: ecom:sku
```

### Upload it in one go

One `PUT` replaces the whole namespace with the file: it creates the namespace, all concepts, and all relationships in the right order.
The Entropy Data CLI has no command for this yet, so call the API directly. It uses the same API key and host as the CLI:

macOS / Linux:

```bash
curl -sS -f -X PUT "$ENTROPY_DATA_HOST/api/semantics/experimental/namespaces/ecommerce/ontology.yaml" \
  -H "x-api-key: $ENTROPY_DATA_API_KEY" \
  -H "Content-Type: application/yaml" \
  --data-binary @semantics.yaml -w "HTTP %{http_code}\n"
```

Windows (PowerShell):

```powershell
Invoke-RestMethod -Method Put `
  -Uri "$env:ENTROPY_DATA_HOST/api/semantics/experimental/namespaces/ecommerce/ontology.yaml" `
  -Headers @{ "x-api-key" = $env:ENTROPY_DATA_API_KEY } `
  -ContentType "application/yaml" `
  -InFile semantics.yaml
```

Output:

```text
HTTP 200
```

On Windows, a successful call prints nothing.

> **Community Edition**
>
> The setup script wrote `ENTROPY_DATA_API_KEY` and `ENTROPY_DATA_HOST` into `.env`, but `curl` and `Invoke-RestMethod` don't read that file. Load the two values into your shell first:
>
> macOS / Linux:
>
> ```bash
> set -a; source .env; set +a
> ```
>
> Windows (PowerShell):
>
> ```powershell
> Get-Content .env | Where-Object { $_ -match '^ENTROPY_DATA_(HOST|API_KEY)=' } | ForEach-Object { $k, $v = $_ -split '=', 2; Set-Item "env:$k" $v }
> ```

Check what arrived:

macOS / Linux:

```bash
entropy-data semantics concepts list ecommerce
```

Windows (PowerShell):

```powershell
entropy-data semantics concepts list ecommerce
```

Output:

```text
               semantic-concepts (page 0)
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┓
┃ ID          ┃ Name        ┃ Kind     ┃ Group ┃ Status ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━┩
│ article     │ Article     │ entity   │       │        │
│ order       │ Order       │ entity   │       │        │
│ order_id    │ Order ID    │ property │       │        │
│ order_total │ Order Total │ property │       │        │
│ sku         │ SKU         │ property │       │        │
└─────────────┴─────────────┴──────────┴───────┴────────┘
```

Want to change something later? Edit `semantics.yaml` and upload it again. Concepts you removed from the file are removed from the namespace, too.

## Link your data contracts

### Link the contract fields by IRI

Link schemas and fields to the concepts with `authoritativeDefinitions` of type `semantics`, using the full IRI as `url`:

```yaml
schema:
  - name: orders
    authoritativeDefinitions:
      - type: semantics
        url: https://learn.datacontract.com/ontology/ecommerce#Order
    properties:
      - name: order_id
        authoritativeDefinitions:
          - type: semantics
            url: https://learn.datacontract.com/ontology/ecommerce#orderId
```

Do the same for every field that has a concept:

| Field | IRI |
|---|---|
| `order_total` | `https://learn.datacontract.com/ontology/ecommerce#orderTotal` |
| `sku` | `https://learn.datacontract.com/ontology/ecommerce#sku` |

Link them in `orders_v1`, `orders_v2`, and `sku_sales_per_year`. Then **remove the copied descriptions** from these fields. The definition now lives in the concept.

Check that the contracts are still valid:

macOS / Linux:

```bash
datacontract lint orders_v1.odcs.yaml
datacontract lint orders_v2.odcs.yaml
datacontract lint sku_sales_per_year.odcs.yaml
```

Windows (PowerShell):

```powershell
datacontract lint orders_v1.odcs.yaml
datacontract lint orders_v2.odcs.yaml
datacontract lint sku_sales_per_year.odcs.yaml
```

Output:

```text
╭────────┬────────────────────────────────────────────┬───────┬─────────╮
│ Result │ Check                                      │ Field │ Details │
├────────┼────────────────────────────────────────────┼───────┼─────────┤
│ passed │ Data contract is valid against ODCS v3.2.0 │       │         │
╰────────┴────────────────────────────────────────────┴───────┴─────────╯
🟢 data contract is valid. Run 1 checks. Took 0.17 seconds.
…
🟢 data contract is valid. Run 1 checks. Took 0.17 seconds.
…
🟢 data contract is valid. Run 1 checks. Took 0.14 seconds.
```

> **The CLI resolves IRIs, too**
>
> `datacontract test` looks up each IRI through `ENTROPY_DATA_HOST` (`/api/semantics?iri=...`) and uses the concept's description where the field has none. This needs `ENTROPY_DATA_API_KEY`.

### Re-publish the contracts

macOS / Linux:

```bash
entropy-data datacontracts put orders_v1 --file orders_v1.odcs.yaml
entropy-data datacontracts put orders_v2 --file orders_v2.odcs.yaml
entropy-data datacontracts put sku_sales_per_year --file sku_sales_per_year.odcs.yaml
```

Windows (PowerShell):

```powershell
entropy-data datacontracts put orders_v1 --file orders_v1.odcs.yaml
entropy-data datacontracts put orders_v2 --file orders_v2.odcs.yaml
entropy-data datacontracts put sku_sales_per_year --file sku_sales_per_year.odcs.yaml
```

Output:

```text
Data contract 'orders_v1' saved.
Open https://app.entropy-data.com/tutorial-simon/datacontracts/orders_v1
Data contract 'orders_v2' saved.
Open https://app.entropy-data.com/tutorial-simon/datacontracts/orders_v2
Data contract 'sku_sales_per_year' saved.
Open https://app.entropy-data.com/tutorial-simon/datacontracts/sku_sales_per_year
```

> **Your CI pipeline now needs the platform**
>
> `datacontract test` and `datacontract ci` resolve the IRIs through the platform. Without `ENTROPY_DATA_API_KEY`, or if the host is unreachable, the run fails with "Could not resolve business definition".
> In your workflow from [CI/CD with GitHub Actions](https://learn.datacontract.com/en/ci-cd/), you have two options:
>
> - **Cloud:** pass the `ENTROPY_DATA_API_KEY` secret and `ENTROPY_DATA_HOST` to the step (see the bonus of the previous exercise).
> - **Community Edition** (unreachable from GitHub): add `--no-inline-references` to the `datacontract ci` command.

**Solution: Contracts with semantic links**

```yaml title=orders_v1.with-semantics.odcs.yaml
# Copy of ../exercise1/orders_v1.odcs.yaml plus semantics links (authoritativeDefinitions)
# and without the text descriptions that the semantic concepts now carry - keep in sync!
version: 1.0.0
kind: DataContract
apiVersion: v3.2.0
id: orders_v1
name: Orders
status: active
domain: ecommerce
description:
  purpose: Order data for analytics and reporting
  limitations: Contains PII (customer email addresses)
tags: ['orders', 'ecommerce']
context:
  instructions: >-
    Orders of the e-commerce platform and their line items.
    order_total is in cents. Timestamps are in UTC.
    Use this data for order volume and revenue analysis.
  verifiedStatements:
  - question: How many orders were placed in 2023?
    answer: SELECT COUNT(*) FROM orders_v1.orders WHERE EXTRACT(YEAR FROM order_timestamp) = 2023;
  - question: What was the revenue per year?
    answer: SELECT EXTRACT(YEAR FROM order_timestamp)::int AS year, SUM(order_total) / 100.0 AS revenue FROM orders_v1.orders GROUP BY 1 ORDER BY 1;
  constraints:
  - constraint: Never output customer_email_address or customer_id. Aggregate the data instead.
    tags: ['pii']
servers:
- server: postgres
  type: postgres
  host: localhost
  port: 5433
  database: workshop
  schema: orders_v1
schema:
- name: orders
  synonyms:
  - synonym: purchases
  - synonym: Bestellungen
    locale: de
  context:
    instructions: One row per order. Join line_items on order_id to get the purchased SKUs.
  physicalType: table
  logicalType: object
  physicalName: orders
  authoritativeDefinitions:
  - type: semantics
    url: https://learn.datacontract.com/ontology/ecommerce#Order
  properties:
  - name: order_id
    physicalType: text
    logicalType: string
    required: true
    primaryKey: true
    authoritativeDefinitions:
    - type: semantics
      url: https://learn.datacontract.com/ontology/ecommerce#orderId
    quality:
    - type: sql
      description: Ensure order_id is unique
      query: SELECT COUNT(*) FROM (SELECT order_id FROM orders_v1.orders GROUP BY order_id HAVING COUNT(*) > 1) duplicates;
      mustBe: 0
  - name: order_timestamp
    physicalType: timestamptz
    logicalType: date
    quality:
    - type: text
      description: Must not be in the future
    - type: sql
      description: Ensure order_timestamp is not in the future
      query: SELECT COUNT(*) FROM orders_v1.orders WHERE order_timestamp > CURRENT_TIMESTAMP;
      mustBe: 0
  - name: order_total
    physicalType: bigint
    logicalType: integer
    authoritativeDefinitions:
    - type: semantics
      url: https://learn.datacontract.com/ontology/ecommerce#orderTotal
    quality:
    - type: sql
      description: Ensure order_total is non-negative
      query: SELECT COUNT(*) FROM orders_v1.orders WHERE order_total < 0;
      mustBe: 0
  - name: customer_id
    physicalType: text
    logicalType: string
    quality:
    - type: text
      description: Must be a non-empty alphanumeric identifier
    - type: sql
      description: Ensure customer_id is not empty
      query: SELECT COUNT(*) FROM orders_v1.orders WHERE customer_id IS NULL OR customer_id = '';
      mustBe: 0
  - name: customer_email_address
    physicalType: text
    logicalType: string
    required: true
    classification: confidential
    examples: ['test394@example.org', 'test4757@example.org']
    quality:
    - type: text
      description: Must be a valid email address
    - type: sql
      description: Ensure email addresses contain @
      query: SELECT COUNT(*) FROM orders_v1.orders WHERE customer_email_address NOT LIKE '%@%';
      mustBe: 0
  quality:
  - type: text
    description: Orders table must not be empty
  - type: sql
    description: Ensure orders table has data
    query: SELECT COUNT(*) FROM orders_v1.orders;
    mustBeGreaterThan: 0
  - type: sql
    description: Ensure every order has at least one line item
    query: SELECT COUNT(*) FROM orders_v1.orders o LEFT JOIN orders_v1.line_items li ON o.order_id = li.order_id WHERE li.order_id IS NULL;
    mustBe: 0
- name: line_items
  synonyms:
  - synonym: order lines
  - synonym: Bestellpositionen
    locale: de
  context:
    instructions: One row per purchased SKU in an order.
  physicalType: table
  logicalType: object
  physicalName: line_items
  properties:
  - name: lines_item_id
    physicalType: text
    logicalType: string
    required: true
    primaryKey: true
    quality:
    - type: text
      description: Must be a valid UUID
    - type: sql
      description: Ensure lines_item_id is unique
      query: SELECT COUNT(*) FROM (SELECT lines_item_id FROM orders_v1.line_items GROUP BY lines_item_id HAVING COUNT(*) > 1) duplicates;
      mustBe: 0
  - name: order_id
    physicalType: text
    logicalType: string
    required: true
    authoritativeDefinitions:
    - type: semantics
      url: https://learn.datacontract.com/ontology/ecommerce#orderId
    relationships:
    - type: foreignKey
      to: orders.order_id
  - name: sku
    synonyms:
    - synonym: article number
    - synonym: Artikelnummer
      locale: de
    physicalType: text
    logicalType: string
    authoritativeDefinitions:
    - type: semantics
      url: https://learn.datacontract.com/ontology/ecommerce#sku
    quality:
    - type: sql
      description: Ensure sku is not empty
      query: SELECT COUNT(*) FROM orders_v1.line_items WHERE sku IS NULL OR sku = '';
      mustBe: 0
  quality:
  - type: text
    description: Line items table must not be empty
  - type: sql
    description: Ensure line_items table has data
    query: SELECT COUNT(*) FROM orders_v1.line_items;
    mustBeGreaterThan: 0
  - type: sql
    description: Ensure every line_items.order_id exists in orders
    query: SELECT COUNT(*) FROM orders_v1.line_items li LEFT JOIN orders_v1.orders o ON li.order_id = o.order_id WHERE o.order_id IS NULL;
    mustBe: 0
team:
  name: order_data_team
  members:
  - username: owner@example.com
    role: Owner
support:
- channel: "#order-data-help"
  url: https://example.slack.com/archives/order-data-help
  tool: slack
slaProperties:
- property: retention
  value: 10
  unit: y
  element: orders.order_timestamp
  description: Orders are deleted after 10 years
- property: frequency
  value: 1
  unit: d
  description: Data updated daily
```

```yaml title=sku_sales_per_year.with-semantics.odcs.yaml
# Copy of ../exercise4/sku_sales_per_year.odcs.yaml plus semantics links (authoritativeDefinitions)
# and with the active status from the end of exercise 5 - keep in sync!
version: 1.0.0
kind: DataContract
apiVersion: v3.2.0
id: sku_sales_per_year
name: SKU Sales per Year
status: active
domain: ecommerce
description:
  purpose: Shows how often each SKU is bought, grouped by year. Built for the purchasing team to support supplier negotiations.
  limitations: Aggregated data only, no PII. Based on the orders data product.
tags: ['sku', 'sales', 'analytics']
context:
  instructions: >-
    One row per SKU and year. Sum order_count or total_quantity across years for totals.
  verifiedStatements:
  - question: Which three SKUs sold the most units in 2024?
    answer: SELECT sku, total_quantity FROM analytics.sku_sales_per_year WHERE year = 2024 ORDER BY total_quantity DESC LIMIT 3;
servers:
- server: postgres
  type: postgres
  host: localhost
  port: 5433
  database: workshop
  schema: analytics
schema:
- name: sku_sales_per_year
  physicalType: view
  logicalType: object
  physicalName: sku_sales_per_year
  properties:
  - name: sku
    semanticType: dimension
    physicalType: text
    logicalType: string
    required: true
    authoritativeDefinitions:
    - type: semantics
      url: https://learn.datacontract.com/ontology/ecommerce#sku
    quality:
    - type: sql
      description: Ensure sku and year combination is unique
      query: SELECT COUNT(*) FROM (SELECT sku, year FROM analytics.sku_sales_per_year GROUP BY sku, year HAVING COUNT(*) > 1) duplicates;
      mustBe: 0
  - name: year
    semanticType: dimension
    physicalType: integer
    logicalType: integer
    required: true
    quality:
    - type: sql
      description: Ensure year is plausible
      query: SELECT COUNT(*) FROM analytics.sku_sales_per_year WHERE year < 2020 OR year > 2100;
      mustBe: 0
  - name: order_count
    semanticType: measure
    transformLogic: COUNT(*)
    physicalType: bigint
    logicalType: integer
    quality:
    - type: text
      description: Number of orders containing the SKU in that year
    - type: sql
      description: Ensure order_count is positive
      query: SELECT COUNT(*) FROM analytics.sku_sales_per_year WHERE order_count <= 0;
      mustBe: 0
  - name: total_quantity
    semanticType: measure
    transformLogic: SUM(line_items.quantity)
    physicalType: bigint
    logicalType: integer
    quality:
    - type: text
      description: Total units bought, never less than the number of orders
    - type: sql
      description: Ensure total_quantity is at least order_count
      query: SELECT COUNT(*) FROM analytics.sku_sales_per_year WHERE total_quantity < order_count;
      mustBe: 0
  quality:
  - type: sql
    description: Ensure the view has data
    query: SELECT COUNT(*) FROM analytics.sku_sales_per_year;
    mustBeGreaterThan: 0
team:
  name: purchasing_analytics_team
  members:
  - username: purchasing@example.com
    role: Owner
support:
- channel: "#purchasing-analytics"
  url: https://example.slack.com/archives/purchasing-analytics
  tool: slack
```

## Explore the ontology

### Explore the ontology

In the web UI, go to **Semantics → E-Commerce** and open **Article → SKU**:

- The diagram shows the ontology: Order with its properties, contains, Article with SKU.
- **Metadata** shows the concept's IRI, both as `ecom:sku` and in full.
- **Data Products** is the reverse lookup: every data product whose contracts link to the concept. *"Which data products contain SKUs?"* is now one click.

![The SKU concept: ontology diagram, IRI, and the data products that use it](https://learn.datacontract.com/screenshots/semantics-concept-sku.webp)

The data product list shows the linked concepts, too:

![Data products with their linked concepts in the Semantics column](https://learn.datacontract.com/screenshots/semantics-data-products.webp)

**Quick check:** Why link contract fields by IRI instead of by the concept's URL on the platform?

- IRIs are shorter to type
- The IRI is a stable global identifier, so the link works on every instance and in other tools (correct)
- The platform cannot show concepts linked by URL
- IRIs make datacontract test check the semantics

Exactly. A URL contains host and organization and breaks when either changes. The IRI names the concept itself, and the platform resolves it wherever the ontology lives.