Part A · The Source Data Product
Exercise 2 · Data Contract Evolution
Release a breaking change as a new major version and migrate.
~30 min0 of 11 steps done
The business wants to know how many units of an item are bought per order.
The orders team adds a column quantity to line_items, defaulting to 1.
Harmless? Not for a pipeline that expects exactly three columns, e.g. a SELECT * into a fixed target table.
So you release the change as a new major version: orders_v2, in its own database schema, next to orders_v1.
You will learn
- Release a change as a new major version of a data contract
- Compare two versions with
datacontract changeloganddatacontract breaking - Understand what counts as a breaking change, and where tools reach their limits
- Announce removals early with the
deprecatedflag (new in ODCS 3.2) - Walk through the contract lifecycle: draft → active → deprecated → retired
Look at the new data
Explore the orders_v2 schema
The orders_v2 schema holds both tables: orders is unchanged, line_items has the new quantity column.
docker compose exec postgres psql -U workshop -d workshop -c '\dt orders_v2.*' -c 'SELECT * FROM orders_v2.line_items LIMIT 5;' List of relations
Schema | Name | Type | Owner
-----------+------------+-------+----------
orders_v2 | line_items | table | workshop
orders_v2 | orders | table | workshop
(2 rows)
lines_item_id | order_id | sku | quantity
--------------------------------------+--------------------------------------+---------------+----------
94aa82c8-50ba-47fb-994a-9b041b4127af | a8c38fec-2acd-4b55-883b-4b48572d4a26 | D3KT74L5EV46T | 1
d67c963f-42a4-4aa8-afff-d7869008e3a9 | 9e44da97-4f72-4bcf-821a-9d9500d06651 | E202K62FT | 3
270ad2c1-f651-438e-a81a-d77713c1d3a3 | 8fc4621c-66ae-4031-91f1-5313beb9f541 | 1O7RID9Y5QJ | 1
cc763a72-cc07-4bc4-8ddf-c88d09db5daa | 98d48daf-3532-4a59-b7c2-3777164bdc65 | 7KJ8466FI39LW | 5
d7ea7f72-a266-469c-a9ef-60063d5ac243 | 2fd9df43-77e8-4d00-b380-ab270e8b73f8 | 7HXBABF0AOT5 | 2
(5 rows)Create v2
Copy the contract
Copy your v1 contract and open the copy in the editor:
cp orders_v1.odcs.yaml orders_v2.odcs.yaml
datacontract edit orders_v2.odcs.yamlBump the version
In Fundamentals, set the new major version:
| Field | Value |
|---|---|
| ID | orders_v2 |
| Version | 2.0.0 |
| Status | draft |
The ID changes with the major version: consumers switch to orders_v2 deliberately, and orders_v1 stays available until they have migrated.
Point the server to orders_v2
In Servers, change the Schema to orders_v2.
Then replace orders_v1. with orders_v2. in all SQL quality checks (e.g. the customer_email_address check). Otherwise they keep testing the old tables.
The copy also kept your AI context block. Its verifiedStatements contain SQL answers that query orders_v1, so switch them to orders_v2 as well. An AI agent would otherwise answer questions from the old version.
Add the quantity column
In Schemas → line_items, add the new property:
| Property | Logical Type | Physical Type |
|---|---|---|
quantity | integer | BIGINT |
Add a quality check that quantity is always greater than 0. Use the "count the bad rows, expect zero" pattern.
Test v2
Save and run the tests:
datacontract test orders_v2.odcs.yamlTesting orders_v2.odcs.yaml
Server: Orders (type=postgres, host=localhost, port=5433, database=workshop, schema=orders_v2)
╭────────┬───────────────────────────────────────────────┬───────────────────────────────┬─────────╮
│ Result │ Check │ Field │ Details │
├────────┼───────────────────────────────────────────────┼───────────────────────────────┼─────────┤
│ passed │ Ensure line_items table has data │ line_items │ │
…
│ passed │ Check that field 'quantity' is present │ line_items.quantity │ │
│ passed │ Check that field quantity has physical type │ line_items.quantity │ │
│ │ bigint │ │ │
│ passed │ Ensure quantity is positive │ line_items.quantity │ │
…
╰────────┴───────────────────────────────────────────────┴───────────────────────────────┴─────────╯
🟢 data contract is valid. Run 37 checks. Took 0.63683 seconds.All checks should pass, including the new quantity check.
Compare the versions
Before a release, you want to know what changed and whether it breaks consumers. The CLI compares two contract files.
Show the changelog
datacontract changelog orders_v1.odcs.yaml orders_v2.odcs.yamlSummary
[ 1 Added ] [ 16 Updated ]
╭─────────┬─────────────────────────────────────────────────────────────────╮
│ Change │ Field │
├─────────┼─────────────────────────────────────────────────────────────────┤
│ Updated │ context.verifiedStatements.How many orders were placed in 2023? │
│ Updated │ context.verifiedStatements.What was the revenue per year? │
│ Updated │ id │
│ Updated │ schema.line_items.properties.lines_item_id.quality.[1] │
│ Added │ schema.line_items.properties.quantity │
…
│ Updated │ servers.Orders │
│ Updated │ version │
╰─────────┴─────────────────────────────────────────────────────────────────╯
Details
…You get a summary and a detailed list of all changes, including the switched quality queries and verified statements. A good base for release notes.
Check for breaking changes
breaking classifies each change by severity:
datacontract breaking orders_v1.odcs.yaml orders_v2.odcs.yaml
echo "exit code: $?"Summary
[ 11 Warning ] [ 6 Info ]
╭──────────┬─────────┬─────────────────────────────────────────────────────────────────╮
│ Severity │ Change │ Field │
├──────────┼─────────┼─────────────────────────────────────────────────────────────────┤
│ INFO │ Updated │ context.verifiedStatements.How many orders were placed in 2023? │
│ INFO │ Updated │ context.verifiedStatements.What was the revenue per year? │
│ INFO │ Updated │ id │
│ WARNING │ Updated │ schema.line_items.properties.lines_item_id.quality.[1] │
│ INFO │ Added │ schema.line_items.properties.quantity │
…
│ INFO │ Updated │ servers.Orders │
│ INFO │ Updated │ version │
╰──────────┴─────────┴─────────────────────────────────────────────────────────────────╯
…
exit code: 0Adding quantity is INFO: adding a column is schema-compatible. The changed quality queries are WARNINGs, because the checks now run against other tables. Nothing is an ERROR, so the exit code is 0.
Now provoke a real breaking change. Keep a copy, change the physicalType of quantity to text (or delete customer_id), and compare:
cp orders_v2.odcs.yaml orders_v2.before.odcs.yaml
# now edit orders_v2.odcs.yaml: change quantity's physicalType to text
datacontract breaking orders_v2.before.odcs.yaml orders_v2.odcs.yaml
echo "exit code: $?"Summary
[ 1 Error ]
╭──────────┬─────────┬───────────────────────────────────────╮
│ Severity │ Change │ Field │
├──────────┼─────────┼───────────────────────────────────────┤
│ ERROR │ Updated │ schema.line_items.properties.quantity │
╰──────────┴─────────┴───────────────────────────────────────╯
Details
╭──────────┬─────────┬────────────────────────────────────────────────────┬───────────┬───────────┬────────────────────────────────╮
│ Severity │ Change │ Path │ Old Value │ New Value │ Message │
├──────────┼─────────┼────────────────────────────────────────────────────┼───────────┼───────────┼────────────────────────────────┤
│ ERROR │ Updated │ schema.line_items.properties.quantity.physicalType │ bigint │ text │ Changed type at │
│ │ │ │ │ │ schema.line_items.properties.… │
│ │ │ │ │ │ from 'bigint' to 'text' │
╰──────────┴─────────┴────────────────────────────────────────────────────┴───────────┴───────────┴────────────────────────────────╯
exit code: 1This time you get an ERROR and exit code 1. That exit code turns the check into a CI/CD gate, see CI/CD with GitHub Actions.
Revert and run the tests again:
mv orders_v2.before.odcs.yaml orders_v2.odcs.yaml
datacontract test orders_v2.odcs.yamlTesting orders_v2.odcs.yaml
…
🟢 data contract is valid. Run 37 checks. Took 0.63683 seconds.Execute the migration
Deprecate a field
The orders team plans to drop customer_id in a future v3. Announce it now. First keep a copy of the current version:
cp orders_v2.odcs.yaml orders_v2.before.odcs.yamlIn orders_v2.odcs.yaml, add deprecated: true to the customer_id property:
YAML
- name: customer_id
deprecated: true
physicalType: text
logicalType: stringCompare both versions:
datacontract breaking orders_v2.before.odcs.yaml orders_v2.odcs.yamlSummary
[ 1 Info ]
╭──────────┬────────┬──────────────────────────────────────╮
│ Severity │ Change │ Field │
├──────────┼────────┼──────────────────────────────────────┤
│ INFO │ Added │ schema.orders.properties.customer_id │
╰──────────┴────────┴──────────────────────────────────────╯
…
│ INFO │ Added │ schema.orders.properties.customer_id.deprecated │ │ True │ Added contract at │
…Only INFO: deprecating is non-breaking. Delete orders_v2.before.odcs.yaml afterwards.
Deprecate v1 (practice)
In orders_v1.odcs.yaml, set status: deprecated and add an endOfSupport SLA property:
YAML
slaProperties:
# ... retention, frequency
- property: endOfSupport
value: "2026-12-31"
description: orders_v1 is replaced by orders_v2. Please migrate until end of 2026.Check that the contract is still valid:
datacontract lint orders_v1.odcs.yaml╭────────┬────────────────────────────────────────────┬───────┬─────────╮
│ Result │ Check │ Field │ Details │
├────────┼────────────────────────────────────────────┼───────┼─────────┤
│ passed │ Data contract is valid against ODCS v3.2.0 │ │ │
╰────────┴────────────────────────────────────────────┴───────┴─────────╯
🟢 data contract is valid. Run 1 checks. Took 0.122955 seconds.Release v2 and retire v1
As a shortcut, jump to the end state:
- set
orders_v2.odcs.yamltostatus: active - set
orders_v1.odcs.yamltostatus: retired
Run the v2 tests one last time:
datacontract test orders_v2.odcs.yamlTesting orders_v2.odcs.yaml
…
🟢 data contract is valid. Run 37 checks. Took 0.63683 seconds.Quick check
Which change to the active orders_v2 contract is a breaking change for consumers?
Bonus
- Export both versions as HTML and compare:
datacontract export html orders_v2.odcs.yaml --output orders_v2.odcs.html - Predict the severity before running
datacontract breaking: rename a field, removerequired: true, add a new table. - Remove the deprecated
customer_idfrom a copy of v2 and rundatacontract breaking. The flag doesn't make the removal compatible.