Progress

Part B · The Consumer-Aligned Data Product

Exercise 6 · Consumer-Driven Contracts

Make your dependencies explicit with a consumer-driven contract.

~30 min0 of 9 steps done

Your view from the previous exercise reads the producer's tables directly. It implicitly depends on the whole orders_v2 contract, but needs only five fields. The orders team can't tell whether a change breaks you.

Make the dependency explicit: a consumer-driven contract for exactly the fields you need, and views that expose only those fields.

You will learn

Understand consumer-driven data contracts
Write a contract with only what a consumer uses
Decouple your data product from the producer's tables with access views
Verify that your own consumers are unaffected

In this exercise: a consumer-driven contract with just the fields SKU Sales needs from orders_v2.

Define what you need

Copy the producer's contract

Copy the orders_v2 contract and open the copy in the editor:

cp orders_v2.odcs.yaml orders_v2.consumer_sku_sales.odcs.yaml
datacontract edit orders_v2.consumer_sku_sales.odcs.yaml

Editing: /Users/you/learn.datacontract.com/orders_v2.consumer_sku_sales.odcs.yaml
Data Contract Editor running at http://localhost:4243
Press Ctrl+C to stop

Set the ID to orders_v2_consumer_sku_sales. Keep the Version at 2.0.0, as it is based on the v2 data.

Strip it down to what you use

Remove all properties your view doesn't use, and the quality checks on them. Keep only:

orders: order_id, order_timestamp
line_items: order_id, sku, quantity

Keep the quantity > 0 check. Your data product relies on it: otherwise total_quantity could be smaller than order_count.

The copy also carries the producer's ODCS 3.2 context and synonyms. Remove the contract-level context: its verified statements query order_total and the producer's tables, and its constraint is about a field you dropped. Schema-level context and synonyms on kept fields may stay.

Make it your contract

This is your contract now, not the orders team's:

set team and support to the purchasing analytics team (as in your sku_sales_per_year contract),
rewrite description.purpose, e.g. "The fields the SKU Sales data product actually needs from orders_v2".

Point it to the access views

Your access views will live in a new schema sku_sales_input:

change the server's Schema to sku_sales_input,
set the physicalType of both schema objects to VIEW,
if the quantity > 0 check is a SQL query, change orders_v2. to sku_sales_input. in it. Otherwise it still tests the old tables.

Run the tests: red again

datacontract test orders_v2.consumer_sku_sales.odcs.yaml

Testing orders_v2.consumer_sku_sales.odcs.yaml
Server: postgres (type=postgres, host=localhost, port=5433, database=workshop, 
schema=sku_sales_input)
╭────────┬───────────────────────────────┬────────────────────────┬────────────────────────────────╮
│ Result │ Check                         │ Field                  │ Details                        │
├────────┼───────────────────────────────┼────────────────────────┼────────────────────────────────┤
│ failed │ Check that field 'order_id'   │ line_items.order_id    │ Could not read model           │
│        │ is present                    │                        │ 'line_items': line_items       │
…
🔴 data contract is invalid, found the following errors:
1) 7 checks on orders.order_id, orders.order_timestamp: Could not read model 'orders': orders
2) 9 checks on line_items.order_id, line_items.sku, line_items.quantity: Could not read model 
'line_items': line_items

They fail, because the views don't exist yet. Contract-first, once more.

Create the access views

Create the views

Create sql/sku_sales_input.sql with views that expose only the contracted fields:

sku_sales_input.sql

CREATE SCHEMA IF NOT EXISTS sku_sales_input;

CREATE OR REPLACE VIEW sku_sales_input.orders AS
SELECT order_id, order_timestamp FROM orders_v2.orders;

CREATE OR REPLACE VIEW sku_sales_input.line_items AS
SELECT order_id, sku, quantity FROM orders_v2.line_items;

Apply it:

docker compose exec -T postgres psql -U workshop -d workshop < sql/sku_sales_input.sql

CREATE SCHEMA
CREATE VIEW
CREATE VIEW

Run the tests: green

datacontract test orders_v2.consumer_sku_sales.odcs.yaml

Testing orders_v2.consumer_sku_sales.odcs.yaml
Server: postgres (type=postgres, host=localhost, port=5433, database=workshop, 
schema=sku_sales_input)
╭────────┬──────────────────────────────────────────────────────┬────────────────────────┬─────────╮
│ Result │ Check                                                │ Field                  │ Details │
├────────┼──────────────────────────────────────────────────────┼────────────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present               │ line_items.order_id    │         │
│ passed │ Check that field order_id has physical type text     │ line_items.order_id    │         │
│ passed │ Check that field order_id has no missing values      │ line_items.order_id    │         │
│ passed │ Check that field 'quantity' is present               │ line_items.quantity    │         │
│ passed │ Check that field quantity has physical type bigint   │ line_items.quantity    │         │
│ passed │ Ensure quantity is positive                          │ line_items.quantity    │         │
…
│ passed │ Check that field 'order_timestamp' is present        │ orders.order_timestamp │         │
╰────────┴──────────────────────────────────────────────────────┴────────────────────────┴─────────╯
🟢 data contract is valid. Run 16 checks. Took 0.545395 seconds.

Rebase your data product

Read from the access views

Change sql/sku_sales_per_year.sql to select from sku_sales_input.orders and sku_sales_input.line_items instead of the orders_v2 tables. Re-apply it:

docker compose exec -T postgres psql -U workshop -d workshop < sql/sku_sales_per_year.sql

NOTICE:  schema "analytics" already exists, skipping
CREATE SCHEMA
CREATE VIEW

Verify your consumers are unaffected

The output of your data product must not change. Your own contract proves it:

datacontract test sku_sales_per_year.odcs.yaml

Testing sku_sales_per_year.odcs.yaml
…
│ passed │ Ensure year is plausible                                 │ year           │         │
╰────────┴──────────────────────────────────────────────────────────┴────────────────┴─────────╯
🟢 data contract is valid. Run 15 checks. Took 0.606872 seconds.

Your data product now touches only the fields in your consumer-driven contract. Everything else in orders_v2 may change without breaking you. Even better: the orders team can run your contract in their CI pipeline and catch a breaking change before it ships. You build such a pipeline in the next exercise.

Quick check

The orders team wants to drop customer_email_address from orders_v2. Does that break the SKU Sales data product?

Bonus

Compare the producer's and the consumer's contract: datacontract changelog orders_v2.odcs.yaml orders_v2.consumer_sku_sales.odcs.yaml
Point the input port in sku_sales_per_year.odps.yaml to orders_v2_consumer_sku_sales. Should the input port reference the producer's contract or yours? Both have good arguments.