DATA ENGINEERING · REPRODUCIBLE PORTFOLIO RUN
From 101,786 source records
to decisions you can inspect.
A fixed seed generates a realistic supply-chain network, round-trips all seven tables through MySQL, applies lossless ETL and quality contracts, then publishes an accessible analytics layer.
Seed 422024-07 → 2026-067 tablesNo account or API
SUPPLIERS · 30PRODUCTS · 150WAREHOUSES · 6INVENTORY · 21,600SALES · 70,000ORDERS · 8,000PRICE · 2,000
→
101,786validated rows
60-SECOND DEMO
Ask the network a question.
Change any filter. Every KPI, chart, table, and fallback is recomputed from the same versioned aggregate cube.
Loading release evidence…
The dashboard data could not be loaded.Run the CLI locally or inspect the committed report.
Demand signal
Monthly sales and seasonality
RevenueUnits
Accessible monthly values
| Period | Revenue | Units |
|---|
Inventory health
Stock positions
A position is one SKU at one warehouse for one monthly snapshot.
Supplier operations
Delivery scorecard
| Supplier | Orders | On-time | Late days |
|---|
Controlled anomalies
Known stress cases
Business anomalies are intentionally flagged; malformed keys and negative measures are never inserted.
ENGINEERING CASE
The dashboard is the last stage, not the system.
The operational contract lives in a Python CLI. MySQL, data contracts, aggregates, downloads, and this interface are outputs that can be regenerated and compared.
Fixed, meaningful data
Demand classes, seasonal weights, promotions, supplier reliability, delays, price changes, and stock risk all affect downstream metrics.
Lossless ETL
Every source row survives extract, transform, validation, and processed load. Derived fields add meaning without hiding rejected records.
Static observability
Pages reads only pre-aggregated JSON. It cannot mutate the database, rerun a pipeline, or publish operational data.
LINEAGE + OBSERVABILITY
One command, eight observable stages.
python -m scripts.orchestration.demo \
--scale portfolio --seed 42 --output site| Stage | State | Rows in | Rows out | Duration |
|---|
DATA MODEL
Dimensions anchor three operational fact streams.
The generated schema uses explicit primary keys, indexed foreign keys, and business constraints. Processed tables mirror the source grain and add derived fields.
REAL, LINKED CODE
The important logic is short enough to inspect.
rng = np.random.default_rng(seed)
promotion = rng.random(config.sales) < 0.14
anomaly = np.arange(1, config.sales + 1) % 997 == 0One seeded generator drives every probabilistic choice. Anomaly positions are deterministic and reviewable.
inventory['quantity_available'] = (
inventory.quantity_on_hand
- inventory.quantity_reserved
)Derived columns preserve grain and source counts. Invalid structure would fail a contract instead of disappearing.
orphaned = int((
~child[column].isin(parent[parent_key])
).sum())Each relationship is checked in Python after extract, in addition to MySQL constraints.
keys = ['period', 'warehouse_id',
'category', 'supplier_id']
cube = sales.merge(orders, on=keys, how='outer')The browser filters a compact, versioned cube rather than downloading 100k source rows.
KPI CONTRACTS
Definitions travel with the output.
Σ(quantity_sold × unit_price)Rounded per transaction, then summed.
on_time / completed × 100In-transit orders are excluded from the denominator.
available / reorder_level < 0.5Distinct product IDs in a critical position at the selected end period.
available / reorder_levelPoint-in-time at the selected end period: <0.5 critical; <1 low; <3 optimal; otherwise excess.
DATA EXPLORER
Inspect samples before downloading.
Twelve rows from each processed table are included in Pages. The complete deterministic CSV set, schema, manifest, seed, and hashes live in the ZIP.
QUALITY + LIMITS
Evidence is explicit; so are the boundaries.
Limitations
- The data is synthetic and tuned to exercise engineering paths, not to benchmark a real company.
- The dashboard is descriptive. It does not forecast demand, optimize orders, or mutate MySQL.
- Stage timings describe the published machine and are operational evidence, not a performance claim.
- MySQL is required for the release command; `--database skip` exists only for fast offline tests.
RUN LOCALLY
Reproduce the same artifact.
Start the bundled database, install the focused portfolio dependencies, then run the public CLI. Use `smoke` for a fast check or `portfolio` for all 101,786 source rows.
docker compose up -d --build --wait mysql
python -m pip install -r requirements-dev.txt
python -m scripts.orchestration.demo \
--scale portfolio --seed 42 --output site
python -m pytest