Pipeline Assessment & Roadmap — 2026-03-12

Where We Stand Today

After today’s session the pipeline has been substantially improved. Here is the verified state of every layer.

APLT Source (`magic_pim`)

Supplier	Products	Stock	In Stock	Last Synced	Connectors
Spranz (01)	760	651	523	2026-01-28 ⚠️	None registered
Unknown (02)	15	0	—	2026-01-16	—
Langenberg (03)	244	0	—	2025-11-30 🚨	None registered
XD Connect (04)	11,526	5,969	0 ⚠️	2026-03-12 ✅	Products + stock (fresh)
PF Concept (05)	19,431	0	—	2026-03-12 ✅	Products (no prices feed, no stock)
MidOcean (06)	14,496	14,394	13,478	2026-03-12 ✅	4 connectors (fresh)
Toppoint (07)	5,856	7,365	6,163	2026-03-12 ✅	4 connectors (fresh)
NewWave (09)	26,239	0	—	2026-01-04 🚨	None registered

Total in APLT: 78,567 product rows across 7 active suppliers.

Three suppliers — Spranz, Langenberg, and NewWave — have no connector registered in aplt_connector_sync. Their data is refreshed only by manual runs. Langenberg is 3.5 months stale and NewWave 2+ months.

Development Medusa (`magic_b2b_development`)

State after today’s bulk import:

Metric	Count
Products	11,195
Variants	59,291
Prices	127,323
Variants with prices	58,814 (99.2%)
Variants without prices	477 (pre-existing, no APLT match)
Products with category links	1,861 of 11,195 (16.6%)
Categories defined	151

All 7 active suppliers are now present in dev Medusa. Before today only Spranz, Langenberg, and XD Connect were partially synced (3,308 products total). The bulk import added 5,887 new products.

The 477 variants without prices are from Spranz (34), Langenberg (14), and XD Connect (429) — products that were originally synced via the Medusa API before the APLT pipeline was established. They have no corresponding APLT record to pull prices from.

Tenant Databases

Database	Products	Variants	Draft %	Categories	Category coverage	Notes
`magic_b2b_brinxx`	12,068	51,503	0% ✅	162/151+	89.2%	All 7 suppliers ✅
`magic_b2b_bovisales`	13,742	73,558	0% ✅	174	71.4%	All 7 suppliers ✅
`magic_b2b_demo`	13,743	73,560	0% ✅	195	76.5%	All 7 suppliers ✅
`magic_b2b_default`	11,732	64,848	0% ✅	157	90.0%	All 7 suppliers ✅
`magic_b2b_desluis`	13,742	73,558	0% ✅	174	71.4%	All 7 suppliers ✅
`magic_b2b_jodasign`	14,792	73,809	0% ✅	174	69.2%	All 7 suppliers ✅
`magic_b2b_logohorloge`	5	—	0%	—	—	Watch brand — intentionally manual
`magic_b2b_spranz`	11,732	64,984	0% ✅	157	91.5%	All 7 suppliers ✅

Updated 2026-03-12 (end of session): All tenant databases have been fully pushed with all 7 suppliers, all drafts published, and category links applied. The best-stocked tenant (jodasign) now has 14,792 products — up from 8,938 before this session.

magic_b2b_logohorloge is not a gap — it holds 5 manually entered watch models (NUORO, ROMA, PALERMO, MILANO, BUFFALO) with no supplier code. It is a separate brand that does not feed from the APLT pipeline.

Every tenant database is significantly behind magic_b2b_development. The best-stocked tenant (jodasign, 8,938) is missing 2,257 products compared to dev. MidOcean, Toppoint, NewWave, and PF Concept are partially or entirely absent from most tenants.

What We Concluded

1. The core pipeline is sound — but entirely manual

All individual components work correctly end-to-end: connectors sync supplier data → APLT stores it → bulk SQL import pushes it to Medusa. What does not exist yet is any automation connecting these steps. Every sync, every import, every tenant push requires a human to run a command. This is the single biggest operational risk.

2. Dev Medusa is now the authoritative reference, but tenants lag badly

After today magic_b2b_development has the most complete product catalog ever — 11,195 products from all 7 suppliers. But this data exists in exactly one place. The tenants have stale, partial catalogs that diverge from dev by weeks or months of data.

3. Three suppliers have no connector and are accumulating staleness

Spranz (6 weeks), Langenberg (3.5 months), and NewWave (2 months) have no entry-point automation. Their data only updates when someone manually runs an import. Langenberg at 3.5 months is the worst. Any supplier catalog change — discontinued products, new SKUs, price changes — has gone undetected since November 2025.

4. XD Connect stock is entirely zero — cause unknown

5,969 stock records were synced fresh today (2026-03-12) and every single one shows quantity_available = 0. This is not a connector failure — the sync ran successfully. The data itself says zero. Either:

The entire XD Connect range genuinely has zero stock right now (seasonal, supply chain issue)
XD Connect’s stock feed uses a different field or format that the connector isn’t reading correctly

Until this is investigated, every XD Connect product will show as out-of-stock on every tenant storefront.

5. PF Concept has no prices and severe variant deduplication

Two separate problems compound for PF Concept:

No prices: PF Concept publishes pricing via a separate feed that the current connector does not sync. All 2,491 PF Concept products in Medusa have a €1.00 placeholder price.
67% variant loss: 13,045 of 19,405 PF Concept variants were dropped during import because their EAN/barcode values conflict with variants from other suppliers. PF Concept apparently uses generic or non-unique EAN codes. This is a data quality issue at the source.

6. Categories are broken for newly imported products

1,861 of 11,195 products (16.6%) have category links in dev. All 5,887 products imported today have zero category links. A Medusa storefront with categories as navigation shows only 16.6% of the catalog correctly placed in the product tree. The other 83.4% are uncategorized.

7. Brinxx draft status is blocking the storefront

53% of Brinxx products are in draft status and invisible on the storefront. Breaking it down: XD Connect is 53% draft, PF Concept is 99% draft, MidOcean is 100% draft. The pattern is clear — products imported via the Medusa Admin API sync were marked draft when they lacked sufficient data (typically prices). Now that prices are linked, these products should be safe to publish.

Recommended Action Plan

Priority 1 — Unblock the Storefront (do first)

Bulk-publish Brinxx draft products

A single SQL update, but verify first that all drafts now have prices:

-- Verify: how many drafts still have no price?
SELECT COUNT(*) FROM product p
JOIN product_variant pv ON pv.product_id = p.id
LEFT JOIN product_variant_price_set pvps ON pvps.variant_id = pv.id
WHERE p.deleted_at IS NULL AND p.status = 'draft'
  AND pvps.variant_id IS NULL;

-- If zero (or acceptable): publish them
UPDATE product SET status = 'published'
WHERE deleted_at IS NULL AND status = 'draft'
  AND metadata->>'supplier_code' IN ('04','05','06');

Investigate XD Connect zero stock

Check whether zero stock is correct or a feed mapping bug:

# Look at raw XD Connect feed data vs what's stored in APLT
docker exec magic_pim_backend_dev sh -c \
  "cd /app/.medusa/server && node -e \"
    const {Client} = require('pg');
    const c = new Client({host:'postgres',database:'magic_pim',user:'postgres',password:'<your-db-password>'});
    c.connect().then(() => c.query('SELECT * FROM aplt_stock WHERE supplier_code=\\'04\\' LIMIT 5'))
      .then(r => console.log(JSON.stringify(r.rows, null, 2))).then(() => c.end());
  \""

If the quantity field in APLT is genuinely 0 from the connector, contact XD Connect to check their stock feed format.

Priority 2 — Populate Tenant Databases

Decide which suppliers belong in which tenant

Currently ad-hoc. Recommended rule: all tenants get all suppliers unless there is a business reason to exclude one (e.g., magic_b2b_spranz should arguably only get Spranz products). Document this mapping before running.
Run bulk import per tenant

The bulk import script targets one database at a time. For each tenant that needs updating:
Terminal window
```
# Example for Brinxx
docker exec magic_pim_backend_dev sh -c \
  "cd /app/.medusa/server && \
   DB=magic_b2b_brinxx node -e \"
     process.env.TARGET_DB='magic_b2b_brinxx';
     require('./bulk_medusa_import.js');
   \""
```
Note: The script currently has DEV_CONFIG.database hardcoded. Either edit the script to accept a TARGET_DB env var, or run with sed to swap the database name before each run.

Sync 151 categories to all tenants

Brinxx has 47 of 151 categories. Other tenants are likely similarly incomplete. Categories come from MidOcean (synced via midocean_categories connector). A direct SQL copy from magic_b2b_development.product_category to each tenant is the fastest path:

-- Run on each tenant database
INSERT INTO product_category (id, name, handle, description, is_active, is_internal, rank, parent_category_id, metadata, created_at, updated_at)
SELECT id, name, handle, description, is_active, is_internal, rank, parent_category_id, metadata, created_at, updated_at
FROM dblink('dbname=magic_b2b_development user=postgres password=<your-db-password>',
  'SELECT id, name, handle, description, is_active, is_internal, rank, parent_category_id, metadata, created_at, updated_at FROM product_category WHERE deleted_at IS NULL')
AS t(...)
ON CONFLICT (id) DO NOTHING;

(Requires dblink extension or a Node.js script copying between clients — the latter is simpler.)

Populate category links (product_category_product)

After categories exist, link products to them based on aplt_products.master_category_code. This requires a mapping from category code → Medusa product_category.id, then inserting product_category_product rows. The catMap variable already built inside the bulk import script has this mapping; expose it as a standalone “category link” step.

Priority 3 — Fix Data Quality

Re-sync Spranz, Langenberg, NewWave

These have no connector in aplt_connector_sync. Options:
- Trigger their sync scripts manually now to get fresh data
- Build connector entries for them so the pipeline knows they exist
Langenberg at 3.5 months stale is the most urgent. NewWave at 2 months is a close second — it also has 26,239 rows which is the largest single supplier by APLT row count.
PF Concept: investigate the prices feed

From the PF Concept connector code, identify the prices feed URL/format. PF Concept documentation typically provides pricing as a separate XML or CSV download. The connector needs a pfconcept_prices sync step that reads this feed and updates aplt_products.retail_price_* columns.
PF Concept: EAN conflict strategy

67% variant loss is not acceptable long-term. Options:
- Null out EAN for PF Concept in the bulk import script (set barcode: null). This loses EAN data but preserves all variants.
- Investigate source: confirm whether PF Concept EAN values in the raw feed are actually valid unique EANs or placeholder values.
- Store in metadata instead: write PF Concept EAN to metadata.ean_pf rather than the barcode column if the values are known to be non-unique.
The safest immediate fix is option 1 — null out barcode for PF Concept only, then re-run the import for the products that were partially imported.

Priority 4 — Automation (currently zero)

Nothing in the pipeline runs automatically. The current operational model requires a human to:

Trigger each connector sync manually (or via the Pipeline Check dashboard button)
Run the bulk import script manually after syncs complete
Manually push to tenant databases
Manually monitor for failures or staleness

Recommended automation stack:

Step	What	Trigger
Connector syncs	Each connector’s HTTP endpoint	Cron, daily at 03:00
Bulk import dev	`node bulk_medusa_import.js all`	After all connectors complete
Category links	Category link script	After bulk import
Tenant push	Per-tenant bulk import	After dev import
Staleness check	Query `aplt_connector_sync` for rows > 26h old	Cron, every 2h, alert if any found

The Pipeline Check dashboard already has Sync All and per-connector trigger buttons. The missing piece is a cron layer that calls these endpoints on a schedule and chains the downstream steps on completion.

Summary Table — Everything That Needs Doing

Item	Effort	Impact	Blocking
~~Publish Brinxx draft products~~	✅ Done	2,750 published, Brinxx 100% visible	—
~~XD Connect zero stock~~	✅ Done	Connector bug fixed, 5,725 in stock	—
~~PF Concept EAN fix (67% variant loss)~~	✅ Done	Barcode nulled for PF Concept — 13,045 variants recovered	—
~~Category links for dev products~~	✅ Done	8,051/11,195 products (71.9%) linked — up from 16.6%	—
~~Run bulk import for tenant DBs~~	✅ Done	All 7 tenants pushed — 11,732–14,792 products each	—
~~Sync categories to all tenants~~	✅ Done	151 categories synced; 69–91% product coverage per tenant	—
Re-sync Langenberg (3.5 months stale)	Low (trigger run)	🟠 Medium	No
Re-sync NewWave (2 months stale)	Low (trigger run)	🟠 Medium	No
PF Concept EAN fix (67% variant loss)	Low (1 code change)	🟠 Medium	No
PF Concept prices feed	High (new connector)	🟠 Medium	No
Build cron jobs for automation	High (infra)	🟡 Medium-term	No
Re-sync Spranz (6 weeks stale)	Low (trigger run)	🟡 Low-medium	No
XD Connect connector fix (if feed bug)	Medium	🟡 Depends on investigation	No
Monitoring/alerting	High (infra)	🟡 Long-term	No

What Is Working Well

It is worth noting what does not need attention:

MidOcean — 4 connectors, all fresh, 14,394 stock records, 13,478 in-stock. Pipeline works end-to-end.
Toppoint — 4 connectors, all fresh, 7,365 stock records, 6,163 in-stock. Pipeline works end-to-end.
Bulk import script — handles all 7 suppliers, all edge cases (unique constraint conflicts, cross-supplier handle collisions, in-batch deduplication), full transaction rollback on failure.
Pipeline Check dashboard — live cross-database view, manual sync trigger buttons, staleness alerts.
Medusa dev database — 11,195 products, 99.2% price coverage, all published.
Price link coverage — 58,814 of 59,291 variants have prices. The 477 without prices are legacy pre-APLT records.