22 — Integration & Processing Orchestration (Blueprint §7)

22 — Integration & Processing Orchestration (Blueprint §7)

Source: 02-Implementation-Blueprint.md — §7) Integration and Processing Plan (steps 1–10).

Product alignment: 01-Post-Purchase-Video-Testimonial-Collector-Plan.md — full operational workflow from order event to published storefront social proof.

This document is a build spec only. No code changes are implied until a task references this file.

Related plans: 06, 07, 10, 11, 13, 14, 20, 21.

---

0) Goal (one sentence)

Define one reliable end-to-end orchestration contract so every subsystem (webhooks, workers, APIs, moderation, storefront rendering) updates data in the right order with idempotency and observable state transitions.

---

1) Blueprint step mapping (authoritative)

This section maps each blueprint step to a concrete app responsibility.

Blueprint §7 step	App responsibility
1. Order webhook -> evaluate active campaigns	webhooks.orders.* route + eligibility service
2. Expand delivered line items	request planner creates one TestimonialRequest per eligible item
3. Create and schedule request rows	write TestimonialRequest with token and scheduledFor
4. Scheduler sends due requests	cron/worker dispatch email/SMS and write events
5. Public page validates token and captures submission	/t/:token loader + submit API
6. Upload media via signed URL	api.testimonial-upload-url + storage provider
7. Worker updates processing status/metadata	media callback/worker updates TestimonialMediaAsset
8. Submission appears in moderation queue	status=pending row visible in Screen 5
9. On approve, bind to PDP product	submission shopifyProductId retained/updated
10. Approved submissions rendered by widget API	public read API filters published + placement

Treat this table as the sequence-of-truth for implementation and debugging.

---

2) System components and boundaries

2.1 Ingestion layer

• Shopify webhooks (orders/paid, orders/fulfilled)
• Validates HMAC and tenant
• Emits request planning operations

2.2 Scheduling/delivery layer

• Due-request worker (cron-driven)
• Template renderer + provider adapters
• Request event writer (queued/sent/failed/...)

2.3 Public capture layer

• Tokenized page (/t/:token)
• Upload URL API
• Submission API

2.4 Media processing layer

• Transcode/thumbnail/metadata worker
• Asset status updates (uploaded -> processing -> ready|failed)

2.5 Moderation/publication layer

• Inbox/detail actions update status/publish flags
• Moderation log append-only writes

2.6 Storefront delivery layer

• Read API returns only eligible published rows
• Theme widget blocks render home/PDP sections

---

3) State machines (must be explicit)

3.1 TestimonialRequest.status

Recommended states:

• scheduled
• queued
• sent
• clicked
• submitted
• expired
• failed

Transition rules:

• scheduled -> queued -> sent
• sent -> clicked -> submitted
• sent -> failed (provider or retry exhaustion)
• scheduled|sent -> expired (token or policy expiry)

No backward transitions without explicit repair action.

3.2 TestimonialMediaAsset.processingStatus

• uploaded
• processing
• ready
• failed

Only ready assets should be storefront-playable.

3.3 TestimonialSubmission.status

• pending
• approved
• rejected
• archived

Publication is orthogonal (published boolean + publishedAt).

---

4) Orchestration contracts between modules

4.1 Webhook -> Request planner

Input:

• shop domain
• order payload
• trigger type

Output:

• deterministic list of planned request rows

Idempotency key:

• (shopId, campaignId, shopifyOrderId, shopifyProductId, variant?)

4.2 Request planner -> Sender worker

Sender worker expects:

• valid submissionToken
• scheduledFor <= now
• channel + template resolvable
• contact destination available

4.3 Public submit -> Moderation queue

Submit API guarantees:

• request status becomes submitted
• submission row exists with product binding
• media asset row linked

Moderation queue consumes status=pending plus media readiness metadata.

4.4 Moderation -> Storefront API

Storefront API should only include rows where:

• status=approved
• published=true
• asset status ready
• placement visibility rules pass

---

5) Failure handling and retries

5.1 Webhook retries

• Must be idempotent and safe for duplicate deliveries.
• Always return 2xx for already-processed payloads.

5.2 Sender retries

• Failed sends should retry with bounded attempts.
• Keep lastError and emit failed event entries.
• Move terminal failures to visible requests log (13).

5.3 Media processing failures

• Set asset to failed with processingError.
• Keep submission in moderation with explicit error badge.
• Allow merchant to request reprocess (future).

5.4 Submission duplicates

• Token submit must be idempotent:
  • second submit returns 409 or “already submitted” response.

---

6) Observability requirements

6.1 Structured logs

Log with correlated IDs:

• shopId
• campaignId
• requestId
• submissionId
• assetId

Do not log full token values.

6.2 Metrics counters

Minimum counters:

• webhook received/processed/duplicate
• requests created
• sends attempted/sent/failed
• submissions created
• media ready/failed
• approvals/published

These counters feed analytics and operational alerting.

---

7) Scheduling and timing semantics

• All persisted timestamps in UTC.
• delayDays interpreted as UTC-day offsets from trigger event time.
• Reminder windows use UTC and are based on initial sentAt.

Document this in code comments to avoid timezone ambiguity.

---

8) Security checkpoints in flow

At each step:

1. Webhook: verify HMAC.
2. Public routes: validate token + expiry + rate limits.
3. Upload URLs: short-lived signed URLs only.
4. Storefront API: return only published/public-safe fields.
5. Compliance hooks: redact/delete tenant and customer-linked data.

Reference 08-security-compliance-and-privacy.md for details.

---

9) Sequence diagram (textual)

1. Shopify sends order webhook.
2. App validates webhook and loads active campaigns.
3. App creates per-product TestimonialRequest rows.
4. Cron worker picks due requests and sends email/SMS.
5. Customer opens submissionUrl, token validated.
6. Customer uploads media via signed URL.
7. Customer submits form; app creates submission + asset + updates request.
8. Worker processes media and marks asset ready.
9. Merchant moderates and publishes.
10. Storefront widget API returns published rows for home/PDP.

---

10) Integration acceptance checklist

• Full happy-path from webhook to storefront render works for one product.
• One order with 3 eligible products creates 3 request rows.
• Duplicate webhook does not create duplicate requests.
• Sent -> clicked -> submitted timeline appears in requests log.
• Media processing transitions are visible in submission detail.
• Published content appears in correct widget placement only.
• Unpublish immediately removes from storefront API output.

---

11) Implementation order (for Cursor)

1. Finalize schema (20) and route map (21).
2. Implement webhook + request creation (07).
3. Implement public capture + upload (06).
4. Implement media processing callback/worker.
5. Implement moderation + publish screens (09,10,11,12).
6. Wire storefront read API + widgets (05).
7. Add observability and analytics (13,14,15,19).

---

12) References

• 02-Implementation-Blueprint.md — Section 7
• 07-email-sms-request-delivery-pipeline.md
• 06-public-submission-page-screen-13.md
• 11-published-content-screen-7.md
• 05-storefront-widgets-and-read-api.md
• 08-security-compliance-and-privacy.md
• 20-database-design-and-migrations-blueprint-section-5.md
• 21-route-and-file-map-blueprint-section-6.md

---

13) Note on numbering

This folder already includes 05 through 21 plans. This file is 22-....