Salesforce Decode
Salesforcedecode
Back to questions
Data CloudAdvancedingestionstreamingbatchlatency

Architect batch vs streaming ingestion for Data Cloud

Real World Scenario

Marketing wants near-real-time cart abandonment segments while finance requires nightly reconciled order totals from ERP. Both feed the same Data Cloud org.

Expected Answer

• Map data freshness requirements per source: streaming for behavioral events, batch for financial snapshots • Use Connector SDK or Ingestion API for high-volume streaming with idempotent event keys • Schedule S3/Azure blob batch loads for ERP with file arrival monitoring and error quarantine • Define SLAs: streaming p95 latency vs batch completion window before downstream activation • Separate data streams into distinct DLOs before harmonization to preserve lineage • Implement reconciliation jobs comparing streaming counts vs batch totals daily • Document backfill procedures when streaming pipelines fail and batch must catch up

Follow-Up Questions & Answers

Click to expand — each follow-up includes a direct, interview-ready answer

Use this approach when volume, complexity, or compliance demands it. Map data freshness requirements per source: streaming for behavioral events, batch for financial snapshots. Avoid it when a simpler declarative solution works. Not everything needs streaming—architects align ingestion mode to business action latency. Optimize for scale and operational observability.

Architect Perspective

Not everything needs streaming—architects align ingestion mode to business action latency. Pay for real-time only where activation or service depends on it.