Change Data Capture (CDC) is a technology that identifies and captures changes (inserts, updates, deletes) in a database, propagating these changes to other systems in real-time or near-real-time.
CDC is critical for data integration, real-time analytics, data synchronization, and microservices architectures, ensuring consistency across disparate data sources.
Core Concepts of CDC
-
Types of Data Changes
- Insert: New records added.
- Update: Existing records modified.
- Delete: Records removed.
-
CDC Objectives
- Realtime Propagation: Minimize latency between change occurrence and delivery.
- Data Integrity: Capture all changes without loss.
- Order Preservation: Maintain the sequence of changes (e.g., prevent an update from overriding a later delete).
see also:
Change Data Capture Pipeline
ETL (Extract, Transform, Load)
