By Benjamin Kwong — Jun 28, 2023

SAP Data & Analytics: Extract S/4HANA Data with Azure Data Factory CDC connector

Azure Data Factory CDC Connnector - SAP Architecture Diagram

Introduction

My last two blog posts focused on two critical parts of data extraction from S/4HANA:

These two frameworks allow external tools like SAP Data Services, SAP Data Intelligence and SAP Datasphere to connect and consume data from S/4HANA.

One alternative method I'm going to give an introduction to is Azure Data Factory, Change Data Capture connector that leverage's SAP Operational Data Provisioning (ODP) framework.

Architecture

Microsoft always provide great documentation on their Azure products and Change Data Capture is no exception. I recommend reading this article to go in depth on how Azure Data Factory CDC connects into S/4HANA

The SAP CDC solution in Azure Data Factory is a connector between SAP and Azure. The SAP side includes the SAP ODP connector that invokes the ODP API over standard Remote Function Call (RFC) modules to extract full and delta raw SAP data.

You can then use Azure data flows to send S/4HANA data to storage destinations like Azure Data Lake Storage Gen2 or databases like Azure SQL Database or Azure Synapse Analytics.

Considerations

Be Cautious with Release Contracts on CDS Views

This framework is still reliant on CDS views data extraction and it's always good practice to start with Released views from SAP which I covered previously. You can get around this with custom CDS views built with Eclipse however depending on your enhancement method you could be introducing technical debt that needs to be maintained long term. This is relevant for on-premise S/4HANA systems.

Azure Data Factory Change Data Capture (CDC) extract from certain SAP Systems

Most modern SAP landscapes are evolving and with the introduction of SAP Business Technology Platform in more and more landscapes you must be strategic about how you extract data from your SAP systems. There are different methods for extracting data in SAP BTP and S/4HANA at the time of writing this article which can complicate your architecture very quickly.

Delta Recovery Scenarios are still dependent on Operational Data Provisioning (ODP)

Delta data is retained for a default setting of 24 hours.

Review SAP note: 2854627 - How to change the Retention periods for Recovery of 'Reorganize delta queues' for more details on how to extend this setting.

SAP can change Operational Data Provisioning

SAP built operational data provisioning to support SAP products first. Using the Azure Data Factory Change Data Capture (CDC) connector you should always be cognizant of the risk that SAP and Microsoft sell competing products that deliver similar data extraction functionality. They have the ability to change these products at any time which could break functionality unexpectedly.