SAP Data Services

What is SAP Data Services (SAP DS)?

SAP Data Services (SAP DS, formerly SAP BusinessObjects Data Services) is a solution to one of the most pressing challenges facing companies: transforming large amounts of data into knowledge that can be used in business.

To this end, the tool offers the possibility of merging, cleansing, enriching and analyzing data from various sources. This applies to structured and unstructured data.

The focus of the application is on data integration on the one hand and on data quality on the other. As a result, companies receive a trustworthy, resilient and continuously updated information base for their business decisions.

Who is SAP Data Services suitable for?

SAP Data Services is aimed primarily at companies that have high requirements for data quality, data integration and data transformation. Existing SAP customers in particular belong to the target group, as SAP DS is very closely integrated with the Walldorf-based manufacturer's business applications and other data tools.

In conjunction with the "SAP (BusinessObjects) Information Stewart" product, the solution is also suitable for data governance tasks and cross-divisional data profiling, for example.

SAP Data Services is particularly helpful when data from SAP and non-SAP systems are to be viewed in an integrated manner. Non-SAP systems include relational databases, NoSQL databases, DBMS (database management systems), third-party enterprise systems, Web services, and even big-data resources (e.g., Hadoop-based). It does not matter whether the source system is operated locally or in the cloud.

SAP DS also includes the appropriate tools for monitoring and improving data quality (across systems). The solution is even suitable for processing and analyzing large volumes of data (Big Data Analytics), as it meets the highest demands on performance and scalability.

What functions does SAP Data Services offer?

The core task of SAP Data Services is to support companies in dealing with fragmented data. The system provides a wide range of features for this purpose. The functional areas are as follows:

• Data integration and data transformation
• Data Quality Management
• Data profiling
• Analysis of text data
• Change Data Capture (CDC)
• Integration with other SAP products

Let us take a closer look at these areas in the following.

1. data integration and data transformation

SAP DS is a proven data integration and data transformation tool. Its core task is therefore to connect, harmonize and process structured and unstructured data from different data sources.

The solution offers several methods for converting and providing the data:

• ETL method (Extract, Transform, Load)
• ELT method (Extract, Load, Transform) in connection with the SAP-HANA database technology
• RTL method (Replicate, Transform, Load)
• SQL push down method

SAP Data Services thus supports not only batch processing, the traditional approach to data transformation. Real-time services can also be used. They allow data services to be queried with defined workflows and immediate responses.

The transformation of data can be performed in parallel (for several sources simultaneously), thus achieving a particularly high data throughput rate.

2. data quality management

Another important area of application for SAP DS is data quality. As a central platform, the solution is suitable for making quality transparent and monitoring it across systems. This also makes quality problems and correlations visible that were previously hidden.

The software also provides information on how quality defects affect downstream applications and systems. All these aspects are visualized on clearly arranged dashboards.

In addition, SAP Data Services comes with predefined, customizable workflows for analyzing, cleansing, standardizing, and enriching data. These features make SAP DS suitable for both a one-off data cleansing project and continuous quality assurance.

3. data profiling

Data profiling is a function that can also be assigned to data quality management. It is an automated analysis that can be used to detect quality problems.

Among other things, data profiling provides information about frequency distributions, invalid values, minimum and maximum values and patterns in existing data sets. This makes it possible to identify patterns and anomalies in a database. This method is especially helpful for the early detection of skewed data.

4. analysis of text data

As mentioned, SAP Data Services also allows the processing of unstructured data. This property is particularly important for the analysis of text data (text data processing).

The method allows key terms, facts, moods, feelings and trends to be extracted from more than 200 different file and text formats, thereby gaining valuable insights. Just a few examples are press releases, social networks, e-mails and blogs.

5. Change Data Capture (CDC)

CDC is used to immediately detect data changes in a source database and to update the target system on this basis.

Thus, the entire data stock does not have to be updated permanently. Instead, only new or changed data records are transferred. This results in significant savings of computing resources.

6. integration with other SAP products

Data Services works together with a whole range of other SAP products from the Enterprise Information Management (EIM) sector. These include, for example:

SAP Information Steward: Application for data profiling, data quality and metadata management

• SAP Master Data Management (MDM): Tool for maintaining SAP master data
• SAP Master Data Governance (MDG): System for the consolidation and central control of master data
• SAP Information Lifecycle Management (ILM): Software for managing data flows and the associated metadata
• SAP Extended Enterprise Content Management from OpenText (Extended ECM): Connecting unstructured content and business applications

How are SAP DS and the ETL process related?

As a data integration and data transformation tool, Data Services is capable of multiple methods of data transformation and delivery. However, the most significant approach so far is ETL.

This abbreviation stands for a multi-stage process that is run through to transfer data from source systems to a data warehouse. In detail, this process consists of the following process steps:

1. extract (extraction)

In this process step, the required data is extracted in its original form from existing data sources. As a rule, companies use a staging area for intermediate storage at this point.

2. transform (transformation)

In the transformation phase, defined rules are used to convert the results of the extraction into the desired output format. Some examples are the conversion of units (e.g. currency), the filtering of data sets and the merging of similar data.

3. load

In this final step, the data is made available in the target system. Usually this is a data warehousing database on which analytics applications then work.

However, if the project involves migration or data cleansing, the data can also be loaded into new applications or back into the source system. Another common ETL scenario is loading data from third-party applications or other external sources into SAP HANA or SAP BW.

Note: If only HANA is used as the target database, the HANA Smart Data Integration tool is a suitable alternative.

What components does SAP Data Services consist of?

From a technical perspective, SAP Data Services consists of several components:

• Designer
• Repository
• Jobserver
• Datendienst-Engine
• Zugriffsserver (Zugriffsserver)
• Verwaltungskonsole

The Designer is a tool for developers. It allows the creation, testing and execution of applications (jobs) to fill a data warehouse (target system). These applications consist of workflows and data flows (data transformation definitions). Users are supported in the development of jobs by an intuitive graphical interface with source-target flow chart.

The repository is a database in which created system objects, source and target metadata and transformation rules are stored.

The so-called Data Service Engine runs on the job server. It executes the jobs defined in the applications (jobs).

The Access Server transfers all messages between the components involved (web applications, data services, job servers and engines).

The browser-based Management Console enables the administrator to manage all resources. This includes the processing, execution and monitoring of batch jobs as well as the configuration and execution of real-time data services, user administration and various configurations.

What is the architecture of SAP Data Services?

The architecture of SAP Data Services is designed for reliability, flexibility, performance and scalability. The components outlined above can be installed on a single computer or distributed across multiple computers. On the one hand, this allows cost-reducing vertical scaling. One computer runs several or even all processes on one server.

On the other hand a horizontal scaling is also possible. In this variant, server processes are distributed over several computers to optimize performance. If companies strive for particularly high reliability, it is also possible to run redundant versions of the same process on several computers simultaneously.

What are the advantages and disadvantages of SAP Data Services?

SAP Data Services maximizes the value of all structured and unstructured data. This is achieved first of all through cleansing and quality assurance. These aspects ensure that decisions are based on valid information.

However, added value is also created through data integration. It ensures that a 360-degree view of important subject areas can be realized. Data Services makes connections visible that companies can use to optimize their business. Even rapidly growing volumes of data can be managed efficiently with the tool without any loss of quality.

One disadvantage of SAP Data Service is its complexity. Thus, implementing the solution is a project with some challenges. Often, support from an external consultant is required to make the implementation a success. However, the subsequent handling in operational use is relatively simple thanks to intuitive interfaces.