Theobald Xtract Universal (XU) offers SAP data over http-json endpoints. XU can be leveraged to extract data from SAP into an Ingest instance using a REST data source.

Benefits

• Schedule executions for all SAP entities (including BAPI, BW Hierarchy, ODP, Query, DeltaQ, etc.) from within TimeXtender. XU supports all SAP entities, unlike the legacy TimeXtender SAP Table Data Source which used Xtract IS, and only supported TimeXtender-scheduled executions for full table extracts.

  

• XU does not require SQL Server Integration Services (SSIS), unlike the legacy TimeXtender SAP Table Data Source which used Xtract IS.

Prerequisites

• Ensure Python (version 3.8 or later) is installed. Install Python directly from the Microsoft Store (by searching for python) or download from https://www.python.org/downloads/
• Buy a Theobald XU license or download a trial.

Steps

1. Install and Configure XU

Install XU on the TimeXtender application server. Read and follow the steps at 

• Requirements
• Installation and Update
• Getting Started with Xtract Universal

2. Create Extractions in XU

Read and follow the steps at

• Defining a Table Extraction
• Alternatively, use this script for generating extractions automatically

You can use all available extraction types that XU offers.

3. Download and unzip the xu-rsd-generator utility

Download the latest version of the xu-rsd-generator utility.
Save and extract the files on the local drive on the TimeXtender application server.

4. Execute install.bat or follow the install instructions from README.md

This will set up python dependencies so that the generator can create rsd files.

5. Edit parameters in xu_rsd_gen\.env

• Adjust XU_BASE_URL to point it to the XU server and port. If you installed XU on the TimeXtender application server then the XU server is simply localhost, and the default port for XU is 8065. 
• Point RSD_TARGET_FOLDER to a local folder that you will use for the TimeXtender REST data source.

6. Execute run.bat

Executing run.bat will create the RSD files in the folder defined under RSD_TARGET_FOLDER.

7. Create REST data source

• Add a data source in the TimeXtender Portal. Select REST from the dropdown.

  

• Set the Location property under the Schema section to the RSD Target Folder directory.

  

• Map the data source to the Ingest instance. For more info see this guide on how to create and map data sources.

8. Add REST data source to an Ingest Instance in TimeXtender Desktop

Read and follow the instructions here: Configuring a Data Source in TimeXtender Desktop

9. Verify the Tables in the Data Source Explorer and Query Tool

Limitations

1. Incremental loading

If you need to load tables with a large amount of records:

1. Use XU to generate a full load of the long table into a parquet file (make sure the Package size property is set on the extraction level, otherwise the extractions can consume extreme amounts of memory on the SAP source system)

2. Configure a timestamp based sliding window for data extraction, either in the RSD files, or on the XU extractions directly e.g. each load will get the last 2 days worth of changed data. This is the equivalent of subtract from value feature as described in the Incremental Load in an Ingest Instance article. 

3. Combine the static full load parquet data source with the sliding window/incremental data source in the MDW table.

2. Pagination

XU does currently not support pagination of long tables. During testing we found that even for millions of records this did not cause a performance problem neither on XU nor on the TimeXtender Ingest Service. Make sure the Package size property is set on the extractions, otherwise the extractions can consume extreme amounts of memory on the SAP source system. If you are concerned with large tables being streamed over http-json, consider the using the approach described in point 1 above regarding Incremental loading.