Prepare

Hi All, Hopefully you can help me with this one. We have a source table from our POS systems containing sales transactions. There are a few value columns we transfer that are stored as a money data type and we see some weird things happening with the values of the columns. Differences of 0.01 up to 0.0001 are appearing from nowhere in the transfer from ODX to DWH and i don't know how this i happening.The columns that have money data type are called ‘Bedrag', ’Korting’ and ‘NettoBedrag’ Take transaction 52. In the source the values are like this: After transfer with ODX, i looked in the parquet file using ParquetViewer and that looks like this: St

Incremental Load Strategy for Data Warehouse Layer (Prepare Views → DW Hi everyone,We’re working on a project for a TPA that provides property and casualty services to smaller P&C companies. The architecture connects to multiple disparate sources that ultimately feed into a standardized data warehouse.Here’s our current setup:Architecture:We’re creating a raw schema in Prepare for each data source, then using views to abstract the transformation logic. The DW maps from those views. So the flow is: Source → Ingest → Prepare (raw) → Prepare (views) → DW.What’s working:Incremental loads are running well from Source to Ingest and from Ingest to Prepare raw — no issues there.The challenge:The DW layer currently runs as a full load every time. We’re moving roughly 200M rows per company. Load times are about 1:45 hours, which is manageable, but we’d like to bring that down with incremental loads. It would also reduce the vCor

Hi team, I need to audit how many rows were moved by batch on the prepare table. Is there in TX how to audit it?ThanksCarlos

When adding a conditional lookup in table A from table B an index is created in table B.This is expected behaviour when the Index generation setting is set to “Automatic”.However, if the lookup operator is set to “None”, no index is created in table B, eventhough index generation is set to automatic.Additionally, if a lookup is at first created with operator “Max” (or any other) an index is created, but if the operator is later changed to “None” the index is automatically removed. This seems like a bug. Is anyone else able to replicate this?I have tested this on version 7026.1

I have 3 big fact tables in the source which i load incrementally. Primary keys are CompanyID and EntryNo (applicable for each table separately, incrementing), the records in these fact tables are never deleted.I need to merge these tables to a single table in DSA layer also incrementally. I think of following setup:Make the DSA table also incremental, check “Don’t handle deletes” (as the records are never deleted in source) Include both CompanyID and EntryNo as primary keys in the table; Incremental Selection Rule → EntryNo > (Last max value); Map all tables from source in this single DSA table The result which i expect is every time DSA table is executed - it incrementally loads data from 3 related fact tables - based on EntryNo for each separate CompanyID (one Company

Hi teamI have created this Schedule Groups,  The ingestion process packages are working fine, but the prepared packages are not working. the message log is:Logfile output: 5:01:53 AM UTC now: 2/20/2026 5:01:53 AM5:01:53 AM TimeXtender Version 25.2.0.57 built 11/10/2025 2:15:00 PM. Running on vm-orc-prod-0075:01:56 AM PID: 42376  - Started with arguments: --startschedulegroup 9 --execute_uat PROD --schedule_group 9 --customerkey uim --use-saas-queue true --token ece94e91-e0af-4588-9844-80e5e7f952cd --is-saas-local true 5:01:56 AM Connected at 2/20/2026 5:01:56 AM5:01:56 AM 5:01:56 AM *** Executing schedule group 165:01:56 AM Running schedule group CortexPolicy on PROD with 4 tasks.5:01:57 AM 2/20/2026 5:01:57 AM Starting: GWPCProd Load Data (Incremental Load) (CortexIn

Hi, I am adding a mapping set (4 tables) to my table and then I get the “Source Table” column. After a deploy and execute the column “Source Table” have only a single dot in the column. I am expecting a table name from the mapping set as before... Whats wrong? Any suggestion? regards,Bjørn A.

I have been trying to sort out something out,My JSON looks like this:addendumGroupName occurs multiple times. Here, its 4 times but in other cases it occurs between 1 to 50.columns can be one or multiple I am trying to list all addendumGroupName and all columns under each addendumGroupName without doing this:json/addendumValues/addendumGroups[2]/addendumGroupName[0]json/addendumValues/addendumGroups[2]/rows[0]/columns/fremdrift_group[0] json/addendumValues/addendumGroups[2]/rows[1]/columns/fremdrift_group[0]json/addendumValues/addendumGroups[2]/rows[2]/columns/fremdrift_group[0]  So i tried this (with an without [*] at the end of column name): <attr name="addendumGroupName" xs:type="string" readonly="false" other:xPath="/js

We are loading data from a SQL Server (onprem) to a Datalake (ingest instance). Data is stored as parquet, but TX is creating a huge parquet file for each table.  We are using ADF to ingest the data into the ingest instance.  Is there a way to load the data in such a way that when it gets into the Datalake the parquet can be divided in parts?  table_xxx_0001.parquet, table_xxx_0002.parquet, etc. (I know this is not the parquet format it is just an example).  Is this possible? We are having performance issues reading from parquet to AZURE SQL DB prepare instance.  We are using ADO.NET from ingest to prepare.

Hi team I was recently working on a table where I added a hash field and configured it for incremental loading. However, the initial deployment failed because the table was missing the required column for the incremental process. When I attempted to redeploy to fix it, I hit a roadblock: a duplicate key error in dbo.DataFields with the value (ea41fe97-e408-4504-9164-239dfcdee390, 148, 99999999).  

Hi team, I am encountering a timeout issue when moving large tables. The problem seems to be originating from the cleanup stored procedure (clean sp), as smaller tables are processed without incident. Could you please look into this issue?  

I’m migrating a large SQL Dedicated Pool data warehouse to Fabric Lakehouse.I’ve run into Fabric’s workspace limit of a thousand items.What’s the best-practices workaround?

Hi all,I'm working in TimeXtender (TDI) and ran into something I can't quite figure out. When I create a new schema in my Prepare instance, I want to assign it to a table that already exists in that same instance. The schema does appear as an option when I try to link it to the table. However, it gives an error upon deployment (of that table), at least until after I deploy the instance. Is this expected behavior?Or am I perhaps creating them in the wrong order or place? 

Hi community,I created a role in the MDW and added Grant permission on one specific database schema. This schema contains some tables and some Stored Procedures. This role need to be able to select data in the tables and execute the stored procedures. I created my Object Security Setup using these settings:  When I login using SSMS (using the user in the role), I see the correct tables and I'm able to select data in the tables. But I'm not able to see the stored procedures. When I check the permissions on the ‘ExcelAddin’ database schema. Only the ‘select’ permission in granted. Is it possible to grant permission for ‘execute’ as well? Using TimeXtender?

Hi,I have a client that uses TDI 7026.1 and i need to install it on another computer, but on the web I can only find 7050.1. Any possibility to find the earlier version somewhere?/Leo

I’m getting a Lakehouse notebook execution timeout in v7142.1 after exactly 10 minutes.My connection and command timeouts in the TX portal are 1200 and 7200 seconds, respectively, for both Ingest and Prepare so that’s not the setting I need to change … and the workspace’s Spark timeout setting is 20 minutes so that’s also not the one.I’ve copied the relevant part of the error message below.Does anyone know how and where to increase the notebook timeout setting?(the Microsoft links aren’t that useful from a TDI perspective, I think) Start Time : 10/15/2025 2:23:08 PM End Time : 10/15/2025 2:37:01 PM on server: V171D008S -Execute Execution Package Default 'Failed'      An error occurred while calling o5570.throwExceptionIfHave.: com.microsoft.spark.notebook.msutils.NotebookExecutionException: Timeout when exe cell - 15 in notebook TimeXtender_PrepareKOMBI_TABLE_dsa_INVENTTRANS, code length = 39808 and it costs

Hello,Is it possible to Extract data from an existing Lakehouse into a Prepare instance without needing to move it into the TimeXtender Ingest instance? The use case is to avoid replicating the data again, if the customer has already ingested data into a data lake or Lakehouse.Thanks in advanceTim

I have two tables with no unique identifer. There a two posts that mentions NEWID() as a way to create a unique identifier, so it seems that it can be used:How to add a default value to a unique identifier field – TimeXtender SupportHow to Add a Default Value to a Unique Identifier Field in TimeXtender Classic | CommunityI have created a filed in each of my table and made it Primary Key.My question is:What happens if we

I’m planning to change a history table in DSA into an incremental table without removing the history and I want to make sure that I understand some of the settings correctly. The ideal outcome for me is the following:New data added incrementally without the history being maintained Regarding the old history: I want to keep all rows with [SCD Is Current]=1 I don’t want to keep rows with [SCD Is Current]=0 Here are some questions I have about options in Table Settings > Data extraction :Does it make sense for this scenario to have the boxes checked for “Truncate raw table before transfer” and “Empty raw table after data cleansing”? What is the best approach for me to take with the delete options (“Don’t handle deletes”, “Use soft deletes” and “Use hard deletes”)? If I’m fine with new rows being overwritten can I then create some kind of incremental rule to make sure that old history is changed (or at

Hi,I obtain a deployment error when I try to make my table historic. Computed column 'SCD Surrogate Hash Key' in table ' <mytable>' cannot be persisted because the column is non-deterministic. I think this has to do with the excessive amount of columns in the table: 695 columns, all varchar(2000). What confuses me is that when I set the hashing algorithms to debug, I still get the same error  I don't have any transformations on this table, its a straight copy from to ODX.  Is there a limit on the amount of fields a History table can have? 

Hello Timextender and communityI’m looking to understand how Timextender Classic deals with collumn statistics that aren’t tied to indexes. Our project is built on Azure SQL DB.Obviously TimeXtender automatically rebuilds indexes as part of its data cleansing procedures. However, rebuilding indexes only updates statistics on the indexed columns—not on the rest. Performance  relies heavily on accurate statistics for query optimization (especially when MAXDOP isn’t explicitly set), this leaves me wondering: Does TimeXtender have any built-in process or configuration to update statistics on non-indexed columns? If not, are there recommended best practices for managing these statistics in a TX solution—either as part of the deployment or as a post-process step? After dealing with a sudden drop in performance, we’ve observed performance improvements aft

I’m looking for a TimeXtender Fabric Lakehouse (pyspark.sql) expert 😀I’ve got a SQL transformation that selects the maximum value of 10 columns:(select max(val) from (values ([Column_1]), ([Column_2]), ..., ([Column_10])) as [Values](val))How do I achieve this in a Prepare Lakehouse custom transformation?I could build the mother/father of all massive case statements, but I’d prefer something simpler and more elegant … if possible!?

I have a client that wants to explore a significant rework of their Prepare instance. They want to duplicate their existing Prepare instance in Dev to create a sandbox instance. Is there a recommended approach to achieve this? They are using version 6898.1

Somebody can explain how to use the custom editor feature?Is it possible to use SSMS to edit the t-sql scripts?How must be setup it?