Prepare

I have been trying to sort out something out,My JSON looks like this:addendumGroupName occurs multiple times. Here, its 4 times but in other cases it occurs between 1 to 50.columns can be one or multiple I am trying to list all addendumGroupName and all columns under each addendumGroupName without doing this:json/addendumValues/addendumGroups[2]/addendumGroupName[0]json/addendumValues/addendumGroups[2]/rows[0]/columns/fremdrift_group[0] json/addendumValues/addendumGroups[2]/rows[1]/columns/fremdrift_group[0]json/addendumValues/addendumGroups[2]/rows[2]/columns/fremdrift_group[0]  So i tried this (with an without [*] at the end of column name): <attr name="addendumGroupName" xs:type="string" readonly="false" other:xPath="/js

We are loading data from a SQL Server (onprem) to a Datalake (ingest instance). Data is stored as parquet, but TX is creating a huge parquet file for each table.  We are using ADF to ingest the data into the ingest instance.  Is there a way to load the data in such a way that when it gets into the Datalake the parquet can be divided in parts?  table_xxx_0001.parquet, table_xxx_0002.parquet, etc. (I know this is not the parquet format it is just an example).  Is this possible? We are having performance issues reading from parquet to AZURE SQL DB prepare instance.  We are using ADO.NET from ingest to prepare.

Hi team I was recently working on a table where I added a hash field and configured it for incremental loading. However, the initial deployment failed because the table was missing the required column for the incremental process. When I attempted to redeploy to fix it, I hit a roadblock: a duplicate key error in dbo.DataFields with the value (ea41fe97-e408-4504-9164-239dfcdee390, 148, 99999999).  

Hi team, I am encountering a timeout issue when moving large tables. The problem seems to be originating from the cleanup stored procedure (clean sp), as smaller tables are processed without incident. Could you please look into this issue?  

I’m migrating a large SQL Dedicated Pool data warehouse to Fabric Lakehouse.I’ve run into Fabric’s workspace limit of a thousand items.What’s the best-practices workaround?

Hi all,I'm working in TimeXtender (TDI) and ran into something I can't quite figure out. When I create a new schema in my Prepare instance, I want to assign it to a table that already exists in that same instance. The schema does appear as an option when I try to link it to the table. However, it gives an error upon deployment (of that table), at least until after I deploy the instance. Is this expected behavior?Or am I perhaps creating them in the wrong order or place? 

Hi community,I created a role in the MDW and added Grant permission on one specific database schema. This schema contains some tables and some Stored Procedures. This role need to be able to select data in the tables and execute the stored procedures. I created my Object Security Setup using these settings:  When I login using SSMS (using the user in the role), I see the correct tables and I'm able to select data in the tables. But I'm not able to see the stored procedures. When I check the permissions on the ‘ExcelAddin’ database schema. Only the ‘select’ permission in granted. Is it possible to grant permission for ‘execute’ as well? Using TimeXtender?

Hi,I have a client that uses TDI 7026.1 and i need to install it on another computer, but on the web I can only find 7050.1. Any possibility to find the earlier version somewhere?/Leo

I’m getting a Lakehouse notebook execution timeout in v7142.1 after exactly 10 minutes.My connection and command timeouts in the TX portal are 1200 and 7200 seconds, respectively, for both Ingest and Prepare so that’s not the setting I need to change … and the workspace’s Spark timeout setting is 20 minutes so that’s also not the one.I’ve copied the relevant part of the error message below.Does anyone know how and where to increase the notebook timeout setting?(the Microsoft links aren’t that useful from a TDI perspective, I think) Start Time : 10/15/2025 2:23:08 PM End Time : 10/15/2025 2:37:01 PM on server: V171D008S -Execute Execution Package Default 'Failed'      An error occurred while calling o5570.throwExceptionIfHave.: com.microsoft.spark.notebook.msutils.NotebookExecutionException: Timeout when exe cell - 15 in notebook TimeXtender_PrepareKOMBI_TABLE_dsa_INVENTTRANS, code length = 39808 and it costs

Hello,Is it possible to Extract data from an existing Lakehouse into a Prepare instance without needing to move it into the TimeXtender Ingest instance? The use case is to avoid replicating the data again, if the customer has already ingested data into a data lake or Lakehouse.Thanks in advanceTim

I have two tables with no unique identifer. There a two posts that mentions NEWID() as a way to create a unique identifier, so it seems that it can be used:How to add a default value to a unique identifier field – TimeXtender SupportHow to Add a Default Value to a Unique Identifier Field in TimeXtender Classic | CommunityI have created a filed in each of my table and made it Primary Key.My question is:What happens if we

I’m planning to change a history table in DSA into an incremental table without removing the history and I want to make sure that I understand some of the settings correctly. The ideal outcome for me is the following:New data added incrementally without the history being maintained Regarding the old history: I want to keep all rows with [SCD Is Current]=1 I don’t want to keep rows with [SCD Is Current]=0 Here are some questions I have about options in Table Settings > Data extraction :Does it make sense for this scenario to have the boxes checked for “Truncate raw table before transfer” and “Empty raw table after data cleansing”? What is the best approach for me to take with the delete options (“Don’t handle deletes”, “Use soft deletes” and “Use hard deletes”)? If I’m fine with new rows being overwritten can I then create some kind of incremental rule to make sure that old history is changed (or at

Hi,I obtain a deployment error when I try to make my table historic. Computed column 'SCD Surrogate Hash Key' in table ' <mytable>' cannot be persisted because the column is non-deterministic. I think this has to do with the excessive amount of columns in the table: 695 columns, all varchar(2000). What confuses me is that when I set the hashing algorithms to debug, I still get the same error  I don't have any transformations on this table, its a straight copy from to ODX.  Is there a limit on the amount of fields a History table can have? 

Hello Timextender and communityI’m looking to understand how Timextender Classic deals with collumn statistics that aren’t tied to indexes. Our project is built on Azure SQL DB.Obviously TimeXtender automatically rebuilds indexes as part of its data cleansing procedures. However, rebuilding indexes only updates statistics on the indexed columns—not on the rest. Performance  relies heavily on accurate statistics for query optimization (especially when MAXDOP isn’t explicitly set), this leaves me wondering: Does TimeXtender have any built-in process or configuration to update statistics on non-indexed columns? If not, are there recommended best practices for managing these statistics in a TX solution—either as part of the deployment or as a post-process step? After dealing with a sudden drop in performance, we’ve observed performance improvements aft

I’m looking for a TimeXtender Fabric Lakehouse (pyspark.sql) expert 😀I’ve got a SQL transformation that selects the maximum value of 10 columns:(select max(val) from (values ([Column_1]), ([Column_2]), ..., ([Column_10])) as [Values](val))How do I achieve this in a Prepare Lakehouse custom transformation?I could build the mother/father of all massive case statements, but I’d prefer something simpler and more elegant … if possible!?

I have a client that wants to explore a significant rework of their Prepare instance. They want to duplicate their existing Prepare instance in Dev to create a sandbox instance. Is there a recommended approach to achieve this? They are using version 6898.1

Somebody can explain how to use the custom editor feature?Is it possible to use SSMS to edit the t-sql scripts?How must be setup it? 

HiLooking to quickly replicate an existing data pipeline in a new prepare instance. Is it possible to clone a table in one prepare instance into another, or somehow export a table from one prepare instance and import into another?I cannot see any option, so my guess is no, but just thought I’d ask.RegardsPaul

I have a table X in the DSA layer from TX and I can't find the script to get the right output.  There is a field FCL that possible have multiple records with different YearFrom and YearEnd. FCL   YearFrom   YearEnd1        2010           20201        2021           20502        2010           2050  I need a data selection rules that only imports from the ODX layer the FCL with the highest YeardEnd group by FCLFCL   YearFrom   YearEnd1        2021           20502        2010           2050  Is there anyone who has a suggestion. The output from is table is userd as lookup table for another table in the same DSA layer. 

This is a simple Deploy and its been running for almost 20 minutes.Whats wrong here?  

Hi, I am facing an issue with duplicate records when using incremental load on a table that receives data via a table insert from another table in the same data area. The incremental load selection rule is based on DW_TimeStamp, and there are three PK in the target table.The issue is that the primary key does not seem to be enforced in the target table, as I am getting duplicate rows in the table. I tested this with another table where data is inserted through a table insert (but without incremental load), and I still see duplicate values for the PK fields and despite setting the table option to "Use instance setting (error)", no error is thrown when duplicates occur.  I have tested this behavior in TDI 6935.1 and 6926.1, and the issue persists. My questions are: 

Dear Sir,In TX version 20 you had a configuration setting Direct Read.I cannot find this setting in the latest and the greatest version in TimeXtender.Question: Is Direct Read still available ? How can I set this up ?Thanx in advance for looking into this. The "Direct Read" option in TimeXtender is particularly useful in scenarios where you need to transfer data between different data areas within the same physical database but using different schemas. Here are some specific scenarios where you might use the Direct Read option:  Large Data Volumes:  When dealing with large tables, Direct Read can be more efficient as it allows for direct data transfer within the same database, reducing the overhead associated with external data move

Hi All, Hopefully you can help me with this one. We have a source table from our POS systems containing sales transactions. There are a few value columns we transfer that are stored as a money data type and we see some weird things happening with the values of the columns. Differences of 0.01 up to 0.0001 are appearing from nowhere in the transfer from ODX to DWH and i don't know how this i happening.The columns that have money data type are called ‘Bedrag', ’Korting’ and ‘NettoBedrag’ Take transaction 52. In the source the values are like this: After transfer with ODX, i looked in the parquet file using ParquetViewer and that looks like this: St

Hey everyone,I have the following question:Is it possible, at the database level, to set the primary key to the one selected in TX (e.g., "Umkod_ID") instead of the default "DW_ID"?The issue arises because we access our table via PowerApps, and since the app existed beforehand, references in the code were already predefined. Currently, the app's primary key is set to the "Umkod_ID" field of the table. However, in our database, the primary key is set to the system field "DW_ID." Since this table is only loaded once and no transformations occur, the system fields are not particularly important to us in this case.We resolved this issue by manually populat