Ingest

Hello,My Azure SQL ERP source does not have a lastmodified datetimestamp on the tables. We’re investigating the options to enable change data capture (CDC) or change tracking (CT) on the biggest tables.How would I proceed with further setting this up with TimeXtender? I can't find any related manuals, blogs or sources for this.Thanks in advance! 

I'm currently loading Parquet "tables" in TX Classic with the following setup: Using Synapse Serverless to read Parquet files and expose them as external tables Connecting via SQL Server authentication This works, but I’m looking for a better approach.Is there a way to read Parquet "tables" directly in TX Classic? The Enhanced Data Source seems limited to reading or aggregating files, not treating them as tables.Alternatively, could I replace Synapse Serverless with Fabric Serverless? The main blocker is authentication—TX doesn't appear to support SQL providers with service principal (OAuth) authentication, only SQL Server authentication, which we’d prefer to avoid.Any suggestions, recommendations or workarounds would be appreciated. 

I’m getting this error in the Sign In proccess in the new version 26.3.0.1  

Hi,We have a Contact Centre based on MS Teams (quite a new solution for us) and now they want to start gathering data and follow up the number of calls per operator and such. The only option that the service provider provides for us to get a hold on that data is to set up an Azure SQL database that they can send the data into.Since we have all our other databases locally, I was looking into how to set up a connection to such a database in the portal but didn’t find any specific option for that. Should I just use a “regular” SQL connection with the Azure server name and database name with Entra authentication or how do I go about this?Anyone who has done this before?Any information is appreciated. :)

I have encountered an issue after upgrading the TX Enhanced REST Connector from version 1.10.2 to version 1.15.0.The problem did not occur when using version 1.10.2.Our environment consists of a single TimeXtender server hosting both DEV and PROD environments.Since upgrading to version 1.15.0, i am no longer able to create new Data Sorce using TX Enhanced REST Connector v. 1.15.0 in PROD (DEV works fine). However, creating connections with version 1.10.2 continues to work without any issues in both PROD and DEVI am a bit reluctuent to re-/uninstall 1.15.0 because i have a datasource in DEV with 15 endpoints that use 1.15.0. 

I am experiencing an issue that I have encountered a few times before in TimeXtender Classic. Previously, the problem was resolved either by restarting TimeXtender or by recreating the affected tables. However, this time I have spent a few hours troubleshooting without success.I have two tables in ODX (BU) that need to be copied to corresponding tables in DSA.The situation is as follows:The source tables in ODX/BU are populated with data. The tables are correctly mapped in DSA. When I perform a Deploy (or Execute and Deploy), the tables are generated in DSA as expected. However, no data is transferred to the DSA tables. The tables are created, but they remain empty. Neither the BU tables nor the DSA tables are guarded on Deploy or Execution.I have verified the mappings and deployment configuration, but I have not been able to identify the cause of the issue.Have you seen this behavior before, or are there any logs or settings you would recommend checking?

Hi Everyone, We want to use Fabric lakehouse as ingest layer, and did the setup according to this article: We are running TimeXtender version 6898.1We created the lakehouse form Fabric, not by using the ‘Create Storage;  option from TDI. When we test the connection in TDI, everything looks fine:However, the metadata import task fails with following error: The execution failed with error:Exception Type: Azure.RequestFailedExceptionMessage: Service request failed.         Status: 400 (BadRequest)         ErrorCode: IncomingOperationUntrusted                  Headers:         x-ms-error-code: IncomingOperationUntrusted         Access-Control-Allow-Origin: *         Access-Control-Allow-Headers: REDACTED         Access-Control-Allow-Methods: REDACTED         Access-Control-Expose-Headers: REDACTED         Strict-Transport-Security: REDACTED         X-Content-Type-Options: REDACTED         x-ms-root-activity-id: REDACTED         Content-Length: 0         Date: Thu, 06 Mar 2025 14:57:14 GMT 

I believe I have read the documentation, in case I missed something. My bad. Im trying to load csv files from the root of a S3 bucket. The 'test connection’ in the portal turns green but when trying to synchronize in timextender it tells me I have to enter a value for ‘path’.I have tried these so far: /.mybucketname/inbound/s3://mybucketname/inbound/But timextender keeps throwing errors saying the directory does not exist. I can succesfully connect to it and download the csv's via Python so Im quite sure the setup is correct.  Does anyone know what to fill in for ‘path’?

Hello,I have a TimeXtender REST data source where some loading tasks sometimes keep running infinitely. This causes issues on my pipelines where I don’t get a message/monitor from, as it doesn't error.I see that in my REST data source settings I don't have a command timeout like for SQL data sources. Is there a way I can put a command timeout? Would it be this Timeout setting in O&DQ?  

A JSON response looks like this [{"ref": 123456,"dim_1": "A","dim_2": "B","peroid": 202601},{"ref": 5E+20242959,"dim_1": "A","dim_2": "B","period": 202602}]The second ref (5E+20242959) is a problem - its neither STRING nor an INT.Ref is defined as NVARCHAR in BU, but as you can see the value is not between “”.I have tried the following approaches under, still i get the same errorInput string '5E+20242959' is not a valid number. Path '[56935]['ref.53e92a42-b13c-4062-87b4-6bd5a89547b1.1992753']', line 1, position 36741783.How do i solve this?Here are the approaches i have tried:Omitting Ref in Table FlattningIn table flattning, i have left it out. I have the other three values, but still landing returns an error:Defining the value as NULL in table flattningField transformationCondition on column ref in BUI have defined the following condition on the column, still i get the same errorCASE WHEN [ref] LIKE '5E+%' THEN NULL ELSE [ref]ENDScript ActionsI have created a script th

I run into this error while loadning csv-files from our s3 bucket.The issue is that the number of fields differs from time to time. We would like to ignore that and read these fields if they exists and otherwise not.Is there a way to do this or is this just an exception in CsvHelper-lib that isnt accessible for me to configure? The error:Executing table [csv].[VOF_UTF_Planeradebesok__aggregated]:failed with error:CsvHelper.MissingFieldException: Field with name 'Planerat besök - konfidentiellt' does not exist. You can ignore missing fields by setting MissingFieldFound to null.IReader state:   ColumnCount: 21   CurrentIndex: -1   HeaderRecord:["Användare - ID2","Användare - personnummer","Användare - profession","Avbruten/inställd","Brukare - ID2","Brukare - personnummer","Planerad resurs - namn","Planerat besök - dubbelbemanning","Planerat besök - ej utfört (anledning)","Planerat besök - sluttid","Planerat besök - starttid","Planerat besök - typ av brukare","Planerat besök - utfört","P

Hi all,We have regular ODX loads that periodically fail because of schema changes in Business Central (e.g. a new column added). The fix is straightforward: sync the metadata and rerun.I'm considering just adding the metadata sync step before every ODX load to avoid these failures altogether. Is there any downside to syncing every time? Could it cause unexpected behavior, longer run times (probably only a few minutes?), or anything else worth watching for?My instinct is that it's still better than having a load fail over a single new column, but wanted to check if this should be standard practice or if there's a reason this isn't already standard practice.Interested to hear your thoughts.

Hi,We're on TDI version 7387.1, using a TimeXtender REST Data Source in an Ingest instance to pull data from a paginated REST API (cursor-based pagination + a changed_after query parameter driven by a Dynamic Value SQL query).We want to run a post-execution SQL step (updating a small watermark/control table used by the Dynamic Value query) only after the transfer task has completed successfully - not before, and not if the run fails partway through.Our specific questions:1. Is there a supported way to attach a pre- or post-script (Script Action) directly to a REST data source's transfer task in an Ingest instance, similar to how Script Actions work on tables in a Data Warehouse/Staging instance?2. If this isn't natively supported on Ingest transfer tasks, what is the recommended pattern for reliably running a SQL step only after a specific ingestion task succeeds, when scheduling through Orchestration? We've seen the "dummy table with pre/post script" workaround mentioned in the commun

Hello. This document has the ports which need to be opened. Configure your Firewall | Community It doesn't reference Fabric Lakehouse.. are you able to advise what we need to open for the ingest service to communicate to Fabric Lakehouse?Thanks Tim

I have a large table with about 50 columns. There is column called projectNumber. This column, i am unable to override.This column is varchar. However in TX with Enhanced Drivers it shows Int32.I have overridden it to Text It even says so in Data type overrides on the DataSource I have even tried adding and removing it in Table builder  

We have a client with a Rest Datasource that does not use URL parameters. Instead they use the post body to pass a series of filters and conditions.We are using this post body to implement Incremental loading from the datasource. It seems to be working correctly in so far as the rest source is definitely loading incrementally.However the Ingest instance does not seem to recognize the tables in the Gen 2 Storage as incremental.We have defined the incremental several times, deleted the entire folder in the Azure Gen2 storage. Still the same result.if we drag the table over to a data area the source in the Ingest instance is not flagged with an “i” indicating that it is incremental.Is this because we are passing the incremental variable through the post body and not as a query parameter?if this is the case is there a work around?The issue we are facing is that we cannot do any downstream incremental loads as TX complains that the source table in the ingets instance is not incremental. Not

Hi,We are consolidating our ingestion setup from one ingest instance per environment to a single shared ingest instance for both DEV and PRD environments.Current situation:ODX DEV → DSA_DEV → SSL_DEVODX PRD → DSA_PRD → SSL_PRDTarget situation: ↗ DSA_DEV → SSL_DEVODX PRD ↘ DSA_PRD → SSL_PRDThe DEV VM is able to communicate with the PRD VM, where the shared ODX instance is hosted.Could you please advise on the next steps required to implement this change?  

Hi,i have a gen2 storage account with a blob container with multiple parquet files. when i do a sync with odx it’s succesfull, the transfer gives me for 4 files the message “ is used by another table”. all other files in the same map go ok. have tried to recreate the blob container, and recreate the parquet files again, same results.Exact message:The table folder 'parquet_taak_status' is used by another table with the id 9a510177-a130-4bcc-aaaa-5abefd42c1b9 how to solve / find the other table that is referenced to? thanks!  

I have followed this guide to set-up an Incremental load on a table in BU:Incremental Loading in a Business Unit (BU) in TimeXtender Classic | CommunityHowever, during the deploy i get an error:Data source Incr LoadsValidation status: ValidMessage: OKStaging Database ODXValidation status: ValidMessage: OKTable: Incr_LoadValidation status: ValidMessage: OKIncremental LoadValidation status: ErrorMessage: Incremental load is not supported on this table.Details:Data source Incr Loads ...TimeXtender.DataManager.UserErrorExceptionIt says in the guide that: When the option is enabled and the table is deployed, TimeXtender performs the following:Initial Full Load: The system fully loads the table. Setting content of the _I Table: This table records the largest incremental field value from the given load I am attachig a video of where i get an error.What am i doing wrong?

Can I preserve my source field names when synchronizeing my fields?I dont want spaces and periods replaces by an underscore, my current workaround is to edit the fields manually when I use the tables in my prepare instance.  

Version: TX 7158.1Hello,I’m currently running into an incremental loading issue where the source system automatically generates invoices in batches. This inserts new rows/Primary keys in the invoice table. It could be that certain invoices are scheduled for a batch. Let’s say from 24-07-2026 scheduled for 29-07-2026.If that’s the case, the new row will get inserted on 29-07-2026 with a datelastmodified of 24-07-2026. Any changes to that document will trigger the datelastmodified correctly, only this process gives this issue. In that case, my incremental loading will completely miss these new document / primary keys (currently -1 day offset).Is there a way to always add new primary keys but only do the updates based on the datelastmodified timestamp?

In TX Enhanced Connector, how do one handle fields in JSON response that has the same name?In the screen-grab below, the fields in marked in red comes multiple times but the values are different.In TX Enhanced Connector, the field always grabs the value of the first occurance of the field i have marked in red.Is there a way to grab the desired value by doing somethng like this:/json/addendumValues/addendumGroups/rows[0]/json/addendumValues/addendumGroups/rows[1]/json/addendumValues/addendumGroups/rows[2]/json/addendumValues/addendumGroups/rows[3] If not, is there a way to grab everything in blue and show in one column?I can then creat a view in BU and grab each saperately The screen grabs are broek here: Understanding and using script parameters in TimeXtender Classic | Community, so i can not fuly understand the documentation

Hello,After Snowflake enabled behavior change bundle 2026_04 on our production account, our execution packages began failing. TimeXtender generates the "SCD Surrogate Hash Key" column as a virtual (computed) column with a NOT NULL constraint.As of bundle 2026_04, Snowflake rejects NOT NULL/CHECK constraints on virtual columns at DDL time, so the CREATE TABLE statement for the TX-generated temporary staging table now fails. The same project ran fine until the bundle was enabled, and runs fine again after we temporarily disabled the bundle.The failures occur only on tables that have History (SCD) settings enabled. Tables without history are unaffected, which is consistent with the surrogate hash key being generated specifically for the history/SCD code path.Environment- TimeXtender Data Integration, product/portal version: 7158.1- Data storage: SnowflakeError messageJavaScript execution error: Uncaught Execution of multiple statements failed on statement "CREATE TABLE ...#<temp>...

I have en endpint with multiple parameters:CaseHandling/External/Cases/GetCases?exportFields=CaseID&exportFields=Closed&exportFields=Position&exportFields=ProjectLevel&exportFields=Anonymous&Key is always exportFields Values are differentIn CData i have no issues getting values in the columns.In TX Enhanced Connection, when i define multiple parametes with key-anme exportFields and different value, only the last is shown: This is how i have defined in TX Endpoint I have tried the whole endoint in Path, which gived empty table  

Dear sir,How can I remap tables to a different ODX instance automatically, instead remapping tables manually for a 100+ tables (V7303.1).Any suggestion is welcome.Thanx.Regards,Arthur