Ingest

We are running TX version 6536.1.Our data source is AX Dynamics 2012 on premise. We have the issue both with the SQL Server connector and the OLEDB connector.After synchronizing, we can't select any tables with right click on the data source → Select Tables → Search.It also takes > 30 minutes before the table selector pops up.Our only option is to select all tables.The Transfer task - Select table option does give us the option to select tables. However we can't select any individual columns. Neither does this give us the option to define any incremental selection rules or row filters.

Hello, TimeXtender: 20.10.40.64ODX: 20.10.31We are using the ODX and TimeXtenders SQL database connector to get data from a synpase database. Sometimes the synpase databas seems to have problems, where a query runs and doesn’t complete (a seperate issue we are working on).However what happens in TimeXtender in the ODX is that the transfer task keeps running indefinitely and never times out, leading timextender execution package to run but just sits there waiting for the odx transfer task to finish. However nothing fails anywhere, so we do not get an alert.We need some way to be able to capture this.Is there anyway to set up one or both of these things in TimeXtender or if you have any other ideas?1. Kill the transfer task if it takes more then X minutes2. Get an alert on failure or long running taskThank you! 

Hi,I try to connect data source to ODX from TimeXtender Portal. I tried with different data provider (Postgresql and CSV). Both data source are stored in the same machine as ODX server and TimeXtender Server. While I got the error of “Test connection request timed out. Please make sure your ODX server is running and try again.”.The TimeXtender server and ODX server are both running. I checked the connection with command, and it is ok. Test-NetConnection timextender-repositories.database.windows.net -Port 1433

HI Team, Please can you help me understand how to connect TX with common data service connector? I am trying to bring the data from D365 field service to TX. With Power BI i can use common data service connector and extract the data from FS. But i want to use TX to do ETL. Please help!!

Hey all,  We use incremental load in the ODX on a datetime field and we use an offset of 3 days (again in ODX). Now we've noticed that when we load a table from ODX to the data warehouse (even when full loading) that the _R table can contain duplicate records. In our Data Lake each increment is a parquet file of the last 3 days of data, since we load every day these files can overlap by 2 days. I believe this overlap is the cause of the duplicates in our _R table.  What happens next is that the Error report gets spammed with “Violation of primary key constraint” (which is true). But our Valid table does have the expected result. A full load transfer task solves the issue, but of course we do not want to do that all the time.   Is this the intended design? Is there a page that explains how the ODX handles these parquet files before inserting them into the Raw table? Any suggestions or insights on how to handle this issue would be greatly appreciated.

EditOffset value in incremental rule is not available for my Tx version I suppose. UpdateDate is a DateTime column and my UpdateTS an Integer. I’ll see if I can combine them in a single field somehow in the backend.TX20.10.35.64I try to set up incremental loading with a separate Date & Time column but can't seem to find the right way. Found an older source below indicating it's not possible, since it results in a OR clause instead of a AND clause.I am also not able to set up a distraction on my date column to say UpdateDate - 1 day for some reason. Is there another way to do this in the meantime instead of only the date column?Is source based incremental load possible on separate date and time fields? – TimeXtender Support  

I have a new Azure VM that I am installing with the latest version of the ODX server in preparation of upgrading from TX 20.10 to the new version.I have a new adlsgen2 storage account available as storage for the ODX. A service principal (enterprise application) in the Entra ID is created with a (confirmed working) secret.I have added the ODX Instance in the portal with the following configuration:  The app reg has Owner rights on the adlsgen2:  Still I get a 403 error when trying to create the storage container (the container does not already exist):  

Hi team,  How is the process to set my account as TX administrator? Now I am a user and I can't run ODX 's transfer option or create a new schedule

 I am trying to set up access to some tables in the ODX. when I try to add a role and search for an Azure AD user I get the error: Service request failed: Code: Authorization_RequestDenied ...Module: TimeXtender.ODX.EngineTimeXtender.ODX.Engine.ODXFaultException   at TimeXtender.ODX.Engine.ODXEngine.SendServiceRequest[C,T](WcfServerSettings serverSettings, Func`3 func)   at TimeXtender.DataManager.AddODXSecurityRoleWizard_MemberSelectStep.<>c__DisplayClass11_0.<SearchClicked>b__0()   at TimeXtender.DataManager.ConnectingThread.ExecuteConnectingThread(Object dummy)Service request failed: Code: Authorization_RequestDenied ...Module: timeXtenderTXModelInterface.ExceptionWrapperException   at TimeXtender.DataManager.ConnectingThread.HandleError()   at TimeXtender.DataManager.ConnectingThread.Execute(String title, Int32 progressSteps, List`1 actions)   at TimeXtender.DataManager.AddODXSecurityRoleWizard_MemberSelectStep.SearchClicked(Object sender, EventArgs e) This setup is a V

We are facing a problem with a ODX data lake storage table. The problems are appearing at two points in our Data Warehouse:The ODX transfer tasks gives the warning “Table folder X is used by another table with id Y” When using the ‘Preview’ option from TX interface in ODX storage we get the error “The azure data lake gen2 data storage table folder has invalid meta data. The ODX must transfer the table again”Using ‘Synchronize objects’ on ODX server does not do anything at all. Moreover, executing the table in DSA still works without errors and seems to contain new data. Also, in our Azure storage account we can see the transfer task adds new versions of the table every day (scheduled transfer task) even though it gives us the warning 1).By the way, the table ID that the error on ODX transfer task gives us does not correspond with the TableId that we can see in the _model.json file in the corresponding folder in our Azure storage account. Where should the table ID from the first error p

Hi, I have created and populated a small table in Dataverse using a Power App.There are no problems accessing and reading data from the table with Power BI Desktop.But I don't really understand how to use CData's connector (Microsoft Dataverse (23.0.8749.0) 64bit). Does anyone know which fields to fill in for the authentication itself? Username and password or equivalent. BRAnders

Hi,I have a working connection to an S3 bucket with CSV files, using the CData CSV connector.I would like to create a solution where I read only the latest file added to the bucket.Calling the stored procedure “ListFiles” (https://cdn.cdata.com/help/RVJ/ado/pg_sp-listfiles.htm) in a query table returns a table of the files, with filename and created timestamp.Is it possible to call ListFiles in an .rsd file? If so, I think it would be possible to access the created timestamp in another .rsd file using a query slicer which can be used to get the latest file.Have anyone solved this problem, or something similar? I am using the latest version, 6512.1. 

We are using the Azure Data Factory - PostgreSQL 14.0.0.0 data source connector and cannot run the synchronize task. We have selected ‘All tables’ in the data source setup but get an error in ADF:“Failure happened on 'Source' side. ErrorCode=UserErrorInvalidValueInPayload,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to convert the value in 'queryTimeout' property to 'System.TimeSpan' type. Please make sure the payload structure and value are correct.,Source=Microsoft.DataTransfer.DataContracts,''Type=System.ArgumentException,Message=Object of type 'System.Int64' cannot be converted to type 'System.String'.,Source=mscorlib”The error seems to be related to the ‘Command timeout’ setting in the TX portal, which we set to 1440 but ADF wants it to be a ‘System.TimeSpan’ type?When we want to use ‘Select tables’ on the ODX data source the screen runs ‘Search tables’ for a split second and then the results remain empty, no tables appear at all.The Linked Ser

I have a rest interface loading via ODX in V20 into an Azure Data Lake.Recently an infrastructure change to the server made the Rest Endpoint timeout (it has an IP whitelist and the server IP was removed).This resulted in the Synchronization task determining that there were no tables available to load, even though there are multiple custom RSD files that have been generated and the CData Rest driver has been configured to Never generate Schema files. The IP has been restored to the whitelist and the Rest interface is working again (after multiple days of the ODX loading nothing). Unfortunately the Meta Data for every table has been flagged as invalid. When I run the ODX load every table comes back with an error that the table has already been used by a table with a different GUID. And If I run the synchronize task on the ODX it deletes the mapping of every table connected to the particular ODX source (40+ tables).I cannot even run a preview of the data from the Gen 2 storage through Ti

Hello,Our ODX gets stuck whenever we make a request to it (be it synch, transfer, or viewing a log). Then we see the error message in attachment and usually we see a second ODX all of the sudden. Any idea what may be causing it? 

Is there a way to get the timetamp of when a table was transferred from source to ODX Server? Like the DW_Timestamp. The only system field i can find is odx_batch_number. 

Is there a way to remove tables from ODX storage that have been deleted in the desktop, similar to the SQL Database cleanup tool available in the DW? My use case is when SQL Storage is used for the ODX.The “Remove unused object” option does not drop tables it seems. The Storage Management Task is good of course, but we can’t use it for deleted data sources.Let’s say I have transferred a large table hundreds of times into my ODX and then deleted the data source. Can I remove them without having to drop the tables manually in SSMS?Best,Pontus 

Hi community,I’ve a question about the ‘PK’ table that is created in the ODX when you enabled the ‘Handle primary key updates’ and ‘Handle primary key deletes’.I’ve noticed the ODX database (SQL Database) is growing rapidly in disk space after enable an incremental load schedule to reload data every 5 minutes. When I used the default SQL ‘Disk usage by top tables’ I noticed the PK tables are the biggest in terms of disk space.My PK table of the GL Entry table contains 4.5 billion rows! And is 95 GB. While my DATA table contains only 118 million records.When I query the PK table and filter the primay key in this table on 1 value. This value is saved 75  times in this table. For every odx_batch_number (odx_batchnumber 0 – 74). Is this normal? I think it is a bit strange that my PK tables are the biggest tables in the ODX in terms of disk space.Even when I run the storage management task. It doesn’t clean the PK table’s. It always contains the primary key for every odx batch load.The cust

Hi,I’m trying to load an Excel file from a OneLake. As OneLake is a rebranded version of Data Lake Gen2, my assumption is that it should be possible to use existing CData features to load data.I have a working setup where we use CData’s Excel provider (version 23.0.8585.0) to load an Excel sheet from a Data Lake Gen2 I explicitly configured the connection type as ADLS Gen2. Actually I was hoping that this configuration would work after changing the [Azure Storage Account] and [URI] to the OneLake parameters, but unfortunately that doesn’t work Error message:  Looking into the logs, I see that CData tries to connect to *.dfs.core.windows.net. However, the uri of OneLake is *.onelake.dfs.fabric.microsoft.com. Manually entering the abfss:// uri did not work here Is it correct that CData is not yet compatible with OneLake, even though techniques under the hood are similar? Documentation on https://cdn.cdata.com/help/RXJ/ado/default.htm does not mention it (yet).Did anyone manage to load da

We have observed an inconsistency in the field order when adding a new table from the ODX to the DSA. Depending on whether we use the "Add New Table" or the "Add New Table with Field Selection" option, the fields are either randomly arranged or aligned with the source database.When we use “Add New Table” we notice that all fields are somewhat randomly arranged. When we use “Add New Table with Field Selection” and then check all the boxes the order is exactly as in the source database (as expected).  How come that there is a difference? Is this on purpose? Thanks (TimeXtender 6346.1)

Hello,We’re getting the following error when executing a table in DSA.ODX object synchronization detects no changes, and our ODX source is synched as well.I’m guessing the ORDER is an order by… Any idea what causes it? Thanks

Hi, Please can you suggest on the following : Export to Data Lake is depreciated by Microsoft in Oct 2024 this year. So, what is the best alternate solution to extract the data from ERP FO solution for reporting? How to extract data from Field Service & CE ERP solution? OData connector is not good option as it is very slow & I cannot build incremental load on this. 

Hi, I created a field StorageReportIndicator in a table to rank the records in the table. This rank is a SQL window function RANK which combines table fields initially loaded from the data source (StorageUnit and ReferenceDateTime below) and lookup fields via table joins (DSA.Datum table, fields WeekKey and DayOfWeek).  Goal is to have a rank to determine the first record of each week for each storageUnit. The outcome afer execution is that all records have rank 1 which is incorrect. I verified this with the same SQL statement in Management Studio.Could it be that this window function does not have the right result since the lookup is done after the rank is calculated? Note, the table is not historical and incremental. Anyone with experience with this case and is there a proper solution?

As shown in Figure Syncronize Task is sucessfully completed, but Transfer Task gets an below error “System.Data.SqlTypes.SqlTypeException: SqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 ”PM even though my date range starts from 2002 and ends at 2023.Even though after suggestion by @Christian Hauggaard i have made changes in setting of Date Conversition and selected MS SQL Min/Max option, but once i change the setting then i gets the below error for same process,“System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out”(This error is geeting for those tables who has Date Column, and date column has such data (for example 03/01/2022 6:42 PM ) )​​​​​​​What are steps to solve above error? 

Hi all, I have an data source that gets daily refreshed in the ODX. With some tables, I have encountered the following error. Does anyone knows how I can resolve this? Thanks in advance!