Ingest

Hello, To my understanding TimeXtender determines the data type of a column based on first x amount of rows, depending on your setup in the portal.  The fields in my source  come in as (dd.mm.yyy) for example: "05.03.2021”. I use a Storage Account with a CSV connection.  I would expect that TimeXtender creates a "date” date type out of this. However, is seems that if a field character starts with a '0’, it skips it. In the ODX it shows an Integer value '5032023’.ODX: Depending if a character starts with a '0’ the fields (dates) show up like: It counts for all date fields and costs me a lot of time to create a work around each time. I tried to override the datatype in my data source but without succes. Help would be much appreciated.    

At one of my clients we are using the ODX and they set it up that for every different type of load they added the data source multiple times. So for the normal full loads there is 1 source . For the incremental loads of the same source there is a different source.In the data source selection the include only the tables they need for the full load. and they do the same for the incremental. But from time to time task wil not execute without showing up in the execution log without an error. I have the feeling that because the includes je indirectly exclude the other tables and if they are running at the same time this cancels each other out. In the windows logs you can see that the package then is skipped but it's no where to be found in TX.Does anyone have an explenation about this and/or an idea how we can fix this?My idea was that maybe putting everything in 1 source and then create 2 different transfer task instead of 2 different sources would fix this problem. Would love to hear from

Why does the app registration for the ODX need owner permissions on the resource? I would think that read/write permissions would be sufficient. Why does TimeXtender need the “extra” rights?

The ODX server stopped (per 25/8) synchronizing it's job status with the portal.Restarting the ODX Server does not solve the issue. Any tips?

Hi team!  I am contacting you because I am trying to transfer data from MariaDB to ODX and I received the following errorfailed with error:System.FormatException: The DateTime represented by the string is not supported in calendar  Any suggestion to solve it? Regards Ignacio 

New fields from source are not showing in the Data Movement pane.New fields are added to the source database, the new fields and data are in the ODX, I can see the new fields in the parquet files, but the fields are not showing in the Data Movement pane.I have tried to run the function in ODX “Synchronize Object”I have tried to run the “Synchronize” task in the odxI’m Running Timextender Version 6346.1

Hi All, Is it possible to use an SAP Theobald Extractor with incremental load and that just for 1 table ( or more, but not all the tables? We tried is on one table and we got an error message saying we needed to change all the tables into incremental load.Kind regards, Eric 

Dear all,I'm trying to set up a completely new ODX Server. I've installed the ODX server and made all the required configuration. When th ODX Server needs to connect to its instance i'm getting this error:I've checked all the firewall settings and I've done all the checks in this document: Configure your Firewall | Community (timextender.com). Furthermore we've added the timeXtender.exe and the OdxServerConfig.exe to the whitelist of the Windows defender firewall.The weird thing is that aswel TimeXtender as the ODX Server config can find the instance, but still we're getting this error. Does anyone know what I'm doing wrong? Error occurred while determining instance version or during performing aut. upgrade:A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error

We are running an environment on two servers. One DEV and one PRD server. The instances are copied from the DEV to PRD server, both ODX and MDW. Apparently the jobs are also copied. After refreshing the list I'm adding jobs to run the production ODX. As we can see below I can choose between both the DEV and PRD ODX's : However. If I select the ODX Just Brands Production / JB NAV / Full Load NAV. It jumps back to ODX Just Brands / JB NAV / Full Load NAV, running the DEV ODX. Why is this?

Our machine runs in UTC+2.00, whilst our Azure databases (by default) run on UTC time. Now when we schedule an ODX Transfer task, the interface let's us schedule these tasks in UTC time. The Execution logs of a transfer task are also in UTC time. However, schedules for execution packages are in local time, UTC 2.00, and therefore there logs are also in UTC 2.00.  I guess my question is, why is this the case? Both the ODX & Execution service are services on the same machine with UTC 2.00. Both the ODX as well as the MDW and the project repo are azure db's in UTC.  Why would one use the machine time and the other the azure db timezone? And is there a way to make them the same other than resetting the machine time to UTC? Version 20.10.37

TX: 6284.1Is there a reason why Lineage generated from the Data Source Explorer in the ODX Server includes the downstream stuff in a DWH instance whereas generating Lineage from a field in a Data Area in the DWH instance does not show the ODX part of the lineage?

Hi all,I have some customers that are reloading data every x minutes in ODX Server (V20.10.x). This generates a lot of execution log entries. When I open the Execution Log in the ODX Server, it takes a long time. I think that removing old execution logs will improve the speed of opening this execution log overview window. I have no idea if it is possible to delete old execution log entries?Another option could be that the execution log view first shows the datetime range you like to see logs and then opens the overview. 

HelloI am wondering, how can I connect to Zip File? I searched and asked Xpilot and then I get this answer: To connect a Zip file in TimeXtender, you can use the "File" data source type. But I couldnt find File datasource. Has the name changed or is there another method?Best Regards

I was interested in this link.Data directly from Data lake to MDW (without ODX) | Community (timextender.com) I am not able to access this link which was posted in the below of the message.  https://support.timextender.com/timextender-product-focus-group-123/virtual-odx-capability-to-read-directly-from-existing-data-lake-container-no-second-copy-614 Please advice,  

Hi all,I have a client that has a datalake with parquet files in an Azure Storage Account Blob container (Datalake Gen2). This datalake has not been created by ODX server, but has been created by Synapse (scripted, delta lake parquet files). Since we want to load multiple tables from multiple folders we use the CData Parquet connector v2023 (TX 20.10.41). The issue we have is that we can’t find the right settings for the connector to recognize that the folder we’re connecting to contains multiple tables and that the parquet files are in a subfolder of the table folder. For the connection we use a Shared Access Signature and the connection is working:The name of the container is ‘development’ which we configure in the Base URL. In the URI we point to the folder of the bronze layer, the system and the database: The database folder contains a folder for each table. In each table folder some tables have a a partition folder followed by the actual parquet file. Since the connector needs to

I have below the API request for Power Shell. Now I want to set it up in TX.  How to set up (Basic) Authentication for a request with a token retrieve?    # Force the use of SSL [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12 ####################### # SETTINGS                                                                       # ####################### # REST API SETTINGS # $API_key = "HIDDEN" $API_username = "HIDDEN" $API_password = "HIDDEN" $API_company = "HIDDEN" $API_URL_Login = "https://secure.xxxxxxxxxxxxx.eu/ta/rest/v1/login" ####################### # LOGIN AND RETRIEVE TOKEN FOR UKG Ready                                         # #######################          Write-Host "### Retrieving token"            $headers_login = @{   'Content-Type' = 'application/json';         'Accept' = 'application/json';         'API-Key' = $API_key;      }     Write-Host "- Headers prepared"         $request_login = '{"credentials": {"username": "'

Hi, I've been trying to add new excel file from a new sharepoint group from the same company link to the same ODX server by using CData connector. Is there any setting to add both sharepoint links or make it visible? Now, I can only see the link with group1.sharepoint page examples;https://company.sharepoint.com/sites/group1https://company.sharepoint.com/sites/group2Best!

Hi,I'm using the CData ADO.NET Provider for OData 2022 (22.0.8322.0) to extract Topdesk data.Due to performance reasons Topdesk made some changes in their feed. Resulting in that, since 19th june 2023, 2 larger tables (IncidentDurationBlocks and IncidentSnapshots) cannot longer be extracted without filtering on a date to make the dataset smaller.How can I add a filter for only one table in my extract?Can I create an OData RSD file?I get this error when I try to extract:System.Data.CData.OData.ODataException (0x80004005): [500] Could not execute the specified command: Unexpected response status code: 503.  ---> rds220I.IC: Unexpected response status code: 503

Hi,we have a customer running incremental loads multiple times a day in the ODX Server (using Data Lake Gen2).  We’re reaching the 10.000th incremental run.I’m wondering how the ODX server will handle that 10.000th increment (since the naming of the parquet files allows a number up to 9999).BR.Peter

Hi community, The CDATA Provider for LinkedIn 2022 contains a bug/error which causes it to call an invalid API URI for the connector’s table ‘OrganizationFollowerCount’. The appropriate URI for this table according to Organization Network Size - LinkedIn | Microsoft Learn is:https://api.linkedin.com/v2/networkSizes/urn:li:organization:1234567?edgeType=CompanyFollowedByMemberHowever, CDATA is calling the following:2023-06-12T11:31:21.793+02:00 2 [5|Q-Id] [HTTP|Req: 3] GET https://api.linkedin.com/rest/networkSizes/1234567?edgeType=CompanyFollowedByMemberWhich results in the following error message:2023-06-12T11:31:22.172+02:00    3    [5|Q-Id]    [HTTP|Res: 3] {"message":"Key parameter value '1234567' is invalid","status":400}2023-06-12T11:31:22.172+02:00    2    [5|Q-Id]    [HTTP|Res: 3] HTTP/1.1 400 Bad Request, 66 Bytes Transferred2023-06-12T11:31:22.194+02:00    1    [5|Q-Id]    [EXEC|Messag] Executed query: [SELECT [FirstDegreeSize], 0 AS [odx_batch_number] FROM [LinkedIn].[Organiz

Hi all,We’re trying to troubleshoot some thorny networking issues, and I need to know more about how ADO.net data transfer works behind the scenes - specifically for old-school business unit SQL Server data sources, if that makes a difference. I know for sure is that data is read from the data source and written into the destination table in the staging database, but I’m fuzzier on what happens between the two places. To start off with, I have two key questions: We’re using a tool server to run TimeXtender, and a different server to host SQL Server and the TimeXtender databases. Is data routed through the that tool server when it is read via the ADO.net package? I assume that it is, but we need to know for sure. The other question I have is what happens after the data is pulled from the data source? Is it pulled into memory somewhere? If so, where? The source server, the tool server, the destination data server, or some combination of the three? Is the data ever written to disk anywher

Hi team, In the business units and data warehouses, deletions in an incremental table are identified by system field IsTombstone = 1. You can use this field in a data selection rule to filter out deletions between the business unit and DSA. This field is not present in ODX Server tables with incremental and delete handling enabled. Is there a way to identify records that have been deleted from incremental tables in ODX Server? And if not, what is the purpose of enabling deletion handling in ODX Server? Kind regards,Andrew - E-mergo

I have made a new app registration in Azure for TimeXtender. Provide the rights exactlly as the old app registration but the secret is expired.Error in TX : AADSTS7000222: The provided client secret keys for app 'OLD_APP_ID' are expired.I filled in the datasource and global enviorment the NEW_APP_ID but the error still is there. Restarted the ODX server, removed the pipelines but still same issue. How can i force that it will use the new application ID becaused I changed it on the places it is used. (Also with the new secret ofcourse)

Hi! I am experiencing an issue with ADO.NET transfer times on incrementally loaded tables where the ADO.NET transfer takes the same amount of time regardless of the amount of new rows coming into the table. A full load of the table containing about 45 million rows takes about 25 minutes, on the next incremental load the load time is still the same with ADO.NET taking up around 24 of these minutes. Our current data flow is as follows: SQL server → TimeXtender → Azure elastic pool (this is where all of our TimeXtender databases resides)  Full load - 45 million rows Incremental load - 11 thousand new rows in the _R table  The amount of new rows coming in to the table Has anyone experienced a similar issue? My best guess is that the problem resides in the azure elastic pool where the ado.net transfer is being throttled. Even if I have 0 rows in the _R table the ADO.NET transfer time is the same. Thank you! 

Hi community, I’m using version V6221.1. I’d like to filter all tables in the data source on column ‘dataAreaId’. I try to set this up using the following settings:  Do I really need to set this filter on every table I load from this data source? Or is this a bug?