Ingest

I have a need to create Managed Queries to sub-select data ranges from a large table source to improve load performance.This is an on-premise SQL instance going to a Gen 2 Storage account in Azure.We have selected ADF because it provides the best performance with the transfer. We are still finding that we have performance issues and so want to break 2 of the larger tables into sub loads so that we can create multiple tables in the Gen 2 storage to leverage parallel loads into the Prepare instance.I was going to use managed queries to accomplish this but it does not appear to be an option on ADF Transfer. Can you confirm?

I have just installed TimeXtender 6892.1 on a client’s site and all of the jobs that call Ingest tasks are all marked as invalid. indicating that “The instance “DataLakeDev” has an invalid repository version - the instance is of a later version than what is supported by the application.Both the Ingest engine and the Prepare / Desktop / Execution engine are 6892.1I have opened every instance and allowed TimeXtender to upgrade the instance in the repository. This is only happening with jobs that execute ingest tasks across both of our ingest instances (dev and Prod). If I try to create a new job it will not display any ingest tasks to add to the job, so I cannot recreate the job.We are using Exmon Orchestration and it still seems able to execute the jobs even though they are marked as invalid. So we are able to run our production loads thankfully. But this needs to be resolved ASAP.

I'm busy with the ingest service configuration, I could sign in, next step is to connect to an instance, that goes well. If i press next the message appear:taking the instance lock, please wait….and then the error message: A network related or instance specific error occured while establishing a connection to Sql server. the server was not found or accesible. my vm for the ingest service configuration can connect to sql instances in my own tenant/resource groups with SSIS. What server/database is the reason that the message appears, and how to solve (ports, rights?)  

HiI have an incremental table with soft deletes enabled in my extract layer (data area). Additionally, I have set up an incremental rule for the SQL data source in the ODX. I was wondering if an hard delete occurs in the SQL data base, will the ODX still be able to detect this such that the incremental table in the data area can mark the record as TombStone?

I have been struggling with this since last week and have tried different methods to solve this.ProjectNumber in my RSD is defined as string but still when i get 123ProjectNumber456 as projectnumber i get an error2025-01-24T15:27:32.136+01:00 3 [153|Q-Id] [HTTP|Res: 1924] {"description":"Budgettype 'approved' does not exist for projectnumber '123ProjectNumber546'"}2025-01-24T15:27:32.136+01:00 2 [153|Q-Id] [HTTP|Res: 1924] HTTP/1.1 404 Not Found, 107 Bytes Transferred2025-01-24T15:27:32.136+01:00 2 [153|Q-Id] [HTTP|Res: 1924] Error: HTTP protocol error. 404 Not Found. Stack: at dwo230y.w.De(String )I have tried sorting out the given project with this code but it didnt work <api:if condition="[touchIn.cleanedProjectNumber | contains('123ProjectNumber456')]"> </api:if> <api:else> <!-- Set revisionType when condition is true --> <api:set attr="detailsIn.revisionType" value="approved"/>I have also tried the follwing bu

In the 20.10 version we have a data source with a .Net Client Data Provider with the settings below: In the SaaS version we have the same source but the network library field is not a drop down and adding TCP/IP or TCP/IP (DBMSSOCN) results in an error that the provided network library does not exist.  What is the value to use when I want it to be TCP/IP?

Getting an error when trying to reterive data.I have created a nested statementapi:set attr="detailsIn.URITemplate" value="[urlbase]/api/Budget/{projectNumber}/currentbudget?api-version=1.0" />I think its because of a blank space after the project number 123456All other projects in TX log comes without a blank space2025-01-23T11:48:43.178+01:00 2 [JSON] Reading JSON file at https://baseurl/Budget/123456 /currentbudget?api-version=1.02025-01-23T11:48:43.184+01:00 2 [53|Q-Id] [HTTP|Req: 1457] GET https://baseURL/Budget/123456 /currentbudget?api-version=1.0I have tried to remove blank spaces with following code but its not working.Anyone has a suggestion? <api:script method="GET"> <api:call op="jsonproviderGet" in="touchIn" out="touchOut"> <!-- Remove all blank spaces from projectNumber and store in a variable --> <api:set attr="cleanedProjectNumber" value="[touchOut.projectNumber | replace('\\s+', '')]" /> <!-- Use the

Hi,I got a data source with multiple databases(22) with equal tables, views, etc.TX ver 20.10 I was able to add databases in the "Additional Connections" for the “SQL Server data source”.  How about the new TX version.. can I do the same? It would save me a lot of work when selecting tables and setting up incr load rulesregards,Bjørn A. 

Hi, I must use tenant_id (company specifict) when authorize my endpoint with OAuth2 (credentials in body and request header).Parameters are client_id, client_secret, tenant_id, etcI really dont want to create a connection for each of my tenant_id's. So my question…How can I use dynamic values to update authentication for each tenant_id? regards,Bjørn A.

Question; I have a project that contains the CDATA Google Ads connector. I need the view ShoppingProducts that was added recently to the connector:Unfortunately I can't find in in Data Selection. Is seems to be added in version / build 9053 of the connector but TX shows me version 9036 is the latest:I have resynchronized the source but as you can see below it’s not in the list:Is the latest CDATA connector available in some way?BR\Dirk

Hi,TimeXtender v.6848.1Can we use Sql version Microsoft SQL Azure (RTM) - 12.0.2000.8   Oct  2 2024 11:51:41   Copyright (C) 2022 Microsoft Corporation as our ingested database?I keep getting this error loading the tablesThe execution failed with error:Exception Type: Microsoft.Data.SqlClient.SqlExceptionMessage: Reference to database and/or server name in 'xxxxxx.sys.extended_properties' is not supported in this version of SQL Server. best regards,Bjørn A.

HiJust upgraded existing TX ingest and Data Integration to latest versions 6848.1 on dev server and now I get a failure when running my ODX Data Source steps, specifically it seems just what used to be know as the synchronise step but now called the import metadata task. Originally each data source has 3 steps, synchronise, transfer and storage management, and they used to run fine. Now I get seemingly random job fail errors on what looks like so far just on what was the synchronise step.If I run the failed step manually it processes without issue. However, when I run the entire ODX job I have set up it seems that one or more data source sync steps now fails. The execution failed with error:Exception Type: Microsoft.Data.SqlClient.SqlExceptionMessage: Transaction (Process ID 76) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.             Stack Trace: at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException e

This is a follow-up of Using XML to ingest data, that i have managed to solve.I need some help with creating a nested statement. The first rsd which lists out all the IDs is this:<api:script xmlns:api="http://apiscript.com/ns?v1" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <!-- See Column Definitions to specify column behavior and use XPaths to extract column values from XML. --> <api:info title="contract" desc="Generated schema file." xmlns:other="http://apiscript.com/ns?v1"> <attr name="contractid" xs:type="string" readonly="false" other:xPath="/Envelope/Body/contracts/contract@contractid" /> <attr name="description" xs:type="string" readonly="false" other:xPath="/Envelope/Body/contracts/contract@description" /> </api:info> <api:set attr="DataModel" value="RELATIONAL" /> <api:set attr="URI" value="https://my.soap.endpoint/service.asmx?WSDL" /> <api:set attr="PushAttributes" value="true" /> <api:s

Anyone have a suggestion about how do i use this XML to ingest data into ODX?<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <soap:Body> <ExportDocsProjectWithTendersResponse xmlns="http://publicprocurement.com"> <ExportDocsProjectWithTendersResult>{ "ProjectBodies": [ { "ExternalId": "666666", "DefaultEvaluationModel": "LowestPrice", "DocumentNumberingSchema": "Numeric", "SectionNumberingSchema": 0, "RequirementNumberingSchema": "Alphabetic", "Metadata": [ { "Name": "ProjectName", "Value": "Ingesting Data into TX" }, { "Name": "Reference", "Value": "123456" }, { "Name": "ContractType", "Value": "XML" } ], "DataFields": [ { "ExternalId": "1",

Hi, I have a customer for whom with PoC, we need to connect Oracle Data Source using API. Need help on either documentation Or how to connect the same. Anyone has experience to connect, please share.

I wanted to ask if others have experienced issues with the TimeXtender_Ingest_Service_6823.1.For the second time, I’ve encountered problems with opening the ODX, where it gets stuck during the process. When attempting to restart the Ingest service, it remains in a "stopping" state and fails to restart properly.To resolve this, I’ve had to manually force the TimeXtender_Ingest_Service_6823.1 to stop using the Command Prompt. Once the service has been forced to stop, I can restart it successfully, and afterward, the ODX opens without issue.Has anyone else encountered similar problems? If so, are there any recommendations or fixes to prevent this from occurring again?Looking forward to your insights.

Hello,I'm facing an issue with how TimeXtender handles special Swedish characters (å, ä, ö) when querying data from an Oracle database through the ODBC data source. I have a view that contains a column with the data type (VARCHAR2) in the Oracle DB. When loading the data to TimeXtender the special characters appear as question marks (å), a (ä), and o (ö).  However, when I create a Query Table and use CAST(CF$_PLAN AS VARCHAR2(1000)) on the same column the special characters are retrieved correctly to TimXtender.   I have queried DUMP from the database to check that the column’s special characters are stored with proper UTF-8 encoded values, and they are.I have also checked the NLS settings for the Oracle database, the TX application server and the ODBC connection.Is there a way for these special characters to be retrieved correctly to TimeXtender without having to use CAST in Query Tables?I use TimeXtender Data Integration 6822.1 and TimeXtender Ingest Service 6823.1. The ODBC Driver u

Its my first SOAP ingestion.The XML looks like this <?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <soap:Body> <ListPublishedFilesResponse xmlns="http://something-something.com"> <ListPublishedFilesResult> <idbdata xmlns=""> <originator> <company name="ABC Corp" url="http://www.abc.corp" /> <contact name="ABC Corp" email="post@abc.corp" /> </originator> <generation date="2024-12-09 10:43:09" dateutc="2024-12-09 09:43:09" /> <contract origin="tendsign.com" tendsigncontractid="2649113" name="Protocol" referencename="1 (ABC-01)" startdate="2018-07-01 00:00:00" startdateutc="2018-06-30 22:00:00" enddate="2025-0

Dear Sir,We are looking for following.E-mail notifications when an Ingest ODX Data Source Data Transfer Task is failing to load the data from the source application/database.Is this possible in TX version 6766.1 ?Or do we need an upgrade. Regards from Venray.Arthur

Hello,We are using TimeXtender classic ODX server (20.10.51) to ingest data. Some of our data sources are located in Azure in form of Azure Synapse / Azure SQL.Are there any guides on how to setup data ingestion from Azure to on-premises ODX server?I.e. what are prerequisites from Azure side (service principal / managed identity?) to connect and what connectors should be used from the ones available in ODX server?I know this is possible from cloud version of TX, but is it possible to do so from classic version? Thank you

Hi All, I try to add my Fabric LakeHouse as a new source in TimeXtender version 20.10.50.64 but I keep getting an error when I try to read objects from datasource eventhough it says connection is succesfull.Data source settingsI copy the SQL connection string from my lakehouse and put it as server name. When I do "Read object from data source” I get the following error. The weird thing is that when I try these exact same stept in SSMS I can see the table from the source with the data in it. Am I missing some steps? 

I’m trying the new timextender excel data source but I keep getting errors. I have upgraded to TimeXtender Ingest Service 6823.1 and TimeXtender Data Integration 6822.1.I am trying to load a really basic excel file. First I tried the ‘Infer data types’ option. It succesfully detected the datatypes but I still end up with an error in the last step of the metadata manager.Service request failed: The given value of type String from the data source cannot be converted to type nvarchar of the specified target column. (System.InvalidOperationException)It does not say which column is the issue.After this I tried to disable the ‘infer data types’. It shows all my source columns as text but I still end up with the same error. Lastly I tried to override the data types to varchar(max). Still the same error. Any ideas on what I am doing wrong here?  

Hi,Have anyone succeeded with a Soap 1.2 call from ODX?We have a source that only have a Soap interface. I've done a lot of testing but so far very little success.For the moment it complains about my content-type header. Don't know why. I'm using the rest connector. Perhaps I should use a xml connector instead. If anyone could share a working example I would be more than happy. Perhaps it's not possible at all. I'm not sure. RegardsAnders Bengtsson

Hi, I have a unity catalog in azure databricks where I want to connect to from Timextender (20.10.44), I installed the Cdata connector for Databricks (2024) and am able to connect via test connection (succesfully connected) but when i try to save TX has issues with the length of the (Protocol ?) version which is empty but also using 1 till 8 which are available in the drop down list all result in the same errorWhat do I need to do in order to get his working? Kind regards,  

Hi I want to download the latest version of TDI and Ingest Service which is 6823.1. But when I use the link to download, it still gives 6822.1. Can you please share the link for the same to download.Also can someone share the link to download 6766.1 with Ingest Services.