Skip to main content
To homepage To homepage

Ingest

Ask questions about Ingest Instances in TimeXtender Data Integration

  • 277 Topics
  • 1,488 Replies
277 Topics
A
Starter
anders.bengtssonStarter
asked in Ingest
Connector for Active Directory - examples

Have anyone an example on a successful connector regarding a normal Active Directory - not Azure AD. Can’t find any information about it but I guess many have done a connection using TX for that. Regards Anders Bengtsson

16422
Christian Hauggaard
Community Manager
M
Visitor
MichielVisitor
asked in Ingest
Including Excel File Names as Data Column

Hi, Perhaps I’m overlooking something, but is it possible to include (part of) the file names of Excel sources as a data column? The Excel files are combined using the aggregation function. TimeXtender version 6848.1, Excel Provider 22.0.0.0. BR, Michiel

80
M
Visitor
oscarcleve
Contributor
oscarcleveContributor
asked in Ingest
TimeXtender REST API override header

We have trouble with an endpoint in a TimeXtender REST API data source using the Override Headers functionality. The problem arose when we upgraded our TX software to the latest version of TDI/TIS and TX REST from 6848.1/6848.1 and 7.1.0.0. We get this error message: “Failed to execute request for endpoint 'NBABIItemWS'.HTTP Status Code 401 UnauthorizedResponse body:<error xmlns="http://docs.oasis-open.org/odata/ns/metadata"><code>Unauthorized</code><message>The credentials provided are incorrect</message></error>” We use the Override Headers functionality to limit page size for the particular endpoint. It seems that for some reason the authentication token is overwritten when using this feature, and there fore we can not authenticate. We have plenty of other endpoints in the same data source not using Override Headers, and those are working fine, so we suspect that the issue has to do with that setting. We use this combination of versions: Is there a way we can alter the call for the

514
oscarcleve
Contributor
A
Problem Solver
aftabpProblem Solver
asked in Ingest
Need some help with ingesting

This is my JSON { "projectNumber": "123456", "name": "TX", "addendumValues": { "addendumGroups": [ { "addendumGroupName": "G1", "rows": [ { "columns": { "revnr_G1": [ "Rev.1" ], "date_G1": [ "2023-06-21T00:00:00.0000000" ] } } ] }, { "addendumGroupName": "G2", "rows": [ { "columns": { "revnr_G2": [ "Rev.1" ], "date_G2": [ "2023-06-21T00:00:00.0000000" ] } }, { "columns": { "revnr_G2": [ "Rev.2" ], "date_G2": [ "2023-07-21T00:00:00.0000000" ] } }, { "columns": { "revnr_G2": [ "Rev.3" ], "date_G2": [ "2023-08-21T00:00:00.0000000" ] } } ] } ] }} xpath: /json/addendumGroups has multiple addendumGroupName, here i present only two.I am trying to extract values for revnr_G1, date_G1, revnr_G2 and date_G2. The code below only shows revnr_G1 and date_G1.I am wondering why not revnr_G2 and date_G2 (please look away from case sensitivty) <api:info title="addendumGroups" desc="Generated schema file for TimeXtender." xmlns:other="http://apiscript.com/ns?v1"> <!-- Attributes for the main JSON struct

9212
A
Problem Solver
M
Explorer
MvestExplorer
asked in Ingest
Full Load Transfer vs. On-Demand Execution

Hey there, We used to have the On-Demand setting enabled for one of our data sources. With this setting turned on, the package that executes the corresponding tables would take roughly one hour to complete. Recently, we switched to a Full Load Transfer Task, followed by an Execute Tables package. However, now the transfer/ingestion process takes almost 2.5 hours (and then another 20 minutes to execute the tables). We thoroughly checked to ensure that the tables included in the Full Load are the same as those processed with On-Demand, and they are. In the logs I can see a lot of these messages (every 5 minutes), so I am wondering how much time the full transfer is idly waiting.

301
Christian Hauggaard
Community Manager
B
Starter
Boglarka BaranyStarter
asked in Ingest
API REST Get Schema function not working

The Get Schema function is not working for the REST endpoints / table flattening. I am trying to use version: 8.0.0.0. I get the following message: Any ideas what is happening?

296
B
Starter
P
Contributor
PieterQContributor
asked in Ingest
Fabric lakehouse ingest - error Import Metadata

Hi Everyone, We want to use Fabric lakehouse as ingest layer, and did the setup according to this article: We are running TimeXtender version 6898.1 We created the lakehouse form Fabric, not by using the ‘Create Storage; option from TDI. When we test the connection in TDI, everything looks fine: However, the metadata import task fails with following error: The execution failed with error:Exception Type: Azure.RequestFailedExceptionMessage: Service request failed. Status: 400 (BadRequest) ErrorCode: IncomingOperationUntrusted Headers: x-ms-error-code: IncomingOperationUntrusted Access-Control-Allow-Origin: * Access-Control-Allow-Headers: REDACTED Access-Control-Allow-Methods: REDACTED Access-Control-Expose-Headers: REDACTED Strict-Transport-Security: REDACTED X-Content-Type-Options: REDACTED x-ms-root-activity-id: REDACTED Content-Length: 0 Date: Thu, 06 Mar 2025 14:57:14 GMT Server: Microsoft-HTTPAPI/2.0 Stack Trace: at Azure.Core.HttpPipelineExtensions.ProcessMessage(HttpPipeline pipe

515
Christian Hauggaard
Community Manager
oscarcleve
Contributor
oscarcleveContributor
asked in Ingest
Limited functionality TimeXtender REST data source

Just recently the TimeXtender REST data source at a client was not able to be edited to the same extent anymore. When pressing “Get Schema” in the Table Flattening Table Builder, we get this error message: No changes or updates were made to the TDI, ODX, or the TX REST data connection as the error started to appear. We use this combination of versions: As suggested here: The exact TX REST version used is 7.1.0.0. If possible we would like a solution to this problem that does not require an upgrade of the TDI software.

511
Christian Hauggaard
Community Manager
A
Participant
annfossParticipant
asked in Ingest
TimeXtender Rest data source with CSV response

Hi I have a REST data source that provides the desired output in CSV format. However, I'm looking for guidance on formatting the XML output to match the expected structure. Currently, the XML output I receive looks like this: <?xml version="1.0" encoding="UTF-8"?><Name_of_table> <value> "name_of_column1","name_of_column2","name_of_column3","000: name_of_column4," "G name,with comma","2000M01","Volumindeks, kalenderjustert",. "G name,with comma","2000M02","Volumindeks, kalenderjustert",. "G name,with comma","2000M04","Volumindeks, kalenderjustert",3.2 "G name,with comma","2000M05","Volumindeks, kalenderjustert",50.2... </value></Name_of_table> Is there any support for CSV responses in the Rest Connector, or do you recommend creating a XSLT file for this purpose?

828
A
Participant
anders.e.jonsson
Contributor
anders.e.jonssonContributor
asked in Ingest
Dynamic Values for parameter in same endpoint REST data source

Hi, I am trying to set up a REST connection. Here are the prerequisites: The API delivers data for a maximum of one month, and no more than 10,000 rows at a time. I'm a bit of a beginner when it comes to the new Rest data source 🙄I wonder what the easiest way to do is to;1. Read all rows for the selected period using the fetchFromActivityId field. That field is in one of the API's tables.2. Loop through month by month until I arrive at today's date.The API works fine. It is no problem to fetch 10,000 rows from a selected period.All ideas are gratefully received 🙂 BRAnders

586
anders.e.jonsson
Contributor
K
Contributor
KajEmergoContributor
asked in Ingest
Connection issue ODX Server upgrade

When launching the newly installed ODX Server configuration (20.10.45), we get the following error: First, we thought this was a Firewall issue on our (Azure VM), because this test-netconnection did not work (see the second one failing): When I try to reach that second address from my local laptop, I get the same failure. Could you explain what is going on here? When using the ‘tracert’ to timextender-api.azurewebsites.net command in the command prompt on my local machine I see that it forwards to waws-prod-am2-061.cloudapp.net. When we try to do the same on the TX Azure VM, it times out on the first step. Do you have any advice on how to proceed here? Could it still be a Firewall issue on our VM?

21510
rory.smith
TimeXtender Xpert
avanrijn
Contributor
avanrijnContributor
asked in Ingest
Working with RSD file with TX version 6892

Dear sir, Is it still possible to work with RSD files in TX version 6892 ? I found this link: Is it possible to do something like this with the latest 8.0 TimeXtender REST provider ? I am trying to work with an API that is refreshing there Bearer Token every hour “expiresIn”: 3600 curl -X 'POST' \ 'https://openapi.binnenbeter.nl/api/v1/login' \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "tenant": "horstaandemaas", "username": "*****", "password": "*****"}' { "token": "eyJwMmMi**********lRW5SWTZKaTFzb1EiLCJhbGciOiJQQkVTMi1IUzUxMitBMjU2S1ciLCJlbmMiOiJBMjU2R0NNIn0.J8cstvlbqRrWzsRulKtnBFv4eKG6r3fvudZI*********gvjw", "tokenType": "Bearer", "roles": "ADMIN, API", "expiresIn": 3600} So I have to come up with a solution that dynamically refreshes the token before it is fetching the data or else the API will deny access because the token is out of date. If you have any ideas please let me know. The plan is: What I am trying is to run a powershell script to fetch

788
avanrijn
Contributor
M
Visitor
MonikaVisitor
asked in Ingest
Missing 'Select tables'

Hi, I cannot find the option to ‘Select tables’ from the source: My TX version is 6848.1Where should I look for that?

352
rvgfox
Problem Solver
A
Participant
annfossParticipant
asked in Ingest
Rest data source - Json-stat format

Hi Does the rest data source support json-stat format? Is it able to interpret it and transform it into a structured table? I tried using the JSON data format, but I received an XML file that the ODX could not extract any columns from.

341
Thomas Lind
Community Manager
Paul McLeod Bespoke Analytics
TimeXtender Xpert
Paul McLeod Bespoke AnalyticsTimeXtender Xpert
asked in Ingest
Manage Query Tables on TimeXtender ADF Ingest Data Source

I have a need to create Managed Queries to sub-select data ranges from a large table source to improve load performance. This is an on-premise SQL instance going to a Gen 2 Storage account in Azure. We have selected ADF because it provides the best performance with the transfer. We are still finding that we have performance issues and so want to break 2 of the larger tables into sub loads so that we can create multiple tables in the Gen 2 storage to leverage parallel loads into the Prepare instance. I was going to use managed queries to accomplish this but it does not appear to be an option on ADF Transfer. Can you confirm?

261
Christian Hauggaard
Community Manager
Paul McLeod Bespoke Analytics
TimeXtender Xpert
Paul McLeod Bespoke AnalyticsTimeXtender Xpert
asked in Ingest
TimeXtender 6892.1 Ingest Jobs Issue

I have just installed TimeXtender 6892.1 on a client’s site and all of the jobs that call Ingest tasks are all marked as invalid. indicating that “The instance “DataLakeDev” has an invalid repository version - the instance is of a later version than what is supported by the application. Both the Ingest engine and the Prepare / Desktop / Execution engine are 6892.1 I have opened every instance and allowed TimeXtender to upgrade the instance in the repository. This is only happening with jobs that execute ingest tasks across both of our ingest instances (dev and Prod). If I try to create a new job it will not display any ingest tasks to add to the job, so I cannot recreate the job. We are using Exmon Orchestration and it still seems able to execute the jobs even though they are marked as invalid. So we are able to run our production loads thankfully. But this needs to be resolved ASAP.

804
Paul McLeod Bespoke Analytics
TimeXtender Xpert
P
Explorer
Pieter-JanExplorer
asked in Ingest
Installation "Ingest Service Configuration"

I'm busy with the ingest service configuration, I could sign in, next step is to connect to an instance, that goes well. If i press next the message appear: taking the instance lock, please wait…. and then the error message: A network related or instance specific error occured while establishing a connection to Sql server. the server was not found or accesible. my vm for the ingest service configuration can connect to sql instances in my own tenant/resource groups with SSIS. What server/database is the reason that the message appears, and how to solve (ports, rights?)

372
Christian Hauggaard
Community Manager
A
Participant
annfossParticipant
asked in Ingest
Soft deletes with incremental load in the ODX

Hi I have an incremental table with soft deletes enabled in my extract layer (data area). Additionally, I have set up an incremental rule for the SQL data source in the ODX. I was wondering if an hard delete occurs in the SQL data base, will the ODX still be able to detect this such that the incremental table in the data area can mark the record as TombStone?

323
Thomas Lind
Community Manager
A
Problem Solver
aftabpProblem Solver
asked in Ingest
How to skip a value with an if-condition

I have been struggling with this since last week and have tried different methods to solve this.ProjectNumber in my RSD is defined as string but still when i get 123ProjectNumber456 as projectnumber i get an error 2025-01-24T15:27:32.136+01:00 3 [153|Q-Id] [HTTP|Res: 1924] {"description":"Budgettype 'approved' does not exist for projectnumber '123ProjectNumber546'"}2025-01-24T15:27:32.136+01:00 2 [153|Q-Id] [HTTP|Res: 1924] HTTP/1.1 404 Not Found, 107 Bytes Transferred2025-01-24T15:27:32.136+01:00 2 [153|Q-Id] [HTTP|Res: 1924] Error: HTTP protocol error. 404 Not Found. Stack: at dwo230y.w.De(String ) I have tried sorting out the given project with this code but it didnt work <api:if condition="[touchIn.cleanedProjectNumber | contains('123ProjectNumber456')]"> </api:if> <api:else> <!-- Set revisionType when condition is true --> <api:set attr="detailsIn.revisionType" value="approved"/> I have also tried the follwing but its not working <api:if condition="[touchIn.cleanedProjectNumber |

699
A
Problem Solver
Roy V
Problem Solver
Roy VProblem Solver
asked in Ingest
Network Library options in sql client data provider

In the 20.10 version we have a data source with a .Net Client Data Provider with the settings below: In the SaaS version we have the same source but the network library field is not a drop down and adding TCP/IP or TCP/IP (DBMSSOCN) results in an error that the provided network library does not exist. What is the value to use when I want it to be TCP/IP?

294
Roy V
Problem Solver
A
Problem Solver
aftabpProblem Solver
asked in Ingest
Trim blank spaces in URI

Getting an error when trying to reterive data. I have created a nested statement api:set attr="detailsIn.URITemplate" value="[urlbase]/api/Budget/{projectNumber}/currentbudget?api-version=1.0" /> I think its because of a blank space after the project number 123456All other projects in TX log comes without a blank space 2025-01-23T11:48:43.178+01:00 2 [JSON] Reading JSON file at https://baseurl/Budget/123456 /currentbudget?api-version=1.02025-01-23T11:48:43.184+01:00 2 [53|Q-Id] [HTTP|Req: 1457] GET https://baseURL/Budget/123456 /currentbudget?api-version=1.0 I have tried to remove blank spaces with following code but its not working.Anyone has a suggestion? <api:script method="GET"> <api:call op="jsonproviderGet" in="touchIn" out="touchOut"> <!-- Remove all blank spaces from projectNumber and store in a variable --> <api:set attr="cleanedProjectNumber" value="[touchOut.projectNumber | replace('\\s+', '')]" /> <!-- Use the cleaned projectNumber in the URI --> <api:set attr="detailsIn.UR

231
Thomas Lind
Community Manager
B
Starter
bjornStarter
asked in Ingest
TimeXtender SQL Data Source - Additional Connections

Hi, I got a data source with multiple databases(22) with equal tables, views, etc. TX ver 20.10 I was able to add databases in the "Additional Connections" for the “SQL Server data source”. How about the new TX version.. can I do the same? It would save me a lot of work when selecting tables and setting up incr load rules regards, Bjørn A.

272
B
Starter
B
Starter
bjornStarter
asked in Ingest
Dynamic value for parameter in OAuth

Hi, I must use tenant_id (company specifict) when authorize my endpoint with OAuth2 (credentials in body and request header). Parameters are client_id, client_secret, tenant_id, etc I really dont want to create a connection for each of my tenant_id's. So my question… How can I use dynamic values to update authentication for each tenant_id? regards, Bjørn A.

242
B
Starter
westdorp
Contributor
westdorpContributor
asked in Ingest
Missing view ShoppingProducts in CDATA Google Ads connector

Question; I have a project that contains the CDATA Google Ads connector. I need the view ShoppingProducts that was added recently to the connector: Unfortunately I can't find in in Data Selection. Is seems to be added in version / build 9053 of the connector but TX shows me version 9036 is the latest: I have resynchronized the source but as you can see below it’s not in the list: Is the latest CDATA connector available in some way? BR\ Dirk

432
westdorp
Contributor
B
Starter
bjornStarter
asked in Ingest
Supported Sql Server version for TX 6848

Hi, TimeXtender v.6848.1 Can we use Sql version Microsoft SQL Azure (RTM) - 12.0.2000.8 Oct 2 2024 11:51:41 Copyright (C) 2022 Microsoft Corporation as our ingested database? I keep getting this error loading the tablesThe execution failed with error:Exception Type: Microsoft.Data.SqlClient.SqlExceptionMessage: Reference to database and/or server name in 'xxxxxx.sys.extended_properties' is not supported in this version of SQL Server. best regards, Bjørn A.

251
rory.smith
TimeXtender Xpert

Most helpful members this week

Badge winners

  • Ingest Expert
    nmohas earned the badge Ingest Expert
  • Ingest Expert
    oscarclevehas earned the badge Ingest Expert
  • Ingest Expert
    fwagnerhas earned the badge Ingest Expert
  • Innovator
    woutersmikehas earned the badge Innovator
  • Innovator
    Roy Vhas earned the badge Innovator
Show all badges

Popular tags

Terms and ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings