Ingest

​Hi,I’m sorry for this somewhat lengthy post but I have a problem that I don’t really know how to solve and I’m just looking to see if anyone have run into the same issue, or maybe already solved it.We are going to fetch data from a time management system that have a SOAP API and all the information I find in the community refers to RSD files, that is not to be used anymore. I don’t know how I would solve the problem that I have even with an RSD file, but that’s somewhat beside the point here. 😊The issue is that we must take the request XML, encrypt the whole thing using AES, into a fingerprint that in turn will need to be inserted into the actual request that is then sent to the API.I have received some code in C# that does the following steps:Generates a timestamp and an MD5 hash of the request XML Generates the fingerprint XML using the timestamp and the newly created MD5 hash with the schema like in the attached picture fingerprin

I am trying to replace the CDATA Dynamics365 adapter with the TX REST connector (8.0.0) and I am having some issues where empty values from the API is interpreted as empty strings rather than NULL and as a result creates tables with incorrect datatypesHow do i set up a way to replace these empty strings with NULL values and keep the correct data types as wellI have tried with various combination of “Search and replace" on the endpoint as well as the individual nodes in the table builder without any luck.I also have the “Empty fields as null” option enabled. 

Dear sir, Our client wants to upgrade to the latest and the greatest of TimeXtender.Right now they are on TX version 6766.1.So they have to migrate their CData datasources to the new TX datasources.Question: Is there a ‘datasource migration-tool’ to help them with that ?Or do they have to setup/configure the new TimeXtender datasources manually ?Hope you can shine a light on this.Regards,Arthur

Hi,I'm using the TimeXtender CSV Data Source provider and was wondering if it's possible to combine this provider with either incremental load settings or history settings. I’ve tried enabling incremental load—both in the ODX and by setting the table as incremental in the first data area—but I receive the error: "The provider does not support incremental load."Interestingly, I don't get the same error when I set the table as a history table instead, but when I try to execute the table, the result is empty.I'm wondering if this could be due to incorrect natural key configuration from my side, or if the CSV provider simply doesn't support historical loading.Any clarification would be appreciated. Thank you!  

The Get Schema function is not working for the REST endpoints / table flattening. I am trying to use version: 8.0.0.0. I get the following message: Any ideas what is happening?

I’ve an Azure storage account with a blob Container that has folders and files. How can I configure a data source to use it?  

I use the CSV provider to load a folder with multiple CSV files. I know one of the files has 17471 rows. Only the ingest task only loads the first 14521 rows. I don't have any filtering in the ingest task.Does anyone know how I can load the full file?

I am extracting data from the MS Graph API using the REST connector in TimeXtender. The issue I am facing is that I can easily retrieve the members of the groups but unfortunately, the GroupId is not included in the Query and I do not see any way to include the {id} parameter into the saved results.If anyone has solved this, can you please help with some information on how to solve this?

Have anyone an example on a successful connector regarding a normal Active Directory - not Azure AD.Can’t find any information about it but I guess many have done a connection using TX for that. RegardsAnders Bengtsson

We have trouble with an endpoint in a TimeXtender REST API data source using the Override Headers functionality. The problem arose when we upgraded our TX software to the latest version of TDI/TIS and TX REST from 6848.1/6848.1 and 7.1.0.0.We get this error message: “Failed to execute request for endpoint 'NBABIItemWS'.HTTP Status Code 401 UnauthorizedResponse body:<error xmlns="http://docs.oasis-open.org/odata/ns/metadata"><code>Unauthorized</code><message>The credentials provided are incorrect</message></error>” We use the Override Headers functionality to limit page size for the particular endpoint. It seems that for some reason the authentication token is overwritten when using this feature, and there fore we can not authenticate. We have plenty of other endpoints in the s

This is my JSON{ "projectNumber": "123456", "name": "TX", "addendumValues": { "addendumGroups": [ { "addendumGroupName": "G1", "rows": [ { "columns": { "revnr_G1": [ "Rev.1" ], "date_G1": [ "2023-06-21T00:00:00.0000000" ] } } ] }, { "addendumGroupName": "G2", "rows": [ { "columns": { "revnr_G2": [ "Rev.1" ],

Hey there,We used to have the On-Demand setting enabled for one of our data sources. With this setting turned on, the package that executes the corresponding tables would take roughly one hour to complete.Recently, we switched to a Full Load Transfer Task, followed by an Execute Tables package. However, now the transfer/ingestion process takes almost 2.5 hours (and then another 20 minutes to execute the tables).We thoroughly checked to ensure that the tables included in the Full Load are the same as those processed with On-Demand, and they are.In the logs I can see a lot of these messages (every 5 minutes), so I am wondering how much time the full transfer is idly waiting.  

Hi Everyone, We want to use Fabric lakehouse as ingest layer, and did the setup according to this article: We are running TimeXtender version 6898.1We created the lakehouse form Fabric, not by using the ‘Create Storage;  option from TDI. When we test the connection in TDI, everything looks fine:However, the metadata import task fails with following error: The execution failed with error:Exception Type: Azure.RequestFailedExceptionMessage: Service request failed.         Status: 400 (BadRequest)         ErrorCode: IncomingOperationUntrusted                  Headers:         x-ms-error-code: IncomingOperationUntru

Just recently the TimeXtender REST data source at a client was not able to be edited to the same extent anymore. When pressing “Get Schema” in the Table Flattening Table Builder, we get this error message:  No changes or updates were made to the TDI, ODX, or the TX REST data connection as the error started to appear. We use this combination of versions: As suggested here: The exact TX REST version used is 7.1.0.0.If possible we would l

Hi I have a REST data source that provides the desired output in CSV format. However, I'm looking for guidance on formatting the XML output to match the expected structure. Currently, the XML output I receive looks like this:<?xml version="1.0" encoding="UTF-8"?><Name_of_table>    <value> "name_of_column1","name_of_column2","name_of_column3","000: name_of_column4," "G name,with comma","2000M01","Volumindeks, kalenderjustert",. "G name,with comma","2000M02","Volumindeks, kalenderjustert",. "G name,with comma","2000M04","Volumindeks, kalenderjustert",3.2 "G name,with comma","2000M05","Volumindeks, kalenderjustert",50.2... </value></Name_of_table>Is there any support for CSV responses in the Rest Connector, or do you recommend creating a XSLT file for this purpose? 

Hi,I am trying to set up a REST connection. Here are the prerequisites: The API delivers data for a maximum of one month, and no more than 10,000 rows at a time.I'm a bit of a beginner when it comes to the new Rest data source 🙄I wonder what the easiest way to do is to;1. Read all rows for the selected period using the fetchFromActivityId field. That field is in one of the API's tables.2. Loop through month by month until I arrive at today's date.The API works fine. It is no problem to fetch 10,000 rows from a selected period.All ideas are gratefully received 🙂BRAnders

When launching the newly installed ODX Server configuration (20.10.45), we get the following error:First, we thought this was a Firewall issue on our (Azure VM), because this test-netconnection did not work (see the second one failing):When I try to reach that second address from my local laptop, I get the same failure. Could you explain what is going on here? When using the ‘tracert’ to timextender-api.azurewebsites.net command in the command prompt on my local machine I see that it forwards to waws-prod-am2-061.cloudapp.net. 

Dear sir,Is it still possible to work with RSD files in TX version 6892 ?I found this link: Is it possible to do something like this with the latest 8.0 TimeXtender REST provider ?I am trying to work with an API that is refreshing there Bearer Token every hour“expiresIn”: 3600curl -X 'POST' \ 'https://openapi.binnenbeter.nl/api/v1/login' \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "tenant": "horstaandemaas", "username": "*****", "password": "*****"}'{ "token": "eyJwMmMi**********lRW5SWTZKaTFzb1EiLCJhbGciOiJQQkVTMi1IUzUxMitBMjU2S1ciLCJlbmMiOiJBMjU2R0NNIn0.J8cstvlbqRrWzsRulKtnBFv4eKG6r3fvudZI*********gvjw", "tokenType": "Bearer", "roles": "ADMIN, API", "expi

Hi,I cannot find the option to ‘Select tables’ from the source:  My TX version is 6848.1Where should I look for that?

Hi  Does the rest data source support json-stat format? Is it able to interpret it and transform it into a structured table? I tried using the JSON data format, but I received an XML file that the ODX could not extract any columns from. 

I have a need to create Managed Queries to sub-select data ranges from a large table source to improve load performance.This is an on-premise SQL instance going to a Gen 2 Storage account in Azure.We have selected ADF because it provides the best performance with the transfer. We are still finding that we have performance issues and so want to break 2 of the larger tables into sub loads so that we can create multiple tables in the Gen 2 storage to leverage parallel loads into the Prepare instance.I was going to use managed queries to accomplish this but it does not appear to be an option on ADF Transfer. Can you confirm?

I have just installed TimeXtender 6892.1 on a client’s site and all of the jobs that call Ingest tasks are all marked as invalid. indicating that “The instance “DataLakeDev” has an invalid repository version - the instance is of a later version than what is supported by the application.Both the Ingest engine and the Prepare / Desktop / Execution engine are 6892.1I have opened every instance and allowed TimeXtender to upgrade the instance in the repository. This is only happening with jobs that execute ingest tasks across both of our ingest instances (dev and Prod). If I try to create a new job it will not display any ingest tasks to add to the job, so I cannot recreate the job.We are using Exmon Orchestration and it still seems able to execute the jobs even though they are marked as invalid. So we are able to run our production loads thankfully. But this needs to be resolved ASAP.

I'm busy with the ingest service configuration, I could sign in, next step is to connect to an instance, that goes well. If i press next the message appear:taking the instance lock, please wait….and then the error message: A network related or instance specific error occured while establishing a connection to Sql server. the server was not found or accesible. my vm for the ingest service configuration can connect to sql instances in my own tenant/resource groups with SSIS. What server/database is the reason that the message appears, and how to solve (ports, rights?)  

HiI have an incremental table with soft deletes enabled in my extract layer (data area). Additionally, I have set up an incremental rule for the SQL data source in the ODX. I was wondering if an hard delete occurs in the SQL data base, will the ODX still be able to detect this such that the incremental table in the data area can mark the record as TombStone?

I have been struggling with this since last week and have tried different methods to solve this.ProjectNumber in my RSD is defined as string but still when i get 123ProjectNumber456 as projectnumber i get an error2025-01-24T15:27:32.136+01:00 3 [153|Q-Id] [HTTP|Res: 1924] {"description":"Budgettype 'approved' does not exist for projectnumber '123ProjectNumber546'"}2025-01-24T15:27:32.136+01:00 2 [153|Q-Id] [HTTP|Res: 1924] HTTP/1.1 404 Not Found, 107 Bytes Transferred2025-01-24T15:27:32.136+01:00 2 [153|Q-Id] [HTTP|Res: 1924] Error: HTTP protocol error. 404 Not Found. Stack: at dwo230y.w.De(String )I have tried sorting out the given project with this code but it didnt work <api:if condition="[touchIn.cleanedProjectNumber | contains('123ProjectNumber456')]"> </api:if> <api:else> <!-- Set revisionType when condition is true -->