Skip to main content
To homepage To homepage

TimeXtender Desktop Q&A

Ask questions and find answers about the TimeXtender Desktop Application

718 Topics
rvgfox
Problem Solver
rvgfoxProblem Solver
asked in Jobs & Executions
Dynamic Perspectives by schema

I’ve created a dynamic perspective based on an schema.If I try to create an execution package to execute all the tables in that perspective, it’s not possible.Is it a bug?  

242
G
Community Manager
A
Visitor
AmrVisitor
asked in Ingest
Invalid column count Excel sheet on storage account

Suddenly all the excel sheets are not loading anymore however some of them no one touched 6 months ago.the error is:Executing table excel_sheet:failed with error:Exception Type: System.ExceptionMessage: Invalid column count             Stack Trace: at DataStorageEngine.DataLakeGen2.DataLakeGen2Transfer.<>c__DisplayClass41_5.<UploadData>b__14(DataTable dataTable, Partition partition)                          at DataStorageEngine.DataLakeGen2.DataLakeGen2Transfer.UploadData(ADLSGen2APIManager api, DataSourceModel dataSourceModel, DataLakeGen2ExecutionMethod executionMethod, Action`2 writeExecutionLog)                          at DataStorageEngine.DataLakeGen2.DataLakeGen2StorageEngine.<>c__DisplayClass35_2.<TransferDataAsync>b__7(DataLakeGen2ExecutionMethod executionMethod)                          at DataSourceEngine.Custom.CustomSourceEngine.InvokeDataSourceExecution(Action`1 action)                          at DataSourceEngine.Custom.CustomSourceEngine.RunExecu

30
A
Visitor
L
Starter
leo.noharetStarter
asked in Prepare
Help with Primary Keys and Incremental Load in Table Inserts

Hi, I am facing an issue with duplicate records when using incremental load on a table that receives data via a table insert from another table in the same data area. The incremental load selection rule is based on DW_TimeStamp, and there are three PK in the target table.The issue is that the primary key does not seem to be enforced in the target table, as I am getting duplicate rows in the table. I tested this with another table where data is inserted through a table insert (but without incremental load), and I still see duplicate values for the PK fields and despite setting the table option to "Use instance setting (error)", no error is thrown when duplicates occur.  I have tested this behavior in TDI 6935.1 and 6926.1, and the issue persists. My questions are:  How do primary keys work with table inserts? Should they prevent duplicates, or do they not function as expected when using table inserts? Can incremental load be achieved using table inserts? If so, what are the necessary

495
rory.smith
TimeXtender Xpert
T
Participant
thzmanParticipant
asked in Ingest
SOAP request with encryption

​Hi,I’m sorry for this somewhat lengthy post but I have a problem that I don’t really know how to solve and I’m just looking to see if anyone have run into the same issue, or maybe already solved it.We are going to fetch data from a time management system that have a SOAP API and all the information I find in the community refers to RSD files, that is not to be used anymore. I don’t know how I would solve the problem that I have even with an RSD file, but that’s somewhat beside the point here. 😊The issue is that we must take the request XML, encrypt the whole thing using AES, into a fingerprint that in turn will need to be inserted into the actual request that is then sent to the API.I have received some code in C# that does the following steps:Generates a timestamp and an MD5 hash of the request XML Generates the fingerprint XML using the timestamp and the newly created MD5 hash with the schema like in the attached picture fingerprint_schema Encrypts the fingerprint XML with AES, usi

288
T
Participant
B
Contributor
bjornContributor
asked in Ingest
Avoid URL encoding in the Query

Hey, I use the REST connector to consume a CSV file. In my query parameters I got one for auth key. So when I try to load the file a get an error and the reason is that parameter get encoded (% → %25) .  How can I avoid encoding in the query? /Bjørn A.  

150
B
Contributor
R
Explorer
r.miltenburgExplorer
asked in Ingest
Schema already exists error when change data soruce connection

Hi,We've updatet our Data Integration version from 6766.1 to 6963.1. Which means we had to update our outdated providers. So we followed the steps mentioned in this post Change data source provider | Community (timextender.com).This results in an error when we try to full load changed data source. It keeps giving is the error: “The schema …. already exists on the SQL storage and cannot be used” Which actually means we can’t update our providers/ data sources. We have to create new ones en have to map all our tables in de odx. Anyone else facing this problem and have a solution for this?

435
Thomas Lind
Community Manager
A
Problem Solver
aftabpProblem Solver
asked in Prepare
Deploy running for over 20 minutes

This is a simple Deploy and its been running for almost 20 minutes.Whats wrong here?  

80
A
Problem Solver
N
Contributor
nmoContributor
asked in Ingest
REST: replace '' with NULL

I am trying to replace the CDATA Dynamics365 adapter with the TX REST connector (8.0.0) and I am having some issues where empty values from the API is interpreted as empty strings rather than NULL and as a result creates tables with incorrect datatypesHow do i set up a way to replace these empty strings with NULL values and keep the correct data types as wellI have tried with various combination of “Search and replace" on the endpoint as well as the individual nodes in the table builder without any luck.I also have the “Empty fields as null” option enabled.  

183
Thomas Lind
Community Manager
avanrijn
Contributor
avanrijnContributor
asked in Prepare
Direct Read option latest TX version

Dear Sir,In TX version 20 you had a configuration setting Direct Read.I cannot find this setting in the latest and the greatest version in TimeXtender.Question: Is Direct Read still available ? How can I set this up ?Thanx in advance for looking into this. The "Direct Read" option in TimeXtender is particularly useful in scenarios where you need to transfer data between different data areas within the same physical database but using different schemas. Here are some specific scenarios where you might use the Direct Read option:  Large Data Volumes:  When dealing with large tables, Direct Read can be more efficient as it allows for direct data transfer within the same database, reducing the overhead associated with external data movement tools like SSIS or ADO.NET.  Incremental Loads:  Direct Read is beneficial for incremental loads where only a subset of data needs to be transferred based on specific criteria (e.g., timestamp or modified date). This can help in optimizing performanc

272
Christian Hauggaard
Community Manager
M
Participant
MMMParticipant
asked in Ingest
Snowflake Ingest instance functionality

Hi! We are currently setting up TimeXtender with Snowflake and have a bit of an issue when it comes TimeXtender not having the capability of managing the ODX in Snowflake (internal stage).  As you know an external stage is needed via Azure Data Lake Gen2.  The issue for us is that we would like to use private endpoint when creating this data lake in Azure but that requires business critical subscription to Snowflake which is 30 percent more expensive.   Are there any plans/timeline for when TimeXtender will be capable of handling the Ingest instance in Snowflake and if so could you please give a rough estimate on when that might be available?  We need to make some decisions on our end on how to proceed and would like to know the answer before we move forward.

93
rory.smith
TimeXtender Xpert
oscarcleve
Contributor
oscarcleveContributor
asked in Jobs & Executions
Orchestration executions failing for Prepare Instance

This is the error I am facing when executing any Prepare package (no issue with executing Ingest packages) from the Orchestration portal: ​​​​​​Error I have tried resolving the problem with troubleshooting below, but I still get the same error:  Suggested troubleshootingAny further suggestions on how to handle the issue?We run Data Integration 6935.1 and Ingest 6935.1 Thanks

585
Christian Hauggaard
Community Manager
J
Contributor
JogchumSiRContributor
asked in Prepare
Columns with source data type money behaving incorrect

Hi All, Hopefully you can help me with this one. We have a source table from our POS systems containing sales transactions. There are a few value columns we transfer that are stored as a money data type and we see some weird things happening with the values of the columns. Differences of 0.01 up to 0.0001 are appearing from nowhere in the transfer from ODX to DWH and i don't know how this i happening.The columns that have money data type are called ‘Bedrag', ’Korting’ and ‘NettoBedrag’ Take transaction 52. In the source the values are like this: After transfer with ODX, i looked in the parquet file using ParquetViewer and that looks like this: Still the same, nothing wrong here.I imported the table without any modifications in TimeXtender and it looks like this: After a transfer the data looks like this: If you look closely we lost 0.0001 on the first record ‘NettoBedrag’. Also wel lost 0.0001 on the Bedrag of the second record.I've tried a few things to mitigate this. Different numeri

15919
Christian Hauggaard
Community Manager
avanrijn
Contributor
avanrijnContributor
asked in Ingest
Data Source migration tool

Dear sir, Our client wants to upgrade to the latest and the greatest of TimeXtender.Right now they are on TX version 6766.1.So they have to migrate their CData datasources to the new TX datasources.Question: Is there a ‘datasource migration-tool’ to help them with that ?Or do they have to setup/configure the new TimeXtender datasources manually ?Hope you can shine a light on this.Regards,Arthur

503
Christian Hauggaard
Community Manager
Emiel van Schaik
Visitor
Emiel van SchaikVisitor
asked in Ingest
Does anyone have experience with Adnamics as a datasource?

As title suggests, I wonder if anyone in this community has experience with ingesting TX with the data source Adnamics and their API. Thanks in advance!

100
Emiel van Schaik
Visitor
A
Participant
annfossParticipant
asked in Ingest
CSV Connector Compatibility with Incremental or History Settings?

Hi,I'm using the TimeXtender CSV Data Source provider and was wondering if it's possible to combine this provider with either incremental load settings or history settings. I’ve tried enabling incremental load—both in the ODX and by setting the table as incremental in the first data area—but I receive the error: "The provider does not support incremental load."Interestingly, I don't get the same error when I set the table as a history table instead, but when I try to execute the table, the result is empty.I'm wondering if this could be due to incorrect natural key configuration from my side, or if the CSV provider simply doesn't support historical loading.Any clarification would be appreciated. Thank you!  

334
Thomas Lind
Community Manager
B
Explorer
Boglarka BaranyExplorer
asked in Ingest
API REST Get Schema function not working

The Get Schema function is not working for the REST endpoints / table flattening. I am trying to use version: 8.0.0.0. I get the following message: Any ideas what is happening?

678
Christian Hauggaard
Community Manager
P
Starter
philippus.asefawStarter
asked in Prepare
Exclude DW_ID from Primary Key on Database

Hey everyone,I have the following question:Is it possible, at the database level, to set the primary key to the one selected in TX (e.g., "Umkod_ID") instead of the default "DW_ID"?The issue arises because we access our table via PowerApps, and since the app existed beforehand, references in the code were already predefined. Currently, the app's primary key is set to the "Umkod_ID" field of the table. However, in our database, the primary key is set to the system field "DW_ID." Since this table is only loaded once and no transformations occur, the system fields are not particularly important to us in this case.We resolved this issue by manually populating the table, but I am curious if anyone knows a workaround that could be applied to similar cases in the future.

472
P
Starter
rvgfox
Problem Solver
rvgfoxProblem Solver
asked in Ingest
Data Source to Azure Blob Containers

I’ve an Azure storage account with a blob Container that has folders and files. How can I configure a data source to use it?  

305
Christian Hauggaard
Community Manager
M
Starter
MiesStarter
asked in Ingest
Not all rows of CSV file are loaded during Full load ingest task

I use the CSV provider to load a folder with multiple CSV files. I know one of the files has 17471 rows. Only the ingest task only loads the first 14521 rows. I don't have any filtering in the ingest task.Does anyone know how I can load the full file?

6612
Christian Hauggaard
Community Manager
C
Explorer
cmynhardtExplorer
asked in Ingest
MS Graph API Group and Group Members do not always return the Group Id

I am extracting data from the MS Graph API using the REST connector in TimeXtender. The issue I am facing is that I can easily retrieve the members of the groups but unfortunately, the GroupId is not included in the Query and I do not see any way to include the {id} parameter into the saved results.If anyone has solved this, can you please help with some information on how to solve this?

181
N
Contributor
A
Starter
anders.bengtssonStarter
asked in Ingest
Connector for Active Directory - examples

Have anyone an example on a successful connector regarding a normal Active Directory - not Azure AD.Can’t find any information about it but I guess many have done a connection using TX for that. RegardsAnders Bengtsson

18323
Christian Hauggaard
Community Manager
oscarcleve
Contributor
oscarcleveContributor
asked in Ingest
TimeXtender REST API override header

We have trouble with an endpoint in a TimeXtender REST API data source using the Override Headers functionality. The problem arose when we upgraded our TX software to the latest version of TDI/TIS and TX REST from 6848.1/6848.1 and 7.1.0.0.We get this error message: “Failed to execute request for endpoint 'NBABIItemWS'.HTTP Status Code 401 UnauthorizedResponse body:<error xmlns="http://docs.oasis-open.org/odata/ns/metadata"><code>Unauthorized</code><message>The credentials provided are incorrect</message></error>” We use the Override Headers functionality to limit page size for the particular endpoint. It seems that for some reason the authentication token is overwritten when using this feature, and there fore we can not authenticate. We have plenty of other endpoints in the same data source not using Override Headers, and those are working fine, so we suspect that the issue has to do with that setting. We use this combination of versions:  Is there

604
oscarcleve
Contributor
A
Problem Solver
aftabpProblem Solver
asked in Ingest
Need some help with ingesting

This is my JSON{ "projectNumber": "123456", "name": "TX", "addendumValues": { "addendumGroups": [ { "addendumGroupName": "G1", "rows": [ { "columns": { "revnr_G1": [ "Rev.1" ], "date_G1": [ "2023-06-21T00:00:00.0000000" ] } } ] }, { "addendumGroupName": "G2", "rows": [ { "columns": { "revnr_G2": [ "Rev.1" ], "date_G2": [ "2023-06-21T00:00:00.0000000" ] } }, {

9512
A
Problem Solver
M
Explorer
MvestExplorer
asked in Ingest
Full Load Transfer vs. On-Demand Execution

Hey there,We used to have the On-Demand setting enabled for one of our data sources. With this setting turned on, the package that executes the corresponding tables would take roughly one hour to complete.Recently, we switched to a Full Load Transfer Task, followed by an Execute Tables package. However, now the transfer/ingestion process takes almost 2.5 hours (and then another 20 minutes to execute the tables).We thoroughly checked to ensure that the tables included in the Full Load are the same as those processed with On-Demand, and they are.In the logs I can see a lot of these messages (every 5 minutes), so I am wondering how much time the full transfer is idly waiting.  

341
Christian Hauggaard
Community Manager
P
Contributor
PieterQContributor
asked in Ingest
Fabric lakehouse ingest - error Import Metadata

Hi Everyone, We want to use Fabric lakehouse as ingest layer, and did the setup according to this article: We are running TimeXtender version 6898.1We created the lakehouse form Fabric, not by using the ‘Create Storage;  option from TDI. When we test the connection in TDI, everything looks fine:However, the metadata import task fails with following error: The execution failed with error:Exception Type: Azure.RequestFailedExceptionMessage: Service request failed.         Status: 400 (BadRequest)         ErrorCode: IncomingOperationUntrusted                  Headers:         x-ms-error-code: IncomingOperationUntrusted         Access-Control-Allow-Origin: *         Access-Control-Allow-Headers: REDACTED         Access-Control-Allow-Methods: REDACTED         Access-Control-Expose-Headers: REDACTED         Strict-Transport-Security: REDACTED         X-Content-Type-Options: REDACTED         x-ms-root-activity-id: REDACTED         Content-Length: 0         Date: Thu, 06 Mar 2025 14:57:14 GMT 

675
Christian Hauggaard
Community Manager

Badge winners

  • Innovator
    oscarclevehas earned the badge Innovator
  • Certified Solutions Architect
    leo.noharethas earned the badge Certified Solutions Architect
  • Innovator
    Bernarddbhas earned the badge Innovator
  • Certified Solutions Architect
    JeroenStahas earned the badge Certified Solutions Architect
  • Ingest Expert
    nmohas earned the badge Ingest Expert
Show all badges
Terms and ConditionsCookie settings