To homepage To homepage

- - Knowledge base overview
Product Updates
Training
Portal
For Partners
TimeXtender Classic

Home
Community overview
TimeXtender Desktop Q&A
Ingest

Ingest

Ask questions about Ingest Instances in TimeXtender Data Integration

253 Topics
1,360 Replies

When you subscribe we will email you when there is a new topic in this category

253 Topics

Most replies

Most views

Newest first

Remy Kamphuis | VictaParticipant

asked in Ingest

JSON/REST Provider Ignores Filters in CBS OData URI

Hi Community,I'm trying to connect the database from CBS (in both old and new version of TX). The original request URI gives an error response that the query exceeds 10.000 rows at test connection. This is a well-known issue with data from CBS, so I adapted the URI to include a filter. I tested this new query both in Postman and Qlik Sense Desktop and there the filter works and I retrieve aprox. 2.500 rows, from 2020 until 2022. Using this query in TX does not work unfortunately. The error that normally encounters when testing connection does not pop-up, however, the table still retrieves 10.000 rows. Probably just random from all available years (1995 t/m 2022).I am using below request URI:https://opendata.cbs.nl/ODataFeed/OData/70072NED/TypedDataSet?$filter=substring(Perioden,0,4) ge '2020' and substring(Perioden,4,2) eq 'JJ'with custom header: accept:application/jsonAny ideas what causes this? Thanks!

22511

1 year ago

rory.smithTimeXtender Xpert

asked in Ingest

Upgrading only one ODX project/environment to new version for testing

At one of our customers we have a separate cloud for testing software / infrastructure changes. We are running 20.10.35 and want to evaluate 20.10.40. I would like to be able to upgrade the ODX Cloud repository for only one of the projects/environments instead of upgrading both to 20.10.40 and needing to risk production issues. Is this possble / can this be requested?

12115

1 year ago

avanrijnContributor

asked in Ingest

Connect to a Datalake through JDBC -> ODBC Bridge

Dear Sir,For one of our customers we need to connect with TX to a Datalake (CSI Cloud).Our customer is able to connect with PowerBI to their Datalake with a JDBC → ODBC Bridge piece of software from ZappySys JDBC Bridge Driver. This works fine.Now we need to connect with TimeXtender without this third party JDBC Bridge Driver software to save our client some ZappySys license money.So I checked with the CData component in TimeXtender, but it seems their is no JDBC → ODBC Bridge Driver available...Question: Is their a way to do this with the best Data Management Platform in the world ?Hope you can shine a light on this.Thanx in advance. Regards from Amsterdam,ArthurFourpoints TimeXtender Partner

1244

1 year ago

ignacioContributor

asked in Ingest

Load automatic process CVS Files

Hi community, I have to design the following issue, every day, I have a new CVS file in the Azure storage account. These file’s name change with a new day, for exampleFormaExtract.net_new_commission_2023-03-01.csvFormaExtract.net_new_commission_2023-03-02.csvFormaExtract.net_new_commission_2023-03-03.csvHow I could create an automatic process to load a new file to the table in DSA? Now I will do it manually adding every day a new file an process the data.ODX DSA Another requirement is to create a new column SourceMainTable in the DSA table with the file’s name. RegardsIgnacio

772

1 year ago

gerrit.degleeExplorer

asked in Ingest

Unable to setup ODX notification

I'm trying to setup ODX critical error notification. I've used the same settings as the regular TX notifications, but the test email never arrives. I do notice that in the ODX I do not have the option to enable SSL/TLS, which is needed. I'm using smtp.office365.com.

592

1 year ago

rory.smithTimeXtender Xpert

asked in Ingest

Upgrading data source provider version

TX: 6143.1 I have a TimeXtender SQL Data Source version 17.0.0.0 that I am trying to upgrade to 17.0.0.1. When I do Manage Data Sources in the ODX Server from the TimeXtender application it tells me there is an update available. When I apply this it tells me to ‘Edit the datasource’ to update the connection string. There does not seem to be an obvious way to do this. The User Portal also does not seem to have a way to update this.What is the process I should apply?

742

1 year ago

sguntupalliContributor

asked in Ingest

datediff function resulted in an overflow

Hi,I am using the ODX server to get data from the SQL server database.After executing the transfer task, I am getting the below error.Any idea how to solve this?“System.Data.SqlClient.SqlException (0x80131904): The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart.System.Data.SqlClient.SqlException (0x80131904): The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart.”

TamimStarter

Hi Support,We are using an ADO Data Source connection to load data in the ODX into Azure.From there we map the ODX table to the next layer in TimeXtender.This seemed to work.But now we changed the source table with added and removed columns.We sync and transfer it in the ODX and when we use preview in the ODX we are seeing the new table as expected. But when we map it in TimeXtender again to the next layer, the changes are not added. It instead maps to what is used to be. It mappes fields to columns that no longer exist, and it does not map the new added columns.We tried clearing the files in Azure to start fresh but it still mappes to the older version that no longer exist. Why would it still mapp to the old table that no longer exist in the source and no longer exist in Azure?Why would it not mapp to the new table thats in Azure now?Is there some metadata in TimeXender that first has to be cleaned before changes can be mapped?Kind regards,Tamim

682

1 year ago

rvgfoxProblem Solver

asked in Ingest

Dynamics 365 F&O Data Lake as a TX source

We are planning to use Export to Azure Data Lake feature in D365 F&OMy question it’s:How can connect to the Data Lake from TX?Or, we can connect to Synapse Analytics?How can we make the data source setup in TX?

51315

1 year ago

KajEmergoContributor

asked in Ingest

Extract CSV from SFTP with public key authentication

We have a SFTP source on which CSV files are stored and which can be authenticated to with a ‘public key’. When using the CData SFTP-connector in ODX Server, we are able to connect with the following settings: The most important setting here is the SSH Auth mode, where we specify that we are using PublicKey authentication. Since I am not familiar with this type of connector, I would not now how to retrieve the data from the CSV files (the only table we get from the source now is the ‘Root’ table that contains the FileNames that can be found in the folder).I would rather use the CData CSV-connector, but that leaves me with a problem when testing the connection: I cannot specify that I want to use the PublicKey authentication mode. That is, it is specified under ‘SSH Auth Mode’ but it is not possible to select under ‘Auth Scheme’: As a result, we keep getting the same error on testing the connection (see attached file). I have tried setting the Auth Scheme to ‘None’, ‘Auto’, ‘SFTP’ or ‘B

1943

1 year ago

KajEmergoContributor

posted in Ingest

CData Excel Online not showing shared Excel files

Hi,We are using the Excel Online connector which is authenticated with a service account and uses delegated permissions to access Excel files on Sharepoint. The idea is that all relevant files will be shared with this account and then loaded into our DWH. The required files are visible on Sharepoint when siging in with the user:But for some reason, they are not when using the Excel Online connector (with option Show shared documents = ‘True’). I know that the connector uses the /SharedWithMe (OneDrive) call to fetch shared items, since that is what I inferred from the logging.This call indeed retrieves no results through the Graph Explorer, but the files are visible through another, similar, call on the Graph Explorer (the one from ‘Insights’): Why are files visible on Sharepoint but not on OneDrive? Is there a way to work around this? I have seen use cases where they actually are visible on both Sharepoint and OneDrive and the connector is working properly.

24420

1 year ago

rvgfoxProblem Solver

asked in Ingest

Excel files in SharePoint

I’ve three excel files in a SharePoint folder:Budget Forecast Price ListWhat it’s the best option to get the files into a TXA DSA?Can you provide me with detailed instructions?

1673

1 year ago

Jan StroeveContributor

asked in Ingest

CSV source loads all fields as bigint

I have created a CSV source in Timextender NG ODX. The data explorer shows the data type correctly for my table.However, when I drag the table to the DSA, all fields appear as bigint type. (by the way, I have setQuote Character to “ and Row Scan Depth to 0). Any help on how to resolve this issue, would be appreciated.

1257

1 year ago

thzmanParticipant

asked in Ingest

ODX API version error

Hi!We have had our installation for a while now but as far as I’m aware the application has the latest updates. As we are becoming more people in the project working at the same time I have installed the client on both a new server and on my own laptop. The problem is that when I try to connect to the ODX service running on one of our “old” servers I keep getting the error as in the attached pictures. Why is this? As you can see the application has the same version on both thte “old” and the “new” server.The correct ports are opened between the servers at least so that should not be an issue but I get the same error when trying to connect from my laptop. Anyone who has an input on this?

1043

1 year ago

sguntupalliContributor

asked in Ingest

Selecting provider

I am using the 20.10.36.64 desktop version.I am trying to select a provider to connect the SQL server using ODX.I am not seeing any SQL-specific providers here.Which one should I use?

221

1 year ago

alain.thireContributor

asked in Ingest

Error connecting to DB2

Trying to connect to DB2 sql server with 20.10.6.64 but get an error (NO error message available) that is not documented. Connection is working in another environment. It is not the CDATA connector but DB2 from data source

261

1 year ago

hugo.winkelhorstParticipant

asked in Ingest

Amazon Redshift Connector SSL mode drop down missing options

For one of our clients I need to connect to a database on Amazon Redshift. The IT Vendor who made the system is telling us to use SSL mode Required ( https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html#connect-using-ssl ). Using this setting in the ODBC connector from Amazon itself works fine but the CData connector only gives a true false dropdown. I've checked if this perhaps was one of those fields where you can overrule the drop down with manual input but that's not the case for this one. Is there any way to get this working in the CData connector or would our only option for now by create DSN and then use the ADO ODBC option the the ODX to connect with the DSN.

1087

1 year ago

rory.smithTimeXtender Xpert

posted in Ingest

CData IBM DB2 - extracting results in The column [xxxx] does not exist

TX: 20.10.39ODX: 20.10.37ODX Storage: SQL Server DB on-premDSA/MDW: SQL Server DB on-prem We are successfully able to connect to a DB2 system and synchronize, we can also successfully use the Data Source Explorer to query tables. Every table we try to extract fails with the following error (schema/table/column names obfuscated): Executing table [schema].[table]:failed with error:edo220W.gm: The column [column] does not exist at edo220W.ai.GetOrdinal(String ) at System.Data.CData.DB2.DB2DataReader.GetOrdinal(String name) at System.Data.SqlClient.SqlBulkCopy.WriteRowSourceToServerCommon(Int32 columnCount) at System.Data.SqlClient.SqlBulkCopy.WriteRowSourceToServerAsync(Int32 columnCount, CancellationToken ctoken) at System.Data.SqlClient.SqlBulkCopy.WriteToServer(IDataReader reader) at DataStorageEngine.SQL.SQLStorageEngine.TransferWithBulkCopy(SqlCommand destinationCommand, String destinationTableName, IEnumerable`1 columnModels, IDbCommand transferCommand, DataTable transfe

21119

1 year ago

hugo.winkelhorstParticipant

asked in Ingest

Using a field from an ERP to use as a Parameter for a REST call

I need to be able to call an API using a reference number from the ERP system ( which is already in the ODX) as part of the URL. It looks like the Dynamic parameters as descibed on the old support site could work. However it is stated that this will only work in the old fashioned business unit. Is there a solution that would work in the ODX or would a Power Automate/Logic App be the only solution.

632

1 year ago

vw-victaContributor

asked in Ingest

Incorrect Job Repository: Expected 1 found 2, after downgrade

Hello,I'm working on a project (new license model) where we want to use the Azure Data Factory for Data Movement. I've installed the TX 6143.1 version but got the message “Data Factory source is out of date”. Looking at the release notes this version of TX doesn't support ADF Data Movement at the moment:“Warning: The new version does not support the data source providers that move data using Azure Data Factory (e.g. "Azure Data Factory - SQL Server (11.0.0.0) 64 bit").”I've decided to downgrade to TX version 6117 but I'm getting the message as shown below (screenshot). It looks like there is another Repository from both installations.How can I solve this and remove one of the repositories. I want to use the ADF Movement so the repository for version 6117 is needed.Thanks in advance!VinceVicta B.V.

1528

1 year ago

sierd.zuiderveldContributor

asked in Ingest

Error validating data storage version. The version is empty

Hi, I am trying out the new TimeXtender version and just set up my environment. Now when I use a transfer task I receive the following error: System.AggregateException: One or more errors occurred. ---> System.Exception: Error validating data storage version. The version is empty at DataStorageEngine.SQL.SQLHelper.GetDataStorageVersion(SqlCommand command) at DataStorageEngine.SQL.SQLStorageEngine.ValidateStorage(SqlCommand command, Boolean tryTakeOwnership, Boolean throwIfInvalidOwner) at DataStorageEngine.SQL.SQLStorageEngine.TransferData(IDataSourceEngine dataSourceEngine, ExecutionTaskModel taskModel, List`1 tableModels, Boolean forceFullLoad, SecurityExecutionModel securityExecutionModel) at ExecutionEngine.Action.StandardAction.Execute() at ExecutionEngine.Action.ExecutionAction.<.ctor>b__9_0() at System.Threading.Tasks.Task.Execute() --- End of inner exception stack trace ------> (Inner Exception #0) System.Exception: Error validating data storage versio

762

1 year ago

johannmStarter

asked in Ingest

ODX cdata XML missing attribute

I'm reading XML files from a folder using cdata XML provider. This is working fine and using incremental loading also works. The issue I'm having is that there is an attribute defined in my rsd file that is not always present in the XML files. When ODX starts reading the folder and the first file it parses doesn't have this field it skips the field for all following files. Similarly if the first file does have the attribute all the rest is parsed ok. Is this a bug or is there a setting I'm not aware of? Or maybe a definition in rds that could help?

1216

1 year ago

sturla.winger.svendsenStarter

asked in Ingest

Change db compatibility_level

Hi, We are running on MS SQL Server 2019, but all of the DBs created from TX are still on compatibility level 120. Will changing the compatibility_level impact the TX solution? br,Sturla

211

1 year ago

dylan brabecVisitor

asked in Ingest

SAP ByDesign Connection suddenly stopped working

Hi Guys, I had an ODX connection to a SAP ByDesign instance that suddenly stopped at the end of last week and spit out the below error log. I am looking for any ideas on the direction to go. From how I am reading the error, I am assuming its originating at the ByDesign end, and I need to find a location to whitelist DiscoveryHub. (doesn't explain why it was working then suddenly not working without any other changes). Thanks for any help. If you need assistance, please submit this with your ticket. Service request failed: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> rds190C.vX: HTTP protocol error. 401 Unauthorized. ---> rds190C.vX: HTTP protocol error. 401 Unauthorized. ---> rds190C.IX: HTTP protocol error. 401 Unauthorized.at rds190C.ERZ.q(String , String , FR , ERk , String , Boolean , String )at rds190C.ERZ.o(Boolean )at rds190C.FZZ.K(WDn )at rds190C.WaZ.q(Wg , WDt )at rds190C.WaZ.A(Wg , Wan )--- End of

471

1 year ago

dror.svartzmanStarter

asked in Ingest

Dynamically Import tables into ODX

Hi, My source database contains over 300 tables, from which only 30 are relevant for our purposes. Instead of manually searching and checking the required tables, Is it possible to define a list of objects of interest(text) and let TX pick it from there? Kind regards, Dror Svartzman

686

1 year ago

Page 9 / 11

Badge winners

Dikshahas earned the badge Certified Solutions Architect
brett.taylorhas earned the badge Certified Solutions Architect
RobvdBrinkhas earned the badge Certified Solutions Architect
aftabphas earned the badge Ingest Expert
Leo Rogemanhas earned the badge Certified Solutions Architect

Show all badges

Terms and Conditions Cookie settings

Sign up

Already have an account? Login

Login with SSO

SSO login

Username *

E-mail address *

Full Name

(Private)

Only you and moderators can see this information

TX Partner / Customer *

Company

(Private)

Only you and moderators can see this information

Password *

I accept the terms & conditions

loginBox.register.email_repeat

Login to the community

No account yet? Create an account

Login with SSO

SSO login

Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.