Skip to main content
To homepage To homepage

Ingest

Ask questions about Ingest Instances in TimeXtender Data Integration

  • 253 Topics
  • 1,360 Replies
253 Topics
P
Contributor
Paul BridgeContributor
asked in Ingest
Overnight Jobs Not Run - Failing ODX Service

HiWe experienced for the second time a failure with our overnight TX jobs not running (first time was on 16 Sep) and on investigation we were not able to open up the ODX in TimeXtender (same issue as already logged here https://support.timextender.com/ingest-89/tx-freezes-when-opening-the-odx-2248)Could not manually stop and restart the  [TimeXtender_Ingest_Service_6744.1] service, so had to manually kill the task from the CMD and rerun the [TimeXtender Ingest Service Configuration 6744.1] tool.In the other post the conclusion seemed to be that this issue had now been resolved, but for us the issue has now occurred twice in the last 6 or so weeks.Is this still a known issue?TimeXtender Ingest 6744.1TimeXtender Data Integration 6745.1

441
Christian Hauggaard
Community Manager
1 month ago
M
Starter
MaarsenStarter
asked in Ingest
GA4 -- Multiple Properties

Hi, I'm currenty developing the GA4 connector in TimeXtender. With some other tips and tricks of other posts it was quite simple to create the connection. At this moment I have 7 different properties from where all data should be loaded. I've read multiple posts but haven't found a solution to extract data dynamically over a list of properties.  Current setup:CData Connector -- Google Analytics In this connector I can filter on property ID. TimeXtender -- v6xxxx + ODX Creating managed Queries to filter the relevant data from [GlobalAccessObject]. But haven't found a property id in this list.  I think that currently the only option is to create for each property one connection. Or has someone found another solution?

494
andrew.gebhard
TimeXtender Xpert
1 month ago
M
Visitor
MarcoVisitor
asked in Ingest
Ingest Data from Infor Data Fabric

Hi, we are in a project to update our on premise ERP system Infor to the current version Infor LN CE which is running on the cloud. Since we cannot access the database then directly we are currently checking how to ingest the data to TimeXtender in future. There is a tool within the ERP suite to provide the data to "Infor Data Fabric" which is an Amazon S3 data lake where the ERP data is stored in JSON files. This data fabric provides a SQL Endpoint to query the data. So similar architecture as in MS fabric with lakehouses. The data can be queried in two ways:- JDBC: using a JDBC driver with API Gateway authentification, see    Link- API: handle API request in a certain order to start a query job, aks its status and get the results, see link:        https://www.youtube.com/watch?v=m4d0awIQ6AgFor TimeXtender, which would be the way to prefer? I could not find the possibility to add a JDBC datasource with API authentification and I am wondering if TimeXtender can handle the process order

542
Thomas Lind
Community Manager
1 month ago
P
Explorer
Pieter-JanExplorer
asked in Ingest
the table folder is used by another table

Hi,i have a gen2 storage account with a blob container with multiple parquet files. when i do a sync with odx it’s succesfull, the transfer gives me for 4 files the message “ is used by another table”. all other files in the same map go ok. have tried to recreate the blob container, and recreate the parquet files again, same results.Exact message:The table folder 'parquet_taak_status' is used by another table with the id 9a510177-a130-4bcc-aaaa-5abefd42c1b9 how to solve / find the other table that is referenced to? thanks!  

1034
rory.smith
TimeXtender Xpert
1 month ago
S
Explorer
svaExplorer
asked in Ingest
Issues Retrieving datetimeoffset Columns with ODBC in TimeXtender

Hello everyone,I'm facing an issue with TimeXtender when trying to retrieve tables containing datetimeoffset columns through the Generic ODBC Data Source connector. The datetimeoffset columns aren’t coming through, and in the ODBC settings, the option to override datetime types is greyed out, which I thought might solve the issue.Has anyone encountered this or found a workaround to retrieve datetimeoffset data through ODBC in TimeXtender?Thanks in advance!

293
rory.smith
TimeXtender Xpert
1 month ago
S
Explorer
svaExplorer
asked in Ingest
Issue with Retrieving a Column with the Same Name as a SQL Function

Hi everyone,I'm facing an issue where one of the columns in my destination table has the same name as a SQL function, and because of this, I'm unable to retrieve it in a normal way. To work around this problem, I wanted to use the Query Table Tool to retrieve the column in a parameterized way.However, for this specific data source (which is a Generic ODBC Data Source), the Query Table Tool option is greyed out. I've noticed that the Query Table Tool is available for other data source types.Has anyone encountered this issue before?Thanks in advance!

606
S
Explorer
1 month ago
D
Contributor
dennishanssenContributor
asked in Ingest
Error when filtering columns BC Online

Using the TimeXtender Dynamics 365 Business Central - Online connector, I'm having trouble filtering columns in the ODX. With all columns selected everything is fine, but once I make a selection in columns I get the error below when running the Transfer Task. Any ideas on why this is happening and how to fix it? (TX Version 6XXX) Executing table customer:failed with error:Exception Type: System.InvalidOperationExceptionMessage: Sequence contains no matching element             Stack Trace: at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)                          at TimeXtender.ODX.Account.BC365CustomDataSource.ExecutionHelper.GetIncrementalLoadPartitionSetups(AccountBC365CustomDataSource accountBC365CustomDataSource, TableModel tableModel)                          at TimeXtender.ODX.Account.BC365CustomDataSource.AccountBC365CustomDataSource.GetIncrementalLoadSetups(TableModel tableModel)                          at DataSourceEngine.Custom.CustomSourceEng

447
D
Contributor
1 month ago
avanrijn
Contributor
avanrijnContributor
asked in Ingest
Cdata Sharepoint connection Data Model feature

Dear Sir,I am connecting through Cdata to Sharepoint Online.I have hierarchies in my Sharepoint data, I can see them when I connect with PowerBI to my Sharepoint site. Can I use the Data Model feature ‘Flatten Documents’ like you can with JSon connectivity ?It looks like CData for Sharepoint Online doesn't have this capability. Can I somehow bring these ‘hidden fields’ into my TimeXtender  project ?In the attachment (Knipsel.PNG) you can see the hierarchical option in PowerBI with PowerQuery. I miss these fields when I connect with TX Cdata to the same Sharepoint site.Hope you can shine a light on this.Thank you in advance for looking into this case. Regards,Arthur 

212
avanrijn
Contributor
2 months ago
Pontus Olausson
Participant
Pontus OlaussonParticipant
asked in Ingest
Error 'Buffer cannot be null' in ODBC Connection

 Hi,I’m am trying to create a ODBC data source connection in the TimeXtender ODX portal.I have created a ODBC data source on the server on which TimeXtender is run, see picture below. This data source seems to work fine, I am able to test the connection with success and I am able to connect to the database and query the data through PSQL control center on the server.ODBC Data Source on TimeXtender VM In the TimeXtender portal, my configuration is as follows:TimeXtender portal configurationWhen trying to test the connection, I get the error “Buffer cannot be null. Parameter name: buffer”.I have tried adding buffer as an additional property without success.I am unable to find documentation or guidance on how to solve this issue. Since I am able to connect to the database and query the data through PSQL control center on the server on which TimeXtender is run, the ODBC data source on the server seems to be working correctly. It is only through the TimeXtender portal I am unable to use the

8210
Pontus Olausson
Participant
2 months ago
S
Contributor
sigsolContributor
asked in Ingest
How to use Rolllup task

Hello Community!When running a rollup task, the resulting rollup file ends up in a separate folder inside the same full load folder: After the rollup, new incremental files continue to appear in the DATA folder and the old incremental files are still there. What have we achieved by doing the rollup? Is the new rollup file being used at all? How can we delete the old incremental files given that the delete Storage Management task only removes entire full load folders and we don’t perform full loads?Currently I don’t understand how we can clean up the many incremental files, both for load performance and storage management.Please note: We cannot rely on doing regular full loads because of the size of the tables.Thanks!TX version: 6745.1Connector: TimeXtender SQL Data Source 21.0.2.1We are using Azure Data Lake and ADF.

918
Christian Hauggaard
Community Manager
2 months ago
J
Explorer
JorisKammaExplorer
asked in Ingest
Business Central Online Connection Error

I’m trying to connect to Business Central Online using the Time Xtender connector. I’m using a service principal which has an app registration in Azure and I’ve run through all the steps in the BC/TX tutorial.I’m getting the following error message when I try to add the data source. Any ideas what might happen Service request failed: Sequence contains no matching element (System.InvalidOperationException)Details:Service request failed: Sequence contains no matching element (System.InvalidOperationException)Module: TimeXtender.ODX.EngineTimeXtender.ODX.Engine.ODXFaultException   at TimeXtender.ODX.Engine.ODXEngine.SendServiceRequest[C,T](WcfServerSettings serverSettings, Func`3 func)   at TimeXtender.DataManager.AddODXDataSourceWizard_SelectConnectionStep.<>c__DisplayClass11_0.<Validate>b__0()   at TimeXtender.DataManager.ConnectingThread.ExecuteConnectingThread(Object dummy)Service request failed: Sequence contains no matching element (System.InvalidOperationException)Modul

778
J
Explorer
2 months ago
R
Participant
Remy Kamphuis | VictaParticipant
asked in Ingest
Error BC365 connection after testing connection

Hi support,Following this ticket I will make one myself to keep things clear. We have set up a Business Central 365 connection, both via OAuth2.0 as service to service authentication. We have succesfully tested the connection with both authentication options. However, when we want to read objects, we receive the error “Invalid response from Business Central retrieving table Field for company”. Full log is added to this ticket as an attachment. We have walked through all steps from the prerequisites guide. Hence, we have a succesful connection. From the other ticket I understand that there are more versions of the Jet Analytics for Business Central extension. It has to be the newest one and I have to keep on trying. In the Extension Marketplace we only find two versions of the Jet Analytics for Business Central extension. The first from Jet Global Data Technologies is not available anymore and they point to the second; the one from insightsoftware. This is the one we have installed; ver

8410
Thomas Lind
Community Manager
2 months ago
T
Participant
thzmanParticipant
asked in Ingest
TimeXtender REST dynamic values - error on Sync

Hi!I’m doing some tests and trying to set up som e form of incremental load with this connector to an API endpoint. It works fine with the two static parameters but as soon as I add the parameter for lastSynchronizedRowVersion with its dynamic value I get an error with I run the Synchronize job.Is there something obvious that I have missed here?I tried to follow the article Connect to Fixer IO with the TimeXtender REST data source | CommunitySetup for the endpoint: Error message: 

372
T
Participant
2 months ago
H
Starter
HansZZVStarter
asked in Ingest
Error when opening project on TX version 20.10.51.64

Hello,when i try to open my project on my development server i get the below message; Details:Error loading steps for execution package Refresh PowerBI - PModule: timeXtenderSystem.Exception   at TimeXtender.DataManager.ExecutionPackage.LoadSteps(HashSet`1 projectSteps)   at TimeXtender.DataManager.Project.LoadExecutionpackages(List`1 listToUpdateWhenAllIsLoaded)   at TimeXtender.DataManager.Project.InitProject(Int32 version, DsProject loadDsProject, DataSet adaptersDataSet, ProjectsRow projectRow)   at TimeXtender.DataManager.Project.LoadProject(DsProject _dsProject)   at TimeXtender.DataManager.Project.<>c__DisplayClass136_0.<LoadProject>b__0()   at TimeXtender.DataManager.ConnectingThread.ExecuteConnectingThread(Object dummy) Error loading steps for execution package Refresh PowerBI - PModule: timeXtenderTimeXtender.DataManager.ExceptionWrapperException   at TimeXtender.DataManager.ConnectingThread.HandleError()   at TimeXtender.DataManager.ConnectingThread.Execute(Strin

222
Thomas Lind
Community Manager
2 months ago
T
Participant
thzmanParticipant
asked in Ingest
TimeXtender REST data source doesn't update endpoint names

Hi!I have set up a REST connection in my Ingest instance to an API with multiple endpoints. The first time I misspelled two of the endpoint names, which I later changed. Then I noticed that the “old” names are still active in the ODX and does not get updated despite multiple refreshes (see attached screenshot). We have already done some work on some of the endpoints so I’m a bit reluctant to delete the whole connection and set it up again just to get these names updated. Is there some other way to fix this?  

423
T
Participant
2 months ago
B
Contributor
BennyContributor
asked in Ingest
Failed to read metadata from XmlDocument for endpoint X with transformation 'T' Input string was not in a correct format.

The sync of my ODX datasource returns the following error.  The datasource is a REST API endpoint of the form URL/{foreign_ID}, where foreign_id is a dynamic value that comes from a different endpoint within the same data source. The option "Perform exhaustive meta data scan” is enabled. The error returned is this:Failed to read metadata from XmlDocument for endpoint X with transformation 'T'Input string was not in a correct format.There is no further information.  The sync does work when I replace foreign_ID by a fixed value.  The endpoint contains a field that has a lot of values that look like integers, but some values contain letters. Since the above error message is the most verbose info I’ve got, I assumed that's probably the cause. However this conflicts with my understanding of what an “exhaustive meta data scan” should do.  Any help is appreciated. 

869
Christian Hauggaard
Community Manager
2 months ago
daniel
TimeXtender Xpert
danielTimeXtender Xpert
asked in Ingest
Connecting to Azure Active Directory (AAD)

Dear Community,I’m having some issues with connecting to the Azure Active Directory. We want to use this data for RLS and feed this from the DWH.I’ve set up the Active Directory Connector and put in credentials:  After a minute or so I get an timeout. If I leave out the OAuth stuff I get an not authorized. What do I need to set up here to get the data from the AAD? Thanks!

483
Thomas Lind
Community Manager
2 months ago
daniel
TimeXtender Xpert
danielTimeXtender Xpert
asked in Ingest
JSON connector only get 1st JSON file from Blob

Dear Comunity,I’ve got a blob folder which contains 10 JSON files. I’ve set everything up but it only gets 1 file (the first) from the blob. Also only 1 RSD files get created. Is there something in the connector that needs to change in order for the connector to get more then 1 file? Thanks in advance

391
Christian Hauggaard
Community Manager
2 months ago
P
Contributor
Paul BridgeContributor
asked in Ingest
Jobs Not Running On Schedule

HiI have recently noticed a problem where some of my scheduled jobs do not run at their scheduled times. When digging about I noticed in the event viewer the following;Running the [TimeXtender Execution Server Configuration 6698.1] tool to unlink and relink the instances has not resolved the issue.Running the jobs manually work fine.Please advise on next steps

491
Christian Hauggaard
Community Manager
2 months ago
H
Starter
HansZZVStarter
asked in Ingest
Filter rows in ODX on multiple substrings

Hi I am trying to find a way to filter rows in ODX on multiple 'like’ expressions.Below the SQL statement i use in SQL Server Manager that i would like to implement;SELECT [autoid], [values]FROM TableAWHERE [autoid] > 61103544AND  ([values] like '<N V="D%' or [values] like '<N V="VAS%')How can we achieve this in ODX so I can limit the amount of records retrieved from the source.Thanks in AdvanceHans

332
H
Starter
2 months ago
Roy V
Problem Solver
Roy VProblem Solver
asked in Ingest
Error when processing data from odx to snowflake

Good afternoonWe are working on the migration to fully deploy to Snowflake. We created a sas token as described in the instruction (https://support.timextender.com/configure-resources-126/use-snowflake-data-warehouse-storage-727) and added the token in the ODX settings in the portal.  However when we deploy and execute to Snowflake the deployment goes fine but when executing we get the following error:Failure using stage area. Cause: [This request is not authorized to perform this operation. (Status Code: 403; Error Code: AuthorizationFailure)]In Snowflake the user has enough priviliges but it is when running the stage that want to read from the data lake storage that this error occures.The storage account has enabled that you can read from it with keys so that should not be the issue.Any ideas on why Snowflake can not access the data lake?

645
Roy V
Problem Solver
2 months ago
P
Explorer
Pieter-JanExplorer
asked in Ingest
loading odx project information hangs

Hi, if i start timextender the first thing doen is “loading odx project information” this hangs. my version is 20.10.46.64 already restarted whole server , didn’t make any difference

443
Bernarddb
TimeXtender Xpert
3 months ago
oscarcleve
Contributor
oscarcleveContributor
asked in Ingest
Error synchronizing TimeXtender Dynamics 365 Business Central - Online data source

I get the following error message when synchronizing a TimeXtender Dynamics 365 Business Central - Online data source: “System.AggregateException: Ett eller flera fel har uppstått. ---> System.ApplicationException: Invalid response from Business Central retrieving table Field for company “ Authorize OAuth as well as Test Connection is successful in TimeXtender desktop. It seems like the Tenant and the Environment parameters are correct as connection fails when I alter these parameters. I am able to authorize and connect regardless what I input in the fields in the Account section. I use the Service to Service authentication with Client ID and Client Secret from the Azure app registration. I use TimeXtender v. 6718.1 and ODX Service v. 6718.1. Any suggestions on what to try next? 

1469
R
Participant
3 months ago
W
Explorer
woutersmikeExplorer
asked in Ingest
Data movement ODX -> DSA with mapping set

HiIt looks impossible to add a new field from the ODX onto the DSA, once you have added fields before. I’m not able to drag a new field (one of the grayed out fields) onto the table. Only solution seems to drop the existing mapping set and restart. Am I missing something? Situation as is:We are using TX-version is V21 Different agencies, all using the same ERP. Have created different ODX-sources for each agency. Created a mapping set to import data from ODX to DSA. Mapping set is a handy feature. Instead of dragging each data source individually, you can create a mapping set that will pull in all the tables with the same name and structure Problem seems to be that when you have created the mapping set, that you can’t change/add something afterwards. This can’t be the case I assume.In the example I have added 14 fields before. I now want to add (1) more but the application won’t let me drop it on the table (or mapping set).  Thanks for your thoughts/solutions

722
Christian Hauggaard
Community Manager
3 months ago
L
Visitor
lindaj.bekkerVisitor
asked in Ingest
Exception from HRESULT: 0xC020801C

Error creating DTS package Exception from HRESULT: 0xC020801C I can't deploy the data and can't syncronize the datasource. I am getting this error. Exception from HRESULT: 0xC020801C Please help

683
B
Starter
3 months ago

Badge winners

  • Certified Solutions Architect
    Dikshahas earned the badge Certified Solutions Architect
  • Certified Solutions Architect
    brett.taylorhas earned the badge Certified Solutions Architect
  • Certified Solutions Architect
    RobvdBrinkhas earned the badge Certified Solutions Architect
  • Ingest Expert
    aftabphas earned the badge Ingest Expert
  • Certified Solutions Architect
    Leo Rogemanhas earned the badge Certified Solutions Architect
Show all badges
Terms and ConditionsCookie settings