To homepage To homepage

ODX

Ask questions about TimeXtender ODX Instances

  • 169 Topics
  • 966 Replies

169 Topics

S
Contributor
sigsolContributor
asked in ODX

Paging with cursor in TimeXtender REST API 6.1

Hello,We are currently utilizing the new TimeXtender REST API 6.1 and leveraging its built-in paging feature to connect to an external REST API. We're not using an RSD configuration for this setup.I've encountered a challenge with pagination, specifically with the use of a parameter named "Cursor." This parameter is critical for the pagination logic in the API we're connecting to. Each row in the data returned by the API has a unique Cursor value, but for pagination purposes, only the first Cursor value from the current page should be used in the subsequent POST request to fetch the next page.Here’s how we've set up the Cursor parameter:Name: cursor Type: XPath Value: /TX_Autogenerated_Root/TX_Autogenerated_Element/data/arTransactions/edges/cursorThe issue arises because I need to ensure that only the first Cursor value from each fetched page is used in the pagination post body for the next request. I'm unsure how to configure TimeXtender to utilize only the first Cursor value for this

8
Thomas Lind
Community Manager
1 day ago
K
Contributor
KCMTContributor
asked in ODX

Set up semi-incremental loading, e.g. UpdateDate >= 01-01-2024

v20.10.35.64I just noticed there’s a bug in the current version of our ERP related to the UpdateDate & UpdateTS field which impacts incremental loading. Hence, I'm investigating the possibilities to set up a semi-incremental loading infrastructure. With my TX version I can also set up the following, which is limited:  My database has data ranging from 2013-today. I'm looking to find a way to load 2013-2023 just once and then only full load the data with a UpdateDate >= 01-01-2024. I'm thinking to achieve this the following way:Set up a Query Table for every table where UpdateDate >= 01-01-2024. Set up a Load task, LOAD_FULL which has the original tables Set up a Load task, LOAD_INCREMENTAL, which has the Query table with just data from 2024. Execute the LOAD_FULL task just once. Schedule & Execute LOAD_Incremental every 30 minutes. If I do it this way, it's not a classic incremental loading process.Can I still utilize TimeXtenders ODX/DSA incremental loading functionaliti

8
Christian Hauggaard
Community Manager
2 days ago
daniel
TimeXtender Xpert
danielTimeXtender Xpert
asked in ODX

ODX server only SQL server authentication

Dear community,Is the only way to connect the ODX server to an Azure DB still on SQL Server authentication?No Windows or Azure integrated security yet?Thanks!= Daniel

2
Christian Hauggaard
Community Manager
3 days ago
S
Contributor
spenningContributor
asked in ODX

Referencing a content of local file in RSD / avoiding credentials in the portal?

Hi, I was wondering if it is possible to reference to the content of local file on the ODX server from within a RSD file. Such as Get-Content with Powershell.  We have some RSD files for some of the api’s call that we are making. We are in the process of including these RSD's in our own source control system in order to keep track of the changes that we do with the RSD files. One of the api's is quite unstable and sometimes needs tweeking in the rsd file. (Source control on the RSD file's / powershell scripts files could be another topic to be honest).Anyway, we have to include an API key in the script that we would not like to have saved in the source control. Therefore, an option to reference a file in which we can store the API key would be great.  I know that you can also include authentication in the portal, but we are looking to maybe avoid putting credentials as much as possible in the portal. The portal also creates a OAUTH map if you use OAUTH in the portal. But I don't know h

2
S
Contributor
4 days ago
S
Explorer
Stijn HensenExplorer
asked in ODX

Importing Exchange attributes in Active Directory

Hi guys,We are creating a users table in Timextender based on Active Directory.  However, there’s also an on premise Exchange server, adding Exchange attributes to the user in Active Directory.  But it seems we cannot read these attributes.  We use the CData Provider for Active Directory in our ODX server.  When trying to add the fields, the Exchange attributes are not in the list.  How can we add these attributes?  Do we need the CData Provider for Exchange?  If so, has anyone set that one up and is willing to provide more details? Or how can we manage to import all the Exchange attributes in our user table? Thanks!Stijn Hensen

15
Christian Hauggaard
Community Manager
4 days ago
rogier.helmus
Contributor
rogier.helmusContributor
asked in ODX

TX not responding when opening ODX task log (v20.10.45)

Hi all,I have v20.10.45 running in Azure with ODX server running for a client. We have about 15 ODX datasources. The synch/loads run daily and the server has been up for about 9 months, so no extraordinary amount of logging I think. The issue that I’m having is that it’s not possible anymore to check the execution logs of  some ODX task runs. The popup doesn’t appear and TX goes into a “Not Responding” state. Any click in the client will crash the application. This happens when I have no TX project opened, only the Manage tab for ODX server (see screenshot).I’ve upscaled the VM from DS2v2 (2 core, 8GB) to D4S_v3 (4 core, 16GB), but this doesn’t change it. Restarting ODX server also doesn’t help.The issue is not with all tasks. Tasks for which all recent executions are successful will show the logging. Checking the event logging at ODX server level will crash and tasks with a lot of recent failures also crash. Edit 1: A connection that has this logging issue is a connection using the CD

6
Christian Hauggaard
Community Manager
8 days ago
F
Explorer
FENExplorer
asked in ODX

SSIS Transfer failure Package because of the packagename length

Hello dear TX Community,We want to load tables from our ERP System, and we are encountering the following error: HRESULT: 0xC0010006.This error indicates that the name of the SSIS Package created is too long, and we need to shorten it. I've attempted to shorten the name by modifying the schema name we agreed upon, but this would only be a temporary solution as there are more tables with longer names. For your information, the package name typically consists of the following parameters: connector name - schema name - table name - SSIS-ID, for example, sql91_DBTest_zzdomAG_DBTest_dbo_SalesAppointment_3d5329a2-54ab-42a1-9126-4c21d569131ce.Now, I have the following questions:1. How can I manipulate/shorten the package name apart from changes I can make to the connector name and schema name (in TX)?2. Is the SSIS-ID always generated with the same string sequence, and is there a specific use case behind it that requires the ID to be formatted this way?kind regards Faras E.Nazeri

4
Christian Hauggaard
Community Manager
9 days ago
S
Contributor
sigsolContributor
asked in ODX

Incremental load in TimeXtender REST API 6.1

Hello,We are looking to implement incremental data fetching from REST API endpoints, specifically by dynamically filtering with an updatedAt parameter. The goal is to use the maximum updatedAt value previously fetched from each API call, which is stored in our ODX tables.While I'm aware of the capability to use dynamic parameters based on endpoints and queries within TimeXtender, I'm struggling to find a method to reference the maximum updatedAt value already loaded in the ODX tables for this purpose.Has anyone implemented a similar dynamic data fetching mechanism in TimeXtender? If so, could you share how you managed to reference data from the ODX tables to set the parameters for the next API call?Thanks!

3
Thomas Lind
Community Manager
9 days ago
P
Contributor
Paul BridgeContributor
asked in ODX

Issue With Data Source Connection No Longer Working

I have a number of Oracle data connections (TimeXtender Oracle Data Source) which have been set up and working against the same data source for a few months now. However, just recently, one of these data connections stopped working and started reporting timeout issues after 1 hour of connecting. I have multiple cloned versions of this same Oracle connection, and it seems to be only one of the cloned connections which is having this issue. I have since cloned one of the working connectors and added the 6 tables that are pulled by the failing connector (to mimic the failing connector setup) and this temporary connector runs without issue, completing in just a few minutes. As far as I can see, the 2 connectors are set up exactly the same way, its just the the original does not work whilst the new temporary test connection does. It may not be related, but this connector started failing one day after I had performed an in-place upgrade of both the TX ODX server and desktop client. However,

17
P
Contributor
10 days ago
E
Contributor
ebayContributor
asked in ODX

multiple Excel files to one

HelloEvery day, I have a new Excel file in the Azure storage or OneDrive account. I use OneDrive to test phase. These file’s name change with a new day. How can I this multiple Excel files to one? Is there a option for Excel like aggregate files option in CSV?Best Regards

14
Thomas Lind
Community Manager
15 days ago
M
Starter
MarcodenijsStarter
asked in ODX

Rename table from REST API

We have a REST API that we use to connect to several endpoints. The default name of all the tables is: REST.Jsondata. This causes issues in our Azure Gen2 datalake. It gives an error saying the table name is already in use by a table with a different ID.I almost feel like there must be some really easy way to rename the table name before it lands in the Gen2 datalake but so far I haven’t been able to figure out how.The workaround I use now is to use a QueryTable but that is not the nicest way to do it I guess. 

2
M
Starter
23 days ago
M
Starter
MvestStarter
asked in ODX

Graphical bug or authorisation issue? Certain functionalities missing or disabled.

Hello,My organisation recently had to redeploy our development virtual machine and reinstall timextender / odx on it. Since then I am not allowed to click on certain buttons inside the application, and options are missing when I right-click on certain objects.For example I am not allowed to click on the [Stop Task] button in the ODX Execution Queue window. And when I right-click on an object in the Manage ODX tab, all I can see it ‘View Properties’ and not any of the many other options that used to be there.I do have these functions available in our other environments. And my collegues have no issues on our dev machine.I have tried the following so far:Enable/disable Deprecated Features. Delete the hidden Default.rdp file and create a new rdp connection to the machine. Any tips?

4
M
Starter
30 days ago
rvgfox
Problem Solver
rvgfoxProblem Solver
asked in ODX

Two ODX in the same Storage container

Currently I’m using one ODX instance with this setup:  All the mapped data sources connections has selected the option “Enabled data on demand”:  I am using three instances of Data Warehouse: Development → Testing → Production.My question is:Can I create a new ODX instance connected to the same Storage container?The idea is that the same data sources in the DW instances for development and test use the "Enable data on demand" OFF option. In this way the storage container only will be refreshed one time (when we refresh the DWH production instance) because the data source is the same. 

5
rvgfox
Problem Solver
30 days ago
M
Starter
majeddigStarter
asked in ODX

Transfer task doesnt work - sql database - doesnt show any table in ODX select table

Hello Guys. I tried to select table from ODX but it deosnt show none (except once query table).the transfer task work fine (it show succefull) but once I triy to select table in order to send them to the datawerhouse I show nothing.- even if it work fine when I use query tool. Any suggestion please? thanks@Thomas Lind   

5
Christian Hauggaard
Community Manager
30 days ago
wouter.goslinga
TimeXtender Xpert
wouter.goslingaTimeXtender Xpert
asked in ODX

CSV load fails with operation could not be found error

I am loading a CSV (comma separated) file with the ODX. After upgrading the ODX to the latest version, transferring any table with this datasource gives the following error: Syncing does work.ODX version: 6521.1TX version: 6536.1CSV Provider version: tried both 22.0.8276.0 and 23.0.8749.0 but same error.

13
T
Starter
1 month ago
christian.koeken
Contributor
christian.koekenContributor
asked in ODX

Incremental loading with APIs

Dear Support,My customer is using AFAS. From this tool they load data into TimeXtender with an API.They have a wish to load the data from AFAS incrementally in the ODX.Is it possible to load incrementally from an API to the ODX server and what are your suggestions to start with? Best regards,Christian Koeken

13
Christian Hauggaard
Community Manager
1 month ago
S
Contributor
spenningContributor
asked in ODX

TimeXtender Desktop 6536.1 issue with ODX incremental load?

Hi,We have only source in which we are using a rest source and incrementally loading the data in the odx. The api cannot handle a date range that is too big. When we trying to preview the data in the odx, we see the data that we need. However, when we run the odx transfer task, with syncing it does not give all the updates in the table like it should. I saw that there were some fixes with the previous versions with incremental loading. We upgraded today to 6536.1. Also we upgraded the rest data source to the newest version. However, because the preview is fine, I don't think the problem is the source connector. 

2
S
Contributor
1 month ago
S
Contributor
sigsolContributor
asked in ODX

ODX Rollup task does nothing

Hi,I have set up a Storage Management Task with only the Rollup option ticked, and filled in the right ADF credentials. One of my source tables has been selected for this task.TX says the runs are successful and they finish in a fraction of a second, but nothing happens in the storage account. The version folders contain the full load file + all the incremental files, as before. I have tried with many different settings for min and max rollup file size. I have not specified “folder (optional)”. I have tried with and without specifying integration runtime.No pipeline run seems to show up in ADF after triggering the task.

8
Thomas Lind
Community Manager
1 month ago
A
Contributor
aftabpContributor
asked in ODX

Getting BLOOB file into DVH

I need some guidence saving contents ofan endpoint as BLOOB. I can see that the version we use doesnot have an option for BLOOB but may be Text or Binary with maximum length can give me what i need.So I have this RSD that i created <api:script xmlns:api="http://apiscript.com/ns?v1" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <!-- See Column Definitions to specify column behavior and use XPaths to extract column values from JSON. --> <api:info title="GetAllStops" desc="Generated schema file." xmlns:other="http://apiscript.com/ns?v1"> <!-- You can modify the name, type, and column size here. --> <attr name="kortnavn" xs:type="string" readonly="false" other:xPath="/json/kortnavn" /> <attr name="tittel" xs:type="string" readonly="false" other:xPath="/json/tittel" /> <attr name="virksomhet" xs:type="string" readonly="false" other:xPath="/jso

8
Thomas Lind
Community Manager
1 month ago
J
Participant
JeroenSParticipant
asked in ODX

No tables visible in Select Table Option on Data Source after synchronizing

We are running TX version 6536.1.Our data source is AX Dynamics 2012 on premise. We have the issue both with the SQL Server connector and the OLEDB connector.After synchronizing, we can't select any tables with right click on the data source → Select Tables → Search.It also takes > 30 minutes before the table selector pops up.Our only option is to select all tables.The Transfer task - Select table option does give us the option to select tables. However we can't select any individual columns. Neither does this give us the option to define any incremental selection rules or row filters.

13
Thomas Lind
Community Manager
1 month ago
V
Participant
VictorParticipant
asked in ODX

ODX Transfer task running indefinitely

Hello, TimeXtender: 20.10.40.64ODX: 20.10.31We are using the ODX and TimeXtenders SQL database connector to get data from a synpase database. Sometimes the synpase databas seems to have problems, where a query runs and doesn’t complete (a seperate issue we are working on).However what happens in TimeXtender in the ODX is that the transfer task keeps running indefinitely and never times out, leading timextender execution package to run but just sits there waiting for the odx transfer task to finish. However nothing fails anywhere, so we do not get an alert.We need some way to be able to capture this.Is there anyway to set up one or both of these things in TimeXtender or if you have any other ideas?1. Kill the transfer task if it takes more then X minutes2. Get an alert on failure or long running taskThank you! 

20
V
Participant
1 month ago
J
Starter
junweiStarter
asked in ODX

Issue of data source connection

Hi,I try to connect data source to ODX from TimeXtender Portal. I tried with different data provider (Postgresql and CSV). Both data source are stored in the same machine as ODX server and TimeXtender Server. While I got the error of “Test connection request timed out. Please make sure your ODX server is running and try again.”.The TimeXtender server and ODX server are both running. I checked the connection with command, and it is ok. Test-NetConnection timextender-repositories.database.windows.net -Port 1433

8
J
Starter
1 month ago
J
Starter
jyothi.dosiStarter
asked in ODX

CommonDataService Connector

HI Team, Please can you help me understand how to connect TX with common data service connector? I am trying to bring the data from D365 field service to TX. With Power BI i can use common data service connector and extract the data from FS. But i want to use TX to do ETL. Please help!!

2
Christian Hauggaard
Community Manager
2 months ago
sierd.zuiderveld
Contributor
sierd.zuiderveldContributor
asked in ODX

Primary Key Violation using Incremental Offset

Hey all,  We use incremental load in the ODX on a datetime field and we use an offset of 3 days (again in ODX). Now we've noticed that when we load a table from ODX to the data warehouse (even when full loading) that the _R table can contain duplicate records. In our Data Lake each increment is a parquet file of the last 3 days of data, since we load every day these files can overlap by 2 days. I believe this overlap is the cause of the duplicates in our _R table.  What happens next is that the Error report gets spammed with “Violation of primary key constraint” (which is true). But our Valid table does have the expected result. A full load transfer task solves the issue, but of course we do not want to do that all the time.   Is this the intended design? Is there a page that explains how the ODX handles these parquet files before inserting them into the Raw table? Any suggestions or insights on how to handle this issue would be greatly appreciated.

8
J
Contributor
2 months ago
A
Contributor
aftabpContributor
asked in ODX

Data capture with multiple API-endpoints

Hi,Has anyone worked with API from Energinet?I need some help extracing data from Energinet. Energinet delivers data with multiple endpoints. It starts with https://www.energinet.net/api/unit which gives me this [ { "unit_id": "1234ABCDfolder", "name": "My Company", "links": { "info": { "verb": "GET", "href": "/api/unitinfo/1234ABCDfolder" }, "drilldown": { "verb": "GET", "href": "/api/unit/1234ABCDfolder" } }, "datasources": [] }]Now, i have to drill down to get some info: https://www.energinet.net/api/unit/1234ACDfolder[ { "unit_id": 1Afolder", "name": "Company1", "links": { "info": { "verb": "GET", "href": "/api/unitinfo/1Afolder" }, "drilldown": { "verb": "GET", "href": "/api/unit/1Afolder" } }, "dat

15
A
Contributor
2 months ago

Badge winners

  • Certified Solutions Architect
    Karankadamhas earned the badge Certified Solutions Architect
  • TimeXtender Xpert
    Muddihas earned the badge TimeXtender Xpert
  • Innovator
    andrew.gebhardhas earned the badge Innovator
  • Certified Solutions Architect
    mello.bogaartshas earned the badge Certified Solutions Architect
  • Innovator
    ekw9has earned the badge Innovator
Show all badges
Terms and ConditionsCookie settings