To homepage To homepage

Data Sources Q&A

Ask questions of about setting up data sources

80 Topics

P
Contributor
pontus.berglundContributor
asked in General

Power BI dataset as data source

Hi,Has anyone used a Power BI dataset as a data source? I have tried using the CData SSAS connector but can not get it to work. Any advice would be appreciated.Best,//Pontus

1
P
Contributor
6 hours ago
A
Starter
AndrewSStarter
asked in General

Multiple CSV files with History (19.11.7.64)

Hi Everyone,---------BackgroundMy organization is trying to build a Supervisory Org tracking database. Our previous HRM system did this natively, and enough departments are tired of waiting for the new system quirks to be ironed out so I’m tasked with a work around. I can’t connect TX directly to the new system, but HR can download a csv that includes employee ID, Employee Name, Supervisor ID, Supervisor Name, Job Title, Work Location and various less critical fields.My plan then was to have an HR rep download the csv file once a week to a specific network folder, point TX at it with the Multiple Text File data source. I’d then use the DSA to perform some basic transformations and enable history on the DSA table. In theory this gives me a weekly update to an employee, tracking changes to key things like job title, supervisor, etc. We have a little over 1200 employees in the report, and 60-70k records a year shouldn’t be a problem. The initial ingest works well. TX reads the file, moves

6
A
Starter
7 days ago
J
Contributor
jacqueline.hofmeijerContributor
asked in RSD File Customization

Using a nested REST API in combination with paging

Hi, I am having some trouble in using the nested REST API in combination with paging. The extensive description on how to use the nested call in the RSD-file was very helpfull. The only thing I cannot get working is the paging at the first level.Two different cases, but I think the same underlying issue. In the first case the pageoffset is used as a parameter, in the other one it uses a pagetoken (the nextmarker in a blobstorage). In the output, I do see that paging is applied, but the problem is that it starts at the first page again. I seems that when using the input- and output files the correct information about the next page is not being picked up by the api-call.I have tried the call without nesting (so only on the first level) and without qualifiying the parameters and then the paging is working correctly, but when I switch to in and out again, I get false results.Does anybody recognize this isse and solved it somehow? Kind regards,Jacqueline Hofmeijer

17
Christian Hauggaard
Community Manager
21 days ago
I
Starter
info.tejavardhanStarter
asked in ODBC

Is TimeXtender tool is capable to support "MultiSubnetFailover=True" for version 20.39.10.64

Hi Team, we are using sql server 2019 databases and servers are configured with HADR cluster and Always ON (AG) with multi-subnet failover option .The MultiSubnetFailover option in database connection string controls how the client attempts to connect to a database server, especially when multiple IPs are configured. To support AG listener alias connection string with multiple IPs, we need to add additional parameter: “MultiSubnetFailover=True” in DB connection string.We need to know TimeXtender  20.39.10.64 tool is capable to support "MultiSubnetFailover=True" parameter in DB configuration or not.

9
Christian Hauggaard
Community Manager
21 days ago
J
Visitor
Johnny YVisitor
asked in RSD File Customization

Using a nested REST API in combination with pagination (using the Graph API)

Hello everyone, I hope someone can help me with this.I am working on getting Azure AD Groups via the Graph API and then retrieving the members through a nested API call.I saw a post concerning a nested REST API call together with pagination in which one of the commenters (Gijs) had the exact same API which I am also using (commented by Gijs):https://support.timextender.com/rsd-file-customization-96/using-a-nested-rest-api-in-combination-with-paging-858However, the suggestion didn't work for my use-case as it only ‘expanded’ the nested values, but didn't paginate. I only got the members of the first 100 groups.I used the RSD-file from the commenter Gijs with the addition of two points:two additional rows to enable pagination; [memberout.userPrincipalName| allownull()] instead of [memberout.userPrincipalName] as I was getting a [500] error with this specific attribute.I've made them bold in the code down below.Would someone be able to give me some pointers to what I'm missing in the RSD

2
Christian Hauggaard
Community Manager
26 days ago
A
Starter
andreas.skagestad
asked in REST APIs

REST connection with JSONROWS - Columns and rows within same array

Hi,I am trying to parse a response using the REST connector from an API which returns a JSONROWS-format where the first row contains the name of the columns, while the following rows contain the values.The response is on the form:{ "response":[ [ {"name":"test1"}, {"name":"test2"} ], [ 1, 2 ], [ 3, 4 ], [ 5, 6 ] ]}I have tried several different xPath-configurations. The following gives the correct columns, but returns zero rows:xPath:        column:/response;columnname:/response.name;row:/responseI was hoping that at least this would have parsed all the values as strings with the first row containing the {“name”:”testN”} in the corresponding columns.How do I work with this response?

6
Christian Hauggaard
Community Manager
26 days ago
erik.van.mastrigt
TimeXtender Xpert
erik.van.mastrigtTimeXtender Xpert
asked in RSD File Customization

Loading multiple Excel files with 1 data connection using RSD

Hi community,I have multiple Excel files on a sFTP location, all having the same structure. I would like to load these files using 1 data connection and create a RSD file for each file/sheet that needs to be loaded.When generating the RSD via TX, it does not contain the api:set attr=”URI” property. An example: On line 2 (api:info) there is a parameter other:uri=”[...]”, but modifying this does not change anything.Manually adding the api:set attr=”URI” does not change anything eitherThe data will always be loaded from the Excel file that you enter in the UI: Emptying the URI field results in an error that either URI or Excel File parameter should be set: Is there a way to load multiple Excel files/sheets with a single data connection using RSD files, same as we do with eg. REST API endpoints and CSV files? It will save us from creating ca. 15 separate connections :)Best,ErikTX: 20.10.43CData Provider for Microsoft Excel 2023, v23.0.8565.0

7
erik.van.mastrigt
TimeXtender Xpert
28 days ago
D
Starter
dpjatStarter
asked in REST APIs

Help with REST API based on table

Hi everyone, I’ve been searching but haven’t been able to find in previous threads. For an API i'm trying to extract to TX, i'm running into some issues. The API extracts some geographical building data, based on an ID. I have these ID's from a seperate source already in the warehouse.What I want to do is get the ID from the ODX-table and for each ID run a call to the API. I'm new to this scripting language and maybe im overlooking it in existing documentation but for me it seems impossible. Any idea's and suggestions would be welcome, thanks!

10
Christian Hauggaard
Community Manager
30 days ago
N
Starter
Niek.GuijtStarter
asked in REST APIs

Rest API rate limit

Hi,I'm currently connecting to a new api.https://ads-api.autotelex.com/swagger/index.html?urls.primaryName=Autotelex%20%7C%20Advertisement%20API%20V2%20(Beta)In my RSD file I call /api/v2/stock to get the ids for a stock, then I call /api/v2/stock/{adsStockId}/summary using the ids from the first call.The problem I run in to is that i can only make 300 calls per minute for the second call. Whenever I run into this limit my execute fails with the message: "Too many requests".Is there any way to limit or reduce the amount of calls I make per minute?Kind regards,Niek

8
J
Contributor
1 month ago
J
Contributor
jacqueline.hofmeijerContributor
asked in REST APIs

Use variables on Query tables

Hi,I have a project that is using the cdata connector as a datasource, and because of issues with paging and nested call we have created query tables to get the correct results (using queryslicer in the rsd-files).Because of performance issues the tables have to be setup for incremental loading, but I cannot get this working because I need the incremental value in the call.Is it somehow possible to use a variable or a field from another table in the Quey tables to get the incremental value?When using a fixed date the logic is working fine (as shown below) but that's not workable:    SELECT *     FROM [REST].[EventRegistration_Detail]      WHERE [file_name] IN         (SELECT [Name]          FROM [REST].[EventRegistration_List]         WHERE [Properties.Last-Modified] > '2023-01-01')  (using TX 20.10.30.64 with Business Units) Any good suggestions on this? Kind regards,Jacqueline

4
Thomas Lind
Community Manager
1 month ago
F
Explorer
FENExplorer
asked in ODBC

How to manage a new field in the staging area without loosing Data through Deployment

Hello,i have the following issue:i would like to save my tables in the staging area so i can store them (because some of my data in the datasource will be deleted soon).Futhermore in the future maybe new fields will be added to those tables in the datasource and then i would be forced to deploy my tables in the staging area.Question :  is there a way to keep the old values from that tables and only add the new field to that table without truncating everything with the deployment in the staging area?ty in advanceFEN

6
F
Explorer
1 month ago
erik.van.mastrigt
TimeXtender Xpert
erik.van.mastrigtTimeXtender Xpert
asked in REST APIs

CData REST API connector loads incomplete data when paging is used

Issue: tables with paging stop loading while data is still incompleteWe have an execution package with loads data from AFAS GetConnectors. There are 7 Business Units in TX, each loading data from a separate AFAS implementation.  Per implementation AFAS supports up to 20 parallel connections. From this perspective the Max. Threads 10 should not be a problem.AFAS uses paging with record offset for tables that are too big to load. All Business Units have a similar set of tables. I’ll use AfasIncluzio as an example, as this one has the most data.This table is currently the only one loaded with pagingTX_Financiele_MutatiesWhen we look at the execution log on package level, we can see that the Total Time varies per day:  The AFAS API is quite stable in terms of speed. Hence I can tell that all executions under 15 minutes did not load all data. Further investigation learned that this only happens on table Tx_Financiele_Mutaties, which is currently the only one with paging.This is the Executio

10
erik.van.mastrigt
TimeXtender Xpert
1 month ago
mnoordam
Contributor
mnoordamContributor
asked in RSD File Customization

REST with POST method reading AppSheet Database

Yes, I’ve read the other posts about this topic But I'm rather unlucky getting it to work. Each time I press the ‘Test Connection' it fires a GET method at the URI, which results in the infamous ‘The requested resource does not support http method 'GET'’ message. RSD file:==================================<api:script xmlns:api="http://apiscript.com/ns?v1" xmlns:xs="http://www.w3.org/2001/XMLSchema">    <!-- See Column Definitions to specify column behavior and use XPaths to extract column values from JSON. -->    <api:info title="Table1" desc="Generated schema file." xmlns:other="http://apiscript.com/ns?v1">        <!-- You can modify the name, type, and column size here. -->        <attr name="_RowNumber" xs:type="string" readonly="false" other:xPath="/json/_RowNumber"/>        <attr name="Assignee" xs:type="string" readonly="false" other:xPath="/json/Assignee"/>        <attr name="Date" xs:type="date" readonly="false" other:xPath="/json/Date"/&g

16
Thomas Lind
Community Manager
2 months ago
A
Visitor
antanVisitor
asked in General

External SQL Connections

In the old version of TX (20.10) it is possible to set up an external SQL connection in a business unit under Sources, so that tables from an external database can be used in a TX project without extracting them from mentioned database. TX will then deploy those external tables as views pointing directly to the source database, which can even be on another server (‘linked server’).I cannot find external SQL connections in the latest version of TX. Can anyone confirm if this feature is available or not, and do we know if this in the product roadmap?

3
Thomas Lind
Community Manager
3 months ago
M
Contributor
Michel van DrunenContributor
asked in General

Multiple XML files including filename

Hi I am using TimeXtender 20.10.37.64 incl. ODX server. I am trying to get some data from XML files with a wildcard which have the filename as a reference to another file. I am using the Cdata XML connector. I managed to get the data from both files in TimeXtender but can’t find a way to include the original file name to the table/query. Is there a standard option for this or maybe a workaround? Hope someone can help me.

9
Thomas Lind
Community Manager
3 months ago
Niels Jonker
Contributor
Niels JonkerContributor
asked in REST APIs

RSD pagination with incrementing value

Hey community,I am trying to set up pagination for my REST connector to ADP. I need to use the CData parameters $skip & $top.My first call should be: https://api.eu.adp.com/hr/v2/workers?$skip=0&$top=100Then I need to increment the $skip parameter by 100 so the next call would be: https://api.eu.adp.com/hr/v2/workers?$skip=100&$top=100In need to end when I receive an empty response. Anyone that has experience with this?

4
Matt Thomas
Starter
4 months ago
V
Contributor
vw-victaContributor
asked in ODBC

Additional connection properties - new TX version(s)

Hello,for one of our customers we are using the “Odbc Data Provider” so that we can a local ODBC with a specific driver called "Progress OpendeEdge” (Source: ProFashionALL). We need to add Additional Connection Properties for this datasource. In the older TX versions (< v20) we could add these Additional Properties. This is missing in the new TX version so we can't connect to this source at the moment. This is the main source and crucial in this project. Anyone ran into this issue or has a workaround?There is a workaround but this must done at the source where they store the credentials in the “_user table". I hoped there would be a solution at our (TX) side. Hope to hear.

11
V
Contributor
4 months ago
H
Explorer
htrimasExplorer
asked in General

Multiple Incremental selection fields

(Using Timextender 20.10.35.64)I have 2 date fields on a table. I want to load the row if one OR the other fields have changed in the data source. How do I do that?I tried doing this (see under)  but that caused duplicates because data had already been loaded (when only the loggedat field was the incremental field)  If I do this (see under) will it only load if both of the fields have changed? 

5
Christian Hauggaard
Community Manager
4 months ago
S
Contributor
SpellingContributor
asked in Oracle

Sync Oracle slow for ODX server

Hi,I have a setup of a TimeXtender 20.10.41.64, ODX, ADL, ADF, Self hosted IR which I get to work just fine. I need to extract data from an IFS10 source, Oracle database, and the sync is too slow so it times out and fail after 2 hours. The data source is provided by a third party and which means that the customer doesn’t have control to change any settings and 2 hours should be enough time for a sync. The customer have a couple of TX environments After many tests and discussing with the customer a similar problem have occurred before, but in an environment not using ODX server, where a sync took over 2 hours and this was fixed in Timextender release 20.10.3, issue “8889: Oracle slow synchronization - affects all bit specific data sources”. After this release the customer says the sync only takes about 10-15 min. Could this be a problem in ODX server as well? I managed to get a successful sync with version 20.10.40 which took about 2 hours but after an upgrade to 20.10.41 I’ve  managed

4
S
Contributor
4 months ago
S
Contributor
sierd.zuiderveldContributor
asked in RSD File Customization

SOAP API setup

Hi, I have some trouble setting up a SOAP API within the RSD file. The envelope returns data as follows : <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"> <s:Body> <getCertificateListByCompanyResponse xmlns="http://tempuri.org/"> <getCertificateListByCompanyResult xmlns:a="http://schemas.datacontract.org/2004/07/" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"> <a:ObtainedCertificate> <a:CategoryName>text1</a:CategoryName> <a:CertificateCode>12345</a:CertificateCode> <a:CertificateID>12345</a:CertificateID> <a:CertificateName>text2</a:CertificateName> <a:CertificateNumber/> <a:DateAssigned>12345</a:DateAssigned> <a:DateExpire/> <a:DateObtained>12345</a:DateObtained> <a:Expired>false</a:Expired&g

2
S
Contributor
5 months ago
M
Starter
MorganHeijdemannStarter
asked in General

Custom Data Provider alternative in latest TX version

Hi everybody,Our client uses a SOAP API from a supplier. While TX supports SOAP the extraction requires multiple steps and string manipulation. The complete process is:For each table in the list of tables: Call the first SOAP endpoint to retrieve the names of the fields in the table Call the second SOAP endpoint to retrieve a page of data for the table Split the data into individual rows by splitting the string on the ^ character Split the rows into individual colums by splitting the string on the | character Fetch until the number of rows returned is less than the page size Move to next table We created a custom data provider for the TX custom data connector. This works fine in TX 20.10, but is not supported in 6221, so we need to create an alternative. I’m aware that you can use RSD files for more advanced logic, but as far as I can see this doesn’t include an option to perform the string manipulation (splitting). For the time being, we’ve ported the logic to a Python script

5
M
Starter
5 months ago
Bitmetric_Maarten
Contributor
Bitmetric_MaartenContributor
asked in General

Data format issues when loading CSV

Hi all,We are trying to load CSV files into the ODX using the CSV data source. However we are getting errors on fields it is trying to interpret as int (housenumber), but some are filled in as 12-18 and therefor failing. We have tried to use the override data type option, but to no avail. What can we do about this?Kind regards,Maarten

2
Bitmetric_Maarten
Contributor
5 months ago
kerry_bespoke
Contributor
kerry_bespokeContributor
asked in General

Historical Table Load Setup

Good DayI am setting up a new Data Source in TimeXtender. This dataset is very large and would like to eventually have this running as an incremental, but I have the task of importing the last 5 years of data first.I have tried to set this up with an Incremental Table setup in TX, but I keep getting a timeout setting. Iam wondering if I could set this table up as Simple for the historical load and then swap to Incremental but am worried my data would be cleared on changing the table type. Does anyone have any experience with loading in data with a similar scenario?Kerry 

4
kerry_bespoke
Contributor
5 months ago
P
Contributor
pontus.berglundContributor
asked in REST APIs

Connecting to App Store Connect API

Hi,My client wants to connect to the App Store Connect API (https://developer.apple.com/documentation/appstoreconnectapi) but we are having trouble with getting the authorization to work.Calls to the API require JSON Web Tokens (JWT) for authorization, specifically using the “ES256” signature algorithm. We tried using the CData Rest connector but found in the documentation that “...the JWT signature algorithm cannot be set directly. Only the RS256 algorithm is supported”  https://cdn.cdata.com/help/DWH/ado/pg_oauth.htm#jwtHas anyone experience connecting to the App Store Connect API?

5
Christian Hauggaard
Community Manager
5 months ago
ignacio
Contributor
ignacioContributor
asked in General

TX customer data source

Hi FriendsI have used TX CDS to connect SQL Server because It was a best practice recommendation, but now I don’t know if CDS will be deprecated, What is the best connector for SQL Server? CDS will be deprecated? RegardsIgnacio

3
Thomas Lind
Community Manager
5 months ago

Badge winners

Show all badges
Terms and ConditionsCookie settings