TimeXtender Desktop Q&A

Hi team, TimeXtender allows adding parameters from a different table to a custom field in a semantic data model (Qlik). The resulting syntax/qlik script combination is always broken.When using adding a custom field parameter from a different table, TimeXtender fully qualifies the Qlik syntax regardless of the settings. The resulting syntax on the Qlik side will no longer match the syntax in the views created by TimeXtender:Qualified setting:Fully qualified setting: The resulting Qlik Script:"Sales_Targets":LOAD"KPI", "Target", "DIM_Boekdatum.DayName" AS "Test";SQL SELECT"KPI", "Target"FROM "Test"."dbo"."Test QVD_SLQV";But the view has the following syntax:CREATE VIEW [dbo].[Test QVD_SLQV]-- Copyright 2011 timeXtender a/s-- All rights reserved---- This code is made available exclusively as an integral part of-- timeXtender. You may not make any other use of it and-- you may not redistribute it without the written permission of-- timeXtender a/s.ASSELECT [KPI] AS [KPI] ,[Target] AS [Targ

I’m using Timextender version 6521.1 and I get an error when using hard deletes on a incremental table. If I choose no deletes it works just fine, soft deletes also throw me and error. As far as I could tell from these notes incremental loading should be fully supported. Things I have checked:Primary key on the table in the ODX Incremental load is setup in the ODX Incremental load is setup on the Snowflake table. I’m not quite sure if this is just a bug or I missed something obvious. This is the first table i’m trying to load incremental in Snowflake. This is the error it throws at me:System.AggregateException: One or more errors occurred. ---> Snowflake.Data.Client.SnowflakeDbException: Error: Error parsing the parquet file: Invalid: Parquet file size is 0 bytes  Row 0 starts at line 0, column   If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, plea

Hi,We have been using ADF for transferring data from our ODX storage account to Azure SQL database for a long time but now our manual and scheduled reloads get stuck in ADF.It looks as if TX does not pass the runtime to ADF. This is required because our storage account is behind a private endpoint so we need to use an IR that resolves in our VNet in Azure.Do you have any idea why the runtime does not appear on the pipeline? As you can see there is no runtime assigned to this run:  Normally the runtime appears in ADF:  We are on TX 20.10.45 by the way.Would love to hear from you!

Hello,We are currently utilizing the new TimeXtender REST API 6.1 and leveraging its built-in paging feature to connect to an external REST API. We're not using an RSD configuration for this setup.I've encountered a challenge with pagination, specifically with the use of a parameter named "Cursor." This parameter is critical for the pagination logic in the API we're connecting to. Each row in the data returned by the API has a unique Cursor value, but for pagination purposes, only the first Cursor value from the current page should be used in the subsequent POST request to fetch the next page.Here’s how we've set up the Cursor parameter:Name: cursor Type: XPath Value: /TX_Autogenerated_Root/TX_Autogenerated_Element/data/arTransactions/edges/cursorThe issue arises because I need to ensure that only the first Cursor value from each fetched page is used in the pagination post body for the next request. I'm unsure how to configure TimeXtender to utilize only the first Cursor value for this

v20.10.35.64I just noticed there’s a bug in the current version of our ERP related to the UpdateDate & UpdateTS field which impacts incremental loading. Hence, I'm investigating the possibilities to set up a semi-incremental loading infrastructure. With my TX version I can also set up the following, which is limited:  My database has data ranging from 2013-today. I'm looking to find a way to load 2013-2023 just once and then only full load the data with a UpdateDate >= 01-01-2024. I'm thinking to achieve this the following way:Set up a Query Table for every table where UpdateDate >= 01-01-2024. Set up a Load task, LOAD_FULL which has the original tables Set up a Load task, LOAD_INCREMENTAL, which has the Query table with just data from 2024. Execute the LOAD_FULL task just once. Schedule & Execute LOAD_Incremental every 30 minutes. If I do it this way, it's not a classic incremental loading process.Can I still utilize TimeXtenders ODX/DSA incremental loading functionaliti

Dear community,Is the only way to connect the ODX server to an Azure DB still on SQL Server authentication?No Windows or Azure integrated security yet?Thanks!= Daniel

When choosing a execution job, just a single execution job, it’s grayed out and I’m not able to choose it.it’s deployed and the execution jobs works, when I run it manually.I’m running Timextender version 6590.1

Hi, I was wondering if it is possible to reference to the content of local file on the ODX server from within a RSD file. Such as Get-Content with Powershell.  We have some RSD files for some of the api’s call that we are making. We are in the process of including these RSD's in our own source control system in order to keep track of the changes that we do with the RSD files. One of the api's is quite unstable and sometimes needs tweeking in the rsd file. (Source control on the RSD file's / powershell scripts files could be another topic to be honest).Anyway, we have to include an API key in the script that we would not like to have saved in the source control. Therefore, an option to reference a file in which we can store the API key would be great.  I know that you can also include authentication in the portal, but we are looking to maybe avoid putting credentials as much as possible in the portal. The portal also creates a OAUTH map if you use OAUTH in the portal. But I don't know h

Hi guys,We are creating a users table in Timextender based on Active Directory.  However, there’s also an on premise Exchange server, adding Exchange attributes to the user in Active Directory.  But it seems we cannot read these attributes.  We use the CData Provider for Active Directory in our ODX server.  When trying to add the fields, the Exchange attributes are not in the list.  How can we add these attributes?  Do we need the CData Provider for Exchange?  If so, has anyone set that one up and is willing to provide more details? Or how can we manage to import all the Exchange attributes in our user table? Thanks!Stijn Hensen

Hi all,I have v20.10.45 running in Azure with ODX server running for a client. We have about 15 ODX datasources. The synch/loads run daily and the server has been up for about 9 months, so no extraordinary amount of logging I think. The issue that I’m having is that it’s not possible anymore to check the execution logs of  some ODX task runs. The popup doesn’t appear and TX goes into a “Not Responding” state. Any click in the client will crash the application. This happens when I have no TX project opened, only the Manage tab for ODX server (see screenshot).I’ve upscaled the VM from DS2v2 (2 core, 8GB) to D4S_v3 (4 core, 16GB), but this doesn’t change it. Restarting ODX server also doesn’t help.The issue is not with all tasks. Tasks for which all recent executions are successful will show the logging. Checking the event logging at ODX server level will crash and tasks with a lot of recent failures also crash. Edit 1: A connection that has this logging issue is a connection using the CD

Hello dear TX Community,We want to load tables from our ERP System, and we are encountering the following error: HRESULT: 0xC0010006.This error indicates that the name of the SSIS Package created is too long, and we need to shorten it. I've attempted to shorten the name by modifying the schema name we agreed upon, but this would only be a temporary solution as there are more tables with longer names. For your information, the package name typically consists of the following parameters: connector name - schema name - table name - SSIS-ID, for example, sql91_DBTest_zzdomAG_DBTest_dbo_SalesAppointment_3d5329a2-54ab-42a1-9126-4c21d569131ce.Now, I have the following questions:1. How can I manipulate/shorten the package name apart from changes I can make to the connector name and schema name (in TX)?2. Is the SSIS-ID always generated with the same string sequence, and is there a specific use case behind it that requires the ID to be formatted this way?kind regards Faras E.Nazeri

I have set up two completly new enviorments in AZURE.Project name is same in both (i also have tried to append _PROD in Production) Why is this an issue? How do i come around this? 

Hi all,My first post here. My company is currently using legacy TX but I started poking around the new version. I was a bit surprised to find out that passwordless authentication methods are not supported yet. Are there plans to support managed identity authentication in the legacy or new version?Our company security policy requires password expiration even on service accounts so you can imagine the hazard to update them in many places regularly.TX MDW SSMS 

Current advice on partitioning on a non-date field calls for creating a partitioning “bucket” using field transformations. While attempting to follow the instructions outlined here I encountered a problem. Partitioning works by creating a calculated field in the valid table that has the capacity to handle NULL values if the partition template is so configured. However, that same field, along with the calculated default value, is also present in the Raw table. This means that when anyone following TimeXtender’s guidance for non-date transformations executes the table, that execution will fail with the error “Cannot insert the value NULL into column ‘DW_PartitionKey’”Is this a bug, or does the documentation for creating partitions need to be updated? 

I have a number of Oracle data connections (TimeXtender Oracle Data Source) which have been set up and working against the same data source for a few months now. However, just recently, one of these data connections stopped working and started reporting timeout issues after 1 hour of connecting. I have multiple cloned versions of this same Oracle connection, and it seems to be only one of the cloned connections which is having this issue. I have since cloned one of the working connectors and added the 6 tables that are pulled by the failing connector (to mimic the failing connector setup) and this temporary connector runs without issue, completing in just a few minutes. As far as I can see, the 2 connectors are set up exactly the same way, its just the the original does not work whilst the new temporary test connection does. It may not be related, but this connector started failing one day after I had performed an in-place upgrade of both the TX ODX server and desktop client. However,

Hello, I added an execution package to my production instance. I want to add it to a certain job, but that job cannot find my package. In fact, it finds all packages except for the one I just added. How can this be?  

Hello,In Timextender there is a report called ODX Execution Log (see image). I was wondering in which tables the underlying data is stored.My company has a dashboard / chat bot that we use to check on our nightly DWH executions. For this we use the tables ExecutionPackageLogs, ExecutionPackageLogDetails, Projects, etc.I want to expand these reports with data regarding the ODX Transfer Tasks. I checked most of the tables in the metadata database (e.g. ScheduleJobs, ScheduleJobLogs) but I was not able to find it.  

HelloEvery day, I have a new Excel file in the Azure storage or OneDrive account. I use OneDrive to test phase. These file’s name change with a new day. How can I this multiple Excel files to one? Is there a option for Excel like aggregate files option in CSV?Best Regards

Good morning All,We are encountering the following:We have history tables that use the DW_SourceCode as part of the primary key. This is because we have multiple sources with the same structure which are merged together in one table after ODX. As primary keys can overlap in the different sources, we need the DW_SourceCode to make distinguish the data coming from one or other source. Also we need history as the sources have a tight cleanup system and we need to keep track of information ourselves. While developing some of the tables, the DW_SourceCode was somehow not included in the primary key in the first deployment and this was corrected afterwards. TimeXtender also reflects this: Although deployment was successful, the computed column definition of the surrogate hashkey was not altered: The problem we're having now is that the history is matched based on the surrogate hashkey. Now wrong records are updated (type 1 history) because the hashkey matches but actually shouldn't because t

One of my colleagues made a change to an incremental field on a very large table, not realizing that this would force a full load. We only realized what had happened halfway through our promotion, but since we needed to make changes to that table anyway, we proceeded. Reloading that table and its downstream tables took five days. Luckily, a long weekend greatly reduced the impact of that reload on our users. We have now begun our next promotion cycle.  Unfortunately, despite the fact that the table in question was reloaded in our development environment, differential deployment is again forcing a full reload of that table. We really can’t afford to spend a week reloading this table a second time. We’d love to know why the flag remains flipped in the first place, how we can avoid this happening in the future, and how we can avoid this extremely costly reload, as it is blocking our ability to promote urgently needed changes into production. 

TimeXtender beings almost all tasks by checking to see if the objects involved currently exist. It does this by querying SQL Server system views. However, this does not take into account a very unusual property of system views: querying them naively can result in SQL Server attempting to get a lock on the underlying system metadata tables. When it does this is wildly inconsistent - sometimes it will try to get a lock on the table when doing only a simple SELECT, other times it will only attempt to get a lock when certain joins are used. This isn’t really documented anywhere I’ve been able to find, but I have observed it in action many times. The problem is that SQL Server schema changes write to those metadata tables, which also locks them. And as long as the change is occurring, the tables stay locked.  This becomes an issue when rebuilding indexes. This is a very frequent occurrence curing TimeXtender execution as it happens whenever a table is fully reloaded. For large tables, this

We have a REST API that we use to connect to several endpoints. The default name of all the tables is: REST.Jsondata. This causes issues in our Azure Gen2 datalake. It gives an error saying the table name is already in use by a table with a different ID.I almost feel like there must be some really easy way to rename the table name before it lands in the Gen2 datalake but so far I haven’t been able to figure out how.The workaround I use now is to use a QueryTable but that is not the nicest way to do it I guess. 

Hello,My organisation recently had to redeploy our development virtual machine and reinstall timextender / odx on it. Since then I am not allowed to click on certain buttons inside the application, and options are missing when I right-click on certain objects.For example I am not allowed to click on the [Stop Task] button in the ODX Execution Queue window. And when I right-click on an object in the Manage ODX tab, all I can see it ‘View Properties’ and not any of the many other options that used to be there.I do have these functions available in our other environments. And my collegues have no issues on our dev machine.I have tried the following so far:Enable/disable Deprecated Features. Delete the hidden Default.rdp file and create a new rdp connection to the machine. Any tips?

Currently I’m using one ODX instance with this setup:  All the mapped data sources connections has selected the option “Enabled data on demand”:  I am using three instances of Data Warehouse: Development → Testing → Production.My question is:Can I create a new ODX instance connected to the same Storage container?The idea is that the same data sources in the DW instances for development and test use the "Enable data on demand" OFF option. In this way the storage container only will be refreshed one time (when we refresh the DWH production instance) because the data source is the same. 

Recently we identified that our Azure SQL Database Backup Costs for our TimeXtender environment are very high, almost even higher then the regular storage costs itself. We now try to minimize these Backup Costs. We already minimized the Backup Retention policies:Now, when analysing some Backup data in the Azure Portal we found that the Log Backup Storage Size is growing the the multiple TeraBytes, even though our PROD database uses around 28GB.  Does anybody know how we can further analyse and minimize these backup costs and data? I found the following regarding the transaction log, but I can't seem to dive in the transaction log/size per table or execution for example. --------------------------------------------------------------------------------------------------------------------In Azure SQL Database, transaction logs record changes made to the database to ensure data durability and provide for features like point-in-time restore and disaster recovery. Various operations within Az