Build better products with our product team
Hi all,Currently there are Azure Data Factory providers for SQL Server, Oracle, MySQL, PostgreSQL and Azure Data Lake.When looking at the available database sources for ADF pipelines, there are plenty more databases available. I would like all these databases to be made available as an Azure Data Factory provider.For a TX client I’m currently working at I’m specifically interested in having the Teradata source added.You might ask why I don’t use the CData Teradata connector? We’re running TX on an Azure VM. This VM has no direct connectivity to the on-premise database servers that are hosted in European datacenters. We only have access to those database servers via a self-hosted integration runtime in Azure Data Factory. Greets, Rogier Databases available as a source for ADF pipelines:
When sorting by Execution Date in the Job monitor, it seems to sort by text instead of datetime: I think datetime would be better 😉
Hi, It would be nice and timesafing to have the automap function available for Add Related Records, similar as the automap function already existing at the Table Insert function. Best regards,Devin
Hi all,I’d like to propose that we start boosting XPilot into an AI that can help us in developing our data warehouses by supporting questions and tasks on the meta data of our model.I think this should be possible since the TimeXtender model can be represented in an XML file and recently ChatGPT has added an XML interpreter which allows you to ask questions about it. ChatGPT does this by translating the question to Python XML parsing code and then running it to get the results, So far it’s been a struggle to get it working but I think it should be possible to create an AI on the TimeXtender model it if it has enough knowledge about how the TimeXtender meta data is structured. I would like for XPilot to not only give me answers but also to make changes to the model. This can be a huge time saver in the case of repetitive tasks (e.g. updating multiple tables). Below is a list of categories and sample questions / actions that I’d like to be able to ask to XPilot. If you like to idea of getting a super boosted XPilot, please upvote my post. What types of questions or tasks would you like to see supported? Write them below in the comments! Greets,Rogier TimeXtender categories and questions / tasksData model analysis List the tables that are missing a primary key definitions List the DSA tables that I can join the DSA OrderLine table to based on the fields available? List the tables are using a post or pre script and that don't have a dependency set List the DSA views that don't have mapped fields List the tables with property [….] enabled/disabled. List the MDW tables that have custom records configured List the DSA tables with object dependencies and list the dependant tables for each table List the tables or fields with […] in the description List the fields that have a TimeXtender or custom transformation but no description Code generation / transformations Create a tables called 'Sales' from the ODX table SalesLines from the AdventureWorks data source and add the following fields […] Set a default value of [...] for the fields […] Create a new varchar 50 field with a fixed value of 1 and a fixed value of 2. Set the batch size to [...] for the DSA tables […] when the current value is [...]. Convert the varchar fields in the Customer table to nvarchar Can you suggest a custom SQL transformation that creates a running total of the Revenue field grouped by the CustomerKey and ordered by the SalesDataKey? (the current XPilot can already answer this) Set the value of the [...] field in the DSA [...] table to [...] when the value is [...] . Change the operator of the LastUpdate field to [...]. Model quality and integrity What is the distribution of the CustomerType field in the DSA table Customers? Does the OrderLine table have duplicate records on the field OrderLineId? Give me a list of all DSA tables that have no rows Does the ProductDescription field in the DSA Product table have NULL values? List the tables with duplicate values Model governance and security Tag all the fields in the data model with the name [...] with the tag [...] List the tables that have a field tagged name [...] Data lineage List the ODX data sources, tables and fields that are used to calculate the [...] field in the [….] table in the [...] semantic layer Which fields in the DSA [...] table aren't used? List the MDW tables that have a [...] surrogate key field Team development Summarize the DWH Production instance changes since version […] Warn me when a colleague starts working on a model I'm currently working on Warn me when a colleague makes a change to a table that I currently also have changes open on
In the definition of the incremental selection rule in MDW it would be very usefull allow variables: And the same when we define the incremental Load Setup Rule in ODX:
The On-demand attribute of Data Sources makes it possible to chain ODX reloads together with DWH/SSL executions. This is (usually) fine for scheduled reloads but in a situation where the same ODX alse serves developers who may need to reload DWH tables without needing fresh ODX data, you may want to turn it off while developing against a source.In that situation you may still want to be able to reload an SSL using auto-generated tasks. Currently you need to turn off On-demand for all sources that have it (which is not easiliy visible) and then generate the end-to-end tasks after which you turn On-demand on again.I would like to be able to generate these Tasks whether On-demand is on or not (via a toggle for instance).As on-demand does not seem to be aware of situations where it unneccesarily reloads tables multiple times, it is something that tends require a lot of fiddling with.
The DAX Snippets doesn’t works like the SQL Snippets.It doesn’t allow in the “Show Usage” section to change the snippet and change too the measures afected.In the document:Updating a SnippetWhen updating a snippet, edit the objects where the snippet is being used. Remove and re-add the snippet for the objects, for the changes to take effect It must be work like SQL snippets.
It would be very useful to Copy/Paste Snippets to be more productive.
When you set up roles and permissions in an Ingest instance, you may want to be able to see what is going on. Currently the UI for this is rather unintuitive: Looking at the screenshot above, you may come to the conclusion that this role has no permissions set up, but you would be wrong. You need to actively click on each Data Source to see what has been defined for that source:In this case I want to ensure my DENY has been set, but can only check if I highlight the specific source. To improve this I would like to suggest the following:upon opening the dialog show all Rules, i.e. do not select a source allow a user to unselect a source by clicking on it again if it is highlighted make bold / italic if a Data Source has rules set think about implementing some kind of ‘effective permission’ visual that allows you to browse through the schemas/tables and marks the access rights (like 20.10.x) give a hyperlink to the resource in the Azure portal
There are a few things that would greatly enhance the Semantic Layer. I've come across some old tickets that address some of these points. It would be highly beneficial to have a library of reusable DAX calculations instead of having to rewrite them each time. This could be similar to the Business Function library found in OLAP cubes. It would be useful to have some form of syntax verification within the DAX script window. Currently, it can be time-consuming to deploy and execute the Tabular model only to discover a syntax error. (I understand that there are tools like Tabular Editor and other solutions that allow for quicker testing.) It would be great to have the ability to create folders within the measure tree and easily drag measures into these folders. This feature would provide a much-improved overview and a more user-friendly approach when dealing with numerous calculations, rather than grouping measures to folders by right-clicking the table. To be able to change the mapping table in the semantic layer, currently you have to delete the table and drag in the new table.
Descriptions in ODX data sources function feel somewhat hidden. There is no bold text to indicate that a data source has a description and there is no tooltip to display it. It would be nice to have the ODX data source descriptions work like descriptions usually works in TX, i.e. tooltip + bold text.The work around currently is to edit individual data sources and check if they have descriptions. This is somewhat sluggish since the provider needs to be loaded.
When teaching people to use TimeXtender or evaluating implementations I usually tell people to convert custom CASE statements into Conditional Fixed transformations because these are clearer to understand and easier to maintain.While writing an article on this topic I discovered that the code TimeXtender generates is no longer as close to a handwritten CASE statement as it used to be.Imagine the following setup: I have a handwritten CASE statement that populates a ProductClassDescription field based on the contents of the [Class] field (syntax in window on bottom-right). The way CASE statements work, the first result expression that returns TRUE for its boolean expression gets returned (see: https://learn.microsoft.com/en-us/sql/t-sql/language-elements/case-transact-sql?view=sql-server-ver16). This means that a record containing ‘H’ in the [Class] field will return ‘High’ for the [ProductClassDescription] field.If I create a ‘proper’ implementation using Fixed Transformation and conditions, the order of the transformations is inverted (bottom up). This means that the last Transformation ‘wins’ and the [ProductClassDescription_Clone] field contains ‘Low’.To exactly recreate the original CASE statement I would need to invert the order of the transformations: You do not run into this if your conditions are disjunct and remember to put the transformation without condition at the top, but you may want to set up a more complex if-then-else with overlapping conditions that makes use of the ordering of WHENs. I would then expect to define the transformations and conditions in a top-down order. If I add another Fixed Transformation without Condition to the bottom, all earlier Transformations are ignored in the code:Generated code:If the order is bottom-up I would expect only the ‘Unknown’ to be applied. I know that you can use a Default transformation to implement the ELSE in an IF-THEN-ELSE construct and avoid some of this, but I think this behaviour is wrong. I think the best solution would be:Use a single CASE with separate WHENs in the case of Fixed Transformations. Order them in the order defined in the UI (top-down) , allow only one Fixed Transformation without a condition and implement this as the ELSE statement. This also avoids the 10-levels of nesting issue with simple statements like this. Of course this would be a behavioural change and would require an option to be added to control this. Potentially use nested CASE statements for Custom Transformations as only scalar expressions use short-circuit logic, but I am not entirely sure what the reason for nested CASE statements is in the first place
Calculation Groups in Tabular models are very versatile. They may for example be used to;- avoid redundancy when creating time intelligence calculation items- dynamic format strings- currency conversions- default values in filters based on UPN et cetera.Calculation Group can be added to a deployed Tabular model (Endpoint) by executing xmla code, and run a Process Recalc command, from a PowerShell script. This can be done in TimeXtender with an External Executable that are run after processing the SSL in a Job.This means that the calculation groups will be reapplied each time a tabular model is processed, it takes no time at all, so the impact on performance is negligible.But there is a catch. A property in the tabular model - discourageImplicitMeasures - has to be set to true. Otherwise the calculation group can’t be created. We found now way to apply this in a script unless the entire model is replaced.The elegant way to solve this would be a new property in the TimeXtender portal where the Tabular endpoint is created. A new check box:Set Discourage Implicit Measures to True ⛝.When checked the property should be applied when the tabular model is deployed.I will not go into how to create calculation groups. There is already a lot published about it. For example, by Frank Wagner in TimeXtender Tuesday's episode 26.But I promise to write a small article on how to create the necessary XMLA and PowerShell scripts, once the checkbox is in place in the portal 😉. (This topic has been rewritten and moved from the Product Focus Group)BRAnders
Since SQL Server 2019 a new feature called calculation groups has been added. Calculation groups help with the reusability of calculations in models, for example, to simplify time calculations.
Currently we need to manually enter DAX for any calculations apart from the Standard Sums Etc. It would be great to have the ability to add Snippets for Tabular so er can develop a library.
The new TX release has an option to add a source table field as a TimeXtender audit field to a table. While all other TX audit fields are without spaces in the fieldnames, this field is [Source Table]. For consistency I would prefer this field to be in the format of the existing audit fields.
After deleting a data source from ODX server the dataset and pipeline objects remain in the data source folders in Azure Data Factory. It would be beneficial if these folders and objects are removed from ADF, since not all developers at clients have access to ADF in Azure to do a manual cleanup.TX release: 20.10.45
When we create a RLS Setup using dynamic like this: TX creates an internal table like: ..Companies_CompanyName_SecurityTableThis table it’s filled in the deploy&run proccess in TX.The first problem is that if we are using an external process to refresh the data, in this case the internal table is not updated, but we could avoid this by using a script action to populate that table.My suggestion is to not create the security table, create a security view instead.With this approach the data will be up to date.
When installing a newer TimeXtender version than the instance, I get this error message: Two jobs are also crossed out and display an error about an incompatible version: Where do I find out which version I should install?And how does the displayed instance version (3.1.0.0) map to the software version (6436.1)?
I need an option to give to my users the possibility to launch a job manually outside of the desktop app.We have some jobs scheduled but some times the user (some key users) needs to launch the job manually before the time that it’s schedule. Some ideas to how to do it?
When defining a "Selection Rule" in a table SSL TX requires the field to be used in the filter to be in the table, but sometimes it is not necessary because it is in the table source: This forces us to include the field in the table and hide it if we do not want to show it.It would be useful to allow the field to be used in the filter without having to include it in the table.
Hi,When using the query tool, TimeXtender does not allow you to use multiple fields with the same name:The table used for referenceAdding another “Country” field to trigger the errorThis was not the case in version 20 and previous. It is a bit annoying because we mostly use the query tool to do ad hoc queries for data investigation, transformation testing etc. Doing this quickly does of course not produce the prettiest queries but that does not matter because they are not intended to be saved.Having this extra limitation set by TimeXtender provides no clear benefit and only slows down aforementioned process.The current work around is to use SSMS, but it is unfortunate to have to jump between softwares. Appreciated if anyone had a look at this.Best,Pontus Berglund
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.