It's almost like you cannot say fall without Fabric. At least this October release of TimeXtender Data Integration (Desktop v. 7142.1) has Microsoft Fabric features as the headliner along with a new logging system for Ingest and a bunch of big and small fixes and improvements. Dive in below!

New and improved

Fabric Lakehouse on Prepare gets views, shortcut tables and much more

With this release, we're taking a major step towards full support for Microsoft Fabric Lakehouse as Prepare instance storage. Validations, history tables, table inserts, custom data, custom hash fields and junk dimensions are now supported on Fabric.

On the programmability side of things, you can now add Notebook views, a new type of custom views that are deployed on a Fabric notebook and can be referenced and used in custom scripts. Regular custom views are also supported through the Fabric storage's SQL endpoint.

Utilizing the OneLake shortcut feature, the Fabric flavor of the enable/disable physical valid table option is now available. Disabling the valid table creates a shortcut table where data is loaded directly from the Ingest instance. This saves execution time and storage space, but on the flipside, the table in Prepare must be 100% identical to the source table in Ingest - no transformations allowed.

As a final improvement, Fabric instances now support service principal authentication, so a non MFA-user is no longer required.

Redesigned Ingest logging

We've taken a fresh look at logging in the Ingest instance in order to make it more useful and more robust. It includes more thorough logs, more options, an improved UI and, last but not least, file-based logging.

Logs are now stored in the local file system and not in the Ingest instance, which is handy when you're investigating, e.g., issues with connecting to the instance in the cloud. It also makes them immediately available if you want to use them for analysis purposes. To aid in that, we're using the open W3C Extended Log File Format for the logs.

When setting up the Ingest service, you can configure the log retention in days, as well as the maximum file size of the logs to keep the logs from hoarding to much storage space. You can also select the minimum severity level for logs to the log file as well as the Windows event log.

The old logging system has been deprecated, but is still available for the time being when Show Deprecated Features is enabled in the View menu.

PowerBI endpoints now support Direct Lake

The Power BI endpoint in Deliver now supports Direct Lake on SQL and Direct Lake on OneLake which facilitates much faster data access. This is especially useful when you have really large amounts of data.

Control access to instance blueprints

Based on partner feedback, we've updated the access controls for instance blueprints. Partners can now control which customers are granted access to specific shared blueprints. The initial implementation worked more like a "shared folder" where all blueprints were available to all customers.

And much more

In addition to the headliners, the release also includes a bunch of smaller improvements:

• Snowflake storage now supports key-pair authentication in preparation for Snowflake's requirement for multi-factor authentication for all users that kicks in by November.
• In the Ingest instance, we've changed the name of the default transfer task to "Load Data" and added a second default task called "Load Data (Full Load)". Previously, the default task was called "Full Load", but would actually used incremental load if it was available.
• The Ingest Service Configuration can now import the configuration of a previously installed version, making upgrading a bit easier.
• In the Portal, we've made deleting data source connections easier. You'll now see what instances the data source is mapped to and those mappings will be deleted along with the connection. Previously, you'd have to delete all mappings before you could delete the data source connection.
• We've tuned the data lineage queries to get you better performance when you want to view data lineage for an object.
• REST data sources now makes URL encoding of query parameters optional.
• ODATA discovery now includes certificates (PFX/PEM) for authentication.

Fixed

Desktop

• Fixed freezing UI and missing feedback when generating End-to-End Tasks and Packages.
• Fixed an issue in the Deploy and Execute dialog which could cause out of memory exception when having deployment and/or execution steps.
• Fixed an issue causing an 'Unauthorized' error.
• Fixed an issue where very large error messages could cause TDI to not correctly communicate with Orchestration
• Zoom in/out should now behave correctly in the data lineage window.
• Fixed an issue where data lineage would find old relation and potentially make the search very large.
• Changed some misleading labels in the Primary Keys and Synchronize windows.
• Added option to disable "windowed incremental load" for ADO and OleDB data sources to make incremental load work for specific ODBC sources.
• Fixed an issue in the remapping dialog in the Metadata Manager where sorting the columns would cause an "Index out of range" error and close the dialog.
• Fixed issue with installing new data source versions in a multi threaded environment.
• Fixed issue where ODBC/Ansi syntax for ADO/OleDB ignores incremental subtract value.
• Search & replace in REST now saves empty replace values as empty strings instead of null.
• Fixed errors with data type conversions on data area transfers using Fabric prepare.
• Fixed an issue with custom transformations and transformations on lookup fields returning null on Fabric prepare.
• Fixed an issue with empty date tables when working with multiple schemas on Prepare instances with Fabric storage.
• Fixed an issue with selection rules not working after renaming a field on Prepare instances with Fabric storage.
• Fixed an issue where the stored procedure used for direct read between data areas could become too long.
• Fixed an issue where adding and deleting a table mapping in a data area could crash the application.
• Fixed an issue where the SQL endpoint in a Prepare instance could cause slow UI updates.
• Fixed an issue where renaming a table where a field is used as a lookup field on a conditional lookup field would not mark the table where the conditional lookup field belongs as modified.
• Fixed an issue where deploying a history table with a dot in the table name would fail.
• Custom selection rules in Deliver instances would loose variables when closing the project.
• Fixed an issue where the order of the columns from Prepare instance would change.
• Made sure the Add Calculation Group and Edit Column Description dialogs have a minimum height and only vertical scrollbars.
• Fixed an issue where the semantic endpoint would resolve the Endpoint Name parameter incorrectly on execution.
• Fixed an issue where Power BI endpoint deployment with RLS fails with error AD Service Principal Authentication is not supported with this SQL Server version.
• Fixed an issue where deploying a Prepare instance could crash the application.
• Fixed an issue with parsing SqlDecimal data in Parquet files in Ingest.
• Fixed an issue where fields on Deliver instances could not be deleted if they contained a custom data selection rule.

Portal

• Fixed a duplicate/incorrect log entry on user (contact person) deletion
• Fixed an issue where activity logs required a refresh to show new entries
• Fixed issue where users were added despite invitation failure; failures return proper error messages now.
• Fixed activity log order for data source creation with simultaneous ingest mapping.
• Standardized table design across most of the Portal.
• Fixed issue where Double value types were not correctly interpreted in data source connection forms, leading to a failure to save.
• Fixed issue where the "Use Microsoft Entra members in Ingest instance security roles" checkbox would automatically reselect itself after being deselected and saved on Ingest instances with Azure SQL storage.