Build better products with our product team
Currently, the incremental load setting needs to be configured at the transfer task level in the Ingest Server. Since Orchestration is now the best product for scheduling reloads in TimeXtender Data Integrator (TDI), it would be much easier to set this option at the package level within Orchestration.This change would allow you to define a single transfer task for a data source in the Ingest Server while maintaining the flexibility to execute that task as either a full load or incremental load using Orchestration.
Hi all, a question from me. We have a customer with a lot of very large tables. We would like to filter these tables in our dev and test systems, to be able to develop and execute faster.Is there any way we can filter tables in the Ingest conditionally? So we can add filters that will not be transferred to production on promotion, or filters that have a condition so that they will only be applied on non-production Ingest instances?
Using jobs is going to be a deprecated feature so we are using Exmon Data Orchestration.However we can not see a process queue or stop executions from out of Exmon.Apparently this can only be seen if you have an additional paid license but this ticket is to make that availble in the default license that comes along with TimeXtender.It is strange that one hand we are forced to be using Exmon but then not have access to the main functionality (starting and stopping processes) to do orchestration properly.Hope you agree on this.
The Challenge:We've probably all encountered the challenge of coordinating releases, especially within bigger development teams. Currently, when we execute a release in TimeXtender, the entire project gets deployed. This can lead to problems, especially when trying to address minor production issues or releasing only the work of a single developer while others are still in progress. This often results in project-wide delays. The Solution: Partial ReleaseWhat if we had the option to release only parts of the ODX, MDW, or SL, rather than the entire project? This "Partial Release" feature has the potential to: Boost Efficiency: It allows us to release only the specific changes that are ready for deployment, making collaboration more seamless. No more waiting for everyone to finish their work simultaneously. Facilitate Agile and SCRUM Practices: For larger teams and those following agile methodologies, this feature is a perfect fit. It aligns with the agile principle of releasing work as soon as it's completed, helping us stay nimble and responsive. Enable Quick Production Issue Fixes: When a production issue arises, quick fixes are crucial. With a partial release feature, we can address these issues in production without inadvertently deploying other unreleased changes, maintaining system stability. Expand TimeXtender's Applicability: This feature opens the doors to larger organizations with multiple TimeXtender developers. It ensures that TimeXtender remains a viable choice for businesses with extensive development teams.The "Partial Release" feature in TimeXtender is more than just an enhancement; it's a solution to a common problem that developers face daily. It aligns with best practices, offering greater flexibility and efficiency in development processes..
There is a very nice new source connector for Dynamics F & O that allows you to bypass the Synapse Link / CDMUtil hassle involved with extracting Dynamics 365 F & O data. Additionally this virtualises the extraction, meaning you no longer need to make a redundant copy of the data involved.At the moment, this requires a Fabric Ingest + Prepare setup which has some performance and functionality drawbacks. There are two improvements that could be made to allow more implementations to make use of this great provider:make it so that other types of prepare can also load through the virtual Onelake Ingest. This way you could add a separate OneLake Ingest or switch over and still be able to use Azure SQL or Snowflake or other Prepare variants. make it so that it is possible to build a data movement from Prepare to Prepare. This way, you could set up a separate Ingest + Prepare backed by Fabric capacity to extract and transform D365 F&O data and move the results to your “master” Prepare instance in, for example, Snowflake. The current “reuse existing objects” functionality does not work for this kind of situation.Apart from D365 F & O, I would expect Business Central to also be able to benefit from a similar approach at some point.
Hi,It would be good if you could choose the encoding/character set for the source files in the connector.I had a problem with a file, but was able to load it after changing the encoding of the file to UTF-8.BRAnders
It would be beneficial if the authentication process for the REST data source in the portal could be expanded a bit further.The addition of an authentication endpoint is a great start, but in our setup, we have a REST API that requires both authentication and a login to execute methods. Moreover, the login also needs to access the authentication token.Currently, we are executing these methods by storing the login and auth token in a table and then querying them using a Dynamic Values Source Query.However, it would be more efficient if we could reference these details directly within the query without needing an extra table. This would streamline our authentication workflow and reduce the complexity of managing multiple data sources.Could we possibly extend this endpoint to include the option to perform both authentication methods and then use them as variables in the methods?This would greatly enhance our ability to manage authentication within the portal and simplify our current process.
I have a customer table that is a union of 7 different sources. Same ERP but in different databases and different Ingest Sources (so 7 different folders in azure storage).When I add a FIELD TRANSFORMATION to the VALID table and re-run it, both the RAW and VALID are dropped and reloaded (I don’t use history). In this case 7 ADF-pipelines are spun up, but it was not really necessary because nothing new came from any of the others VALID tables.When you deploy the changes, you see that TX only deploys the TRANSFORMATION view, so it “sees” that only a change at the level of the TRANSFORMATION view was added.I don’t know if this is feasible in all cases, but it would be more cost-efficient as I don’t always need to spin up those (expensive) ADF-pipelines
If I send tables to the Execution Queue from here: I can not see the name of the table in the Execution Queue: This makes it difficult to manage and monitor the Execution Queue. If i drag-and-drop the table to the Execution Queue I see the table name: I think this should be the case for all executions.
It would make it easier for business analysts and report developers if it were possible to see data lineage and documentation in the portal.A new role would also be needed in the portal. A role which only gives the right to see data lineage and documentation.2024-09-05 Added wishes:- Data catalogue- Graphic representation of data models
With the support for Fabric Lakehouse in Ingest and Prepare instances, there is only one missing: being able to have a Fabric Lakehouse endpoint in Fabric Lakehouse.This would make reusing Deliver instances / datamarts in a Fabric context much more usable.
This screen it’s crazy. Please redesign it to be more “human friendly”.Adding filters in the colums for example.
It would be great to have the option to add the table of a mapping set to a transfer task in the ODX.I have a lot of data sources with the same ERP (these are all separate plants across Europe) when importing 1 table from 1 data source, you have the option to convert that to a mapping set, which is a very good addition to the software in these cases when right clicking the mapping set, you get a few options. It would be great to also see an option to add this new table to a specific transfer task when your process is created end-to-end (from ODX/Ingest to SSL/Delivery), you have the option to ‘generate end-to-end Taks and schedules’. All relevant tables will be added to a transfer task as in my examples (Auto_Orders…. and Auto_Sales)However, when you develop, you don’t automatically do it end-to-end. In order to get data from the new table into the DSA, you first have to add that new table to a transfer task. A transfer task that you also have to create first. In my case, I have 7 different plants, so I need to repeat this action 7 times. When I add more new tables to the DSA, I always have to repeat this step. Then I have to manually execute those 7 transfer tasks (1 time per data source). So, an option to add a table to a transfer task (where that transfer task is immediately created) would be a great addition.
It would be great if the option of loggin on to Sharepoint through the Data provider in Exmon DG would allow the option of having OAUTH. Currently the user has to be without MFA to be able to log in.
If you create a Data Area for a Fabric Lakehouse Prepare Instance with a space in the name, you will discover this is not allowed (because a schema with the same name is created automatically). If you then change the Data Area name, the schema name is unaffected.It would be useful to have TDI pop up a question asking whether you want the schema name to be changed as well. Similar to how name changes for Conditional Lookups are suggested.
Whe you asing a custom insert to a new table it would be very usefull to get an option to create the fields in the table automatically.
We use supernatural keys for tabular links … it’s such a pain tabular doesn’t support composite joins.(please let me know if this is not best practices)If we use supernatural keys for e.g. open sales orders (Company + SO), we’ll build a repository of orphaned keys over time as the open sales orders get closed and removed from the data warehouse.Will this impact performance over time? Not sure … but I think it would be a nice feature if we could right-click a key store and get the option to delete orphaned keys (in both v20 and v21).The alternative is of course to redeploy the key store and regenerate all required keys.
When developing over multiple data areas (e.g. DSA and MDW) it would be great to have the option to deploy and execute only the modified tables and views all in one single task. However since the new version this is only available on data area level and not on instance level. Adding this feature back again would be nice!
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.