Copy of item in the TimeXtender Product Focus Group with some edits to summarize.
In the world of BI tools, vendors are moving to SaaS solutions just as in other domains. Qlik has done the same and is focussing development on Qlik Cloud (QC). As Qlik is delivering nice improvements with respects to Qlik Sense Enterprise (QSE) and QlikView (QV) in this platform, customers are quite motivated to migrate towards this solution.
Improvements of note that motivate customers to migrate are:
- Can host both QSE and QV apps in one solution
- Notifications and Alerts are included (costly expansion for QSE) Development of new features is done first here (if developments ever land in the older tools)
- Qlik Application Automation adds powerful features missing in the older solutions
- Integration of QDI features into the platform
The idea is of course that you use all the other tools in Qlik's suite and work entirely in the cloud, but you can also push data from on-prem. The options Qlik delivers on this topic are the following:
- Qlik Data Gateway (analogue to the Power BI Gateway). See: here . Coupled with using the correct APIs, this seems the easiest way to get support to work. This approach is also more performant than Qlik DataTransfer.
- Qlik Automation can expose REST endpoints that can be called from TX to do things like reload apps.
- Qlik DataTransfer allows you to create a DataSet in QC that can (on a schedule) reload data from a source (for instance the views created by a Semantic Model in a Datawarehouse instance's Data Area) into QC and reload an app.
- Qlik DataTransfer can also "watch" a folder and upload files (QVD, CSV, etc.) to the Data Files area and reload apps.
- Qlik DataTrasfer can also reload apps in QSE and move them to QC.
- Qlik APIs create/import/reload apps programmatically (with or without data) qlik-cli commandline tool to interact with Qlik APIs
- Qlik Sense Enterprise customers with QSE deployments can synchronise apps from on-prem to QC based on rules
- QlikView Publisher customers with QlikView Publisher deployements can publish apps to QC
Within our active customers and in a lot of our pre-sales discussions with prospective customers we see a large motivation to move to QC. The question that then pops up is "How will TimeXtender support / work with Qlik Cloud?".
Given that there are APIs that would allow programmatic integration with the platform, we feel TimeXtender should be able to support QC as a first-class citizen. This would mean that an endpoint for QC should be implemented that allows the generation of QVD or front-end apps for:
- customers with only QC --> use the APIs to facilitate this
- customers with QSE and QC --> use the APIs or use the existing QSE endpoint and programmatically configure the synchronisation capability to synch apps with QC
- customers with QV Publisher --> can set up the push to QC in their environment as there is probably no API for this
The current QSE endpoint implementation would also benefit from a review: you point TimeXtender to a QSE node (QSE deployments can consist of many nodes) and TimeXtender talks directly to the Engine Service on this node. This may overrule any load balancing defined in your QSE deployment: you might have reload nodes set up to perform heavy lifting on appropriate hardware but TimeXtender may be talking to the central node of a cluster causing reloads to happen on the wrong node.
Qlik publishes APIs to programmatically work with their system and allow any rules set up in the cluster to be applied. Given that QSE can work with QC, it stands to reason that the same approach QSE takes to push apps to QC could be taken by TX.
Taking this approach means customers will use Semantic Models and automate from within TimeXtender instead of setting up something manually that will likely pull data from a Datawarehouse layer instead of using a Semantic Model.
One of the most powerful points of Semantic Models is being able to define them and push them to whatever tools you support. Needing to bypass Semantic Models for tools likely to be very common is unfortunate. This also makes it easier for customers to transition between BI solutions without needing to re-engineer or to feel that effort spent developing Semantic Models was wasted in retrospect.
What problem is this solving?
There is no direct support for QC in TimeXtender, work-arounds are much more manual than was the case before Power BI Premium Capacity support or require QSE next to QC and are therefore more costly.
As Qlik fleshes out Qlik Application Automation and tools like Qlik DataTransfer, customers may decide that the added benefits of the things QC does will outweigh the automation TimeXtender would bring if you then need to manually integrate the two.
TimeXtender has the (Shared) Semantic Layer as a major component of the solution, it would be strange if QC customers would then need to bypass this. And this becomes especially tricky if things like row-level security (Section Access) would then require complicated work-arounds. The current QSE implementation bypasses QSE handling appropriate for clustered environments meaning that deployments with larger Qlik installs need to change the approach and do more development in Qlik instead of in TimeXtender than would normally be required.
How is this problem currently solved (inside or outside of TimeXtender)?
In the case of QC: Customers would need to deploy Qlik DataTransfer to pull data to the cloud (time-consuming manual process) or monitor a folder with CSV exports (not efficient) or keep a QSE instance (costly) running to be able to push to QC.
Another option is to pull data from TimeXtender into QC (potentially through Application Automation) if the database is reachable from QC. This way of integrating data is generally more robust than using Qlik DataTransfer, but does require the database being reachable from outside and needing to write or push script.
How does this idea improve the business value for NEW and EXISTING customers?
New customers would expect support for the current platform offered by one of the largest BI vendors. Not having to worry about the chosen BI solution or the migration between them would be a great relief.
New customers with large QSE deployments would not need finicky workarounds to get their load balancing to work. Existing customers considering migration would not need to worry about how to deal with this and would be able to fulfill the wishes of business quickly. Existing customers would also not need to re-engineer parts of their implementation when they decide to work with QC instead of QSE.
For QSE deployments, working with the proper Qlik APIs would make it possible to make much more use of the features provided by TimeXtender and bundle more of the orchestration there instead of needing to dive into QSE.
Provide references or explain why this approach is best practice
We have tried out the architecture where QC and Qlik DataTransfer were chained together to push data to Qlik Cloud (in PoCs). This is in no way a "pretty" process, especially compared to the ease with which you can work with the currently supported endpoints. We have also tried out the Qlik Data Gateway approach and this works well, though currently in a pull scenario.
Being able to orchestrate as much of the data chain from source to end-product from TimeXtender is a great USP that should be strengthened further.
