What is TimeXtender Data Integration?
TimeXtender Data Integration automates, orchestrates and documents data solutions. To properly setup your environment for TimeXtender, it’s important to first understand how the platform works. The platform can be understood as an abstraction layer where data is ingested and business logic is defined, independently of the underlying infrastructure. This makes the business logic resilient, and facilitates a "single version of truth", while achieving fluidity in the underlying components. In other words, infrastructure changes can be made quickly (which is often times necessary in an ever-changing technical landscape) without lengthy migrations, rebuilding business logic or compromising the "single version of truth". Learn even more and get up to speed quickly with Training.
Anatomy of a TimeXtender Data Estate
1. Ingest Server
Windows-based server software installed on a centrally located physical or virtual machine. This software orchestrates the ingestion of data from various sources. This separates data movement from the desktop software, while ensuring data never leaves a users environment.
2. Ingest Storage
A SQL database or Azure Data Lake where raw data is stored after ingestion from sources. By storing data in it's raw form, it can be used for later analysis, or retrieval.
3. Prepare Storage
A SQL Database, Synapse Dedicated SQL Pool, or Snowflake Data Warehouse where data from multiple sources is cleansed, transformed, and consolidated into a single version of truth.
4. Semantic model endpoints
A subset of related data combined into a single model or "mart". This model can be exported to multiple endpoints such as Power BI, Qlik, Tableau, Analysis Services, or CSV.
5. TimeXtender Portal
A web portal handling the administration of your TimeXtender Data Estate. Instances hold the storage connection details as well as maintain the configuration information implemented in TimeXtender Desktop.
6. TimeXtender Data Integration
Windows-based Desktop-client software where each instance can be configured & implemented through a single, integrated user-interface.
TimeXtender Instances
Instances are the metadata components of your TimeXtender Data Estate which are configured in the portal and
- store the connection info to the associated target storage.
- contain the metadata for how this area of TimeXtender is configured. (e.g. What tables are selected in an Ingest Data Source, What relations & transformations exist in your Prepare Instance, etc.)
- maintain versioning, so each individual instance can be rolled back a previous version.
- can be copied to another instance of the same type. Allowing you to migrate your instance configuration to another storage type or environment.
Ingest Instance
This is where you ingest raw data from disparate sources into a centralized storage. An infrastructure choice has to be made regarding Ingest storage, although this can always be easily adjusted at a later point.
Prepare Instance
This is where you prepare, cleanse, transform, and consolidate data in your Data Warehouse to create a "single version of truth". An infrastructure choice has to be made regarding storage for your Prepare Instance, although this can always be easily adjusted at a later point.
Deliver Instance
Also referred to as semantic models, this is where combine and deliver relevant data into data marts or models and publish to various endpoints for business consumption. A Deliver instance can have multiple endpoints, but a choice has to be made as to which endpoint(s) should be used as a starting point.
Learn more about adding Instances and Data Sources here
Select a Reference Architecture
TimeXtender supports various storage options, so environments can be configured in many different ways. Below are 4 suggested reference architectures based on common scenarios:
Sandbox
- Ideal for non-production use cases such as prototyping, demonstration and training
- Can be setup for FREE, using SQL Server Developer Edition on an existing windows machine in less than 20 minutes
Azure SQL Database
- Balance cost and performance in an Azure cloud solution
- Ideal for production data solutions smaller than 1 TB
Azure Synapse Dedicated SQL Pool
- Geared towards big data solutions in Azure
- Ideal for production data solutions greater than 1 TB
- Near-infinite scalability, up to PB scale
- Extremely powerful, distributed, analytics workloads
- Not suitable for smaller data solutions, as performance, in these cases, is better served by Azure SQL Database
Snowflake
- Snowflake’s multi-cloud compatibility allows users to deploy on AWS or GCP.
- Automatic usage-based scaling of compute and storage resource provides ideal cost/resource optimization.
- Suitable for medium to large data solutions. A great choice for mid-size data solutions (from 500GB and up), or in cases where estimated size of data solution is uncertain and/or might rapidly grow.
AWS
- Balance performance and cost using Amazon Web Services (AWS)
- Use AWS VM for the application server and Amazon RDS SQL database for ODX and MDW storage
On-Premise SQL Server
- Ideal for production on-premise data solutions in SQL server
Keep in mind that it is quick and easy to deploy TimeXtender instances to a different environment architecture at any time. For example, if you build your TimeXtender instance on-premise, you can quickly and easily deploy your instance onto Azure SQL DB or Azure Synapse, with little to no downtime. So, don't feel like you will need to live with your decision forever.
If you're having trouble deciding which option to select, please reach out to your TimeXtender Partner or Solution Specialist to determine the best course of action for your goals.
To learn more about installing TimeXtender and setting up your instances see the below article:
