Skip to main content

    Filter by idea status

    Filter by product

    859 Ideas

    rory.smith
    TimeXtender Xpert
    rory.smithTimeXtender Xpert

    Make it clear whether a table set to "Automatic" extraction is actually incrementalSUBMITTED

    ContextWhen starting on a new TimeXtender implementation, you are usually making a lot of transformations and changes to tables. If you are new to TimeXtender, or coming from a 20.10.x deployment, you may not realise that all tables pulled from ODX (in ADLS) to DWH instance will automatically be incremental. This means that after an initial Deploy & Execution any subsequent changes will not materialise unless you force a Full Reload of the table.In a running implementation, where you are changing less at the same time (usually) this is slightly less annoying. It is still needlessly confusing in my opinion. ChallengeNew users and users that are used to the “old way” may not realise that the Automatic setting for Data Extraction on a table means that they will be incremental. The fact that this is the case is only visible if you expand mappings and notice the I on the mapping icon. This can result in surprises: either nulls for new fields for existing data, or old values for modified transformations. This means that you really need to cover this in a training context before you start any deploy and execution actions, substantially frontloading complexity.If a table has multiple mappings, some of which may not be incremental, it becomes even more confusing. Current solution / work-aroundCurrently you either need to change the Data Extraction setting of tables to Full from the default Automatic, or do a “Review Tasks” followed by selecting the full load steps required (I am not certain whether always selecting all options for a table is always safe, you may need to specifically select the required steps). If you are doing a more complex change where you want to Deploy and Execute multiple objects, this becomes tedious Proposed solutionMake it explicit when Automatic makes a table incremental by giving a visual cue. There are probably three states:Full load (none of the mappings are incremental) Incremental (all of the mappings are incremental) Mixed (at least one of the mappings is incremental)I'm sure someone can come up with nice pixel-art to make clear icons.In addition it would be good if an optional warning can be added / extra options can be added when performing a Deploy & Execute after a change. Something like “This table has incremental mappings, not all records may reflect changes” and an extra checkbox in the Deploy & Execute menu “Perform full load”. This should be an option in the TimeXtender settings so that people who are used to walking uphill to school in a snowstorm can do things the hard way.