Solved

Dynamically Import tables into ODX


Hi, 

My source database contains over 300 tables, from which only 30 are relevant for our purposes. 

Instead of manually searching and checking the required tables, Is it possible to define a list of objects of interest(text)  and let TX pick it from there? 

Kind regards, 

Dror Svartzman

icon

Best answer by Christian Hauggaard 17 January 2023, 11:34

View original

6 replies

Hi Dror

I often create a selection template after I'm done manually selecting tables and fields, so I can re-use my selection elsewhere. 

An alternative is to manually add the objects of interest directly into a selection template file (XML) and import it. You would need to know the exact spelling of the table names and also minimum 1 fieldname per table.

You can also add additional logic like setup primary key and incremental loading.

Makes sense? 

Example: WideWorldImporters demo database:

<?xml version="1.0" encoding="utf-16"?>
<SelectionTemplate xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <TableSelectionTemplates>
    <TableSelectionTemplate>
      <SchemaName>Sales</SchemaName>
      <TableName>BuyingGroups</TableName>
      <FieldTemplates>
        <FieldSelectionTemplate>
          <FieldName>BuyingGroupID</FieldName>
        </FieldSelectionTemplate>
        <FieldSelectionTemplate>
          <FieldName>BuyingGroupName</FieldName>
        </FieldSelectionTemplate>
        <FieldSelectionTemplate>
          <FieldName>LastEditedBy</FieldName>
        </FieldSelectionTemplate>
        <FieldSelectionTemplate>
          <FieldName>ValidFrom</FieldName>
        </FieldSelectionTemplate>
        <FieldSelectionTemplate>
          <FieldName>ValidTo</FieldName>
        </FieldSelectionTemplate>
      </FieldTemplates>
    </TableSelectionTemplate>
  </TableSelectionTemplates>
</SelectionTemplate>

Hi, 

Sounds good!

Does it mean that you need to explicitly name every field? how do you prevent field name change?

Can you use the same method to upload other object type other than tables and view(stored procedures for example)? 

Yes, you would need to explicitly name fields. If possible, I use a fieldname that are available on every table like "TimeStamp", "ModifiedDate" or "RECID". After import, I then filter my Data Selection pane to only show selected tables. Then I can easily select the additional fields or click the table box twice to select all.

You cannot import more than tables/views and I'm curious what procedures running in the data source that you also want included in your BI platform? Not something I've ever had the need for. Can you provide an example?

 

Hi Jacob, 

Firstly, your help is much appreciated.

The platform on which the source database resides, generate insights using stored-procedures but not all of the data is stored back in tables. The stored-procedures have a "deep", nested dependencies (not only table/view typed). 

I'm afraid I need to build up a proxy database where the result of the sp will flow into and then use this proxy as the source of TX. Not the prettiest solution but then again in our profession not everything is...

Kind regards, 

Dror

I have very similar question...I used to limit number of fields under a table when using 'BusinessUnits'. We don't have such option on ODX transfer task ? Also, Cant we grab Transfer Task on ODX to TX Execution Package where I can do 'Transfer Task' + Staging + MDW' in one -go.

Userlevel 6
Badge +5

Hi @satish.padarthi this functionality to select columns in the ODX is available in the latest version of TimeXtender. Also, we now have an advanced rule based selection as well as a simple selection option in order to choose relevant tables. Please see the following article for more information

 

Reply