To homepage To homepage

- - Knowledge base overview
Product Updates
Training
Portal
For Partners
TimeXtender Classic

Home
Community overview
TimeXtender Desktop Q&A
Prepare

Prepare

Ask questions about Prepare Instances in TimeXtender Data Integration

222 Topics
1,002 Replies

When you subscribe we will email you when there is a new topic in this category

222 Topics

Recently active

Most replies Most views Newest first

rvgfoxProblem Solver

asked in Prepare

Add custom table insert feature

Can you explain how and for what it can be used?

3266

1 year ago

rvgfoxProblem Solver

asked in Prepare

Revese Data Area feature

Can you provide some documentation about this?How can I use it?For what it must be used?

692

1 year ago

rvgfoxProblem Solver

posted in Prepare

Copy between instances

When you arte coping between instances, the destination insatance name doesn’t show complete.

430

1 year ago

rvgfoxProblem Solver

posted in Prepare

TX allows to add data selection rules over calculated fields but gets an error in execute

As you can see in the screenshot, I think that this situation must be controled:

496

1 year ago

richardrayStarter

asked in Prepare

Troubleshooting data cleansing rules

v20.10.36.64Hi,I have a project that is loading data from the ledger table into the staging database with approx, 5m records.When manually executing the load, the ADO.NET transfer is taking 5-6 minutes, but the cleansing rules can take hours, to the point I have to terminate the load.Is there any troubleshooting I can perform on the cleansing steps? I know I have taken the easy option on joins where the data types differ, and wonder if that is now compounding the issue?Any tips or pointers greatly appreciated.Thanks,Richard

LorenzoContributor

Hi,I would like to do a “range lookup”. Because I can't solve it using a conditional lookup, I used a custom field transformation. My code looks like this:CASE WHEN substring([SBIcode_cleansed], 1, 1) like '[^0-9]' THEN (SELECT [kvk_groep_omschrijving] FROM [dsa].[SBIcode_groepering] where [SBIcode] = [SBIcode_cleansed] ) --'test1' ELSE (SELECT [kvk_groep_omschrijving] FROM [dsa].[SBIcode_groepering] where substring([SBIcode_cleansed], 1, 2) >= [min_sbi_code] and substring([SBIcode_cleansed], 1, 2) <= [max_sbi_code] )--'test2' END It seems that a subquery is not possible because I get an error message that more than one value is retrieved using the subquery but that is not possible.Testing it in SSMS works fine. (Not using between because above seems to work faster when testing

1286

1 year ago

rvgfoxProblem Solver

posted in Prepare

Adding a new field to an existing table with data

If you add a new field with the deploy only option, to an existing table with data, there are two diferent behaviors, depending if the table has incremental rule defined or not.This is controled by this setting: By default it’s cleared for “normal” tables and setted to incremental tables.I think the behavior should always be the same or at least get a warning message

rvgfoxProblem Solver

I’ve detected that if you have the same table in DSA and MDW. let say DSA.Invoices and MDW.Invoices for example, and you define partition for that, both tables share the same partition schemas and partition functions in the SQL Server database.It’s a problem when you try to add or remove field from whatever table.

1068

1 year ago

rory.smithTimeXtender Xpert

asked in Prepare

Replace transformation with char type requires looking up the int value of a char

I discovered that when using the replace transformation with the CharType that you are required to know or look up the int value of the character you want to replace and use that as the parameter.This does not seem to be documented anywhere or be useful in any way I can think of. I assume REPLACE('abc’, char(65), 'b’) is functionally equivalent to REPLACE('abc’, 'a’, 'b’). In a Case Insensitive collation there is no difference between the two anyhow.There is also no COLLATION or binary type support, which are things that you can do with REPLACE() that might actually be useful in certain rare cases.

1515

1 year ago

rvgfoxProblem Solver

asked in Prepare

Problems with "Manage Query Tables"

I’ve a query table created to my D365 F&O data source like you can see in this screenshot: The query returns data as you can see in the query tool (it would be nice to be able to run it in the “Manage Query Tables” window too): But, If try to Synchronize ODX Objects: I get an error and there is not the table if I try to remap it: And when I deploy and execute the table, it’s empty: Is this a bug or I’m doing something wrong?

1062

1 year ago

malose.motlhasediStarter

asked in Prepare

SQL Server Analysis Services Error

Greetings,I have an odd error and trying to eliminate where this issue is coming from.I have a server running SQL server 2016 and TX is running on top of that. The version of TX I am running is 20.10.15.64 in Server 1.The errors I am getting are attached. TX is running ok in Server 1 which is DB server.There is Analysis Server 2 which seems to be an issue and is creating the errors above.The attached error is coming from PowerBI Report Server interface. We are using Microsoft PowerBI tool for our reports. I have increased memory allocation in Server 1 where TX is running and issue seems to still be there.Our products are all 64bit version.Any Idea or view of what could be causing this.

3155

1 year ago

sierd.zuiderveldContributor

asked in Prepare

Object Security for Raw table (_R)

Hi, Is it possible to add object security rules on a _R table from within TimeXtender? Thx!

574

1 year ago

rvgfoxProblem Solver

posted in Prepare

Data platform with TimeXtender data architecture

I would like to share with you this diagram that shows the architecture of our data platform with TX.Perhaps it will be another recomended architectures and I would like to know it and start a discussion about this topic.

250

1 year ago

MarcoJStarter

asked in Prepare

How to remove tables from a MDW

When I remove a MDW-table from the project it remains available in the MDW-database.How do I completely remove the table so I can start over (create a new table using the old name)

rvgfoxProblem Solver

We are implementing a data warehouse with data from Dynamics 365 F&O and I’m thinking that we will have money data from UK in pounds and from Spain in euros and we will need currency conversion (based on time) to resolve this situation.Some body has considered this situation?How must it be implemented?

571

1 year ago

Peter DijkstraStarter

asked in Prepare

Data selection rule is not working with Mapping Sets

If i try to add a dataselection rule my tables stay empty.Tried allready the selection rule on the table and with an without ‘ ‘ If i do the same in the query tool it looks good. Is it a bug?Using version 6143.1

mchinskyExplorer

After manually executing a package, where do I find a log that shows the # of rows processed per table? I'm trying to verify my incremental loads are working properly

1083

1 year ago

Jan StroeveContributor

asked in Prepare

Is Custom Data Masking possible?

The following article describes several types of data masking: https://legacysupport.timextender.com/hc/en-us/articles/5463363539101-Dynamic-Data-Masking .Does TimeXtender allow custom data masking as well? The example table shown at the top of this article is exactly what I am trying to accomplish. However, the article doesn't describe this type of masking, does it?

852

1 year ago

alain.thireContributor

asked in Prepare

Loading transactions

I have a transaction table in odx with the following fields (simplified)Key,Value, transactiondate Example100, V100, 2023-01-01100, V100A,2023-01-31 09100, V100B,2023-01-31 10 Each day there are multiple transactions added , of course with another transaction date.Note in the example I have only shown the date part (not hours, minutes, seconds) What I need is a table with the latest update in DSAI created the same table. Set primary key to KEY and added an incremental selection rule on transactiondate . Added history settings to update the Value and transaction date based on the keyWhere it goes wrong is if we get in one day multiple transactions 100, V100A,2023-01-31 09100, V100B,2023-01-31 10I traced down the cleansing procedure and Tx detects that there are twice the same key. So far so good.Next it puts the ID of the latest transcation into <table>_L and only processes the ID’s that ar not in <table>_LThe result is that I get 100, V100A,2023-01-31 09 which is incor

50511

1 year ago

danielTimeXtender Xpert

posted in Prepare

Supernatural keys & Data Cleansing performance

Dear Community,I like to build my data estates with supernatural keys but in lager datasets the data cleansing starts to take very, very long. Do you happen to have the same issues? Is there a way to make the supernatural keys load faster? Even with incremental loading it begins to be super slow:I've ran a test on 435,397 records. This is on a Azure SQL with 10 vCores1. is a full load on the table with 7 supernatural keys.2. is a full load on the same table without the supernatural keys.1 has data cleansing of 1 second. 2 has a data cleansing of 104 seconds!Second I've done a test on the same tables but now incremental loads:1. incremental load with 7 supernatural keys2. incremental load without supernatural keys1 has a data cleansing time 1,6 seconds and 2 a data cleansing of a whopping 129 seconds!I'm not so sure I want to keep using the supernatural keys. What do you guys do? Take care= Daniel

2262

1 year ago

sierd.zuiderveldContributor

asked in Prepare

Force Supernatural key value

Hi, I have a list that maps product_codes to product_ID's.What I would like to create is a Key Store that, when fed a product_code from the list, produces the same product_ID and when fed a new code (so not in list) produces a supernatural_key as usual. Is it possible to force the keystore to create the ID's as shown above?

1082

1 year ago

rvgfoxProblem Solver

asked in Prepare

Setup Pre and Post script

The setup phases are confusing, can anyone explain where exactly they run? The Pre script runs previously to insert the records in the valid table, it’s correct in my case, but the post script that must be run after the records inset, doesn’t run.My script it’s:

893

1 year ago

bas.hopstakenTimeXtender Xpert

posted in Prepare

Batch cleansing option - best practices, tips and tricks

Hi all,I'd like to start a discussion about the batch cleansing option. You can find this option in: Table Settings → Performance → Enable batch data cleansing. This option is disabled by default. I'm looking for best practices when you should use it and how to determine the best batch cleansing size.I normally enable it on big tables (5.000.000 + rows), this reduces the disk space used for logging. But in some cases it also improves performance. It can result in faster loading of a table. However, I'm always struggling determining the correct batch size. I know the answer will be ‘it depends’. But I'm trying to find some best practices for this option.For now I start at 250.000 rows for batch cleansing and increase with 250.000 rows. I write down the timings and pick the best setting. This setting depends on the number of rows and number of transformations in the table. I'm curious if there are any practices or tips and tricks for this?

5121

1 year ago

brandon.spencerContributor

asked in Prepare

Are "Over Ride Data Types" applied in order like a SQL case statement?

Does anyone know whether or not the rules you put in the over ride data types area are applied sequentially like they would be in a SQL case statement (i.e. once a field evaluates to TRUE it ignores the subsequent logic)?

501

1 year ago

rvgfoxProblem Solver

asked in Prepare

How to use variables to usage conditions

I would like to run a job only if the number of records in the raw table it’s diferent of the number of records in the valid table wit a condition.How can I do it?

904

1 year ago

Page 7 / 9

Badge winners

avanrijnhas earned the badge Ingest Expert
danielhas earned the badge Deliver Expert
p.ruijterhas earned the badge Jobs & Executions Expert
lstegmeierhas earned the badge Certified Solutions Architect
Bennyhas earned the badge Certified Solutions Architect

Show all badges

Terms and Conditions Cookie settings

Sign up

Already have an account? Login

Login with SSO

SSO login

Username *

E-mail address *

Full Name

(Private)

Only you and moderators can see this information

TX Partner / Customer *

Company

(Private)

Only you and moderators can see this information

Password *

I accept the terms & conditions

loginBox.register.email_repeat

Login to the community

No account yet? Create an account

Login with SSO

SSO login

Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.