Skip to main content
Question

Help with Primary Keys and Incremental Load in Table Inserts


Hi, 

I am facing an issue with duplicate records when using incremental load on a table that receives data via a table insert from another table in the same data area. The incremental load selection rule is based on DW_TimeStamp, and there are three PK in the target table.

The issue is that the primary key does not seem to be enforced in the target table, as I am getting duplicate rows in the table.

 

I tested this with another table where data is inserted through a table insert (but without incremental load), and I still see duplicate values for the PK fields and despite setting the table option to "Use instance setting (error)", no error is thrown when duplicates occur.

 

 

I have tested this behavior in TDI 6935.1 and 6926.1, and the issue persists.

 

My questions are: 

  1. How do primary keys work with table inserts? Should they prevent duplicates, or do they not function as expected when using table inserts?

  2. Can incremental load be achieved using table inserts? If so, what are the necessary configurations to ensure that primary key constraints are respected and duplicates do not occur?

Any insights, guidance, or best practices would be greatly appreciated. Thanks in advance for your help!

 

Best regards,

Leo

3 replies

rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 662 replies
  • April 2, 2025

Hi,

 

in my opinion Table Inserts should respect PK checks. If that does not work that may be a bug. You can use Related Record inserts to add an Exists or Not Exists constraint on the insert. So if you are doing a sort of incremental load where you pull a delta from a source and have a large (historical) table as well, you can map the delta to the target table and use a Not Exists (use an equals on the key fields) to insert any records with keys from the historical table that do not exist in the target table.

If you use PK checking, you are also inserting the errors into error tables. The additional I/O for that might be more costly than the Related Records route.


Thomas Lind
Community Manager
Forum|alt.badge.img+5
  • Community Manager
  • 1034 replies
  • April 3, 2025

Hi ​@leo.noharet 

This is actually a known bug. We had a ticket in zendesk about this.

It will be fixed in a future release.


  • Author
  • Starter
  • 1 reply
  • April 3, 2025

Thank you for your quick reply ​@Thomas Lind, do you have any estimate for when this patch is planned to be released? 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings