Skip to main content
Solved

Help with Primary Keys and Incremental Load in Table Inserts


Forum|alt.badge.img

Hi, 

I am facing an issue with duplicate records when using incremental load on a table that receives data via a table insert from another table in the same data area. The incremental load selection rule is based on DW_TimeStamp, and there are three PK in the target table.

The issue is that the primary key does not seem to be enforced in the target table, as I am getting duplicate rows in the table.

 

I tested this with another table where data is inserted through a table insert (but without incremental load), and I still see duplicate values for the PK fields and despite setting the table option to "Use instance setting (error)", no error is thrown when duplicates occur.

 

 

I have tested this behavior in TDI 6935.1 and 6926.1, and the issue persists.

 

My questions are: 

  1. How do primary keys work with table inserts? Should they prevent duplicates, or do they not function as expected when using table inserts?

  2. Can incremental load be achieved using table inserts? If so, what are the necessary configurations to ensure that primary key constraints are respected and duplicates do not occur?

Any insights, guidance, or best practices would be greatly appreciated. Thanks in advance for your help!

 

Best regards,

Leo

Best answer by Thomas Lind

Hi ​@leo.noharet 

The bug fix was released in the 6963.1 release.

https://support.timextender.com/product-updates/timextender-data-integration-6963-1-3337
  • 23785: Primary key check is getting skipped when a table has no mappings

    This issue was corrected - It affected tables with no mappings but having custom data, table inserts, and/or related records.

View original
Did this topic help you find an answer to your question?

rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • April 2, 2025

Hi,

 

in my opinion Table Inserts should respect PK checks. If that does not work that may be a bug. You can use Related Record inserts to add an Exists or Not Exists constraint on the insert. So if you are doing a sort of incremental load where you pull a delta from a source and have a large (historical) table as well, you can map the delta to the target table and use a Not Exists (use an equals on the key fields) to insert any records with keys from the historical table that do not exist in the target table.

If you use PK checking, you are also inserting the errors into error tables. The additional I/O for that might be more costly than the Related Records route.


Thomas Lind
Community Manager
Forum|alt.badge.img+5

Hi ​@leo.noharet 

This is actually a known bug. We had a ticket in zendesk about this.

It will be fixed in a future release.


Forum|alt.badge.img

Thank you for your quick reply ​@Thomas Lind, do you have any estimate for when this patch is planned to be released? 


Forum|alt.badge.img+1
  • Contributor
  • April 25, 2025

@leo.noharet, I stumbled upon the same problem (with the duplicate key values). The problem seems to be that the code TimeXtender should generate to do a duplicates check for the PKs is simply not generated. The checks for null values are generated as expected. 

So in short, currently when you fill a table with a table insert, the check for nulls is done, but the check for duplicates isn't. 


rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • April 25, 2025

I heard from colleagues that if you add a mapping to the table it will enforce the PK check, might be worth trying out if that is a workaround until it is fixed.


Thomas Lind
Community Manager
Forum|alt.badge.img+5

Hi ​@leo.noharet 

The bug fix was released in the 6963.1 release.

https://support.timextender.com/product-updates/timextender-data-integration-6963-1-3337
  • 23785: Primary key check is getting skipped when a table has no mappings

    This issue was corrected - It affected tables with no mappings but having custom data, table inserts, and/or related records.


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings