Skip to main content
To homepage To homepage

We are using the provider Azure Data Factory - SQL Server for a lof connections where the instances for these connections contains almost the same data but each time another customer.

 

What we have noticed when the incremental load is setup to look at a datetime field called DW_MODIFIED_date. it will never load that 1 record which is in the table.

 

When I exclude that table from the incremental load then I have 1 record loaded.

When I include the table again in the incremental load it is removing the first folder from the previous load and don’t write a new file because it was not updated.

When I remove the folder from the container and I incremental load that client table again I get no record at al while it should be the first load. Even if the incremental load is on and there is no file  then I expect to have in the initial load the data.

Hi ​@jermaine.cotino 

It could be a precision issue. I assume you are using a datetime field for your rule.

For example if this row is in a second, it will use that as the rule and then loose it on subsequent runs due to it not being larger or equal to this exact time as the row was added in this second.

Have you tried to add a 1 minute subtraction on the rule?

Terms and ConditionsCookie settings