Skip to main content
To homepage To homepage

Hello,

Is there any posibility to know how much data was loaded through the data warehouse during a scheduled execution package? 

Either through TX repository, logs or through system files in the azure database?

Let’s say my customer has a databasewith 500gb worth of data, but we load only certain tables (some full, some incremental) and would like to know how much data we push every scheduled load.

Thank you,
Victor

Hi @Victor ,

 

there are many ways to keep track of this. If you are on 20.10.x you can use the TX Dictionary project as a baseline and expand it with table size information. In v21 you don't have an equivalent yet, but you can also just look at your resources: ADF will tell you how much data went through pipelines, your PaaS databases will tell you what the backup delta sizing is (the larger your backup, the more delta you have pushed), etc.

Hi @Victor 

Yeah as suggested by Rory, your options change depending on what version you are in.

A execution package has a setting for logging the row count. If you use that it will save how many rows are transferred on each execution.

Hi @Victor do the above comments answer your question? If so please help us by marking a best answer above

Thank you for your help, I think I can build something good around this!

Reply


Terms and ConditionsCookie settings