Hello,
Is there any posibility to know how much data was loaded through the data warehouse during a scheduled execution package?
Either through TX repository, logs or through system files in the azure database?
Let’s say my customer has a databasewith 500gb worth of data, but we load only certain tables (some full, some incremental) and would like to know how much data we push every scheduled load.
Thank you,
Victor
Best answer by rory.smith
View original
+7
Hi
there are many ways to keep track of this. If you are on 20.10.x you can use the TX Dictionary project as a baseline and expand it with table size information. In v21 you don't have an equivalent yet, but you can also just look at your resources: ADF will tell you how much data went through pipelines, your PaaS databases will tell you what the backup delta sizing is (the larger your backup, the more delta you have pushed), etc.
+5
Hi
Yeah as suggested by Rory, your options change depending on what version you are in.
A execution package has a setting for logging the row count. If you use that it will save how many rows are transferred on each execution.
+5
Hi
Thank you for your help, I think I can build something good around this!
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
