I have a project that is loading data from the ledger table into the staging database with approx, 5m records.
When manually executing the load, the ADO.NET transfer is taking 5-6 minutes, but the cleansing rules can take hours, to the point I have to terminate the load.
Is there any troubleshooting I can perform on the cleansing steps? I know I have taken the easy option on joins where the data types differ, and wonder if that is now compounding the issue?
Any tips or pointers greatly appreciated.
Best answer by Thomas LindView original