We are receiving some weird error messages when executing our project, which uses Azure Data Factory to copy tables from our Data Lake storage to the Azure SQL db (ODX → DWH). The error can be found in the attachments. When inspecting the pipeline run on ADF, it seems to be missing the configured activities (see second attachment). When executing the table manually from TX, the pipeline runs succesfully. Also the pipeline seems correctly configured on ADF and the pipeline run that is created when executing the table manually shows the correct activities.
Any ideas as to what might be causing this?
Best regards,
Kaj
Page 1 / 1
Hi @KajEmergo which TimeXtender version are you using?
Hi Christian, we are on version 20.10.38!
is it a particular table/data source that fails? is the issue intermittent (i.e. sometimes the package executes successfully, sometimes it fails)?
The issue is not related to a specific table but several ones (from Lake to DWH). It might be worth mentioning that it is mostly on DSA tables that have mappings from 3 different sources in ODX Server (the same ERP systems but from different sub-companies). It is indeed intermittent, the issue only happens occasionally.
Hi,
We are having the same issue on DSA tables that are merged from 3-4 different sources into one table. ADF gives 'Internval Server Error’ with no activity in the specific pipeline. When runing them seperately, they run succesfully all the time, but not in the system execution packages. It became an quite urgent issue for the system that fails often.
TX version 20.10.38!
Best,
Kerem
Hi @KajEmergo, it seems that you opened a ticket regarding the same issue in November 2022, at which point a reboot of the server seemed to fix the issue. Have you tried rebooting the server recently?
@KeremOG can you please share the full error message?
Some follow-up questions to troubleshoot both issues further:
How many threads are you running the package in? Have you tried reducing the number of threads?
Have you setup retries in the execution package?
Have you tried increasing the timeouts on the DSA (by right clicking on the DSA and selecting “edit data warehouse”)?
Hi @Christian Hauggaard ,
Yes, we were using 4 then decreased to 3 threads.
No, we setted up just.
No they are setted to Command timeout 1800s, connection timeout 60s.
Error message for the specific pipeline is:
"InternalServerError executing request”
@KeremOG ok thanks for confirming. What is the full error message returned within TimeXtender? Can you please try reducing to 2 threads to see if this helps? Also try increasing the connection timeout to 180s
@Christian Hauggaard, we will indeed adjust to 2 threads. Retries are configured now as well.
Please find 2 different error messages from TX attached.
@KajEmergo thanks for providing error messages. Did you reboot the server recently? Can you also increase the timeout settings on the DSA? Please let me know if the problem persists with these new settings
@Christian Hauggaard , yes we rebooted. We did the changes, it worked without failures for now. Thanks for the help.
For the future solutions, we would like to hear more about the cause. If you would have any tips, we would be glad to hear!
Glad to hear the issue has been resolved. Can you please help us by selecting and marking a best answer?
