Question

Why is my execution to much slow?


Badge

I am working in a project and the execution is to mach slow, 

 

It has take 35 minutes for 5 tables, that all of them has 10000 rows. 

 

 

 


9 replies

Userlevel 6
Badge +5

Hi @ignacio 

Which version of TX are you using?

Are you using ODX or business units?

Are you seeing performance issues on the transfer from ODX to DSA or data cleansing in the DSA?

What storage are you using for ODX/BU and DSA? e.g. SQL Server, Azure SQL db, data lake, etc.

Badge

Hi @Christian Hauggaard,

We work with  version 20.10.36.64

We use ODX, the  transfer from ODX to DSA  is slower in some ocations  return the following error

The timeout period elapsed prior to completion of the operation or the server is not responding.

 

Userlevel 6
Badge +5

Hi @ignacio 

Which ODX storage are you using? SQL Server, Azure db or data lake?

Please also confirm which DSA storage are you using? i.e. on-prem SQL Server or Azure sql db?

Can you please try increasing the connection and command timeouts for your ODX and for your DSA?

Please also try to reduce the number of threads to 2 in the execution package, and deploy (i.e. by right-clicking on the project and selecting “Save project as deployed version” under advanced)

 

Please also monitor the CPU and memory on the app server during execution to see if there are any bottlenecks

Badge

@Christian Hauggaard 

about your question 

Which ODX storage are you using? SQL Server

which DSA storage are you using? on-prem SQL Serve

I changed the time out according you advisor. 

I dont understand why Query Tool working. I am frustrated

Userlevel 6
Badge +5

Hi @ignacio I have created a support ticket for this

Userlevel 4
Badge +5

Dear @ignacio ,

I believe I've had the same issue before. The thing is that you;ve put the max rows to copy (or the batch size) to 0, which means all the rows.

in this case TX will try to take ALL the rows from the ODX and transfer them in 1 batch. hich is useally huge and will most of the time will fail and result in timeouts (because it takes to long). Also the machine you are working on might run outt of RAM really easily and quickly.

Maybe if you change the max rows to copy to 300000 (default TX setting) this will fix your problem?

 

Hope this helps

= Daniel

Badge

​ Hi @daniel , I made the change you mentioned and the process takes 1 hour and 37 minutes. I stopped the scheduler service to only run my execution

 

 

Userlevel 4
Badge +5

Dear @ignacio ,

can you screenshot the gantt chart? Im wondering where the execution is so slow. 

Badge

Yes @daniel 

 

Reply