To homepage To homepage
Solved

Transfering multiple tables from Theobald Error while copying content to stream

Niels Jonker
Rank
Userlevel 1
Badge +1

I am setting up TimeXtender to extact tables from SAP with Theobald. I have a selection of tables that I can succesfuly extract when I add just one table to a transfer task. However, when I add all relevant tables to a single transfer task I run into errors. The extraction seems to be succesful on the Theobald side but moving the data to the ODX storage gives the following error “Error while copying content to a stream”. I can figure out why the tranfer is working when I do it per table and gives an error when I do multiple tables..

 

Full error sample below:

 

Executing table rest_mara_generalarticledata:

failed with error:

System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.ObjectDisposedException: Cannot access a closed Stream.

   at System.IO.__Error.StreamIsClosed()

   at System.IO.MemoryStream.get_Position()

   at System.Net.Http.StreamToStreamCopy.StartAsync()

   --- End of inner exception stack trace ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.REST.RestWebRequestManager.<SendRequestAsync>d__7.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.REST.RestWebRequestManager.<SendRequest>d__6.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at Azure.Management.REST.RestWebRequestManager.<SendRequest>d__5.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.API.ADLSGen2.PathAPI.<AppendData>d__14.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.API.ADLSGen2.PathAPI.<AppendData>d__13.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at ODX.Parquet.ParquetDataUploader.<UploadData>d__31.MoveNext()

   --- End of inner exception stack trace ---

---> (Inner Exception #0) System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.ObjectDisposedException: Cannot access a closed Stream.

   at System.IO.__Error.StreamIsClosed()

   at System.IO.MemoryStream.get_Position()

   at System.Net.Http.StreamToStreamCopy.StartAsync()

   --- End of inner exception stack trace ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.REST.RestWebRequestManager.<SendRequestAsync>d__7.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.REST.RestWebRequestManager.<SendRequest>d__6.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at Azure.Management.REST.RestWebRequestManager.<SendRequest>d__5.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.API.ADLSGen2.PathAPI.<AppendData>d__14.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Azure.Management.API.ADLSGen2.PathAPI.<AppendData>d__13.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at ODX.Parquet.ParquetDataUploader.<UploadData>d__31.MoveNext()<---

 

icon

Best answer by Christian Hauggaard 15 May 2023, 14:33

View original

10 replies

Christian Hauggaard
Rank
Userlevel 6
Badge +5

Hi @Niels Jonker 

Just to confirm - if you execute the table rest_mara_generalarticledata on its own in a transfer task it works successfully?

Can you please let us know which version of TimeXtender you are using?

Which data source provider and provider version are you using?

 

Niels Jonker
Rank
Userlevel 1
Badge +1

Hey Christian,

 

Yes I can succesfuly extract the table with it’s own transfer task. 

I am using Theobald to extract the SAP data. I set it up as described in the article about Theobald & TimeXtender. 

It’s using the REST connector version 22.0.8257.0

It also looks like the extraction is succesful on the Theobald side so I think the error is happening somewhere in the ODX transfer proces.

Thomas Lind
Rank
Userlevel 5
Badge +5

Hi Niels

In my experience, when you get the cannot access closed stream message it is a sort of timeout based on an external one. So it could be the Theobald REST provider that timed out or similar to that.

Check your timeout settings and see if you are able to increase them.

Christian Hauggaard
Rank
Userlevel 6
Badge +5

@Niels Jonker as Thomas stated, can you please try increasing the timeout setting?

Also, can you please confirm how long the transfer task ran before the error occurred?

Niels Jonker
Rank
Userlevel 1
Badge +1

@Thomas Lind I have increased timeouts to the maximum on ODX, MDW, Data Source and in TX Desktop under advanced settings on the data source. I still get the error. I’ve also upped all timeouts on the Theobald side as far as I can see.. The task runs for about 50 minutes and the errors occur after about 20 minutes till the end of the run.

Christian Hauggaard
Rank
Userlevel 6
Badge +5

Hi @Niels Jonker could you please try reducing the number of execution threads to 1 under advanced settings? And perhaps try only selecting 2 tables for the transfer task in order to troubleshoot further?

 

Niels Jonker
Rank
Userlevel 1
Badge +1

@Christian Hauggaard I’ve tried a subset of tables that were giving errors and set the execution threads to 1. It worked that way but if I want to do it this way with all tables it’s going to be slow ofcourse… Any solution for this?

Christian Hauggaard
Rank
Userlevel 6
Badge +5

Hi @Niels Jonker

Thanks for testing. Have you increased the timeouts for the Azure Data Lake ODX storage?

After these have been increased, could you please try gradually increasing the concurrent execution threads and seeing how this affects the transfer time?

Niels Jonker
Rank
Userlevel 1
Badge +1

@Christian Hauggaard I was now able to extract all tables with one transfer task. I already increased all those timeout settings before. I see you are also using a different version of TimeXtender, I’m using the latest version. I will try with 2 threads, I tried 3 earlier but that did not work.

Niels Jonker
Rank
Userlevel 1
Badge +1

@Christian Hauggaard 2 threads also works. I guess that the timeout time is cumulative and when multiple large tables are extracted at the same time some timeout is exceeded. With the current setup, 2 threads is acceptable so for now the issue is resolved.

Reply


Terms and ConditionsCookie settings