Skip to main content
Solved

varchar(max) fields not coming through in ADLS Gen2


Forum|alt.badge.img+1

I have a table with a varchar(max) field (stored in Dedicated Pool, which is a data source). and it won’t come through to ADLS Gen2 (ODX, version 6284.1). I’m using the Azure Synapse provider.

The longest field contains almost 4 million characters, but I’ve been unable to figure out whether this is a limitation in TX or Azure. Does anyone know?

Best answer by Christian Hauggaard

I replicated the steps described, creating a field varchar(max) on synapse with 4,000,000 characters


After which I received the following error message on the transfer task that "completed with warnings"

I managed to resolve the issue by increasing the connection timeout for the Azure DL storage on my ODX instance in the TimeXtender Portal

I then transferred the table to my MDW and confirmed that the field was brought over in SSMS

 

View original
Did this topic help you find an answer to your question?

5 replies

rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 651 replies
  • October 23, 2023

Hi @Søren Sørensen ,

it may be a limitation - I do not know whether PolyBase is used in your scenario but having 1MB limitations on LOB data is fairly common in this field. A varchar(max) will store a pointer in the record (there are usually also limits in record size as well) to the actual data. Do you get an error or a timeout? I guess TimeXtender may be able to confirm.


Forum|alt.badge.img+1

Hi @rory.smith .

There are no error messages in the UI or logs because I load data “on demand”. But your question triggered something I should have thought of … to do a simple ODX load, which I just did … and ran into this one:

System.NullReferenceException: Object reference not set to an instance of an object

We might need @Christian Hauggaard or one of the other Community Managers to have a look at this.


Thomas Lind
Community Manager
Forum|alt.badge.img+5
  • Community Manager
  • 1017 replies
  • October 23, 2023

@Søren Sørensen 

It may be better to add a ticket in the old system for this.

It may be about the size, but we will have to get a similar sized field to replicate the issue.


Forum|alt.badge.img+1

Thanks @Thomas Lind 

I’ll do that...!.


Christian Hauggaard
Community Manager
Forum|alt.badge.img+5

I replicated the steps described, creating a field varchar(max) on synapse with 4,000,000 characters


After which I received the following error message on the transfer task that "completed with warnings"

I managed to resolve the issue by increasing the connection timeout for the Azure DL storage on my ODX instance in the TimeXtender Portal

I then transferred the table to my MDW and confirmed that the field was brought over in SSMS

 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings