Skip to main content
Question

TimeXtender CSV Data Source - PathNotFound (aggregation pattern)

  • November 28, 2024
  • 7 replies
  • 108 views

Hi,

We are trying to connect to several CSV files stored in a local folder. While we can successfully synchronize the data source and perform a full load in the ODX, we encounter an error when attempting to add the table to our data area (DSA).

The issue lies in the path to the Parquet file stored in Azure. The correct path should be:

CSV_DNB/csv_*/DATA_2024_11_28__11_09_50_2219585/DATA/DATA_0000.parquet

However, the path Timextender is looking for is:

CSV_DNB/csv_^*/DATA_2024_11_28__11_09_50_2219585/DATA/DATA_0000.parquet

It seems that Timextender is misinterpreting the automatically generated name and adds a ^ character.

I also attempted to use a specific file aggregation pattern, such as;
H100.*.csv
(all files in folder have the prefix H100 followed by a random number). However, I encountered the same error. Is there a way to specify the name of the table generated in the ODX? It seems like the “File aggregation pattern” is the issue. Do you have any idea how to fix this? 

 

-Execute Execution Package Execution Queue Default 'Failed'
    System.AggregateException: Pipeline execution failed ---> System.Exception: {
      "errorCode": "ActionFailed",
      "message": "Activity failed because an inner activity failed; Inner activity name: TIMEXTENDER COPY DATA, Error: ErrorCode=UserErrorFileNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code 'NotFound'. Account: 'xxx'. FileSystem: 'containerodx'. Path: 'CSV_DNB/csv_^*/DATA_2024_11_28__11_09_50_2219585/DATA/DATA_0000.parquet'. ErrorCode: 'PathNotFound'. Message: 'The specified path does not exist.'

 

 

7 replies

Thomas Lind
Community Manager
Forum|alt.badge.img+5
  • Community Manager
  • 1017 replies
  • November 28, 2024

Hi ​@annfoss 

I do it like this.

 

I would use H100*.csv.

Did you try that and are you updated to version 22.0.0.1 for your CSV provider?


  • Author
  • Participant
  • 14 replies
  • November 29, 2024

Hi ​@Thomas Lind  Yes we are on version 22.0.0.1 for our CSV provider. I tried with: H100*.csv but I got the same error unfortunately:

 

Path: 'CSV_DNB/csv_H100^*/DATA_2024_11_29__07_45_31_0797599/DATA/DATA_0000.parquet'. ErrorCode: 'PathNotFound'.

 

It still adds the: ^


Christian Hauggaard
Community Manager
Forum|alt.badge.img+5

Hi ​@annfoss 

I am unable to reproduce the issue you describe

I have 3 csv files

These are my datasource settings

 

I execute the import metadata task and synchronize

and then execute the transfer task and bring into my prepare instance

I am using Ingest instance with data lake storage and prepare instance with Azure SQL db storage and ADF transfer from Ingest instance to Prepare instance

Do you notice any differences in our setup?

I am using version 22.1.0.0 of the provider, can you perhaps try upgrading, just to rule this out?


  • Author
  • Participant
  • 14 replies
  • December 2, 2024

Hi ​@Christian Hauggaard we have the same setup as you have. The only difference is that we are on version 6766.1, so we don’t have the import metadata task. Do we perhaps need to upgrade?

Some additional information: We are able to do a Full load in the ODX and preview the data. We also get a parquet file in the data lake storage. 


Christian Hauggaard
Community Manager
Forum|alt.badge.img+5

@annfoss please try to upgrade TDI and the data source provider to the latest version, please let me know if you still encounter the issue after upgrading


  • Author
  • Participant
  • 14 replies
  • December 4, 2024

@Christian Hauggaard we tried to upgrade to 6822.1, but I am sorry to let you know that we still encounter the same error: 

 

Path:'CSV_DNB/csv_^*/DATA_2024_12_04__14_03_26_1213665/DATA/DATA_0000.parquet'ErrorCode: 'PathNotFound'. Message: 'The specified path does not exist.'


Christian Hauggaard
Community Manager
Forum|alt.badge.img+5

@annfoss I have created a support ticket for this issue


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings