Skip to main content
SUBMITTED

CSV Data Source - Skip Top X rows when determining Header record

Related products:TimeXtender Data Integration
  • March 14, 2025
  • 2 replies
  • 49 views

Stijn Verhoeff
Explorer

In the current TimeXtender CSV Data source, the Header Setup only allows the first record to be marked as the header. In some cases, files may contain one or more redundant rows, before the actual data starts:

<Row>      
1 Some irrelevant data    
2 FieldA FieldB FieldC
3 1 A ...
4 2 B ...
5 ... ... ...

 

In those cases, you would want to skip the top X rows and mark row x+1 as the header

2 replies

Stijn Verhoeff
Explorer

Just to provide a workaround for the time being, you can abuse the 'Skip comment rows’ feature, depending on your file's data structure. 

In the sample table above, enabling the 'Skip comment rows’ and setting the 'Comment row character’ to 'Some’, skips the first row entirely and parses the second row for headers. That got it working for my specific case, but this workaround will likely not solve all scenario's. 


Forum|alt.badge.img+1

This functionality is also essential when connecting Excel files, as they may similarly contain irrelevant rows at the top before the actual header row appears. The ability to skip the top X rows and explicitly define the true header row is critical for ensuring accurate data ingestion and proper field mapping in both CSV and Excel data sources.

Unlike the workaround currently possible for CSV files—where the 'Skip comment rows' feature can be abused by setting the 'Comment row character' to match a known pattern in the irrelevant rows—there is no native workaround for Excel files. As a result, users are forced to either manually clean the Excel files before ingestion or resort to scripting the table structure themselves, which undermines the low-code/no-code approach TimeXtender is known for.