Skip to main content
Question

Is there a way to skip invaild value in BU

  • September 4, 2026
  • 6 replies
  • 59 views

Forum|alt.badge.img+2

A JSON response looks like this 

[
{
"ref": 123456,
"dim_1": "A",
"dim_2": "B",
"peroid": 202601
},
{
"ref": 5E+20242959,
"dim_1": "A",
"dim_2": "B",
"period": 202602
}
]

The second ref (5E+20242959) is a problem - its neither STRING nor an INT.
Ref is defined as NVARCHAR in BU, but as you can see the value is not between “”.
I have tried the following approaches under, still i get the same error

Input string '5E+20242959' is not a valid number. Path '[56935]['ref.53e92a42-b13c-4062-87b4-6bd5a89547b1.1992753']', line 1, position 36741783.

How do i solve this?


Here are the approaches i have tried:

Omitting Ref in Table Flattning

In table flattning, i have left it out. I have the other three values, but still landing returns an error:

Defining the value as NULL in table flattning


Field transformation

Condition on column ref in BU

I have defined the following condition on the column, still i get the same error

CASE 
WHEN [ref] LIKE '5E+%' THEN NULL
ELSE [ref]
END

Script Actions

I have created a script that i have run pre-load - still the same error

UPDATE [ODX].[_test]
SET [ref] = NULL
WHERE [ref] LIKE '5E%';

 

6 replies

rory.smith
TimeXtender Xpert
Forum|alt.badge.img+8
  • TimeXtender Xpert
  • September 7, 2026

HI,

can you convince the source system to not use scientific notation in outputs, or to produce believable output? I assume 5 × 10²⁰²⁴²⁹⁵⁹ is not a valid value in most fields outside of science.

Syntactically I think it is valid JSON, there is just no way to store that value in a “normal” type. You can pre-process the data as a file and simply add "” around those values or perhaps filter those rows out with a data selection rule before even ingesting them


Forum|alt.badge.img+2
  • Author
  • Champion
  • September 7, 2026

For the vendor of the source system its not a priority at the moment
As i have mentioned in my initial post, i have omitted it completely when creating a flttened table, but i still get an error about it when lading i BU. Why is that?

How do i filter these rows with a data selection rule?
I am using TX Enhanced Connectors and i get an error when i try using data selection:

 


rory.smith
TimeXtender Xpert
Forum|alt.badge.img+8
  • TimeXtender Xpert
  • September 7, 2026

Hi,

I would see if the API allows for filtering as it looks like there is no support for those kind of selection rules in that source connector.


Forum|alt.badge.img+2
  • Author
  • Champion
  • September 10, 2026

I have now got the possibility to filter 5E values in POST body 

May be it’s a good idea to allow filtering values while landing !


Thomas Lind
Community Manager
Forum|alt.badge.img+5
  • Community Manager
  • September 22, 2026

Hi ​@aftabp 

Currently filtering the content in the Get or Post call is the only option for the REST provider.

The reason it fails in Table flattening is that it is the second step. First step is the raw file, that then gets transformed, so if it isn’t able to add it to the raw part it will fail despite adding a conversion in the table flattening setup.

You can create this issue as an idea then it may be added as a feature in a future release.


Forum|alt.badge.img+2
  • Author
  • Champion
  • September 24, 2026

​@Thomas Lind 
Thank you for the clarification
As i wrote in my previous post, i have got the vendor to add a filter in the POST so that i can filter out the 5E-values