REST Data source - How to set replace value in Table builder?

Hi ​@anders.e.jonsson 

I just did a quick google on XLST functions and found this.

fn:substring(string,start,len) fn:substring(string,start)

Returns the substring from the start position to the specified length. Index of the first character is 1. If length is omitted it returns the substring from the start position to the end Example: substring('Beatles',1,4) Result: 'Beat' Example: substring('Beatles',2) Result: 'eatles'

I don’t know exactly how it works, but it seems to me that this could be an option to do something like this.

Hi ​@Thomas Lind ,

Thanks, but that made no difference. 

At what stage is the transformation performed?
What I really want to do is to hash sensitive data.

BR
Anders

Hi ​@anders.e.jonsson 

How did you set it up with this suggestion?

And you have to hash it immediately?

You can block access to the data in the Data Lake/SQL DB and then apply the hash in the table that is added to the data area.

Hi Thomas,

Yes, we will do the hashing in the DSA until it’s possible to do it in the Rest Data source. Before the data reaches the Ingest storage.

Have you found out were/in what stage the transformation is being done?

I hope the transformations are in no way dependent on the api we are reading from.
Many of the APIs our customers pull data from are very unreliable.
From our perspective, it would be best if it could be done right before data is written to the Ingest storage, then you could use SQL, and hashing wouldn't be a problem 😊

BR
Anders

Hi ​@anders.e.jonsson 

You can do all sorts of queries. There is the search and replace option, the connection variables and the ability to string multiple endpoints together using the endpoint query option.

So there is options besides the table builder.

Hi Thomas,

Thanks!

The reason för using the Table builder is flattening. Without that we get 9 small tables that proved difficult to join.

And replace value seemed like the perfect place to do transformations 😊But if that’s not working, I think we will continue to do hashing in the DSA for now.

BR
Anders

Hi ​@anders.e.jonsson 

I will ask if it is supposed to work or not. I don’t know myself, I only attempt to provide alternatives if something isn’t giving the expected result.

Hi Thomas,

I appreciate all your attempts! 😊

But the API’s we are using now are unreliable, and are changing quite often, so I don’t want to get it more complicated than necessary.

Several functions in the Rest Data source that works well when I try them with the API’s in your examples, but creates different errors, and sometimes duplicate rows and other strange things, when used in these API’s.

BR
Anders

HI Thomas,

Thanks!

Case closed then. 😊

I’ll add a link to this topic as well
 

Please upvote ☺️

BR
Anders