Skip to main content
Question

Max value of multiple columns in Lakehouse

  • July 5, 2025
  • 1 reply
  • 24 views

Søren Sørensen
Contributor
Forum|alt.badge.img+1

I’m looking for a TimeXtender Fabric Lakehouse (pyspark.sql) expert 😀

I’ve got a SQL transformation that selects the maximum value of 10 columns:

(select max(val) from (values ([Column_1]), ([Column_2]), ..., ([Column_10])) as [Values](val))

How do I achieve this in a Prepare Lakehouse custom transformation?
I could build the mother/father of all massive case statements, but I’d prefer something simpler and more elegant … if possible!?

1 reply

rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 713 replies
  • July 7, 2025

Hi ​@Søren Sørensen ,

 

I have not tried but theoretically you should be able to either use Spark SQL: https://spark.apache.org/docs/latest/api/sql/index.html#max

or use the post-script feature with pySpark, i.e. something along these lines: https://stackoverflow.com/questions/57531778/how-to-find-the-max-value-of-multiple-columns


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings