Hi team how to run data bricks script from TX
Solved
Hi team how to run data bricks script from TX
Best answer by fwagner
Hi
Thank you for your question!
To include Databricks jobs in your TimeXtender orchestration, I recommend looking at the “jobs API” of Databricks. You can either create the jobs in the UI and trigger them with the API or you can create the jobs on the fly using the API.
Either way you would create power shell scripts in your TimeXtender environment to trigger the job, and include them in your execution packages.
When authoring the power shell script, consider a secure way to store access tokens, e.g. in windows environment variables in an azure key vault, so you don’t include secrets in your power shell code.
The above is assuming you’re using the latest version of TimeXtender (v6024.1 or later).
Does that help to guide your next steps?
Please let us know how it goes.
Best Regards,
Frank
+4Hi
Thank you for your question!
To include Databricks jobs in your TimeXtender orchestration, I recommend looking at the “jobs API” of Databricks. You can either create the jobs in the UI and trigger them with the API or you can create the jobs on the fly using the API.
Either way you would create power shell scripts in your TimeXtender environment to trigger the job, and include them in your execution packages.
When authoring the power shell script, consider a secure way to store access tokens, e.g. in windows environment variables in an azure key vault, so you don’t include secrets in your power shell code.
The above is assuming you’re using the latest version of TimeXtender (v6024.1 or later).
Does that help to guide your next steps?
Please let us know how it goes.
Best Regards,
Frank
+5Hi
Hi
Could you please specify the exact version details where I can perform above steps.
latest version is :
where I am not able to found this functionality.
+5You will need to figure out a way to run the Data Bricks script from a script action.
https://legacysupport.timextender.com/hc/en-us/articles/360051848111-Scripting
I don’t know why, but in the past external tasks like this was done by creating a SQL job that would start the external task. This job can be started with a script action like this.
+5Frank’s solution above is based on the latest version of TimeXtender (v6024.1 or later). While Thomas’ solution applies to the version you are using 20.10.40 (the legacy version).
Have you been able to try Thomas’ suggestion using a script action? If so could you please help us to mark the best answer above
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
