Hi team how to run data bricks script from TX
Best answer by fwagner 17 March 2023, 20:12
Thank you for your question!
To include Databricks jobs in your TimeXtender orchestration, I recommend looking at the “jobs API” of Databricks. You can either create the jobs in the UI and trigger them with the API or you can create the jobs on the fly using the API.
Either way you would create power shell scripts in your TimeXtender environment to trigger the job, and include them in your execution packages.
When authoring the power shell script, consider a secure way to store access tokens, e.g. in windows environment variables in an azure key vault, so you don’t include secrets in your power shell code.
The above is assuming you’re using the latest version of TimeXtender (v6024.1 or later).
Does that help to guide your next steps?
Please let us know how it goes.
Hi @anishekkamal26, did @fwagner’s reply help in setting this up? if so can you please mark the response as best answer, or let us know if we can provide further assistance :)
Hi @Christian Hauggaard @fwagner ,
Could you please specify the exact version details where I can perform above steps.
latest version is :
where I am not able to found this functionality.
You will need to figure out a way to run the Data Bricks script from a script action.
I don’t know why, but in the past external tasks like this was done by creating a SQL job that would start the external task. This job can be started with a script action like this.
Frank’s solution above is based on the latest version of TimeXtender (v6024.1 or later). While Thomas’ solution applies to the version you are using 20.10.40 (the legacy version).
Have you been able to try Thomas’ suggestion using a script action? If so could you please help us to mark the best answer above
Already have an account? Login
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.