Skip to main content
To homepage To homepage

In this article you will learn about the new Azure Cloud Optimizer package type, how to create one in TimeXtender Data Orchestration and run it.

When using cloud resources, such as Azure virtual machines and Azure SQL databases, leaving them idle at high resource usage levels, or running at all, when they are not truly needed can quickly become very expensive. The Azure Cloud Optimizer package type for TimeXtender Data Orchestration allows you to create packages that can switch Azure Virtual Machines on or off, and reboot them should that be required, or scale Azure SQL databases up or down, between purchasing models, tiers and the different available capacities. These packages can be run before and after other processes, making sure those processes that need powerful virtual machines running that nothing else uses, or that require databases to process much more data than they otherwise do very quickly, have what they need, but then going back to the less expensive configuration afterwards.

Create a Data Provider

  1. Open up TimeXtender Orchestration and Data Quality (O&DQ) Desktop client
  2. Click General > Data Providers
  3. Right-click Data Providers, select New -> Data Provider
  4. Type in a Name for your Data Provider
  5. Type in a Description for your Data Provider
  6. Select the Systems in which your Data Provider should be accessible
  7. Click Data source Type and choose Azure Connection
  8. Type in the Tenant-Id, App Id and the Client Secret from the Azure App Registration
  9. Click Save

Create Azure Virtual Machine package

  1. Click Orchestration
  2. Right-click Packages, select New -> Azure Cloud Optimizer.
  3. Type in a Name for the Azure Cloud Optimizer package.
  4. Select the recently created Data Provider.
  5. Select the appropriate resource type, “Virtual machine start/stop”. This will show the appropriate Configuration panel, and populate the list of Azure virtual machines. Depending on the amount of resources in your Azure subscription this step may take a long time.
  6. Select the virtual machine name you wish to configure the package for.
  7. Select the appropriate operation for the virtual machine, Start, Restart or Shut down.
  8. Click Save, and the package is ready to be run.

Create Azure SQL Database scaling package

  1. Right-click Packages, select New -> Azure Cloud Optimizer.
  2. Type in a Name for the Azure Cloud Optimizer package.
  3. Select the recently created Data Provider.
  4. Select the appropriate Resource Type “Azure SQL database scaling”. This will show the appropriate Configuration panel and populate the list of Azure SQL databases. Depending on the Amount of resources in your Azure subscription this step may take a long time.
  5. Select the SQL database name you wish to configure the package for.
  6. Select the purchasing model you wish to configure the database to (vCore or DTU).
  7. Select the tier you wish to configure the database to (“General Purpose”, “Business Critical” or “Hyperscale” for the vCore purchasing model, “Basic”, “Standard” or “Premium” for the DTU purchasing model.
  8. Select the capacity you wish to change the Database to, depending on the purchasing model and tier, there are different capacities available.
  9. Click Save, and the package is ready to be run.

For advanced users there is an option to write the JSON payload of the REST command to scale the database directly (for information on the parameters that can be configured this way consult the Azure documentation from Microsoft). Note that when a package using custom JSON to scale a database is run, the package run is marked as successful once Azure has responded to the scaling REST command with a success code.

Hi,

 

this is really great! I have a few questions:

  • what permissions does the App Registration require?
  • is it possible to choose the Hardware Type for databases? Premium hardware is often required for Hyperscale databases

Hi Rory,

Just to notify, this feature will be available in our new 24.3 version which will be released at the beginning of November.
 

The required permissions are as follows:

To scale an Azure SQL database, you need specific Azure Role-Based Access Control (RBAC) permissions. The required roles include:

  • Contributor
  • SQL DB Contributor
  • SQL Server Contributor

To start or stop an Azure Virtual Machine (VM) via the REST API, you need specific Azure Role-Based Access Control (RBAC) permissions. The required permissions include:

  • Microsoft.Compute/virtualMachines/start/action: To start the VM.
  • Microsoft.Compute/virtualMachines/deallocate/action: To stop (deallocate) the VM.
  • Microsoft.Compute/virtualMachines/read: To read the VM properties.

These permissions are typically included in the following built-in roles:

  • Contributor
  • Virtual Machine Contributor
  • Owner


Regarding Hardware Types:

 

Changing between hardware types (or families) using this package type can be done, using the Advanced JSON feature, by adding in the “family” parameter and choosing the correct sku name for the desired tier. An example, changing to the Fsv2 family, General Purpose tier, capacity 2, could be done using this JSON:

{"location": "your-region","properties": {"sku": {"name": "GP_Fsv2_2","tier": "GeneralPurpose","family": "Fsv2","capacity": 2}}}
 

Switching between hardware types for an Azure SQL Database does involve a brief period of downtime. During this switch, there is a short connectivity loss to the database, which can be mitigated using retry logic in your application. This downtime occurs because the database engine process is restarted and may be moved to a different virtual machine if needed.

To minimize the impact, it’s recommended to perform such operations during low-usage periods and ensure that your application is designed to handle transient errors gracefully.

Another questions about this. Is it included for the standard TX customers?

It will be possible to resume Azure SQL database from paused to online?

Hi @rvgfox 
This is a part of the full version, so it will not be available for the TX light version.
Pausing Azure SQL databases will not be available for this first version of this package.

Hi @Dagný Björk Stefánsdóttir , 

We use Identity authentication for scaling Azure resources. Advantage is that you do not need to save credentials in Exmon only in Azure. You need to create a Identity in Azure for the Virtual Machine that is running the script, this VM needs to have Contributor permissions on the Azure SQL Database.

Is this also supported? Or will this be supported in feature?

Hi @bas.hopstaken,

Currently that is not an option. This is of course a newly developed feature so we are looking for feedback from everyone. I do expect that we will revisit this feature to add in more options, so any ideas or use cases you think are missing from the feature are well appreciated.

For your setup Bas, I have only used Identities for calls from within the Azure tenant but I’d expect that the Orchestrator would always need to save the credentials else it’d not be able to authenticate, or would you not think so?

Hi @gtgunnarsson ,

Currently we are using PowerShell to scale Azure resources. For now we have only experience with scaling Azure SQL Database and Azure Analysis Services.

For both we use Identity authentication. The authentication part of the PowerShell is only one line of code:

Connect-AzAccount -Identity

Other resource parameters needed are server name, resource group name and database name. 

Reply


Terms and ConditionsCookie settings