Instance Manager

The Instance Manager displays the Datatailr Virtual Machines (VM) that are running various runnables and the costs of running these VMs. Datatailr dynamically creates VMs in order to ensure just enough computing power to support the hardware requirements of all its jobs scheduled in the Job Scheduler as defined by the settings in the Auto-scaling app in the Datatailr Settings app. Therefore, typically there’s no need for you to do anything in the Instance Manager.

Note – A global Datatailr administrator (Admin) and a member of the instance_admin group can access the Instance Manager.

Note – You may refer to the Datatailr Cost Manager for information about the cost centers in your organization, such as the cost incurred by specific runnables and by specific teams of employees.

Tip – Datatailr automatically spawns just enough VMs to support job requirements so that there’s generally no need to spawn even more VMs, because it won’t improve performance.

Tip – Typically, there is no need to terminate VMs, unless notified by the cloud service provider (such as AWS) that their servers will be down. If you terminate VMs, Datatailr will automatically restart VMs in order to support user requirements. If needed, for larger VMs, you might decide to save costs by stopping a VM when it is not in use.

WARNING! – The only VMs that you should ever touch (such as to stop or terminate) are those that start with the string Datatailr-compute-####. For example, as shown below. Terminating any other type of VM will have a detrimental effect on Datatailr’s functionality. The Datatailr-compute-#### VMs are stateless so that if the app/services running on a specific VM are not performing as expected, then you can terminate and restart them without any detrimental effect.

Monitoring Running VMs

Click the Instance Manager icon. A row is displayed for each VM.

Hovering over the name of the VM displays a variety of information about it. For example, as shown below –

The following columns of information are shown for each VM –

Instance – Shows the hardware specifications of this VM, such as its processor and amount of memory.

Architecture/GPU – Specifies the architecture on which the VM is running and whether it has a GPU.
Environment – IMPORTANT! Specifies the Datatailr environment in which the image is running on the VM (Dev, Pre or Prod), as described in Datatailr Environments.
Status – Specifies whether this VM is Running or Stopped or in another transitional states (such as Terminating).
Accumulated Cost – Specifies the cost of running this VM since it was created according to the cost in the Hourly Price column.

Tip – It is highly recommended to use the Auto Scaling feature to save costs because it will ensure that you are paying for just enough VM resources and not too much.

Tip – Sometimes, you may need to run jobs on a very powerful VM which is very costly. In this case, you may decide not to run this VM all the time. Instead, you may manually activate jobs on it and then terminate this VM when it is not needed in order to save costs. This type of VM should not be set to Auto Scaling.

Note – If you schedule a job manually on a specific VM, the Auto-scaling app never terminates it.

To stop or terminate a VM –

Click on the Instance Manager icon.
Right-click on the relevant VM’s row to display a dropdown menu of options.
Select Stop or Terminate.

To spawn a new instance –

Click the + in the top right corner of the window. The following is displayed –

Click the + button in the top right of this window to display an additional row, as shown below –

In the Type column, select Compute or a GPU machine, as shown above.
Fill out the other rows of this new instance to define the VM and click OK. For example, as shown below –

Datatailr will then automatically support this VM. Datatailr will use this VM before automatically spawning new VMs. This means that Datatailr will only spawn additional VMs if they are required to run the jobs defined in the Job Scheduler.

Right-clicking on an instance displays a dropdown menu of options. For example, select the Jobs option to see which jobs are actually running.

Tip – The Jobs option enables you to see which jobs are running on which VMs in order to determine whether meaningful jobs are running on each. The Jobs option is not available if no jobs are running on the VM, meaning that the VM is idle.

Cost Savings Tip – Datatailr’s Auto Scaling feature dynamically allocates just enough compute when it is required. It closely monitors compute demand and allocates additional VMs as needed. As soon as commute demand is no longer needed, Datatailr automatically shuts down the relevant VMs to save costs. You can mark certain VMs as exempt from Auto scaling. Datatailr does not terminate VMs that were scheduled manually or VMs that have been configured by their owners not to be terminated.

Cost Savings Tip – Auto-scaling can result in a large job being assigned a costly, large VM (because it needs it), and a smaller job(s) that was defined as exempt from termination being manually scheduled on that same large VM, thus preventing automatic termination of the large VM. In such cases, you should manually terminate the large VM. Datatailr will then automatically and immediately relocate these smaller job(s) to other VMs.