Image Builder

The Image Builder enables you to build container images that can be used by other developers as is or which they can use as a basis for creating a customized environment by adding (or removing) packages. A container image is a Docker image with all Python and OS packages that were included into it at build time.

Note – Existing images can be managed in the Image Manager.

🚧
Note – The Image Builder app will be deprecated soon. Instead, it is recommended to use Image Builder VSCode extension in Datatailr IDE. To access the extension, click on the Datatailr icon in the left ribbon of your IDE and then click on "Datatailr Image Builder". The instructions on this page are also relevant for the extension.

Click the Image Builder icon. The loading window will be displayed – it may take a couple of minutes for the Image Builder to load if a new VM has to be provisioned.

Once the Image Builder loads, the following is displayed –

Base Config Tab

In the Base Config tab, in the Base Image area in the top left corner, in the Build Type field, select New Build to build a new image. Alternatively, you can select Build from Image to select a previously created image to create a new image. The list of images from which to select is displayed in the bottom left corner of this window.

In the Image OS field, select Fedora or Alpine or leave the default value.
- Fedora is a free and open-source Linux operating system developed by the Fedora Project and sponsored by Red Hat.
- Alpine Linux is a lightweight, security-oriented, and resource-efficient Linux distribution based on musl libc and busybox.
  Tip – Fedora is the more generic image in which a wider variety of packages operate properly out-of-the-box. You might want to use this if your image has a wider set of dependencies. Alpine, on the other hand, produces substantially smaller images that have a quicker startup time.
In the Architecture field, select amd64 or arm64 or leave the default – amd64.
- amd64 architecture is a 64-bit microprocessor architecture developed by Advanced Micro Devices (AMD), which is commonly used in most modern personal computers and servers.
- arm64 architecture is a 64-bit variation of the Advanced RISC Machine (ARM) architecture, which is commonly used in devices that have low power consumption.
  Tip – Using arm64 is typically less expensive. amd64 is more general-purpose and hence more costly.
  Tip – Alpine produces more compact and lightweight images that are quick to start up, but less flexible because some packages might not support this type of the architecture out-of-the-box. The light and fast way is to run Alpine and arm64 in some cases, but it might require more optimization efforts.
In the Versions area in the center of the window, in the Compiler field, typically leave the default value unless you have specific compiler requirements.
You can mark the Install Compiler checkbox to install the compiler into the image, which later allows you expert options. Only use this expert option if you’re familiar with it.
In the Python field, select the version of the Python, Julia or Rust interpreter of the image. The available versions to choose from are determined by the permissions set by the Datatailr admin.
Tip – If you need a version that is not listed, check with your Datatailr admin before contacting support because your Datatailr admin may have restricted it.

Python Packages Tab

Select the Python Packages tab or click the Next button in the bottom right corner. The following is displayed –

The Python Packages area on the left of the window lists the ~380,000 Python external packages from pypi.org.

Note – PyPI (Python Package Index) is a repository for open-source Python software, providing an easy way to install and manage packages for use in Python projects – https://pypi.org/.

The Julia Packages tab and the Rust Packages tab operate in the same manner as the Python packages tab which is described on the following pages.

In the Python Packages area, check the checkbox on the left of each Python package to include it in the image to be used by developers in Datatailr. You can type any string in the search box to filter the list, such as numpy, as shown below. The Python package that is an exact match appears first in the list.

Tip – Mark the Show selected checkbox to filter the list to only show those that you have selected.

Note – You can always add additional Python packages to the image later, if needed.

[Optional] For each Python package that you selected (as described above), in the Version column of that row, you can select the Python package version. If you do not select a version, then a default version is selected for you.
Tip – We recommend that you do select the version in order to ensure that you’re using the latest Python package version that is suitable for you.
Tip – You can mark the Change constraints option at the top of the list to change the constraint that applies to all the selected Python packages in the list.

Each Python package is listed in the Selected Packages area on the right, as shown above.

Each of these Python packages may include multiple files in the version. The versions of these may have interdependencies.
If needed, you can use the Constraints column to impose conditions on the Python packages versions and their dependencies to be included in the image. See Python Package Version Constraints in Step 2 – Defining a Default IDE Imageof your options for defining the version constraints. Alternatively, you can leave the caret (^) symbol, which is the default.

The Internal Python Packages area in the center of the window lists all the Python packages that either you have created or your colleagues have created, and you’re permitted to use. Typically you will choose a runnable that you developed in order to build a container image for it. For example simple-app, as shown below –

You can choose specific versions using the Version and Constraint columns in the same way as they are used in the Python packages area, as described on the previous pages.

The Selected Packages area on the right lists all the packages that you have selected to be installed in the image.
Later, when this image is built as described in the next steps, it will evaluate the dependencies required by the app you selected to build a container image and will install those too.
If you selected the New Build option in the Base Config tab, then at this point, these dependencies are not listed here.
If you selected the Build from Image option in the Base Config tab, then these dependencies will be listed in the Selected Packages area even though you did not select them, because they were included in the image on which this image was based.
For example, the following shows various packages in the Selected Packages area even though only the simple-app option is selected in the Internal Python Packages area –

OS/C++ Packages Tab

Select the OS/C++ Packages tab, which enables you to define operating system packages to be included in the image. The following is displayed –

The OS packages area on the left lists the OS packages you can select.

Mark the checkboxes to select packages that appear in the Selected Packages column.

For example, Python might not automatically detect that it requires a postgres OS package. In this case, you can type postgres into the Search field, as shown below –

Finding Missing Packages

If an operating system library is not installed in the image, then during the build process, an error message is displayed specifying the missing library.

This error message cannot specify which package contains the missing library. Typically (without Datatailr), a developer would have to search on the Internet to try and figure out which package contains this library. However, with Datatailr, it’s much easier!

Simply, come back to this window and type in that library name into the OS Packages Search field, which will intelligently search inside the packages for the relevant library and then will list the packages that contain this library.

For example, you can type libstdc++.so into the Search field, as shown below –

It’s optional, but if you want to verify that this package contains the relevant library, you can right-click on one of those listed packages and select the Files option from the dropdown menu to see the list of libraries that this package contains. For example, as shown below –

Note – At Datatailr, we have made every effort to provide you with these tools to help you find the packages that contain the libraries you need yourself or with the help of your colleagues.

Build Tab

Select the Build tab. The following is displayed, summarizing all the selections –

[Optional] You can also click the Extra Build Script button to display the following window in which you can append any command to a Docker build script. It is not recommended to enter any passwords here.

Click the Build button to build a container image for the platform that was selected (amd64 or arm64). Alternatively, you can select the Build (amd64 or arm64) button, which builds the image for both platforms. A variety of messages are displayed while Datatailr builds the image, along with a success message at the end.
After the image has been built successfully, the Save button is enabled so that you can click it to save the image.

Click the Save button. The following window is displayed –

Assign the image a name and description, specify the Datatailr users and groups that are allowed to view/modify it and click OK.
Image Builder will push the image into a Container Registry and display a message with results.

Managing Images

This new image is then displayed as a row in the Image Manager. Double-click on it to display additional information or right-click to get access to a context menu.

Note – You can attach an IDE to any image by right-clicking on it in the Image Manager and selecting Launch IDE. You can also configure any image as a default IDE image from the same context menu.

Note – See Changing IDE Settings to learn more about setting a default image for the IDE.