Image Builder

The Image Builder enables you to build an instance image that can be used by data developers as is or which they can use as a basis for creating a customized environment by adding (or removing) packages. A container image contains the Docker images of the environments (Python packages containing all of the files, libraries, and settings) for running the Datatailr IDE.

If modifications are made to a container image that is being used by developers, then those development environments are only affected after they terminate the Datatailr IDE and restart it.

You can create a new version of a container image and then assign it to developers as needed.

Note – Images can be managed in the ImageManager.

Note – Anyone who belongs to the developer group can access the Image Manager.

Click the Image Builder icon. The following may be displayed –

The following is then displayed –

Note – On the first day of using Datatailr, a Datatailr admin must create a default image (package environment containing all of the files, libraries, and settings) to be used by developers so that they can start working in the DatatailrIDE. See Step 2 – Defining a Default IDE Image for details.

To define a container image –

Base Config Tab

1. In the Base Config tab, in the Base Image area in the top left corner, in the Build Type field, select New Build to build a new image. Alternatively, you can select Build from Image to select a previously created image to create a new image. The list of images from which to select is displayed in the bottom left corner of this window.

2. In the Image OS field, select Fedora or Alpine or leave the default value.

• Fedora is a free and open-source Linux operating system developed by the Fedora Project and sponsored by Red Hat.
• Alpine Linux is a lightweight, security-oriented, and resource-efficient Linux distribution based on musl libc and busybox.

Tip – Fedora is the more generic image in which a wider variety of packages operate properly out-of-the-box. You might want to use this if your image has a wider set of dependencies. Alpine produces substantially smaller images that have a quicker startup time.

3. In the Architecture field, select amd64 or arm64 or leave the default – amd64.

• amd64 architecture is a 64-bit microprocessor architecture developed by Advanced Micro Devices (AMD), which is commonly used in most modern personal computers and servers.
• arm64 architecture is a 64-bit variation of the Advanced RISC Machine (ARM) architecture, which is commonly used in devices that have low power consumption.

Tip – Using arm64 is typically less expensive. amd64 is more general-purpose and more costly.

Tip – Alpine produces more compact and lightweight images that are quick to start up, but less flexible because some packages might not support this type of the architecture out-of-the-box. The light and fast way is to run Alpine and arm64 in some cases, but it might require more optimization efforts.

4. In the Versions area in the center of the window, in the Compiler field, typically leave the default value unless you have specific compiler requirements.
5. You can mark the Install Compiler checkbox to install the compiler into the image, which later allows you expert options. Only use this expert option if you’re familiar with it.
6. In the Python field, select the version of the Python, Julia or Rust interpreter of the image. The available versions to choose from are determined by the permissions set by the Datatailr admin.

Tip – If you need a version that is not listed, check with your Datatailr admin before contacting support because your Datatailr admin may have restricted it.

The selection of Python packages available to you may have been restricted by the Block/Allow list defined by the admin. Contact your local Datatailr admin if you require access to a Python package that you do not see in the list.

Python Packages Tab

1. Select the Python Packages tab or click the Next button in the bottom right corner. The following is displayed –

The Python Packages area on the left of the window lists the ~380,000 Python external packages from pypi.org.

Note – PyPI (Python Package Index) is a repository for open-source Python software, providing an easy way to install and manage packages for use in Python projects – https://pypi.org/.

The Julia Packages tab and the Rust Packages tab operate in the same manner as the Python packages tab which is described on the following pages.

2. In the Python Packages area, check the checkbox on the left of each Python package to include it in the image to be used by developers in Datatailr. You can type any string in the search box to filter the list, such as numpy, as shown below. The Python package that is an exact match appears first in the list.

Tip – Mark the Show selected checkbox to filter the list to only show those that you have selected.

Note – Later, you can add additional Python packages to the image if needed.

3. [Optional] For each Python package that you selected (as described above), in the Version column of that row, you can select the Python package version. If you do not select a version, then a default version is selected for you.

Tip – We recommend that you do select the version in order to ensure that you’re using the latest Python package version that is suitable for you.

Tip – You can mark the Change constraints option at the top of the list to change the constraint that applies to all the selected Python packages in the list.

Each Python package is listed in the Selected Packages area on the right, as shown above.

4. Each of these Python packages may include multiple files in the version. The versions of these may have interdependencies.

If needed, you can use the Constraints column to impose conditions on the Python packages versions and their dependencies to be included in the image. See Python Package Version Constraints for full description of your options for defining the version constraints. Alternatively, you can leave the caret (^) symbol, which is the default.

5. The Internal Python Packages area in the center of the window lists all the Python packages that either you have created or your colleagues have created, and you’re permitted to use. Typically you will choose a runnable that you developed in order to build a container image for it, such as Hello_World. For example, as shown below –

You can choose specific versions using the Version and Constraint columns in the same way as they are used in the Python packages area, as described on the previous pages.

6. The Selected Packages area on the right lists all the packages that you have selected to be installed in the image.

Later, when this image is built as described in the next steps, it will evaluate the dependencies required by the app you selected to build a container image and will install those too.

If you selected the New Build option in the Base Config tab, then at this point, these dependencies are not listed here.

If you selected the Build from Image option in the Base Config tab, then these dependencies will be listed in the Selected Packages area even though you did not select them, because they were included in the image on which this image was based.

For example, the following shows various packages in the Selected Packages area even though only the simple-app option is selected in the Internal Python Packages area –

7. Select the OS/C++ Packages tab, which enables you to define operating system packages to be included in the image. The following is displayed –

The OS packages area on the left lists the OS packages you can select.

Mark the checkboxes to select packages that appear in the Selected Packages column.

For example, Python might not automatically detect that it requires a postgres OS package. In this case, you can type postgres into the Search field, as shown below –

Finding Missing Packages

If an operating system library is not installed in the image, then during the build process, an error message is displayed specifying the missing library.

This error message cannot specify which package contains the missing library.

Typically (without Datatailr), a developer would have to search on the Internet to try and figure out which package contains this library. However, with Datatailr, it’s much easier!

Simply, come back to this window and type in that library name into the OS Packages Search field, which will intelligently search inside the packages for the relevant library and then will list the packages that contain this library.

For example, you can type libstdc++.so into the Search field, as shown below –

It’s optional, but if you want to verify that this package contains the relevant library, you can right-click on one of those listed packages and select the Files option from the dropdown menu to see the list of libraries that this package contains. For example, as shown below –

Note – At Datatailr, we have made every effort to provide you with these tools to help you find the packages that contain the libraries you need yourself or with the help of your colleagues.

8. Select the Build tab. The following is displayed, summarizing all the selections –

9. [Optional] You can also click the Extra Build Script button to display the following window in which you can insert any Docker build command. It is not recommended to enter any passwords here.

10. Click the Build button to build a container image for the platform that was selected (amd64 or arm64). Alternatively, you can select the Build (amd64 or arm64) button, which builds the image for both platforms. A variety of messages are displayed while Datatailr builds the image, along with a success message at the end.

11. After the image has been built successfully, the Save button is enabled so that you can click it to save the image.

12. Click the Save button. The following window is displayed –

13. Assign the image a name and description, specify the Datatailr users and groups that are allowed to run it and click OK.

This new image is then displayed as a row in the Image Manager. Click on it to display additional options.

14. Select the Copy to option and then select pre to copy this image to the pre-testing environment, as shown above.

Note – The Pre environment may also be referred to as the user acceptance testing (UAT) environment.

Note – This image can be launched in the IDE, by right clicking on it in the Image Manager and selecting Settings, which is not relevant for now.

15. A container image at this stage can be launched in the Datatailr IDE by any developer from the Image Manager for testing by selecting the Launch IDE option shown below.

16. Select the Copy to option again and then select prod to copy this image to the production environment. This means that this image is now available for selection when a developer right‑clicks on the Datatailr IDE icon and selects Settings shown below –

Note – A container image must first be assigned the Datatailr pre environment tag, before it can be assigned the Datatailr prod environment tag, from where it can be made available for developers.

The IDE Settings window is displayed in which this image can be selected from the Image dropdown menu, as shown below –

This means that the Datatailr IDE is ready for developers to launch it and start developing Datatailr runnables. For more information about the IDE, see Datatailr IDE.