Full API Reference

Datatailr - Python SDK

The datatailr package provides a Python SDK for interacting with the Datatailr platform.

For a quickstart guide please launch the default workstation on your installation and open the README file.

Datatailr provides a simple interface for scheduling and managing data tasks, handling user access control, working with data blobs and secrets, and more.

App

Deploy a web application or dashboard to Datatailr.

This supports Streamlit, Dash, or any framework that can be started via a standard entrypoint (for example, streamlit run app.py).

Parameters:

Name	Type	Description	Default
name	str	App display name.	required
entrypoint	Callable | str	Callable that starts the app server.	required
environment	Optional[Environment]	Target environment for the job.	DEV
image	Optional[Image]	Container image to use for execution.	None
run_as	Optional[Union[str, User]]	User to execute the job as.	None
resources	Resources	Resource requirements for the job.	Resources()
acl	Optional[ACL]	Access control settings for the job.	None
framework	Optional[str]	Framework to use for the app.	None
python_version	str	Python version to use.	'3.12'
python_requirements	str | List[str]	Requirements to install (string or list).	''
build_script_pre	str	Shell script to run before build.	''
build_script_post	str	Shell script to run after build.	''
env_vars	Optional[Dict[str, str | int | float | bool]]	Environment variables to set.	None
update_existing	bool	Update an existing job with the same name.	False

Examples:

Minimal Streamlit app.

# app.py
import streamlit as st

def main():
    st.title("Hello Datatailr App")

if __name__ == "__main__":
    main()

Test locally.

streamlit run app.py

Deploy to Datatailr.

from app import main
from datatailr import App

app = App(
    name="Simple Dashboard App",
    entrypoint=main,
    python_requirements="streamlit")

app.run()

__init__(name, entrypoint, environment=Environment.DEV, image=None, run_as=None, resources=Resources(), acl=None, framework=None, python_version='3.12', python_requirements='', build_script_pre='', build_script_post='', env_vars=None, update_existing=False, app_section='')

Initialize an App deployment.

Parameters:

Name	Type	Description	Default
name	str	Display name for the app.	required
entrypoint	Callable | str	The callable (function) that starts the application.	required
environment	Optional[Environment]	Target environment for the deployment.	DEV
image	Optional[Image]	Pre-configured container Image. When None, an image is built from python_version, python_requirements, and the build scripts.	None
run_as	Optional[Union[str, User]]	User or username to run the app as. Defaults to the currently signed-in user.	None
resources	Resources	CPU and memory resources for the container.	Resources()
acl	Optional[ACL]	Access control list. Defaults to standard permissions for the current user.	None
framework	Optional[str]	Optional framework name (e.g., 'streamlit', 'dash', 'flask'). If not provided, the framework is inferred from the entrypoint.	None
python_version	str	Python version for the container image.	'3.12'
python_requirements	str | List[str]	Python dependencies (see Image).	''
build_script_pre	str	Dockerfile commands to run before installing requirements.	''
build_script_post	str	Dockerfile commands to run after installing requirements.	''
env_vars	Optional[Dict[str, str | int | float | bool]]	Environment variables passed to the running container.	None
update_existing	bool	If True, load and update an existing job definition with the same name instead of creating a new one.	False
app_section	str	The section to which the app belongs. If not provided, the app will be assigned to the default section. This affects the app launcher page.	''

Blob

Interface for Datatailr's binary large object (blob) storage.

Blob storage lets you persist and retrieve arbitrary files and binary data. Each blob is identified by a unique name (key/path).

Notes

This class wraps Datatailr blob operations (list/copy/read/write/delete). Large reads/writes use temporary files internally.

Examples:

Basic file copy to/from blob storage:

>>> from datatailr import Blob
>>> b = Blob()
>>> _ = open("/tmp/data.txt", "wb").write(b"qwerty123")
>>> b.put_file("mybucket/data.txt", "/tmp/data.txt")
>>> b.get_file("mybucket/data.txt", "/tmp/data_copy.txt")

Read/write blob contents as bytes:

>>> payload = b"hello world"
>>> b.put("mybucket/hello.bin", payload)
>>> data = b.get("mybucket/hello.bin")
>>> data == payload
True

List and delete:

>>> sorted(b.ls("mybucket/"), key=lambda x: x["name"])
[{'is_file': True, 'last_modified': ..., 'name': 'data.txt', 'size': 9}, {'is_file': True, 'last_modified': ..., 'name': 'hello.bin', 'size': 11}]
>>> b.delete("mybucket/hello.bin")
>>> b.exists("mybucket/hello.bin")
False

acls(path)

Retrieve the access control list (ACL) for a specific blob path.

Parameters:

Name	Type	Description	Default
path	str	The blob path for which to retrieve the ACL.	required

Returns:

Name	Type	Description
ACL	ACL	The ACL for the blob path.

delete(name)

Delete a blob.

Parameters:

Name	Type	Description	Default
name	str	The name of the blob to delete.	required

exists(name)

Check if a blob exists.

Parameters:

Name	Type	Description	Default
name	str	The name of the blob to check.	required

Returns:

Name	Type	Description
bool	bool	True if the blob exists, False otherwise.

get(name)

Get a blob object.

Parameters:

Name	Type	Description	Default
name	str	The name of the blob to retrieve.	required

Returns:

Name	Type	Description
Blob	bytes	The blob object.

get_file(name, path)

Copy a blob file to a local file.

Parameters:

Name	Type	Description	Default
name	str	The name of the blob to retrieve.	required
path	str	The path to store the blob as a file.	required

ls(prefix, recursive=False)

List files in the specified prefix. If recursive is True, list files in subdirectories recursively.

Parameters:

Name	Type	Description	Default
prefix	str	The prefix to list files from.	required
recursive	bool	Whether to list files in subdirectories recursively.	False

Returns:

Name	Type	Description
list	list[str]	A list of file names in the specified path. If recursive is True, the list will contain files in subdirectories recursively.

put(name, blob, acl=None)

Put a blob object into the blob storage.

Parameters:

Name	Type	Description	Default
name	str	The name of the blob to create.	required
blob	bytes | str	The blob object to store.	required
acl	ACL	The access control list for the blob.	None

put_file(name, path, acl=None)

Copy a local file to a blob.

Parameters:

Name	Type	Description	Default
name	str	The name of the blob to create.	required
path	str	The path of the local file to copy.	required
acl	ACL	The access control list for the blob.	None

set_acls(path, acls)

Set the access control list (ACL) for a specific blob path.

Parameters:

Name	Type	Description	Default
path	str	The blob path for which to set the ACL.	required
acls	ACL	The ACL to apply.	required

Environment

Enum representing different environments for Datatailr jobs.

ExcelAddin

Represents an Excel add-in deployment on Datatailr.

An Excel add-in exposes Python functions as Excel worksheet functions, allowing users to call server-side computations directly from Excel spreadsheets.

Example

from datatailr import ExcelAddin
from datatailr.excel import Addin

addin_def = Addin("Options Pricer", "Option pricing functions")

@addin_def.expose(description="Black-Scholes price")
def price_option(spot, strike, vol, rate, expiry):
    ...

def __excel_main__(port=8080, ws_port=8000):
    addin_def.run(port, ws_port)

addin = ExcelAddin(
    name="Options Pricer",
    entrypoint=__excel_main__,
    python_requirements=["numpy", "scipy"],
)
addin.run()

__init__(name, entrypoint, environment=Environment.DEV, image=None, run_as=None, resources=Resources(), acl=None, python_version='3.12', python_requirements='', build_script_pre='', build_script_post='', env_vars=None, update_existing=False, app_section='')

Initialize an Excel add-in deployment.

Parameters:

Name	Type	Description	Default
name	str	Display name for the add-in.	required
entrypoint	Callable	The callable (function) that starts the add-in server.	required
environment	Optional[Environment]	Target environment for the deployment.	DEV
image	Optional[Image]	Pre-configured container Image.	None
run_as	Optional[Union[str, User]]	User or username to run the add-in as.	None
resources	Resources	CPU and memory resources for the container.	Resources()
acl	Optional[ACL]	Access control list.	None
python_version	str	Python version for the container image.	'3.12'
python_requirements	str | List[str]	Python dependencies (see Image).	''
build_script_pre	str	Dockerfile commands to run before installing requirements.	''
build_script_post	str	Dockerfile commands to run after installing requirements.	''
env_vars	Optional[Dict[str, str | int | float | bool]]	Environment variables passed to the running container.	None
update_existing	bool	If True, update an existing job definition.	False
app_section	str	The section to which the app belongs. If not provided, the app will be assigned to the default section. This affects the app launcher page.	''

Group

Representing a Datatailr Group.

This class provides methods to interact with the Datatailr Group API. It allows you to create, update, delete, and manage groups within the Datatailr platform.

Attributes:

Name	Type	Description
name	str	The name of the group.
members	list	A list of members in the group.
group_id	int	The unique identifier for the group.

group_id property

The unique numeric identifier of the group.

members property

List of user IDs that are members of this group.

name property

The name of the group.

__init__(name)

Create a Group instance by fetching group data from the platform.

Parameters:

Name	Type	Description	Default
name	str	The name of an existing group.	required

Raises:

Type	Description
ValueError	If a group with the given name does not exist.

add(name) staticmethod

Create a new group with the specified name.

Parameters:

Name	Type	Description	Default
name	str	The name for the new group.	required

Returns:

Type	Description
Optional[Group]	The newly created Group, or None if the operation failed.

add_users(usernames)

Add users to the group.

Parameters:

Name	Type	Description	Default
usernames	list	A list of usernames to add.	required

Raises:

Type	Description
ValueError	If the group name is not set.

exists(name) staticmethod

Check if a group exists by its name.

Parameters:

Name	Type	Description	Default
name	str	The group name to look up.	required

Returns:

Type	Description
bool	True if the group exists, False otherwise.

from_dict(data) classmethod

Construct a Group instance from a dictionary.

Parameters:

Name	Type	Description	Default
data	dict	A dictionary with keys "name", "members", and "group_id".	required

Returns:

Type	Description
Group	A Group instance populated from the dictionary.

Raises:

Type	Description
ValueError	If required keys are missing from data.

get(name_or_id) staticmethod

Retrieve an existing group by name or numeric ID.

Parameters:

Name	Type	Description	Default
name_or_id	Union[str, int]	The group name (str) or group ID (int).	required

Returns:

Type	Description
Group	The matching Group instance.

Raises:

Type	Description
ValueError	If no group with the given name or ID exists.

is_member(user)

Check whether a user belongs to this group.

Parameters:

Name	Type	Description	Default
user		A User instance or a user ID (int).	required

Returns:

Type	Description
bool	True if the user is a member, False otherwise.

Raises:

Type	Description
ValueError	If the group name is not set.

ls() staticmethod

List all groups available in the Datatailr platform.

Returns:

Type	Description
list	A list of Group instances.

remove(name) staticmethod

Remove a group by its name.

Parameters:

Name	Type	Description	Default
name	str	The name of the group to remove.	required

remove_users(usernames)

Remove users from the group.

Parameters:

Name	Type	Description	Default
usernames	list	A list of usernames to remove.	required

Raises:

Type	Description
ValueError	If the group name is not set.

Image

Represents a container image for a job.

The image is defined based on Python dependencies and two build scripts expressed as Dockerfile commands.

Parameters:

Name	Type	Description	Default
python_version	str	Python version string or "auto".	'auto'
python_requirements	str | list[str]	Requirements spec, file path, list, or "auto".	''
build_script_pre	str	Pre-build Dockerfile commands or file path.	''
build_script_post	str	Post-build Dockerfile commands or file path.	''
branch_name	str | None	Git branch name for the repo, if applicable.	None
commit_hash	str | None	Git commit hash for the repo, if applicable.	None
path_to_repo	str | None	Path to the repository root, if applicable.	None
path_to_module	str | None	Path to the Python module, if applicable.	None

Attributes:

Name	Type	Description
python_version	str	Resolved Python version string.
python_requirements	str	Requirements string (newline-delimited).
build_script_pre	str	Pre-build Dockerfile commands.
build_script_post	str	Post-build Dockerfile commands.
branch_name	str | None	Git branch name.
commit_hash	str | None	Git commit hash.
path_to_repo	str | None	Path to the repository root.
path_to_module	str | None	Path to the Python module.
build_stdout	str | None	Stdout captured from the last build, if any.
build_stderr	str | None	Stderr captured from the last build, if any.
name	str | None	Built image name, if any.
last_build_succeeded	bool | None	Whether the last build succeeded.
bundle_name	str | None	Bundle name, if any.
registry	str | None	Registry URL or name, if any.
run_after_build	bool | None	Whether to run after build.
tag	str | None	Image tag, if any.

__init__(python_version='auto', python_requirements='', build_script_pre='', build_script_post='', branch_name=None, commit_hash=None, path_to_repo=None, path_to_module=None)

Initialize an Image configuration.

Parameters:

Name	Type	Description	Default
python_version	str	Python version for the container (e.g. "3.12"). Use "auto" to match the current interpreter version.	'auto'
python_requirements	str | list[str]	Python dependencies. Accepts a pip-freeze string, a list of package specifiers, a path to a requirements.txt file, or "auto" to run pip freeze.	''
build_script_pre	str	Dockerfile commands to execute before installing Python requirements. Can be a string or a file path.	''
build_script_post	str	Dockerfile commands to execute after installing Python requirements. Can be a string or a file path.	''
branch_name	Optional[str]	Git branch name (populated automatically during build).	None
commit_hash	Optional[str]	Git commit hash (populated automatically during build).	None
path_to_repo	Optional[str]	Absolute path to the source repository root.	None
path_to_module	Optional[str]	Relative path from the repository root to the Python module containing the entrypoint.	None

from_dict(data)

Create an Image instance from a dictionary representation.

to_dict()

Convert the Image instance to a dictionary representation.

to_json()

Convert the Image instance to a JSON string representation.

update(**kwargs)

Update one or more Image attributes.

Parameters:

Name	Type	Description	Default
**kwargs		Attribute names and their new values.	{}

Raises:

Type	Description
TypeError	If a string-only attribute receives a non-string value.
AttributeError	If an attribute name does not exist on the Image.

KV

A distributed key-value store for storing and retrieving configuration data, application settings, and other key-value pairs.

Note that different environments (dev/pre/prod) have separate key-value stores.

Each key-value pair can have its own access control list (ACL) to manage permissions for reading and writing.

Example:

from datatailr import KV, ACL, User, Group, Permission
kv = KV()
kv.put("db_url", "postgresql://user:password@host:port/dbname")
acls = ACL({Permission.READ: [User.get("user1"), Group.get("group1")],
            Permission.WRITE: [User.get("user2")]})
kv.add_acl("db_url", acls)
db_url = kv.get("db_url")
all_keys = kv.ls()
kv.delete("db_url")

acls(key)

Retrieve the access control list (ACL) for a specific key in the key-value store.

Parameters:

Name	Type	Description	Default
key	str	The key for which to retrieve the ACL.	required

Returns:

Type	Description
ACL	dict[str, list[str]]: A dictionary containing lists of users/groups with 'read' and 'write' permissions.

Example:

from datatailr import KV
kv = KV()
acl = kv.acls("db_url")

add_acl(key, acls)

Add an entry to the access control list (ACL) for a specific key in the key-value store.

Parameters:

Name	Type	Description	Default
key	str	The key for which to add the ACL entry.	required
entity	str	The user or group to which the permission is granted.	required
permission	str	The permission to grant ('read' or 'write').	required

Returns:

Type	Description
None	None

Example:

from datatailr import KV, ACL
kv = KV()
acls = ACL(read=["user1", "group1"], write=["user2"])
kv.add_acl("db_url", acls)

delete(key)

Delete a key-value pair from the key-value store by its key.

Parameters:

Name	Type	Description	Default
key	str	The key of the value to delete.	required

Returns: None Example:

from datatailr import KV
kv = KV()
kv.delete("db_url")  # Deletes the key-value pair with key "db_url"

get(key)

Retrieve a value by its key from the key-value store. Note that different environments (dev/pre/prod) have separate key-value stores.

Parameters:

Name	Type	Description	Default
key	str	The key of the value to retrieve.	required

Returns:

Name	Type	Description
str	str	The value associated with the key.

Example:

from datatailr import KV
kv = KV()
db_url = kv.get("db_url")

ls()

List all keys stored in the key-value store for the current environment (dev/pre/prod).

Returns:

Type	Description
list[dict[str, str | ACL]]	list[dict[str, str|ACL]]: A list of all keys in the key-value store along with their ACLs.

Example:

from datatailr import KV
kv = KV()
all_keys = kv.ls()

put(key, value, acl=None)

Store a key-value pair in the key-value store. Note that different environments (dev/pre/prod) have separate key-value stores.

Parameters:

Name	Type	Description	Default
key	str	The key under which to store the value.	required
value	str	The value to store.	required
acl	ACL	The access control list for the key-value pair.	None

Returns:

Type	Description
None	None

Example:

from datatailr import KV, ACL, Permission, User, Group
kv = KV()
acls = ACL({Permission.READ: [User.get("user1"), Group.get("group1")],
            Permission.WRITE: [User.get("user2")]})
kv.put("db_url", "postgresql://user:password@host:port/dbname", acls)

set_acls(key, acls)

Set the access control list (ACL) for a specific key in the key-value store, replacing any existing ACL entries.

Parameters:

Name	Type	Description	Default
key	str	The key for which to set the ACL.	required
entity	str	The user or group for which to set the permission.	required
permission	str	The permission to set ('read' or 'write').	required

Returns:

Type	Description
None	None

Example:

from datatailr import KV, ACL
kv = KV()
acls = ACL(read=["user1", "group1"], write=["user2"])
kv.set_acls("db_url", acls)

Permission

Enumeration of permission types used in Access Control Lists.

Each permission controls a different aspect of access to a Datatailr resource.

Attributes:

Name	Type	Description
READ		Allows reading or viewing the resource.
WRITE		Allows modifying or updating the resource.
OPERATE		Allows executing or operating the resource (e.g., running a job).
ACCESS		Allows accessing the resource (e.g., opening an app).
PROMOTE		Allows promoting the resource to the next environment.

Resources dataclass

Represents the compute resources allocated to a job container.

Attributes:

Name	Type	Description
memory	str	Memory limit as a string (e.g. "256m", "1g").
cpu	float	Number of CPU cores to allocate (e.g. 1, 0.5).

Schedule

Build a schedule object for batch/workflow jobs using friendly fields.

Parameters:

Name	Type	Description	Default
cron_expression	str	Raw cron expression to use (string).	''
at_minutes	list[int] | None	Specific minutes within the hour, e.g. [0, 30].	None
every_minute	int | None	Run every N minutes.	None
at_hours	list[int] | None	Specific hours within the day, e.g. [0, 12].	None
every_hour	int | None	Run every N hours.	None
weekdays	list[str] | None	Weekdays by name, e.g. ["Mon", "Wed", "Fri"].	None
day_of_month	int | None	Day of the month (1-31).	None
in_month	list[str] | None	Months by name, e.g. ["Jan", "Jul"].	None
every_month	int | None	Run every N months.	None
timezone	str | None	Time zone name, e.g. "UTC".	None
run_after_job_uuid	str | None	Job UUID to run after.	None
run_after_job_name	str | None	Job name to run after.	None
run_after_job_condition	str | None	Condition for dependency, e.g. "on failure".	None

Examples:

schedule = Schedule(at_hours=[0])
schedule = Schedule(at_minutes=[0, 30], weekdays=["Mon", "Wed", "Fri"])

__init__(cron_expression='', at_minutes=None, every_minute=None, at_hours=None, every_hour=None, weekdays=None, day_of_month=None, in_month=None, every_month=None, timezone=None, run_after_job_uuid=None, run_after_job_name=None, run_after_job_condition=None)

Initialize a Schedule.

You can either provide a raw cron_expression or use the human-readable helper parameters (which are compiled into cron syntax automatically).

Parameters:

Name	Type	Description	Default
cron_expression	str	A raw cron string (e.g. "0 */2 * * *"). If provided alongside helper parameters, the helpers take precedence.	''
at_minutes	list[int] | None	Specific minutes past the hour to run (e.g. [0, 30] for ":00" and ":30").	None
every_minute	int | None	Run every N minutes (e.g. 5 for every 5 minutes).	None
at_hours	list[int] | None	Specific hours of the day to run (0 -- 23).	None
every_hour	int | None	Run every N hours.	None
weekdays	list[str] | None	Days of the week to run (e.g. ["mon", "wed", "fri"]).	None
day_of_month	int | None	Day of the month to run (1 -- 31).	None
in_month	list[str] | None	Months to run in (e.g. ["jan", "apr", "jul", "oct"]).	None
every_month	int | None	Run every N months.	None
timezone	str | None	IANA timezone name (e.g. "America/New_York").	None
run_after_job_uuid	str | None	UUID of a job that must complete before this schedule triggers.	None
run_after_job_name	str | None	Name of a job that must complete before this schedule triggers.	None
run_after_job_condition	str | None	Required completion status of the predecessor job (e.g. "success").	None

Example

from datatailr import Schedule
# Every weekday at 08:00 and 16:00 UTC
s = Schedule(at_hours=[8, 16], weekdays=["mon","tue","wed","thu","fri"])

get_cron_string()

Return the compiled cron string.

Returns:

Type	Description
str	Cron string (str).

Examples:

>>> Schedule(
...     at_minutes=[0, 15, 30, 45],
...     at_hours=[0, 12],
...     weekdays=["Mon", "Wed", "Fri"],
...     day_of_month=15,
...     in_month=["Jan", "Jul"],
... ).get_cron_string()
'0 0,15,30,45 0,12 15 1,7 1,3,5'

Secrets

Interface for managing secrets stored in the Datatailr platform.

Secrets are sensitive key-value pairs (passwords, API keys, tokens, etc.) that are stored securely and scoped to the current environment (dev/pre/prod). Access is controlled by per-secret ACLs.

Example

from datatailr import Secrets
secrets = Secrets()
db_password = secrets.get("database_password")
all_secrets = secrets.ls()

get(key)

Retrieve a secret value by its key. Only secrets that are available at the current environment (dev/pre/prod) can be accessed. The user must have the appropriate permissions to access the secret.

Parameters:

Name	Type	Description	Default
key	str	The key of the secret to retrieve.	required

Returns:

Name	Type	Description
str	str	The value of the secret.

Example:

from datatailr import Secrets
secrets = Secrets()
db_password = secrets.get("database_password")

ls()

List all available secret keys in the current environment (dev/pre/prod). The user must have the appropriate permissions to list the secrets.

Returns:

Type	Description
list[dict[str, str | ACL]]	list[dict[str, str|ACL]]: A list of all secret keys along with their ACLs.

Example:

from datatailr import Secrets
secrets = Secrets()
all_secrets = secrets.ls()

Service

Represents a long-running background service deployment on Datatailr.

A service runs continuously (e.g., an API server, a message consumer, or any always-on process). It is restarted automatically if it exits.

Example

from datatailr import Service

# service.py
from flask import Flask

app = Flask(__name__)

@app.route("/health")
def health_check():
    return "OK"

# Service entrypoints receive the port from Datatailr.
def run_server(port):
    app.run("0.0.0.0", port=int(port), debug=False)

svc = Service(
    name="Simple Service",
    entrypoint=run_server,
    python_requirements=["flask"],
)
svc.run()

__init__(name, entrypoint, environment=Environment.DEV, image=None, run_as=None, resources=Resources(), acl=None, python_version='3.12', python_requirements='', build_script_pre='', build_script_post='', env_vars=None, update_existing=False)

Initialize a Service deployment.

Parameters:

Name	Type	Description	Default
name	str	Display name for the service.	required
entrypoint	Callable	The callable (function) that starts the service.	required
environment	Optional[Environment]	Target environment for the deployment.	DEV
image	Optional[Image]	Pre-configured container Image.	None
run_as	Optional[Union[str, User]]	User or username to run the service as.	None
resources	Resources	CPU and memory resources for the container.	Resources()
acl	Optional[ACL]	Access control list.	None
python_version	str	Python version for the container image.	'3.12'
python_requirements	str | List[str]	Python dependencies (see Image).	''
build_script_pre	str	Dockerfile commands to run before installing requirements.	''
build_script_post	str	Dockerfile commands to run after installing requirements.	''
env_vars	Optional[Dict[str, str | int | float | bool]]	Environment variables passed to the running container.	None
update_existing	bool	If True, update an existing job definition.	False

Task

Represents a job within a batch job.

This class can be extended to define specific configurations for each job in the batch.

args property writable

Returns the arguments for the Task instance.

id property

Returns the unique identifier of the Task instance.

__call__(*args, **kwds)

Allows the Task instance to be called like a function, returning itself. This is useful for chaining or functional-style programming.

alias(name)

Set an alias for the Task instance.

:param name: The alias name to set.

run()

Execute the job's entrypoint.

set_resources(resources=None, memory=None, cpu=None)

Set the resources for the Task instance.

:param resources: The Resources instance to set.

to_dict()

Convert the Task instance to a dictionary representation.

to_json()

Convert the Task instance to a JSON string representation.

translate_dependencies()

Translate the dependencies of the Task instance into a format suitable for the batch job.

update_env_vars(env_vars)

Update the environment variables for the Task instance.

:param env_vars: A dictionary of environment variables to update.

User

Representing a Datatailr User.

This class provides methods to interact with the Datatailr User API. It allows you to create, update, delete, and manage users within the Datatailr platform.

Attributes:

Name	Type	Description
first_name	str	The first name of the user.
last_name	str	The last name of the user.
name	str	The username of the user.
email	str	The email address of the user.
user_id	int	The unique identifier for the user.
groups	list	List of groups the user belongs to.
is_system_user	bool	Indicates if the user is a system user.

email property

The email address of the user.

first_name property

The first name of the user.

groups property

List of group names the user belongs to.

is_system_user property

Whether this user is a system (non-interactive) user.

last_name property

The last name of the user.

name property

The username.

user_id property

The unique numeric identifier of the user.

__init__(name=None, id=None)

Create a User instance by fetching user data from the platform.

At least one of name or id must be provided. When both are given, name takes precedence.

Parameters:

Name	Type	Description	Default
name	str | None	The username to look up.	None
id	int | None	The numeric user ID to look up.	None

Raises:

Type	Description
ValueError	If neither name nor id is provided.

add(name, first_name, last_name, email, password, groups=None, is_system_user=False) staticmethod

Create a new user on the Datatailr platform.

Parameters:

Name	Type	Description	Default
name	str	The username for the new user.	required
first_name	str	The user's first name.	required
last_name	str	The user's last name.	required
email	str	The user's email address.	required
password	str	The login password. Ignored for system users.	required
groups	list[str] | None	Optional list of group names to add the user to.	None
is_system_user	bool	If True, create a non-interactive system user.	False

Returns:

Type	Description
Optional['User']	The newly created User, or None if the operation failed.

Raises:

Type	Description
Warning	If a password is provided for a system user.

Notes

Passwords are ignored for system users.

exists(name) staticmethod

Check whether a user with the given username exists.

Parameters:

Name	Type	Description	Default
name	str	The username to look up.	required

Returns:

Type	Description
bool	True if the user exists, False otherwise.

get(name_or_id) staticmethod

Retrieve an existing user by username or numeric ID.

Parameters:

Name	Type	Description	Default
name_or_id	str | int	The username (str) or user ID (int).	required

Returns:

Type	Description
User	The matching User instance.

Raises:

Type	Description
ValueError	If no user with the given name or ID exists.

ls() staticmethod

List all users available in the Datatailr platform.

Returns:

Type	Description
list	list[User]: Users available in the platform.

Examples:

>>> from datatailr import User
>>> User.ls()
[...User(name=root, first_name=Super User, last_name=, email=None, user_id=0, groups=[], is_system_user=True), ...]

remove(name) staticmethod

Remove (delete) a user from the platform.

Parameters:

Name	Type	Description	Default
name	str	The username of the user to remove.	required

signed_user() staticmethod

Retrieve the currently signed-in (authenticated) user.

Returns:

Type	Description
User	The User instance of the currently authenticated user.

Raises:

Type	Description
PermissionError	If no signed-in user is found.

to_cookie()

Return a cookie string for the user.

verify()

Verify the user's authentication signature with the platform.

This refreshes and validates the user's credentials against the Datatailr authentication service.

Workflow

Represents a workflow in the scheduler.

Inherits from Job and is used to define workflows with specific configurations.

next_job_id property

Returns a generator for the next task ID in the workflow.

add_job(job)

Adds a task to the workflow.

:param job: The Task instance to add.

get_env_vars_copy()

Returns a copy of the environment variables for the Workflow instance.

rerun(run_id, version=None, full=False)

Rerun a run id with specific version (latest version by default). If full is true then all the tasks will be rerun, including the ones that already ran successfully.

:param run_id: The run id to rerun. :param version: The version to rerun. If None, it will rerun the latest version. :param full: If True, it will rerun the full workflow, including the ones that already ran successfully. :return: A tuple of (success: bool, message: str).

reset()

Reset the Workflow instance to its initial state.

set_local_run(local_run)

Set the local run flag for the Workflow instance.

:param local_run: A boolean indicating whether to run locally.

to_dict()

Convert the Workflow instance to a dictionary representation.

to_json()

Convert the Workflow instance to a JSON string representation.

construct_remote_base_url(service_name, environment=None)

Construct the base URL of the remote Datatailr instance for a given service name.

Parameters:

Name	Type	Description	Default
service_name	str	The name of the service.	required
environment	Optional[str]	The environment of the service.	None

Returns:

Type	Description
str	The base URL of the remote Datatailr instance for the given service name.

Raises:

Type	Description
RuntimeError	If not logged in or config is invalid.

get_dt_env()

Get the current environment.

get_remote_base_url()

Return the base URL of the remote Datatailr instance.

Raises:

Type	Description
RuntimeError	If not logged in or config is invalid.

get_remote_http_headers()

Return default HTTP headers for calling the Datatailr API as the remote CLI does (Cookie, Accept, User-Agent).

Raises:

Type	Description
RuntimeError	If not logged in or config is invalid.

get_remote_oidc_cookie_line()

Return a single X-Datatailr-Oidc-Data=<jwt> string (same line as datatailr export-auth prints when run without --shell or --fish).

Raises:

Type	Description
RuntimeError	If not logged in or config is invalid.

get_remote_oidc_cookie_pair()

Return (cookie_name, jwt) for building custom headers or cookies.

Raises:

Type	Description
RuntimeError	If not logged in or config is invalid.

get_remote_oidc_jwt()

Return the OIDC JWT cookie value used for authenticated HTTP requests.

Raises:

Type	Description
RuntimeError	If not logged in or config is invalid.

is_dt_installed()

Check if the native Datatailr CLI is available on PATH.

Returns:

Type	Description
bool	True if the resolved dt binary is /opt/datatailr/bin/dt; otherwise False.

Examples:

>>> is_dt_installed()
True

is_remote_logged_in(validate_server=True, timeout_s=5)

Return whether the remote CLI has a usable authenticated session.

Parameters:

Name	Type	Description	Default
validate_server	bool	When true, also call the API to confirm the token is still valid. When false, only check that local config exists and looks valid.	True
timeout_s	int	Timeout for server validation requests.	5

load_remote_client_config()

Load remote client configuration from disk.

Returns:

Type	Description
RemoteClientConfig	Validated config with base_url and jwt_cookie.

Raises:

Type	Description
RuntimeError	If the config file is missing or invalid (run datatailr login).

task(memory=DEFAULT_TASK_MEMORY, cpu=DEFAULT_TASK_CPU)

Decorator that marks a function as a workflow task (batch job).

Use this via the public alias @task() to declare individual computational steps inside a @workflow-decorated function.

Parameters:

Name	Type	Description	Default
memory	str	Memory limit for this task's container (e.g. "256m", "1g").	DEFAULT_TASK_MEMORY
cpu	float	Number of CPU cores to allocate for this task (e.g. 1, 0.5).	DEFAULT_TASK_CPU

Returns:

Type	Description
	A decorator that wraps the target function so it can participate
	in DAG-based workflow orchestration.

Example

from datatailr import task

@task(memory="512m", cpu=2)
def heavy_computation(x, y):
    return x ** y