Data Engine API Reference

Class: DataEngine

Initialization

DataEngine()

Initializes a new DataEngine object, sets up the connection to the Data Engine service, and prepares the HTTP session with the appropriate headers.

Properties

connection

@property
def connection

Returns the underlying Trino connection object.

Methods

ibis_connection(catalog=None, schema=None)

def ibis_connection(self, catalog=None, schema=None)

Creates and returns an Ibis connection to the Data Engine.

• Parameters:
  • catalog (optional): The catalog to use.
  • schema (optional): The schema to use.
• Returns: An Ibis connection object.

execute(query: str, params=None)

def execute(self, query: str, params=None)

Executes the provided SQL query.

• Parameters:
  • query: SQL query string.
  • params (optional): Query parameters.
• Returns: The DataEngine instance (for chaining), or None on error.

fetch_one()

def fetch_one(self) -> Optional[List[Any]]

Fetches a single row from the last executed query.

• Returns: A list of values for the row, or None if no more data is available.

fetch_all()

def fetch_all(self) -> List[List[Any]]

Fetches all rows from the last executed query.

• Returns: A list of rows, each row being a list of values.

description()

def description(self) -> List[DescribeOutput]

Returns metadata about the columns in the result set of the last executed query.

• Returns: A list of column descriptions.

column_names()

def column_names(self) -> List[str]

Returns the names of the columns in the result set.

• Returns: A list of column names.

to_pandas()

def to_pandas(self)

Converts the result of the last executed query to a pandas DataFrame.

• Returns: A pandas DataFrame, or None on error.

to_polars()

def to_polars(self)

Converts the result of the last executed query to a Polars DataFrame.

• Returns: A Polars DataFrame, or None on error.

to_arrow()

def to_arrow(self)

Converts the result of the last executed query to a PyArrow Table.

• Returns: A PyArrow Table, or None on error.

Example Usage

from dt.data_engine import DataEngine

engine = DataEngine()
engine.execute("SELECT * FROM my_table")
df = engine.to_pandas()
print(df)

Notes

• The class uses a custom HTTP session with special headers for authentication.
• The connection is established to a Trino/Presto-compatible backend.
• Error handling is performed via print statements and returning None on failure.