astro.databases.sqlite

Module Contents

Classes

SqliteDatabase

Handle interactions with Sqlite databases. If this class is successful, we should not have any Sqlite-specific

Attributes

DEFAULT_CONN_ID

astro.databases.sqlite.DEFAULT_CONN_ID
class astro.databases.sqlite.SqliteDatabase(conn_id=DEFAULT_CONN_ID, table=None)

Bases: astro.databases.base.BaseDatabase

Handle interactions with Sqlite databases. If this class is successful, we should not have any Sqlite-specific logic in other parts of our code-base.

Parameters

  • conn_id (str) –

  • table (BaseTable | None) –

property sql_type: str
Return type

str

property hook: airflow.providers.sqlite.hooks.sqlite.SqliteHook

Retrieve Airflow hook to interface with the Sqlite database.

Return type

airflow.providers.sqlite.hooks.sqlite.SqliteHook

property sqlalchemy_engine: sqlalchemy.engine.base.Engine

Return SQAlchemy engine.

Return type

sqlalchemy.engine.base.Engine

property default_metadata: astro.table.Metadata

Since Sqlite does not use Metadata, we return an empty Metadata instances.

Return type

astro.table.Metadata

static get_table_qualified_name(table)

Return the table qualified name.

Parameters

table (astro.table.BaseTable) – The table we want to retrieve the qualified name for.

Return type

str

populate_table_metadata(table)

Since SQLite does not have a concept of databases or schemas, we just return the table as is, without any modifications.

Parameters

table (astro.table.BaseTable) –

Return type

astro.table.BaseTable

create_schema_if_needed(schema, location=None)

Since SQLite does not have schemas, we do not need to set a schema here.

Parameters

  • schema (str | None) –

  • location (str | None) –

Return type

None

schema_exists(schema)

Check if a schema exists. We return false for sqlite since sqlite does not have schemas

Parameters

schema (str) –

Return type

bool

static get_merge_initialization_query(parameters)

Handles database-specific logic to handle index for Sqlite.

Parameters

parameters (tuple) –

Return type

str

merge_table(source_table, target_table, source_to_target_columns_map, target_conflict_columns, if_conflicts='exception')

Merge the source table rows into a destination table. The argument if_conflicts allows the user to define how to handle conflicts.

Parameters

  • source_table (astro.table.BaseTable) – Contains the rows to be merged to the target_table

  • target_table (astro.table.BaseTable) – Contains the destination table in which the rows will be merged

  • source_to_target_columns_map (dict[str, str]) – Dict of target_table columns names to source_table columns names

  • target_conflict_columns (list[str]) – List of cols where we expect to have a conflict while combining

  • if_conflicts (astro.constants.MergeConflictStrategy) – The strategy to be applied if there are conflicts.

Return type

None

get_sqla_table(table)

Return SQLAlchemy table instance

Parameters

table (astro.table.BaseTable) – Astro Table to be converted to SQLAlchemy table instance

Return type

sqlalchemy.sql.schema.Table

openlineage_dataset_name(table)

Returns the open lineage dataset name as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md Example: /tmp/local.db.table_name

Parameters

table (astro.table.BaseTable) –

Return type

str

openlineage_dataset_namespace()

Returns the open lineage dataset namespace as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md Example: file://127.0.0.1:22

Return type

str

openlineage_dataset_uri(table)

Returns the open lineage dataset uri as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md

Parameters

table (astro.table.BaseTable) –

Return type

str