astro.databases.sqlite
Module Contents
Classes
Handle interactions with Sqlite databases. If this class is successful, we should not have any Sqlite-specific |
Attributes
- astro.databases.sqlite.DEFAULT_CONN_ID
- class astro.databases.sqlite.SqliteDatabase(conn_id=DEFAULT_CONN_ID, table=None, load_options=None)
Bases:
astro.databases.base.BaseDatabaseHandle interactions with Sqlite databases. If this class is successful, we should not have any Sqlite-specific logic in other parts of our code-base.
- Parameters:
conn_id (str) –
table (BaseTable | None) –
load_options (LoadOptions | None) –
- property sql_type: str
- Return type:
str
- property default_metadata: astro.table.Metadata
Since Sqlite does not use Metadata, we return an empty Metadata instances.
- Return type:
- hook()
Retrieve Airflow hook to interface with the Sqlite database.
- Return type:
airflow.providers.sqlite.hooks.sqlite.SqliteHook
- sqlalchemy_engine()
Return SQAlchemy engine.
- Return type:
sqlalchemy.engine.base.Engine
- static get_table_qualified_name(table)
Return the table qualified name.
- Parameters:
table (astro.table.BaseTable) – The table we want to retrieve the qualified name for.
- Return type:
str
- populate_table_metadata(table)
Since SQLite does not have a concept of databases or schemas, we just return the table as is, without any modifications.
- Parameters:
table (astro.table.BaseTable) –
- Return type:
astro.table.BaseTable
- create_schema_if_needed(schema)
Since SQLite does not have schemas, we do not need to set a schema here.
- Parameters:
schema (str | None) –
- Return type:
None
- schema_exists(schema)
Check if a schema exists. We return false for sqlite since sqlite does not have schemas
- Parameters:
schema (str) –
- Return type:
bool
- static get_merge_initialization_query(parameters)
Handles database-specific logic to handle index for Sqlite.
- Parameters:
parameters (tuple) –
- Return type:
str
- merge_table(source_table, target_table, source_to_target_columns_map, target_conflict_columns, if_conflicts='exception')
Merge the source table rows into a destination table. The argument if_conflicts allows the user to define how to handle conflicts.
- Parameters:
source_table (astro.table.BaseTable) – Contains the rows to be merged to the target_table
target_table (astro.table.BaseTable) – Contains the destination table in which the rows will be merged
source_to_target_columns_map (dict[str, str]) – Dict of target_table columns names to source_table columns names
target_conflict_columns (list[str]) – List of cols where we expect to have a conflict while combining
if_conflicts (astro.constants.MergeConflictStrategy) – The strategy to be applied if there are conflicts.
- Return type:
None
- get_sqla_table(table)
Return SQLAlchemy table instance
- Parameters:
table (astro.table.BaseTable) – Astro Table to be converted to SQLAlchemy table instance
- Return type:
sqlalchemy.sql.schema.Table
- openlineage_dataset_name(table)
Returns the open lineage dataset name as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md Example: /tmp/local.db.table_name
- Parameters:
table (astro.table.BaseTable) –
- Return type:
str
- openlineage_dataset_namespace()
Returns the open lineage dataset namespace as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md Example: file://127.0.0.1:22
- Return type:
str
- openlineage_dataset_uri(table)
Returns the open lineage dataset uri as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md
- Parameters:
table (astro.table.BaseTable) –
- Return type:
str