`astro.databases.postgres`

Postgres database implementation.

Module Contents

Classes

PostgresDatabase

Handle interactions with Postgres databases. If this class is successful, we should not have any Postgres-specific

Attributes

DEFAULT_CONN_ID

astro.databases.postgres.DEFAULT_CONN_ID

class astro.databases.postgres.PostgresDatabase(conn_id=DEFAULT_CONN_ID)

Bases: astro.databases.base.BaseDatabase

Handle interactions with Postgres databases. If this class is successful, we should not have any Postgres-specific logic in other parts of our code-base.

Parameters: conn_id (str) –

illegal_column_name_chars :List[str] = ['.']

illegal_column_name_chars_replacement :List[str] = ['_']

property sql_type

Return type: str

property hook

Retrieve Airflow hook to interface with the Postgres database.

Return type: airflow.providers.postgres.hooks.postgres.PostgresHook

property default_metadata

Fill in default metadata values for table objects addressing postgres databases

Return type: astro.sql.table.Metadata

schema_exists(schema)

Checks if a schema exists in the database

Parameters: schema – DB Schema - a namespace that contains named objects like (tables, functions, etc)
Return type: bool

load_pandas_dataframe_to_table(source_dataframe, target_table, if_exists='replace', chunk_size=DEFAULT_CHUNK_SIZE)

Create a table with the dataframe’s contents. If the table already exists, append or replace the content, depending on the value of if_exists.

Parameters

source_dataframe (pandas.DataFrame) – Local or remote filepath
target_table (astro.sql.table.Table) – Table in which the file will be loaded
if_exists (astro.constants.LoadExistStrategy) – Strategy to be used in case the target table already exists.
chunk_size (int) – Specify the number of rows in each batch to be written at a time.

Return type

None

static get_table_qualified_name(table)

Return table qualified name. This is Database-specific. For instance, in Sqlite this is the table name. In Snowflake, however, it is the database, schema and table

Parameters: table (astro.sql.table.Table) – The table we want to retrieve the qualified name for.
Return type: str

table_exists(table)

Check if a table exists in the database

Parameters: table (astro.sql.table.Table) – Details of the table we want to check that exists
Return type: bool

merge_table(source_table, target_table, source_to_target_columns_map, target_conflict_columns, if_conflicts='exception')

Merge the source table rows into a destination table. The argument if_conflicts allows the user to define how to handle conflicts.