astro.databases.databricks.load_options

Module Contents

Classes

DeltaLoadOptions

Helper class for rendering options into COPY_INTO Databricks SQL statements

class astro.databases.databricks.load_options.DeltaLoadOptions

Bases: astro.options.LoadOptions

Helper class for rendering options into COPY_INTO Databricks SQL statements

Parameters

• copy_into_format_options – File format options for COPY INTO. Read more at: https://docs.databricks.com/sql/language-manual/delta-copy-into.html#format-options

• copy_into_copy_options – Options to control the operation of the COPY INTO command. Read more at: https://docs.databricks.com/sql/language-manual/delta-copy-into.html

• existing_cluster_id – Existing cluster ID on databricks.

• new_cluster_spec – New cluster specifications.

• if_exists – default override an existing Table. Options: fail, replace, append

• secret_scope – Secrets scope

• load_secrets – Defaults to False.

• load_mode – defaults to autoloader. Options: autoloader, COPY INTO

• autoloader_load_options – Options for load using autoloader. Read more at: https://docs.databricks.com/ingestion/auto-loader/options.html

copy_into_format_options: dict

copy_into_copy_options: dict

existing_cluster_id: str

new_cluster_spec: dict

if_exists: astro.constants.LoadExistStrategy = 'replace'

secret_scope: str = 'astro-sdk-secrets'

load_secrets: bool = False

load_mode: astro.constants.DatabricksLoadMode

autoloader_load_options: dict

get_existing_cluster_id()

empty()

convert_format_options_to_string()

convert_copy_options_to_string()

classmethod get_default_delta_options()