astro.databases.databricks.load_options

Module Contents

Classes

DeltaLoadOptions

Helper class for rendering options into COPY_INTO Databricks SQL statements

class astro.databases.databricks.load_options.DeltaLoadOptions

Bases: astro.options.LoadOptions

Helper class for rendering options into COPY_INTO Databricks SQL statements

Parameters:

• copy_into_format_options – File format options for COPY INTO. Read more at: https://docs.databricks.com/sql/language-manual/delta-copy-into.html#format-options

• copy_into_copy_options – Options to control the operation of the COPY INTO command. Read more at: https://docs.databricks.com/sql/language-manual/delta-copy-into.html

• existing_cluster_id – Existing cluster ID on databricks.

• new_cluster_spec – New cluster specifications.

• if_exists – default override an existing Table. Options: fail, replace, append

• secret_scope – Secrets scope

• load_secrets – Defaults to False.

• load_mode – defaults to autoloader. Options: autoloader, COPY INTO

• autoloader_load_options – Options for load using autoloader. Read more at: https://docs.databricks.com/ingestion/auto-loader/options.html

copy_into_format_options :dict

copy_into_copy_options :dict

existing_cluster_id :str

new_cluster_spec :dict

if_exists :astro.constants.LoadExistStrategy = replace

secret_scope :str = astro-sdk-secrets

load_secrets :bool = False

load_mode :astro.constants.DatabricksLoadMode

autoloader_load_options :dict

get_existing_cluster_id()

empty()

convert_format_options_to_string()

convert_copy_options_to_string()

classmethod get_default_delta_options()