astro.databases.databricks.load_options
Module Contents
Classes
Helper class for rendering options into COPY_INTO Databricks SQL statements |
- class astro.databases.databricks.load_options.DeltaLoadOptions
Bases:
astro.options.LoadOptions
Helper class for rendering options into COPY_INTO Databricks SQL statements
- Parameters:
copy_into_format_options – File format options for COPY INTO. Read more at: https://docs.databricks.com/sql/language-manual/delta-copy-into.html#format-options
copy_into_copy_options – Options to control the operation of the COPY INTO command. Read more at: https://docs.databricks.com/sql/language-manual/delta-copy-into.html
existing_cluster_id – Existing cluster ID on databricks.
new_cluster_spec – New cluster specifications.
if_exists – default override an existing Table. Options: fail, replace, append
secret_scope – Secrets scope
load_secrets – Defaults to False.
load_mode – defaults to autoloader. Options: autoloader, COPY INTO
autoloader_load_options – Options for load using autoloader. Read more at: https://docs.databricks.com/ingestion/auto-loader/options.html
- copy_into_format_options: dict
- copy_into_copy_options: dict
- existing_cluster_id: str
- new_cluster_spec: dict
- if_exists: astro.constants.LoadExistStrategy = 'replace'
- secret_scope: str = 'astro-sdk-secrets'
- load_secrets: bool = False
- load_mode: astro.constants.DatabricksLoadMode
- autoloader_load_options: dict
- get_existing_cluster_id()
- empty()
- convert_format_options_to_string()
- convert_copy_options_to_string()
- classmethod get_default_delta_options()