astro.options

Module Contents

Classes

LoadOptions
LoadOptionsList
SnowflakeLoadOptions	Load options to load file to snowflake using native approach.
WASBLocationLoadOptions

Functions

contains_required_option(load_options, option_name)	Check required options in load_option class
list_to_dict(value)	Convert list object to dict

class astro.options.LoadOptions

empty()

to_dict()

Convert options class to dict

Return type:

dict

astro.options.contains_required_option(load_options, option_name)

Check required options in load_option class

Parameters:

• load_options (Optional[LoadOptions]) –

• option_name (str) –

Return type:

bool

astro.options.list_to_dict(value)

Convert list object to dict

Parameters:

value (Optional[List[LoadOptions]]) –

Return type:

Optional[Dict[str, LoadOptions]]

class astro.options.LoadOptionsList

get(option_class)

Check LOAD_OPTIONS_CLASS_NAME attribute and select the correct load_options :param option_class: FileType | FileLocation | BaseDatabase

Return type:

Optional[LoadOptions]

get_by_class_name(option_class_name)

Get load_option by class name :return:

Return type:

Optional[LoadOptions]

class astro.options.SnowflakeLoadOptions

Bases: LoadOptions

Load options to load file to snowflake using native approach.

Parameters:

• file_options – Depending on the file format type specified, use one or more of the format-specific options as key-value pair. Read more at: https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html#format-type-options-formattypeoptions

• copy_options – Specify one or more of the copy option as key-value pair. Read more at: https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html#copy-options-copyoptions

• storage_integration – Specify the previously created Snowflake storage integration

• validation_mode – Defaults to validation_mode=None. This instructs the COPY command to validate the data files instead of loading them into the specified table; i.e. the COPY command tests the files for errors but does not load them. Supported validation mode; RETURN_n_ROWS | RETURN_ERRORS | RETURN_ALL_ERRORS

Note

Specify the supported validation mode;

• RETURN_n_ROWS: validates the specified n rows if no errors are encountered; otherwise, fails at the first error encountered in the rows.

• RETURN_ERRORS: returns all errors (parsing, conversion, etc.) across all files specified in the COPY statement.

• RETURN_ALL_ERRORS: returns all errors across all files specified in the COPY statement, including files with errors that were partially loaded during an earlier load because the ON_ERROR copy option was set to CONTINUE during the load.

Read more at: https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html#optional-parameters

file_options :dict

copy_options :dict

validation_mode :str

storage_integration :str

empty()

class astro.options.WASBLocationLoadOptions

Bases: LoadOptions

storage_account :str