astro.databases.databricks.load_file.load_file_job
Module Contents
Functions
|
Load a file object into a databricks delta table |
Attributes
- astro.databases.databricks.load_file.load_file_job.cwd
- astro.databases.databricks.load_file.load_file_job.supported_file_locations
- astro.databases.databricks.load_file.load_file_job.log
- astro.databases.databricks.load_file.load_file_job.load_file_to_delta(input_file, delta_table, databricks_job_name, delta_load_options, if_exists='replace')
Load a file object into a databricks delta table
- Parameters:
if_exists (astro.constants.LoadExistStrategy) – How to handle the data if the table exists. Defaults to replace (but can be set to “append”)
delta_load_options (astro.databases.databricks.load_options.DeltaLoadOptions) – Loading options for passing data into delta. Things like COPY_OPTIONS, cluster_id, etc.
databricks_job_name (str) – The name of the job as it will show up in databricks.
input_file (astro.files.File) – File to load into delta
delta_table (astro.table.BaseTable) – a Table object with necessary metadata for accessing the cluster.