export_file operator
When to use the export_file
operator
The export_file
operator allows you to write SQL tables to CSV or parquet files and store them locally, on S3, or on GCS. The export_file
function can export data from Supported Databases or a Pandas dataframe.
There are two main uses for the export_file
operator.
Case 1: Export data from a table.
aql.export_file( task_id="save_file_to_gcs", input_data=t1, output_file=File( path=f"{gcs_bucket}/{{ task_instance_key_str }}/all_movies.csv", conn_id="gcp_conn", ), if_exists="replace", )
Case 2: Export data from a Pandas dataframe.
aql.export_file( task_id="save_dataframe_to_gcs", input_data=t2, output_file=File( path=f"{gcs_bucket}/{{ task_instance_key_str }}/top_5_movies.csv", conn_id="gcp_conn", ), if_exists="replace", )
Default Datasets
Input dataset - Source table for the operator.
Output dataset - Target file of the operator.