export_file operator

When to use the export_file operator

The export_file operator allows you to write SQL tables to CSV or parquet files and store them locally, on S3, or on GCS. The export_file function can export data from Supported Databases or a Pandas dataframe.

There are two main uses for the export_file operator.

Case 1: Export data from a table.

    aql.export_file(
        task_id="save_file_to_gcs",
        input_data=t1,
        output_file=File(
            path=f"{gcs_bucket}/{{ task_instance_key_str }}/all_movies.csv",
            conn_id="gcp_conn",
        ),
        if_exists="replace",
    )

Case 2: Export data from a Pandas dataframe.

    aql.export_file(
        task_id="save_dataframe_to_gcs",
        input_data=t2,
        output_file=File(
            path=f"{gcs_bucket}/{{ task_instance_key_str }}/top_5_movies.csv",
            conn_id="gcp_conn",
        ),
        if_exists="replace",
    )