cleanup operator

When to use the cleanup operator

The cleanup operator allows you to clean up temporary tables(Tables) created as part of your Astro SDK pipeline. The cleanup function monitors the status of all the tasks within your DAG, and deletes any temporary tables(Tables) at the end of the DAG run. Astronomer recommends adding a cleanup operator to every DAG that uses the Astro SDK.


You can also specify the temporary tables you want to delete by providing a list of tables with the parameter tables_to_cleanup. Only temporary tables provided in this list will be deleted; persistent tables will be ignored.