astro.utils.dataframe
Module Contents
Functions
|
Convert cols of a dataframe to required case. Options - lower/Upper |
Passes a dataframe into a File using parquet as an efficient storage format. This allows us to use |
- astro.utils.dataframe.convert_columns_names_capitalization(df, columns_names_capitalization)
Convert cols of a dataframe to required case. Options - lower/Upper
- Parameters:
df (pandas.DataFrame) – dataframe whose cols will be altered
columns_names_capitalization (astro.constants.ColumnCapitalization) – String Literal with possible values - lower/Upper
- astro.utils.dataframe.convert_dataframe_to_file(df)
Passes a dataframe into a File using parquet as an efficient storage format. This allows us to use Json as a storage method without filling the metadata database. the values for conn_id and bucket path can be found in the airflow.cfg as follows:
[astro] xcom_storage_conn_id=… xcom_storage_url=/// :param df: Dataframe to convert to file :return: File object with reference to stored dataframe file
- Parameters:
df (pandas.DataFrame) –
- Return type:
astro.files.File