astro.files.types.parquet

Module Contents

Classes

ParquetFileType

Concrete implementation to handle Parquet file type
class astro.files.types.parquet.ParquetFileType(path, normalize_config=None, load_options=None)

Bases: astro.files.types.base.FileType

Concrete implementation to handle Parquet file type

Parameters:

  • path (str) –

  • normalize_config (dict | None) –

  • load_options (LoadOptions | None) –

property name

get file type

LOAD_OPTIONS_CLASS_NAME = PandasLoadOptions
export_to_dataframe(stream, columns_names_capitalization='original', **kwargs)

read parquet file from one of the supported locations and return dataframe

Parameters:

  • stream – file stream object

  • load_options – Pandas option to pass to the Pandas lib while reading parquet

  • columns_names_capitalization – determines whether to convert all columns to lowercase/uppercase in the resulting dataframe

create_from_dataframe(df, stream)

Write parquet file to one of the supported locations

Parameters:

  • df (pandas.DataFrame) – pandas dataframe

  • stream (io.TextIOWrapper) – file stream object

Return type:

None