astro.files.types.ndjson
Module Contents
Classes
Concrete implementation to handle NDJSON file type |
- class astro.files.types.ndjson.NDJSONFileType(path, normalize_config=None)
Bases:
astro.files.types.base.FileTypeConcrete implementation to handle NDJSON file type
- Parameters
path (str) –
normalize_config (Optional[dict]) –
- export_to_dataframe(stream, **kwargs)
read ndjson file from one of the supported locations and return dataframe
- Parameters
stream – file stream object
- create_from_dataframe(df, stream)
Write ndjson file to one of the supported locations
- Parameters
df (pandas.DataFrame) – pandas dataframe
stream (io.TextIOWrapper) – file stream object
- Return type
None
- property name
get file type
- static flatten(normalize_config, stream)
Flatten the nested ndjson/json.
- Parameters
normalize_config (dict) – parameters in dict format of pandas json_normalize() function. https://pandas.pydata.org/docs/reference/api/pandas.json_normalize.html
stream (io.TextIOWrapper) – io.TextIOWrapper object for the file
- Returns
return dataframe containing the loaded data
- Return type
pandas.DataFrame