astro.dataframes.load_options
Module Contents
Classes
Pandas load options while reading and loading different files. Some common used params are added to the class for |
|
Pandas load options while reading and loading csv file. |
|
Pandas load options while reading and loading json file. |
|
Pandas load options while reading and loading Ndjson file. |
|
Pandas load options while reading and loading Parquet file. |
- class astro.dataframes.load_options.PandasLoadOptions
Bases:
astro.options.LoadOptions
- Pandas load options while reading and loading different files. Some common used params are added to the class for
other valid options ref below. They can be passed in kwargs param: 1. CSV file type - https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html 2. NDJSON/JSON file type - https://pandas.pydata.org/docs/reference/api/pandas.read_json.html 3. Parquet file type - https://pandas.pydata.org/docs/reference/api/pandas.read_parquet.html 4. Excel file type: https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html
- Parameters:
delimiter – Delimiter to use. Defaults to None
dtype – Data type for data or columns. E.g.
{"a": np.float64, "b": np.int32, "c": "Int64"}
Use str or object together with suitable na_values settings to preserve and not interpret dtype. If converters are specified, they will be applied INSTEAD of dtype conversion.encoding – Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings: https://docs.python.org/3/library/codecs.html#standard-encodings
normalize_sep – separator used to normalize nested ndjson. ex -
{"a": {"b":"c"}}
will result in:column - "a_b"
wherendjson_normalize_sep = "_"
columns – If not None, only these columns will be read from the file.
- delimiter: str | None
- normalize_sep: str | None
- columns: list[str] | None
- dtype: pandas._typing.DtypeArg | None
- encoding: str | None
- kwargs: dict
- populate_kwargs(kwargs)
- class astro.dataframes.load_options.PandasCsvLoadOptions(*args, **kwargs)
Bases:
PandasLoadOptions
Pandas load options while reading and loading csv file.
- class astro.dataframes.load_options.PandasJsonLoadOptions(*args, **kwargs)
Bases:
PandasLoadOptions
Pandas load options while reading and loading json file.
- Parameters:
encoding – Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings: https://docs.python.org/3/library/codecs.html#standard-encodings
- class astro.dataframes.load_options.PandasNdjsonLoadOptions(*args, **kwargs)
Bases:
PandasLoadOptions
Pandas load options while reading and loading Ndjson file.
- class astro.dataframes.load_options.PandasParquetLoadOptions(*args, **kwargs)
Bases:
PandasLoadOptions
Pandas load options while reading and loading Parquet file.