astro.dataframes.load_options

Module Contents

Classes

PandasLoadOptions

Pandas load options while reading and loading different files. Some common used params are added to the class for

PandasCsvLoadOptions

Pandas load options while reading and loading csv file.

PandasJsonLoadOptions

Pandas load options while reading and loading json file.

PandasNdjsonLoadOptions

Pandas load options while reading and loading Ndjson file.

PandasParquetLoadOptions

Pandas load options while reading and loading Parquet file.
class astro.dataframes.load_options.PandasLoadOptions

Bases: astro.options.LoadOptions

Pandas load options while reading and loading different files. Some common used params are added to the class for

other valid options ref below. They can be passed in kwargs param: 1. CSV file type - https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html 2. NDJSON/JSON file type - https://pandas.pydata.org/docs/reference/api/pandas.read_json.html 3. Parquet file type - https://pandas.pydata.org/docs/reference/api/pandas.read_parquet.html 4. Excel file type: https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html

Parameters:

  • delimiter – Delimiter to use. Defaults to None

  • dtype – Data type for data or columns. E.g. {"a": np.float64, "b": np.int32, "c": "Int64"} Use str or object together with suitable na_values settings to preserve and not interpret dtype. If converters are specified, they will be applied INSTEAD of dtype conversion.

  • encoding – Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings: https://docs.python.org/3/library/codecs.html#standard-encodings

  • normalize_sep – separator used to normalize nested ndjson. ex - {"a": {"b":"c"}} will result in: column - "a_b" where ndjson_normalize_sep = "_"

  • columns – If not None, only these columns will be read from the file.

delimiter: str | None
normalize_sep: str | None
columns: list[str] | None
dtype: pandas._typing.DtypeArg | None
encoding: str | None
kwargs: dict
populate_kwargs(kwargs)
class astro.dataframes.load_options.PandasCsvLoadOptions(*args, **kwargs)

Bases: PandasLoadOptions

Pandas load options while reading and loading csv file.

class astro.dataframes.load_options.PandasJsonLoadOptions(*args, **kwargs)

Bases: PandasLoadOptions

Pandas load options while reading and loading json file.

Parameters:

encoding – Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings: https://docs.python.org/3/library/codecs.html#standard-encodings

class astro.dataframes.load_options.PandasNdjsonLoadOptions(*args, **kwargs)

Bases: PandasLoadOptions

Pandas load options while reading and loading Ndjson file.

class astro.dataframes.load_options.PandasParquetLoadOptions(*args, **kwargs)

Bases: PandasLoadOptions

Pandas load options while reading and loading Parquet file.