astro.sql.operators.dataframe

Module Contents

Functions

_get_dataframe(table[, identifiers_as_lower])

Exports records from a SQL table and converts it into a pandas dataframe

load_op_arg_table_into_dataframe(op_args, ...)

For dataframe based functions, takes any Table objects from the op_args

load_op_kwarg_table_into_dataframe(op_kwargs, ...)

For dataframe based functions, takes any Table objects from the op_kwargs

Classes

DataframeOperator

astro.sql.operators.dataframe._get_dataframe(table, identifiers_as_lower=False)

Exports records from a SQL table and converts it into a pandas dataframe

Parameters
Return type

pandas.DataFrame

astro.sql.operators.dataframe.load_op_arg_table_into_dataframe(op_args, python_callable, identifiers_as_lower)

For dataframe based functions, takes any Table objects from the op_args and converts them into local dataframes that can be handled in the python context

Parameters

  • op_args (Tuple) –

  • python_callable (Callable) –

  • identifiers_as_lower (bool) –

Return type

Tuple

astro.sql.operators.dataframe.load_op_kwarg_table_into_dataframe(op_kwargs, python_callable, identifiers_as_lower)

For dataframe based functions, takes any Table objects from the op_kwargs and converts them into local dataframes that can be handled in the python context

Parameters

  • op_kwargs (Dict) –

  • python_callable (Callable) –

  • identifiers_as_lower (bool) –

Return type

Dict

class astro.sql.operators.dataframe.DataframeOperator(conn_id=None, database=None, schema=None, identifiers_as_lower=True, **kwargs)

Bases: airflow.decorators.base.DecoratedOperator

Parameters

  • conn_id (Optional[str]) –

  • database (Optional[str]) –

  • schema (Optional[str]) –

  • identifiers_as_lower (bool) –

execute(context)
Parameters

context (Dict) –

Return type

Union[astro.sql.table.Table, pandas.DataFrame]