`astro.lineage.extractor`

Module Contents

Classes

`OpenLineageFacets`	OpenLineageFacets are pieces of metadata that can be attached to the core entities: Run,
`PythonSDKExtractor`	This extractor provides visibility on what different python-sdk operator does by

class astro.lineage.extractor.OpenLineageFacets

OpenLineageFacets are pieces of metadata that can be attached to the core entities: Run, Job and Dataset as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/OpenLineage.md#facets

inputs :list[astro.lineage.OpenlineageDataset]

outputs :list[astro.lineage.OpenlineageDataset]

run_facets :dict[str, astro.lineage.BaseFacet]

job_facets :dict[str, astro.lineage.BaseFacet]

class astro.lineage.extractor.PythonSDKExtractor

Bases: astro.lineage.BaseExtractor

This extractor provides visibility on what different python-sdk operator does by extracting operator specific facets by calling get_openlineage_facets on each operator

classmethod get_operator_classnames()

Return type: list[str]

extract()

Empty extract implementation for the abstractmethod of the BaseExtractor class.

Return type: astro.lineage.TaskMetadata

extract_on_complete(task_instance)

Callback on get_openlineage_facets(ti) task completion to fetch metadata extraction details that are to be pushed to the Lineage server.

Parameters: task_instance (airflow.models.taskinstance.TaskInstance) –
Return type: TaskMetadata | None

astro.lineage.extractor

Module Contents

Classes

`astro.lineage.extractor`