`astro.lineage.extractor`

Module Contents

Classes

`OpenLineageFacets`	OpenLineageFacets are pieces of metadata that can be attached to the core entities: Run,
`PythonSDKExtractor`	This extractor provides visibility on what different python-sdk operator does by

class astro.lineage.extractor.OpenLineageFacets

OpenLineageFacets are pieces of metadata that can be attached to the core entities: Run, Job and Dataset as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/OpenLineage.md#facets

inputs :list[openlineage.client.run.Dataset]

outputs :list[openlineage.client.run.Dataset]

run_facets :dict[str, openlineage.client.facet.BaseFacet]

job_facets :dict[str, openlineage.client.facet.BaseFacet]

class astro.lineage.extractor.PythonSDKExtractor

Bases: openlineage.airflow.extractors.base.BaseExtractor

This extractor provides visibility on what different python-sdk operator does by extracting operator specific facets by calling get_openlineage_facets on each operator

classmethod get_operator_classnames()

Return type: list[str]

extract()

Empty extract implementation for the abstractmethod of the BaseExtractor class.

Return type: openlineage.airflow.extractors.TaskMetadata

extract_on_complete(task_instance)

Callback on get_openlineage_facets(ti) task completion to fetch metadata extraction details that are to be pushed to the Lineage server.

Parameters: task_instance (airflow.models.taskinstance.TaskInstance) –
Return type: TaskMetadata | None

astro.lineage.extractor

Module Contents

Classes

`astro.lineage.extractor`