astro.files.locations.amazon.s3
Module Contents
Classes
Handler S3 object store operations |
- class astro.files.locations.amazon.s3.S3Location(path, conn_id=None)
Bases:
astro.files.locations.base.BaseFileLocationHandler S3 object store operations
- Parameters
path (str) –
conn_id (str | None) –
- property hook: airflow.providers.amazon.aws.hooks.s3.S3Hook
- Return type
airflow.providers.amazon.aws.hooks.s3.S3Hook
- property transport_params: dict
Structure s3fs credentials from Airflow connection. s3fs enables pandas to write to s3
- Return type
dict
- property paths: list[str]
Resolve S3 file paths with prefix
- Return type
list[str]
- property size: int
Return file size for S3 location
- Return type
int
- property openlineage_dataset_namespace: str
Returns the open lineage dataset namespace as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md
- Return type
str
- property openlineage_dataset_name: str
Returns the open lineage dataset name as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md
- Return type
str
- location_type