astro.files.locations.azure.wasb
Module Contents
Classes
Handler WASB object store operations |
- class astro.files.locations.azure.wasb.WASBLocation(path, conn_id=None, load_options=None)
Bases:
astro.files.locations.base.BaseFileLocationHandler WASB object store operations
- Parameters
path (str) –
conn_id (str | None) –
load_options (LoadOptions | None) –
- property hook: airflow.providers.microsoft.azure.hooks.wasb.WasbHook
- Return type
airflow.providers.microsoft.azure.hooks.wasb.WasbHook
- property transport_params: dict
get WASB credentials for storage
- Return type
dict
- property paths: list[str]
Resolve WASB file paths with prefix
- Return type
list[str]
- property smartopen_uri: str
SmartOpen does not support URIs prefixed with wasb, so we need to change them to azure.
- Returns
URI compatible with SmartOpen for Azure BlobStorage.
- Return type
str
- property size: int
Return file size for WASB location
- Return type
int
- property openlineage_dataset_namespace: str
Returns the open lineage dataset namespace as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md
- Return type
str
- property openlineage_dataset_name: str
Returns the open lineage dataset name as per https://github.com/OpenLineage/OpenLineage/blob/main/spec/Naming.md
- Return type
str
- location_type
- supported_conn_type
- exists()
Check if the file exists or not
- Return type
bool