append operator
When to use the append
operator
The append
operator allows you to append data from a source table to a target table.
aql.append(
target_table=load_main,
source_table=load_append,
)
If a columns parameter is not provided, the append
operator assumes that the source and target tables have the same schema.
Tables have the same schema
- Case 1: When the complete table needs to be appended, the
columns
parameter can be omitted. aql.append( target_table=load_main, source_table=load_append, )
- Case 1: When the complete table needs to be appended, the
- Case 2: When only a subset of columns needs to be appended, you can pass a
list
of columns to thecolumns
parameter. record_results = append( source_table=filtered_data, target_table=homes_reporting, columns=["sell", "list", "variable", "value"], )
- Case 2: When only a subset of columns needs to be appended, you can pass a
Tables have different schemas
When the source and target tables have different schemas, you can map different column names by passing a dict
of source columns to target columns.
aql.append(target_table=load_main, source_table=load_append, columns={"beds": "baths"})
Conflicts
The append
operator can’t handle conflicts that may arise while appending data. If you want to handle those scenarios, you can use the merge operator.
Default Datasets
Input dataset - Source table for the operator.
Output dataset - Target table of the operator.