append operator

When to use the append operator

The append operator allows you to append data from a source table to a target table.

    aql.append(
        target_table=load_main,
        source_table=load_append,
    )

If a columns parameter is not provided, the append operator assumes that the source and target tables have the same schema.

Tables have the same schema

  1. Case 1: When the complete table needs to be appended, the columns parameter can be omitted.
        aql.append(
            target_table=load_main,
            source_table=load_append,
        )
    
  2. Case 2: When only a subset of columns needs to be appended, you can pass a list of columns to the columns parameter.
        record_results = append(
            source_table=filtered_data,
            target_table=homes_reporting,
            columns=["sell", "list", "variable", "value"],
        )
    

Tables have different schemas

When the source and target tables have different schemas, you can map different column names by passing a dict of source columns to target columns.

    aql.append(target_table=load_main, source_table=load_append, columns={"beds": "baths"})

Conflicts

The append operator can’t handle conflicts that may arise while appending data. If you want to handle those scenarios, you can use the merge operator.

Default Datasets

  • Input dataset - Source table for the operator.

  • Output dataset - Target table of the operator.