When to use the get_value_list operator

We can use get_value_list when you want to execute a SQL query on a database table and get the result. This might be useful when you want to create dynamic tasks on the response of this operator.


Using this operator without limit in the SQL query can push lots of data in XCOM and also can create lots of parallel tasks if using in dynamic task map expand method.

    def custom_task(rating_val):
            return float(rating_val[0])
        except ValueError:
            # If value is not valid then ignore it

    def avg_rating(rating_list):
        rating_list = [val for val in rating_list if val]
        return sum(rating_list) / len(rating_list)

    rating = custom_task.expand(rating_val=get_value_list(sql=QUERY_STATEMENT, conn_id=ASTRO_GCP_CONN_ID))