Skip to content

Azure Test Plans example for Azure Data Factory

Reading Time: 4 minutes

In this post I want to cover an Azure Test Plans example for Azure Data Factory as part of a series of posts. For a few reasons.

Firstly, because I want to do a series of posts about how Azure Test Plans can be used for deployments to various Data Platform services. So that others gain ideas of how to use Azure Test Plans.

Secondly, I want to introduce some Azure Data services to people because I know some people have not seen them before. Finally, because I promised a Data Platform MVP that I would do a Power BI example, which I will cover in a later post.

In reality, I have already done a couple of SQL Server examples. Including one in my post about using Azure Test Plans for Data Platform deployments.

With this in mind, I am going to cover examples based on other Data Platform services. Which are Azure Data Factory, Azure Synapse Analytics and Power BI.

You can find links to the other two posts in this series below:

In addition, each example in this series will use a different type of test suite. For more about the different test suites you can read my post about Azure Test Plans jargon for Data Platform professionals.

Azure Test Plans example for Azure Data Factory

In this example I use a static test suite with Azure Data Factory. Which is a cloud-based service which allows you to move data around. Those of you with a SQL Server background can think of it along the lines of a cloud-based version of SSIS. However, it is different in both functionality and use.

For this basic example I created a mapping data flow, which allows me to get data from two datasets and put them into an Azure SQL database.

In the example below, the two sources are two tables in the AdventureWorksLt sample database.

If you go from left to right below you can see that I first filter the Product table for only road bikes. Afterwards, I perform a lookup to the Model table. I then select which columns I actually want to keep with a Select modifier. Once done it goes to a new table in the RoadBike database.

Azure Data Factory Data flow example
Data flow example

In reality, the two sources above can be from a number of other places instead of Azure SQL Databases. For example, flat files and Snowflake. You can view the list for which sources you can use in the Azure Data Factory and Azure Synapse Analytics connector overview.

Azure Test Plans

In Azure Test Plans within Azure Devops we can create a static test suite to check that this data flow works after creating it in various ways.

Static test suite

For example, we can test that Linked services exist and that the connections are setup correctly. Which can be useful if your company policy states they have to be setup using key vault.

From there you can build up your test case to test various stages of your data flow as below.

Azure Test Plans example for Azure Data Factory

Once you are happy with your data flow you can add a step to test it like I have done as the final step above. Which you can do by adding it as a data flow activity in a pipeline and clicking ‘Debug’.

Selecting Debug for a pipeline in Azure data factory
Data flow activity in a pipeline

Of course, this is just a basic example. You can expand on this by having multiple test cases inside your static test suite for various pipelines. Because it’s a static test suite you can add as many as different ones as needed.

Additional test cases in same test suite in Azure Test Plans
Additional test cases in same test suite

More about Azure Data Factory

If you want to learn more about Azure Data Factory, Microsoft Learn has a learning path for you. It’s called Data integration at scale with Azure Data Factory or Azure Synapse Pipeline.

In addition, there are various books available. Including a recent one by Richard Swinbank called Azure Data Factory by Example. Which I have been recommending to others who are learning Azure Data Factory. It contains hands-on examples.

Final words

I hope this Azure Test Plans example for Azure Data Factory has been an interesting insight.

Because, I had two aims for this post. To give others’ ideas about how Azure Test Plans can be used with Azure Data Factory and to introduce some to Azure Data Factory.

Of course, if you have any comments or queries about this post feel free to reach out to me.

Published inAzure Data FactoryAzure DevOpsBusiness Intelligence & analytics

10 Comments

  1. Hi Kevin,

    thanks for the post!

    I had a question. We are evaluating how to implement manual tests for synapse pipelines and database projects using Az test plans. The idea is to integrate them also with CD pipelines in DevOps, so before or after some tasks a successfully execution of one or more test cases is required in order to go to the next step. We tried out vs test task in devops but it seems it only works with automated tests, which are not available acording to our research.
    Do you know if there is a way to accomplish that or the Az Test Plans for ADF/Synapse are more intended to document tests together with the correspondent PBIs?
    Hope you understood my question 😉
    Best regards,
    Paul

  2. […] Since it was my first visit back to the UK since February 2020 it gave me the opportunity to network and meet people in-person for the first time. Such as Richard Swinbank, whose Azure Data Factory book I mention in my post that showed an Azure Test Plans example for Azure Data Factory. […]

  3. […] Since it was my first visit back to the UK since February 2020 it gave me the opportunity to network and meet people in-person for the first time. Such as Richard Swinbank, whose Azure Data Factory book I mention in my post that showed an Azure Test Plans example for Azure Data Factory. […]

Leave a Reply

Your email address will not be published. Required fields are marked *