Skip to content

Deploying a dacpac to a serverless SQL pool

Reading Time: 5 minutes

In this post I want to cover deploying a dacpac to a serverless SQL pool using Azure DevOps. By showing you examples based on both classic pipelines and YAML pipelines.

Yes, you are reading that right. It is now possible to do this thanks to a SqlPackage update.

You can download the SqlPackage update on the ‘Download and install SqlPackage‘ page. To download it onto an Azure Pipelines Agent during deployment you can read a newer post which covers how to install SqlPackage to work with dacpacs for serverless SQL Pools.

To clarify, a dacpac file is a special file that you can use to deploy database schema updates to SQL Server related databases using a state-based deployment. Plus, when I say serverless SQL pool I mean an Azure Synapse Analytics serverless SQL Pool.

Just to be clear, you can only deploy a dacpac that has been created especially for serverless SQL Pools. Because serverless SQL Pools only support a limited number of objects.

You can read more about this in detail in the Microsoft page about Transact-SQL features supported in Azure Synapse SQL.

As always, this post contains plenty of links. Plus, a link to a GitHub repository that you can use as a template for YAML pipelines in Azure DevOps.

SqlPackage update

A long requested feature is for there to be support to deploy the contents of a dacpac file to databases in serverless SQL Pools. So that state-based deployments can be done.

Which led to Johannes Vink (l/t) raising a GitHub issue about Synapse Serverless SQL Pool support.

It is now finally possible thanks to the latest SqlPackage update that was announced this week. You can read more about this and other updates in a post by Microsoft about updates to DacFx and Microsoft.Build.Sql.

As soon as I found out about the SqlPackage update I downloaded and tested it. I first ran the below code to extract the contents of a database in a serverless SQL Pool into a dacpac file.

SqlPackage /Action:Extract /TargetFile:sqlpackagetest.dacpac /p:VerifyExtraction=true /SourceServerName:"{MY SERVERLESS SQL POOL ENDPOINT}.sql.azuresynapse.net" /SourceDatabaseName:sqlpackagetest /SourceUser:{A SQL POOL LOGIN} /SourcePassword:{REDACTED P/W}

Which created a dacpac on my local machine. So far so good. From there, I ran the below code to publish the contents of the dacpac to another database in the serverless SQL Pool.

SqlPackage /Action:Publish /SourceFile:sqlpackagetest.dacpac /TargetConnectionString:"Server=tcp:{MY SERVERLESS SQL POOL ENDPOINT}.sql.azuresynapse.net;Initial Catalog=destinationdb;Persist Security Info=False;User ID={A SQL POOL LOGIN};Password={REDACTED P/W}

Which worked, as you can see below.

New external table in a serverless SQL Pool created by a dacpac
New external table in serverless SQL Pool created by a dacpac

Deploying a dacpac to a serverless SQL pool using Azure DevOps

In reality, you can figure out how to do the above yourselves by reading the SqlPackage documentation about the SqlPackage extract parameters and publish parameters.

Which is why I decided to take this post a bit further and cover how to deploy a dacpac to a serverless SQL pool using Azure DevOps in this post.

Plus, I wanted to share a link to a new GitHub repository called AzureDevOps-SynpaseServerlessSQLPool-dacpac.

Which is available for anybody to download and use as a template to deploy a dacpac to a serverless SQL pool using Azure DevOps. If you find it useful please give it a star in GitHub.

Note that for both of the methods that I show below I am using a self-hosted agent that has the latest version of SqlPackage installed. In addition, they both use the same variable group.

I will show how to do this using the Classic Editor and Releases feature in Azure Pipelines first. Which is the GUI-based way to do this.

Deploying a dacpac to a serverless SQL Pool using classic pipelines

I first use the Classic Editor in Azure Pipelines to extract the schema from a database in a serverless SQL Pool.

As you can see below, it first uses a PowerShell task to extract the schema into a dacpac file. From there it publishes the dacpac back into Azure DevOps.

Extracting a database to a dacpac
Extracting a database to a dacpac using Classic Editor

For those wondering, the above PowerShell task uses the below code.

# PowerShell to extract contents of a database in a serverless SQLPool into a dacpac file
            
SqlPackage /Action:Extract /TargetFile:$(Build.ArtifactStagingDirectory)\$(TargetFile) /p:VerifyExtraction=true /SourceServerName:$(SQLPoolEndPoint) /SourceDatabaseName:$(SourceDB) /SourceUser:$(SQLPooluser) /SourcePassword:$(SQLPoolpw)

Once the pipeline had completed, I created the below Release in Azure Pipelines. With the artifact being the artifact published by the Classic Editor.

Release to deploying a dacpac to a serverless SQL pool
Release to deploy a dacpac

In the stage to deploy the dacpac I created the below tasks. Which is where things get interesting,

Tasks to deploying a dacpac to a serverless SQL pool
Tasks to deploy the dacpac

As you can see, I used the Azure SQL Database deployment task to deploy the dacpac instead of using a PowerShell task. This is because this behind the scenes this task uses SqlPackage.

As you can see below, this release succeeded.

Succeeded deployment of a dacpac
Completed release

Deploying a dacpac to a serverless SQL Pool using a YAML pipeline

Now I want to cover how to deploy this using a YAML pipeline.

You can see the code shown below in the AzureDevOps-SynpaseServerlessSQLPool-dacpac repository. Which you can download and use as you see fit.

Basically, this is a code-based version of the above. I first extracted the contents of the database into a dacpac and then published the dacpac using the below code.

steps:
 - task: PowerShell@2
 inputs:
   targetType: 'inline'
   script: |
       # PowerShell to extract contents of a database in a serverless SQLPool into a dacpac file
            
 SqlPackage /Action:Extract /TargetFile:$(Build.ArtifactStagingDirectory)\$(TargetFile) /p:VerifyExtraction=true /SourceServerName:$(SQLPoolEndPoint) /SourceDatabaseName:$(SourceDB) /SourceUser:$(SQLPooluser) /SourcePassword:$(SQLPoolpw)

 - task: PublishBuildArtifacts@1
 displayName: 'Publishes dacpac as an artifact'
  # Publishes the dacpac as part of an artifact within Azure DevOps
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)'
          ArtifactName: $(SQLPoolartifactname)
          publishLocation: 'Container'

Afterwards, I deployed the contents of the dacpac using the below code.

 jobs:
  - deployment: 'SQLPool'
  displayName: 'Serverless SQL Pool'
  environment: Production

  pool: 
  name: $(agentpool)

  strategy:
   runOnce:
      deploy:
       steps:
          - task: DownloadBuildArtifacts@0
          displayName: 'Download Artifacts'
          inputs:
          buildType: 'current'
           downloadType: 'specific'
           artifactName:  '$(SQLPoolartifactname)'
           downloadPath: '$(System.ArtifactsDirectory)'

          - task: SqlAzureDacpacDeployment@1
          displayName: 'Install DACPAC on serverless SQL Pool'
          inputs:
           azureSubscription: $(AzureSubscription)
           AuthenticationType: 'server'
           ServerName: $(SQLPoolEndPoint)
           DatabaseName: '$(DestinationDB)'
           SqlUsername: '$(SQLPooluser)'
           SqlPassword: '$(SQLPoolpw)'
           deployType: 'DacpacTask'
           DeploymentAction: 'Publish'
           DacpacFile: '$(System.ArtifactsDirectory)\$(SQLPoolartifactname)\$(Targetfile)'

One key point I want to highlight is that I am using a deployment job in the above code. This is so that I can specify an environment. Because in the real-world you might want to consider approving deployments to production.

As you can see below, the pipeline completed with an artifact published. Which contains the dacpac file.

Completed YAML pipeline that extracts to a dacpac and then deploys it
Completed YAML pipeline that extracts to a dacpac and then deploys it

Final words

I hope this post about deploying a dacpac to a serverless SQL pool helps some you set this up. In addition, I hope the same goes for the AzureDevOps-SynpaseServerlessSQLPool-dacpac that is now available.

If you prefer For migration-based deployments you can still use the method I covered in my post about CI/CD for serverless SQL Pools.

Since publishing this post I have also shared how to deploy a dacpac to a serverless SQL pool using GitHub Actions. Plus, how you can create a database project which you can use to create dacpac files in a post about my homemade serverless SQL Pool database project.

Of course, if you have any comments or queries about this post feel free to reach out to me.

Published inAzure DevOpsAzure Synapse Analytics

45 Comments

  1. Manish Kumar Manish Kumar

    Hi Kevin, I tried using your script and I am getting following error . I first want to make a build pipelines in DevOps but it is complaining about SqlPackage. I have installed sqlpackage at C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\Extensions\Microsoft\SQLDB\DAC

    I copied all the sqlpackage dll and exe to C:\Windows\System32\WindowsPowerShell\v1.0 but still same error. Can you help me here?

    ========================== Starting Command Output ===========================
    “C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe” -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command “. ‘D:\a\_temp\94e0c6a4-f4a4-42b6-9ccf-c533308ce686.ps1′”
    SqlPackage : The term ‘SqlPackage’ is not recognized as the name of a cmdlet, function, script file, or operable
    program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
    At D:\a\_temp\94e0c6a4-f4a4-42b6-9ccf-c533308ce686.ps1:6 char:1
    + SqlPackage /Action:Extract /TargetFile:D:\a\1\a\TREAPOC.dacpac /p:Ver …
    + ~~~~~~~~~~
    + CategoryInfo : ObjectNotFound: (SqlPackage:String) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : CommandNotFoundException

    ##[error]PowerShell exited with code ‘1’.
    Finishing: Extract Content Synapse Serverless Database

    • Kevin Chant Kevin Chant

      Sure, looks like you need to add the folder for sqlpackage to your environment variables so you can run it from any folder.

      • Manish Kumar Manish Kumar

        Hi Kevin, Thanks for the prompt reply. I added sqlpackage PATH in the environment variables of my PC and tried running my build pipeline. I got the same error as above. I tried running following on command prompt and this also result into error but it gives different error:

        SqlPackage /Action:Extract /TargetFile:DBNAME.dacpac /p:VerifyExtraction=true /SourceServerName:”abc-dev-environment-ondemand.sql.azuresynapse.net” /SourceDatabaseName:DBNAME/SourceUser:sqluser /SourcePassword:sqlpwd

        Extracting schema
        Extracting schema from database
        *** Error extracting database:An error occurred while attempting to connect to the server: FUNCTION ‘SESSIONPROPERTY’ is not supported..

        I am not sure how you made it working. If you show it step by step in your blog than it would help many 🙂

        • Kevin Chant Kevin Chant

          Hi, from the look of it you are trying to use an older version of SQLPackage. You need to install the new one and point the path to it.

          Hope that resolves this issue.

          • Manish Kumar Manish Kumar

            Hi Kevin,

            I copied the latest zip file and I can run SqlPackage from command prompt from my PC and it works. It shows some unresolved reference and that I need to check. The issue is with DevOps Pipeline. I still get error. I use Inline Powershell script and the error is

            “========================== Starting Command Output ===========================
            “C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe” -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command “. ‘D:\a\_temp\e3d6e283-40df-4390-ab63-ae2762f0080a.ps1′”
            SqlPackage : The term ‘SqlPackage’ is not recognized as the name of a cmdlet, function, script file, or operable
            program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
            At D:\a\_temp\e3d6e283-40df-4390-ab63-ae2762f0080a.ps1:6 char:1
            + SqlPackage /Action:Extract /TargetFile:D:\a\1\a\TREAPOC.dacpac /p:Ver …
            + ~~~~~~~~~~
            + CategoryInfo : ObjectNotFound: (SqlPackage:String) [], ParentContainsErrorRecordException
            + FullyQualifiedErrorId : CommandNotFoundException

            ##[error]PowerShell exited with code ‘1’.
            Finishing: Extract Content Synapse Serverless Database”
            It seems powershell is still not recognzing SqlPackage command.

  2. Manish Kumar Manish Kumar

    Hi again,

    I can run the build pipeline powershell from command prompt from my pc and it works. I took snippet from your blog and replaced the variables. When I am using the same PowerShell script in the Powershell Task in Build Pipeline, it fails. It keeps giving same error.

    The scrip is same as per your blog:

    SqlPackage /Action:Extract
    /TargetFile:$(Build.ArtifactStagingDirectory)\$(TargetFile) /p:VerifyExtraction=true /SourceServerName:$(SQLPoolEndPoint) /SourceDatabaseName:$(SourceDB) /SourceUser:$(SQLPooluser) /SourcePassword:$(SQLPoolpw)

    Error:
    SqlPackage : The term ‘SqlPackage’ is not recognized as the name of a cmdlet, function, script file, or operable

    It seems , PS Task on Build Pipeline is unable to find the path of SqlPackage. Any idea how to solve it? It will be very much appreciated.

  3. Mark Waelterman Mark Waelterman

    Kevin, can you post your SQL database project and/or publish profile? I’m seeing some errors trying to publish the DACPAC to a serverless database endpoint; ex. Table hint NOLOCK is not allowed.

    • Kevin Chant Kevin Chant

      Hi Mark, are you trying to publish a dacpac that you extracted from a serverless SQL pool? Because there are issues if you try and publish a dacpac created another way?

      • Mark Waelterman Mark Waelterman

        Yeah, I’m trying to use code in a DevOps repo vs. something extracted from serverless…

        • Kevin Chant Kevin Chant

          OK, what error message are you getting?

    • Romain Romain

      Hello
      I have the same error
      It’s not yet possible to create a database project from scratch in visual studio 2022, and to publish it to a serverless database using sqlpackage ?
      I choose “Microsoft Azure SQL Data Warehouse” as target database type.
      Regards,
      Romain

  4. Jesse Jesse

    Hi Kevin, great post!
    I’m looking to add your piece of code to my YAML, being used in a devops pipeline. How do I download the newest version of SQLPackages to my Devops environment?

  5. Thomas Thomas

    Hi,

    Thanks for your work and effort.

    I also get the same error:

    Error:
    SqlPackage : The term ‘SqlPackage’ is not recognized as the name of a cmdlet, function, script file, or operable

    Should I install it in devops pipeline or something?

    • Kevin Chant Kevin Chant

      Hi, you have a couple of options.

      First one is that you use a self-hosted Agent which has the latest version of SQLPackage already installed (e.g. the agent running on your laptop.

      Second option is to add an additional task to install the latest version of SQLPackage on the Agent before you look to extract the pipleine.

      I hope this helps? If I get the chance I might write a post about this part.

      • Thomas Thomas

        Thank you – I actually solved it before you wrote (Im new in devops pipelines) But i added a task installing SqlPackage Via dotnet, so thanks!

        But actually I have a new question.

        When trying to extract the dacpac I get an error on all views saying:
        Error SQL71501: Error validating element [dbo].[xxxx]: SqlView: [dbo].[xxxx] has an unresolved reference to object [r].[PLAYER_ID].

        What could be the issue?

        • Kevin Chant Kevin Chant

          Does the view itself still work? Because looks like it is referencing a column in a table that no longer exists.

          • Thomas Thomas

            Yes and thats the strange thing.

            My view looks like this:

            CREATE VIEW dbo.xxxx AS SELECT
            CONVERT( DATETIME2(7), a ) as a
            , CONVERT( VARCHAR(10), bb ) as bb
            , CONVERT( VARCHAR(100), c ) as c
            , CONVERT( VARCHAR(10), d ) as d
            , CONVERT( VARCHAR(100), e ) as e

            FROM OPENROWSET( BULK N’abc’, DATA_SOURCE = ‘somearea’, FORMAT = ‘DELTA’ ) AS r;

            I can perfectly select from the view without any errors.

            This here is the error from the pipeline:
            Extracting schema
            Extracting schema from database
            Resolving references in schema model
            Validating schema model
            *** Error extracting database:Validation of the schema model for data package failed.
            Error SQL71501: Error validating element [dbo].[xxxx]: SqlView: [dbo].[xxxx] has an unresolved reference to object [r].[a].

            And it does it with all the columns.

          • Thomas Thomas

            Ahh its because views are only supported with a with clause !

          • Kevin Chant Kevin Chant

            Well glad you figured it out. Fingers crossed there will be a smoother process for it in the future.

  6. Max Max

    Hi Kevin we’re facing an issue regarding view using an external data source when deploy..

    Cannot find the CREDENTIAL ‘https://*****.dfs.core.windows.net/***/***/metadata/environment/*.json’, because it does not exist or you do not have permission.

    It seems that because the sqluser has not access on the datalake, the view cannot be created.

    So How to manage that ? Any solution ?

    Thanks Max

    • Kevin Chant Kevin Chant

      Does it error also supply an ID value of the account that does not have authorization?

      • Max Max

        Hi Kevin,

        In fact, it seems that we are in a catch 22 situation. I need to create in order :

        1. Credential
        2. External Data Source
        3. Views on the EDS

        But it seems that the deployment sequence of the EDS are not managed well, and it cause the issue.

        the script deploy in the following order: 1. credential 2. view (error) 3. ***

        We’ve tried to implement a workaround in pre-deploy without success. any idea ?

        • Max Max

          FYI, I finally found a workaround, nota pretty one but it works !!!

          I created 2 dacpac :
          1. “first deploy fix” that contain the credential and the EDS only
          2. the “default/main” dacpac that contain the other objects.

          And I deploy those dacpac in sequence.

          If someone have another better solution, let me know ! 🙂

          • Kevin Chant Kevin Chant

            Glad you figured out a workaround Max

  7. Erfan Erfan

    Hi Kevin,

    Good to know that they support serverless SQL in SqlPackage. But it seems that it does not support views over the partitioned parquet format as I receive the following error when I try to Extract a bacpac:

    SqlView: [dbo].[CoreView] has an unresolved reference to object [r].[filepath].

    we need filepath to return the partitioned value as a column in the view.
    Do you have any suggestion?

    • Kevin Chant Kevin Chant

      Hi Erfan,

      Apologies for the delay, does the view work if you deploy a dacpac instead?

      Kind regards

      Kevin

      • Erfan Erfan

        Hi Kevin,

        The issue is that SQLPackage does not create the dacpac because of the issue that I have mentioned.

        Best Regards,
        Erfan

      • Erfan Erfan

        Hi Kevin,

        I have created the views already in the serverless SQL on DEV environment which work fine and I want to extract them into dacpac in order to create a CI/CD pipeline for other environments.
        Using the extract from SQLPackage, I received the error that I have mentioned and it seems it can not detect the partitioning on the Parquet file and FilePath syntax of OpenRowset

        Best regards,
        Erfan

        • Kevin Chant Kevin Chant

          Hi Erfan,

          I will check a few things shortly as being busy with some other commitments recently.

          Kind regards

          Kevin

          • Kevin Chant Kevin Chant

            Hi Erfan,

            Just to check, are you using the latest version of SqlPackage (162.0.52)?

            Kind regards

            Kevin

          • Erfan Erfan

            Hi Kevin,

            Yes, my SQLPackage version 162.0.52.1.

            Kind regards,

            Erfan

  8. Erfan Erfan

    Hi Kevin,

    The issue is that SQLPackage does not create the dacpac because of the issue that I have mentioned.

    Best Regards,
    Erfan

  9. Nani Lucky Nani Lucky

    Hi Kevin,

    The above script looks great, can you help me to use the release pipeline as a classic one instead of Yaml script for the release.

    • Kevin Chant Kevin Chant

      Hi, in the post it also covers how to do the deployment with the classic pipelines.

      Kind regards

      Kevin

Leave a Reply

Your email address will not be published. Required fields are marked *