Operationalize Fabric workspaces with Azure DevOps using Fabric CLI and fabric-cicd

Reading Time: 5 minutes

In this post I want to show one way you can operationalize Fabric workspaces with Azure DevOps using Fabric CLI and fabric-cicd.

To clarify, when I say the Fabric CLI I mean the Fabric Command Line Interface. Which is a command line offering by Microsoft. Whereas fabric-cicd is a Python library that allows you to perform CI/CD of various Microsoft Fabric items into Microsoft Fabric workspaces.

I want to show how you can utilize both Fabric CLI and fabric-cicd within YAML Pipelines in Azure DevOps. In order to create new workspaces and then populate them. Like in the below diagram.

I decided to do this since the ability to copy workspace items to another one currently only works in interactive mode. You can find links to a number of Fabric CLI examples by following the links within the Fabric CLI cheatsheet.

To manage expectations, this post shows how to do perform deployments with YAML Pipelines in the Azure Pipelines service within Azure DevOps.

Plus, I covered how to perform fabric-cicd deployments in a previous post about operationalizing fabric-cicd with YAML Pipelines. So, details about that stage will be kept to a minimum to avoid repetition. Along the way I share plenty of links.

If you need help with any jargon used in this post, then I recommend that you read one my other posts. Which is a Microsoft Fabric Git integration jargon guide for Fabricators.

Preparing to operationalize with Fabric CLI and fabric-cicd

I utilized the same variable groups that I worked with in a previous post that covered operationalizing fabric-cicd with YAML Pipeline. However, this time around I added two new variables.

• CapacityName – Name of the Fabric capacity assigned for the new workspace.
• EntraObjectId – Obect Id of the Microsoft Entra user to assign permissions for.

In addition, I decided to utilize the same Git repository that contained sample items as in the aforementioned post.

You can view a sample of the “fabric-cli-and-fabric-cicd-demo.yml” YAML pipeline created for this post in the AzureDevOps-fabric-cicd-sample repository. Which you can download/clone and work with as a template.

Operationalize Fabric workspaces with Azure DevOps using Fabric CLI and fabric-cicd

Once prepared, I created a new YAML pipeline in Azure DevOps. I selected a starter pipeline. However, if you download my sample GitHub repository you can create a pipeline from an existing YAML file instead.

Once my YAML pipeline was created I specified a parameter called WorkspaceName. So that every time the pipeline was started a name for the new workspace had to be entered.

parameters:
- name: WorkspaceName
  type: string
  default: "FabricCLI"

You can also add a parameter for the capacity name as well if you so wish.

I then specified the two variable groups that I had created. Plus, the fact that I did not want a trigger at this moment in time so that I can manually run the pipeline. In addition, I specified I wanted an Azure Pipelines Agent with the latest windows image.

variables:
- group: fabric-cicd-ns
- group: fabric-cicd-s

trigger: none
# In this pipeline I use a Microsoft-hosted agent
pool: 
  vmImage: 'windows-latest'

Stage to create a new workspace with Fabric CLI

I first created a stage to create a new workspace. Which contained a single job.

- stage: CreateWorkspace
  displayName: 'Create Workspace'
  
  jobs:
  - job: 'CreateWorkspace'
    displayName: 'Create Workspace'

My first task in this job was to specify the Python version. Which is essential when working with Python based offerings to avoid any errors about invalid syntax.

- task: UsePythonVersion@0
  displayName: 'Use Python 3.12'
  inputs:
    versionSpec: 3.12

Afterwards I installed the Fabric CLI library.

- task: PowerShell@2
  displayName: 'Install necessary libraries'
  inputs:
    targetType: 'inline'
    script: |
      python -m pip install --upgrade pip
      pip install ms-fabric-cli
    pwsh: true

It is worth noting that you can bundle the above PowerShell task and the subsequent ones in this stage together. However, I separated them to make the process easier to understand.

Anyway, afterwards I added the below PowerShell task to login to Fabric CLI and authenticate a service principal.

- task: PowerShell@2
  displayName: 'Authenticate as Service Principal'
  inputs:
    targetType: 'inline'
    script: |
      fab auth login -u $(azure_client_id) -p $(azure_client_secret) --tenant $(azure_tenant_id)
    pwsh: true

Next, I added a PowerShell task to create the workspace. Specifying the provided parameter for the workspace name.

- task: PowerShell@2
  displayName: 'Create workspace'
  inputs:
    targetType: 'inline'
    script: |
      fab create ${{parameters.WorkspaceName}}.Workspace -P capacityname=$(CapacityName)
    pwsh: true

Afterwards, I added a task to add permissions to the new workspace. I suggest doing this straight afterwards so that it can be accessed even if a later task fails.

- task: PowerShell@2
  displayName: 'Add permissions to workspace'
  inputs:
    targetType: 'inline'
    script: |
      fab acl set ${{parameters.WorkspaceName}}.Workspace -I $(EntraObjectId) -R admin -f
    pwsh: true

Final task created in the stage was to get the id value of the newly created workspace. Which can then be passed on to the next stage.

- task: PowerShell@2
  displayName: 'Get Workspace Id'
  name: GetWorkspaceId  
  inputs:
    targetType: 'inline'
    script: |
      $WorkspaceId = fab get ${{parameters.WorkspaceName}}.Workspace -q id
      Write-Host "##vso[task.setvariable variable=WorkspaceId;isOutput=true]$($WorkspaceId)"
    pwsh: true

I tested this stage and checked that the new workspace had been created before adding the second stage.

Stage to populate the new workspace with Fabric CLI

In reality, this stage is very similar to the stages I mentioned in my previous post. Where I covered how to operationalize fabric-cicd to work with Microsoft Fabric and YAML Pipelines.

However, I made two changes to allow the WorkspaceId variable created in the previous stage to work.

First of all, I specified to work with the variable I had previously created at the start of the stage.

- stage: PopulateWorkspace
  displayName: 'Populate Workspace'
  dependsOn: CreateWorkspace
  jobs:
  - job: 'PopulateWorkspace'
    displayName: 'Populate Workspace'
    variables:
      WorkspaceId: $[ stagedependencies.CreateWorkspace.CreateWorkspace.outputs['GetWorkspaceId.WorkspaceId'] ]
    steps:

For those looking to do the same, Microsoft documents how you can use outputs in a different stage.

Second change was to change the variable required for WorkspaceId.

- task: PythonScript@0
  displayName: 'Run script to deploy with fabric-cicd to Test'
  inputs:
    scriptPath: 'auth_spn_secret_AzDo.py'
    arguments: '--WorkspaceId $(WorkspaceId) --Environment "Prod" --RepositoryDirectory "$(Build.SourcesDirectory)\workspace" --ItemsInScope  $(ItemsInScope)'

Testing the pipeline in Azure DevOps

To test running the pipeline I selected the branch that the existing Fabric items were stored in and a name for my new Fabric workspace.

Which completed as you can see below.

When I checked Microsoft Fabric, I found my new workspace along with all the deployed items.

Final words

I hope that showing this way to Operationalize Fabric workspaces with Azure DevOps using Fabric CLI and fabric-cicd helps some of you get started. Plus, inspire some of you to experiment in your own environments.

Because I do believe that both the Fabric CLI and fabric-cicd will be able to help with your automation and CI/CD requirements. As you can imagine if you go through the Fabric CLI cheatsheet I mentioned earlier in this post.

Of course, if you have any comments or queries about this post feel free to reach out to me.