Pipelines in Azure DevOps are extremely useful and its something I used daily for Infrastructure Builds with Terraform on Azure and AWS. Normally these are triggered automatically as part of a pull request and merge to main, but there are times when you want to interact manually with them, such as starting or cancelling pipelines in bulk, and this is where this tip comes in handy.

If you want to start/cancel a single pipeline, via the Azure DevOps portal is quick and easy. But what if you want to control hundreds of pipelines at once? This happened to me recently where I updated a shared module and triggered 300 pipelines (with build gates so no risk), but rather than click through the portal for an hour, I found a way to handle pipelines with Powershell.

First create yourself a Personal Access Token or PAT for short, on Azure DevOps and grant this enough permissions to read/manage build and release pipelines.

Cancel all Pending Jobs

Say you triggered too many pipelines like I did, and you want to get all Pipelines that are stuck in a pending state. This powershell will do the following:-

  1. Connect to Azure DevOps with your PAT Key
  2. Get all your pending pipelines
  3. Cycle through them and cancel each one
  # Replace the values in this for the variables with your own.
  $myDevOpsPAT = "123456789abcedfghijklmnopqrstuvwxyz"
  $myDevOpsOrganization = "StarkIndustries"
  $myDevOpsProjectName = "ArcReactorProject"

  # Generate your auth header
  $myDevOpsAuthHeader = @{Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($myDevOpsPAT)")) }
 
  # Build your URL using your custom variables
  $Uri= https://dev.azure.com/$($myDevOpsOrganization)/$($myDevOpsProjectName)/_apis/build/builds?statusFilter=notStarted&api-version=6.0-preview.1
 
  # Get all the pending jobs
  $PendingJobs = Invoke-RestMethod -Uri $Uri -Headers $myDevOpsAuthHeader -Method get -ContentType "application/json"
  $JobsToCancel = $PendingJobs.value
 
  # Run through each pipeline from above and cancel them
  ForEach($pipeline in $JobsToCancel)
  {
     $pipeline.status = "Cancelling"
     $body = $pipeline | ConvertTo-Json -Depth 10
     $urlToCancel = https://dev.azure.com/$($myDevOpsOrganization)/$($myDevOpsProjectName)/_apis/build/builds/$($pipeline.id)?api-version=6.0-preview.1
     Invoke-RestMethod -Uri $urlToCancel -Method Patch -ContentType application/json -Body $body -Header $myDevOpsAuthHeader
  }

Starting Multiple Pipelines

The reverse of that is starting multiple pipelines. An example I use this all the time is triggering specific pipelines for Terraform. So say I have 300 pipelines and I only want to trigger all my pipelines that have Network in the title, I would run a powershell select statement to filter out what I want and then post this to a slightly different url like below.

  # Replace the values in this for the variables with your own.
  $myDevOpsPAT = "123456789abcedfghijklmnopqrstuvwxyz"
  $myDevOpsOrganization = "StarkIndustries"
  $myDevOpsProjectName = "ArcReactorProject"

  # Generate your auth header
  $myDevOpsAuthHeader = @{Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($myDevOpsPAT)")) }

  # Build your URL using your custom variables
  $Uri= https://dev.azure.com/$($myDevOpsOrganization)/$($myDevOpsProjectName)/_apis/pipelines?api-version=6.0-preview.1
  
  # Get all my Pipelines
  $Pipelines = Invoke-RestMethod -Uri $Uri -Headers $myDevOpsAuthHeader -Method get -ContentType "application/json"

  # Fine all the Pipelines I want that have a name containing Network
  $JobsToStart = $Pipelines.value  | Where-Object {$_.Name -like "*base*"}
 
  # Start all those Pipelines
  ForEach($pipeline in $JobsToStart)
  {
     $body = $pipeline | ConvertTo-Json -Depth 10
     $urlToStart = https://dev.azure.com/$($myDevOpsOrganization)/$($myDevOpsProjectName)/_apis/pipelines/$($pipeline.id)/runs?api-version=6.0-preview.1
     Invoke-RestMethod -Uri $urlToStart -Method post -ContentType application/json -Body $body -Header $myDevOpsAuthHeader
  }

I hope you find that useful and it saves you as much time as it saves me each day!

NOTE As these are based around Preview URLs for Azure DevOps they may change over time.

Andy