Azure Pipelines Validator

Validates, lints, and security-scans Azure DevOps Pipeline configurations (azure-pipelines.yml / azure-pipelines.yaml). Use when working with ADO pipelines, YAML pipeline files, or CI/CD configurations in Azure DevOps: including validating YAML syntax and schema, detecting hardcoded secrets or credentials, checking for deprecated or unpinned task versions, enforcing best practices (caching, timeouts, display names), performing pipeline security audits, or reviewing azure-pipelines.yml before merging. Trigger terms: azure-pipelines.yml, ADO pipeline, Azure Pipelines, YAML pipeline, CI/CD validation, pipeline security scan, DevOps configuration review.

npx skills add github:pantheon-org/tekhne --skill ci-cd--azure-pipelines--validator

Validates, lints, and security-scans Azure DevOps Pipeline configurations (azure-pipelines.yml, azure-pipelines.yaml). Runs four validation layers via a single orchestrator script.

Philosophy

Static analysis is cheap and runs before the pipeline does, so catch problems at review time, not at queue time.
Fix findings in layer order (lint, syntax, best practices, security): a syntax error can hide later findings.
Treat security findings as merge blockers and best-practice findings as guidance that hardens over time.
Re-run only the layer you just fixed, then run the full validation once before merging.

When to Use

Reviewing azure-pipelines.yml or azure-pipelines.yaml before merging a change.
Auditing a pipeline for hardcoded secrets, :latest container tags or curl | bash patterns.
Checking task version pinning, displayName, caching and timeouts on an existing pipeline.
Adding a validation gate to an Azure Pipelines CI job.

When Not to Use

Executing or testing a pipeline run: this skill does static analysis only.
Looking up current task inputs or versions: use the live documentation sources listed below.
Validating GitLab CI or GitHub Actions files: use the matching validator skill instead.
Generating a new pipeline from scratch: use the Azure Pipelines generator skill instead.

Basic Usage

# Full validation (all layers)
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml

Layers executed in order: 0. YAML lint (yamllint): formatting, indentation, trailing spaces

Syntax validation: schema, required fields, stages/jobs/steps hierarchy, task format, dependencies
Best practices: display names, task version pinning, pool image specificity, caching, timeouts
Security scan: hardcoded secrets/API keys/AWS/Azure credentials, dangerous script patterns, SSL bypasses, container :latest tags

Common Options

# Targeted runs
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --syntax-only
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --best-practices
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --security-only

# Skip layers
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --skip-yaml-lint
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --no-best-practices
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --no-security

# Strict mode (fail on warnings)
bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --strict

Individual Scripts

python3 scripts/validate_syntax.py azure-pipelines.yml
python3 scripts/check_best_practices.py azure-pipelines.yml
python3 scripts/check_security.py azure-pipelines.yml

Output and Error Recovery

════════════════════════════════════════════════════════════════════════════════
  Azure Pipelines Validator
════════════════════════════════════════════════════════════════════════════════

[1/3] Running syntax validation...
✓ Syntax validation passed

[2/3] Running best practices check...
SUGGESTIONS (2):
  INFO: Line 15: Job 'BuildJob' should have displayName [missing-displayname]
  💡 Add 'displayName: "Your Job Description"' to job 'BuildJob'

  WARNING: Line 25: Task 'Npm@1' could benefit from caching [missing-cache]
  💡 Add Cache@2 task to cache dependencies and speed up builds

[3/3] Running security scan...
MEDIUM SEVERITY (1):
  MEDIUM: Line 8: Container 'linux' uses ':latest' tag [container-latest-tag]
  🔒 Pin container images to specific versions or SHA digests

When validation fails:

Note the rule code in brackets (e.g., [missing-displayname]): see references/ for rule details.
Fix the flagged line and re-run the same layer (--syntax-only, --security-only, etc.) to iterate quickly.
Run full validation once all targeted fixes are applied to confirm no regressions.
For MEDIUM/HIGH security findings, do not merge until resolved; INFO findings are advisory.

Common Scenarios

New pipeline validation

bash scripts/validate_azure_pipelines.sh new-pipeline.yml

Security audit before merge

bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --security-only --strict

Pipeline optimisation

bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --best-practices

CI/CD self-validation in Azure Pipelines

steps:
- script: |
    bash scripts/validate_azure_pipelines.sh azure-pipelines.yml --strict
  displayName: 'Validate Pipeline Configuration'

Auto-detection

Run without arguments to auto-detect azure-pipelines*.yml files in the current directory (up to 3 levels deep).

Fetching Live Documentation

The validator performs static analysis only. For dynamic lookups (task versions, input parameters, feature docs), use:

# Context7 MCP
mcp__context7__resolve-library-id("azure-pipelines")
mcp__context7__get-library-docs(context7CompatibleLibraryID, topic="deployment")

# Or WebSearch / WebFetch
WebSearch("Azure Pipelines Docker@2 task documentation 2025")
WebFetch("https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/docker-v2")

Requirements

Python 3.10+, Bash
PyYAML and yamllint: auto-installed in a persistent .venv if not available system-wide: no manual setup required.

# Optional manual install
pip3 install PyYAML yamllint

Troubleshooting

Problem	Fix
`ModuleNotFoundError: PyYAML`	`pip3 install PyYAML`
`Permission denied`	`chmod +x scripts/.sh scripts/.py`
Unexpected validation errors	Check `references/azure-pipelines-reference.md` or Microsoft Learn

Anti-Patterns

NEVER skip validation on template files

WHY: Templates called from the main pipeline can contain schema violations, missing parameters, or invalid task references that only surface at runtime. Validating only the entry-point file leaves template errors undetected until the pipeline actually runs. BAD: Validate only azure-pipelines.yml and skip all files under templates/*.yml. GOOD: Run the validator on every .yml file in the pipeline directory, including all templates.

NEVER treat YAML lint warnings as informational

WHY: Azure Pipelines YAML is whitespace-significant. Indentation errors and trailing spaces can silently restructure the pipeline: turning a step into a job property, or merging two separate blocks: without any obvious parse error. BAD: Ignore yamllint warnings such as “trailing spaces” or “wrong indentation” because the pipeline appears to run. GOOD: Fix all YAML formatting issues before submitting; a clean yamllint pass is a prerequisite for a trustworthy pipeline.

NEVER use the strict mode flag on an unreviewed pipeline

WHY: --strict fails on all warnings, which is the correct setting for a CI gate. Applied to a brand-new pipeline with dozens of warnings, it produces so much noise that engineers discard the output entirely and disable validation rather than fix the root causes. BAD: Run --strict on a new pipeline, see 30 warnings, and remove validation from the workflow because “it’s too noisy.” GOOD: Run without --strict first, fix critical errors, then warnings, then graduate to strict mode as a CI gate.

NEVER skip security scanning before merging pipeline changes

WHY: A passing syntax validation does not imply a secure pipeline. Security scanning catches hardcoded credentials, injection vectors, and insecure patterns that syntax checks are not designed to detect. BAD: Merge a pipeline change after syntax validation passes with no security check. GOOD: Always run --security as part of the validation workflow, treating MEDIUM and HIGH findings as merge blockers.

NEVER merge a pipeline with hardcoded secrets in YAML

WHY: The security layer flags hardcoded passwords, API keys, tokens, AWS and Azure credentials (for example [hardcoded-password], [hardcoded-api-key]). Anything committed to the repository stays in history even after the line is removed. BAD: password: 'P@ssw0rd' written directly in a variables: block. GOOD: Reference a secret variable or variable group, for example $(DB_PASSWORD), and mark it secret in Azure DevOps.

NEVER leave task versions unpinned or at version zero

WHY: The best-practices layer reports [task-no-version], [task-version-zero] and [task-old-version]. An unpinned or outdated task can change behaviour between runs or run deprecated code. BAD: - task: Npm with no @ version. GOOD: - task: Npm@1 with the major version pinned.

NEVER use `:latest` container images or `ubuntu-latest` blindly

WHY: The scanner reports [container-latest-tag] and [pool-latest-image]. A floating tag means the same commit can build differently on different days. BAD: container: ubuntu:latest. GOOD: Pin the image to a specific version or a SHA digest.

NEVER pipe remote downloads straight into a shell

WHY: Patterns such as curl ... | bash and wget ... | sh are flagged by the security scan ([curl-pipe-shell], [wget-pipe-shell]). The pipeline executes whatever the remote host serves at that moment. BAD: - script: curl -sSL https://example.com/install.sh | bash. GOOD: Download the file, verify it, then run it, or use a pinned pipeline task.

NEVER disable SSL verification in pipeline scripts

WHY: The security scan flags SSL bypasses such as [insecure-ssl] and [git-disable-ssl]. Disabling verification exposes the build to man-in-the-middle tampering. BAD: - script: git config --global http.sslVerify false. GOOD: Fix the certificate chain or install the corporate CA on the agent.

NEVER ignore a missing timeout or cache finding on long jobs

WHY: The best-practices layer reports [missing-timeout] and [missing-cache]. A hung job without a timeout holds an agent until the platform default expires, and an uncached dependency install slows every run. BAD: A build job with no timeoutInMinutes that reinstalls npm packages on every run. GOOD: Set timeoutInMinutes on the job and add a Cache@2 step for dependencies.

References

references/azure-pipelines-reference.md: full YAML syntax reference and rule definitions
assets/examples/basic-pipeline.yml: simple CI pipeline
assets/examples/docker-build.yml: Docker build and push
assets/examples/deployment-pipeline.yml: multi-environment deployment with approval gates
assets/examples/multi-platform.yml: multi-platform build matrix
assets/examples/template-example.yml: reusable templates

# Test with a bundled example
bash scripts/validate_azure_pipelines.sh assets/examples/basic-pipeline.yml

Extending the Skill

Add custom rules to the appropriate script:

Syntax rules → scripts/validate_syntax.py
Best practice rules → scripts/check_best_practices.py
Security rules → scripts/check_security.py

# Example custom best-practice rule in check_best_practices.py
def _check_custom_rule(self):
    for job in self._get_all_jobs():
        job_name = job.get('job') or job.get('deployment')
        if 'tags' not in pool:
            self.issues.append(BestPracticeIssue(
                'warning',
                self._get_line(job_name),
                f"Job '{job_name}' should specify agent tags",
                'custom-missing-tags',
                "Add 'tags' to pool to select appropriate agents"
            ))

Note: This skill validates pipeline configurations but does not execute pipelines. Use Azure DevOps Pipeline validation or Azure CLI to test actual pipeline execution.

118/ 140

Explicit NEVER/WHY/consequence triplets with side-by-side bad/good examples. Generic warnings score low — concrete failure modes with real impact (security, correctness, performance) score high.

Runtime validation via eval scenarios proving the skill actually changes agent behaviour. Requires evals/ directory, instructions.json, ≥3 scenarios, and ≥80% instruction coverage in summary.json.

Appropriate rigidity for the skill type: strict rules for critical/security skills, balanced steps for process skills, flexible options for tool skills. Mismatched calibration (too rigid or too loose) reduces score.

Expert-only knowledge the AI couldn't derive on its own — domain-specific patterns, production gotchas, decision frameworks, and non-obvious tool behaviour. High scores mean <5% redundancy with what the model already knows.

A clear philosophy plus numbered step-by-step workflows with explicit entry/exit points. Should answer: what is the core principle, how to execute it, and when (and when not) to apply it.

Rich activation keywords and trigger scenarios in the description so the skill fires when needed. Best descriptions read like an exhaustive list of the tasks and concepts that should invoke this skill.

Copy-paste-ready code, complete runnable examples, real file paths and commands. Pseudocode and abstract descriptions score low — concrete, executable content scores high.

SKILL.md as a <100-line navigation hub pointing to focused references/ files. Content should be lazy-loaded on demand, not front-loaded. Includes actionable 'When to Use' conditions per reference.

Proper frontmatter, single-task focus, activation keywords, and cross-harness portability. Skills must work across 40+ agentic harnesses without hardcoded paths or agent-specific references.

262 lines · 1 reference

Azure Pipelines YAML Reference

Comprehensive reference for Azure Pipelines YAML syntax and structure.

Pipeline Structure

Azure Pipelines supports three main structures:

1. Multi-Stage Pipeline

stages:
- stage: Build
  jobs:
  - job: BuildJob
    steps:
    - script: echo "Building"

2. Multi-Job Pipeline

jobs:
- job: Job1
  steps:
  - script: echo "Job 1"
- job: Job2
  steps:
  - script: echo "Job 2"

3. Single-Job Pipeline

steps:
- script: echo "Single job"

Top-Level Keywords

trigger

Defines CI triggers (push events):

trigger:
  branches:
    include:
    - main
    - develop
  paths:
    exclude:
    - docs/*

pr

Defines PR triggers:

pr:
  branches:
    include:
    - main
  paths:
    include:
    - src/*

schedules

Defines scheduled triggers:

schedules:
- cron: "0 0 * * *"
  displayName: Daily midnight build
  branches:
    include:
    - main

pool

Defines agent pool:

pool:
  vmImage: 'ubuntu-22.04'
  demands:
  - npm

Or use specific pool:

pool:
  name: 'My Agent Pool'

variables

Defines variables:

variables:
  configuration: 'Release'
  platform: 'x64'

Or variable groups:

variables:
- group: 'my-variable-group'
- name: myVar
  value: myValue

resources

Defines external resources:

resources:
  repositories:
  - repository: templates
    type: git
    name: MyProject/Templates

  pipelines:
  - pipeline: upstream
    source: UpstreamPipeline
    trigger: true

  containers:
  - container: linux
    image: ubuntu:22.04

Stage Definition

stages:
- stage: StageName
  displayName: 'Stage Display Name'
  dependsOn: PreviousStage
  condition: succeeded()
  variables:
    stageVar: value
  jobs:
  - job: JobName
    steps:
    - script: echo "Hello"

Job Definition

Regular Job

jobs:
- job: JobName
  displayName: 'Job Display Name'
  dependsOn: PreviousJob
  condition: succeeded()
  timeoutInMinutes: 60
  cancelTimeoutInMinutes: 5
  pool:
    vmImage: 'ubuntu-22.04'
  variables:
    jobVar: value
  steps:
  - script: echo "Job step"

Deployment Job

jobs:
- deployment: DeploymentName
  displayName: 'Deploy to Environment'
  environment: 'production'
  pool:
    vmImage: 'ubuntu-22.04'
  strategy:
    runOnce:
      deploy:
        steps:
        - script: echo "Deploying"

Deployment Strategies

runOnce

strategy:
  runOnce:
    preDeploy:
      steps:
      - script: echo "Pre-deploy"
    deploy:
      steps:
      - script: echo "Deploy"
    routeTraffic:
      steps:
      - script: echo "Route traffic"
    postRouteTraffic:
      steps:
      - script: echo "Post-route"
    on:
      failure:
        steps:
        - script: echo "Rollback"
      success:
        steps:
        - script: echo "Success"

rolling

strategy:
  rolling:
    maxParallel: 2
    deploy:
      steps:
      - script: echo "Deploy to rolling targets"

canary

strategy:
  canary:
    increments: [10, 20, 50]
    deploy:
      steps:
      - script: echo "Deploy canary"

Step Types

task

Executes a pipeline task:

- task: TaskName@MajorVersion
  displayName: 'Task Display Name'
  inputs:
    input1: value1
    input2: value2
  env:
    ENV_VAR: value
  condition: succeeded()
  continueOnError: false
  timeoutInMinutes: 10

script

Runs a shell script:

- script: |
    echo "Multi-line"
    echo "script"
  displayName: 'Run Script'
  workingDirectory: $(Build.SourcesDirectory)
  failOnStderr: false

bash

Runs a bash script:

- bash: |
    #!/bin/bash
    echo "Bash script"
  displayName: 'Bash Script'

pwsh / powershell

Runs PowerShell:

- pwsh: |
    Write-Host "PowerShell Core"
  displayName: 'PowerShell Script'

- powershell: |
    Write-Host "Windows PowerShell"
  displayName: 'Windows PowerShell'

checkout

Checks out repositories:

- checkout: self
  clean: true
  fetchDepth: 1
  lfs: false
  submodules: false
  persistCredentials: false

download

Downloads artifacts:

- download: current
  artifact: artifactName

publish

Publishes artifacts:

- publish: $(Build.ArtifactStagingDirectory)
  artifact: drop

template

References a template:

- template: templates/build-steps.yml
  parameters:
    param1: value1

Common Tasks

Npm@1

- task: Npm@1
  inputs:
    command: 'install' # or 'ci', 'custom'
    workingDir: '$(System.DefaultWorkingDirectory)'
    customCommand: 'run build'

DotNetCoreCLI@2

- task: DotNetCoreCLI@2
  inputs:
    command: 'build' # or 'restore', 'test', 'publish'
    projects: '**/*.csproj'
    arguments: '--configuration Release'

Docker@2

- task: Docker@2
  inputs:
    command: 'build' # or 'push', 'login'
    repository: 'myrepo/myimage'
    dockerfile: '$(Build.SourcesDirectory)/Dockerfile'
    tags: |
      $(Build.BuildId)
      latest

PublishPipelineArtifact@1

- task: PublishPipelineArtifact@1
  inputs:
    targetPath: '$(Build.ArtifactStagingDirectory)'
    artifact: 'drop'
    publishLocation: 'pipeline'

AzureWebApp@1

- task: AzureWebApp@1
  inputs:
    azureSubscription: 'Azure-Connection'
    appName: 'mywebapp'
    package: '$(System.DefaultWorkingDirectory)/**/*.zip'

Conditions

# Always run
condition: always()

# Run on success
condition: succeeded()

# Run on failure
condition: failed()

# Run on success or failure
condition: succeededOrFailed()

# Custom condition
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))

Variable Syntax

# Pipeline variable
$(variableName)

# Environment variable (bash)
$VARIABLE_NAME

# Environment variable (PowerShell)
$env:VARIABLE_NAME

# Runtime expression
${{ variables.variableName }}

# Predefined variables
$(Build.BuildId)
$(Build.SourceBranch)
$(Agent.OS)
$(System.DefaultWorkingDirectory)

Templates

Variable Template

# variables/common.yml
variables:
  configuration: 'Release'
  platform: 'x64'

Step Template

# templates/build-steps.yml
parameters:
- name: buildConfiguration
  type: string
  default: 'Release'

steps:
- script: echo "Building with ${{ parameters.buildConfiguration }}"

Job Template

# templates/test-job.yml
parameters:
- name: jobName
  type: string
- name: pool
  type: string

jobs:
- job: ${{ parameters.jobName }}
  pool:
    vmImage: ${{ parameters.pool }}
  steps:
  - script: echo "Testing"

Best Practices

Always pin task versions: Use TaskName@2 not TaskName@*
Use specific VM images: Use ubuntu-22.04 not ubuntu-latest
Use displayName: Add descriptive names for stages, jobs, and steps
Use caching: Cache dependencies to speed up builds
Use templates: Reuse common configurations
Use variable groups: Organize variables for different environments
Set timeouts: Prevent hung jobs with timeoutInMinutes
Use conditions: Control when stages/jobs run
Clean checkout: Use clean: true for consistent builds
Use deployment jobs: For deployments to environments

References

Edit this page on GitHub

Azure Pipelines Validator

Philosophy

When to Use

When Not to Use

Basic Usage

Common Options

Individual Scripts

Output and Error Recovery

Common Scenarios

New pipeline validation

Security audit before merge

Pipeline optimisation

CI/CD self-validation in Azure Pipelines

Auto-detection

Fetching Live Documentation

Requirements

Troubleshooting

Anti-Patterns

NEVER skip validation on template files

NEVER treat YAML lint warnings as informational

NEVER use the strict mode flag on an unreviewed pipeline

NEVER skip security scanning before merging pipeline changes

NEVER merge a pipeline with hardcoded secrets in YAML

NEVER leave task versions unpinned or at version zero

NEVER use :latest container images or ubuntu-latest blindly

NEVER pipe remote downloads straight into a shell

NEVER disable SSL verification in pipeline scripts

NEVER ignore a missing timeout or cache finding on long jobs

References

Extending the Skill

Azure Pipelines YAML Reference

Pipeline Structure

1. Multi-Stage Pipeline

2. Multi-Job Pipeline

3. Single-Job Pipeline

Top-Level Keywords

trigger

pr

schedules

pool

variables

resources

Stage Definition

Job Definition

Regular Job

Deployment Job

Deployment Strategies

runOnce

rolling

canary

Step Types

task

script

bash

pwsh / powershell

checkout

download

publish

template

Common Tasks

Npm@1

DotNetCoreCLI@2

Docker@2

PublishPipelineArtifact@1

AzureWebApp@1

Conditions

Variable Syntax

Templates

Variable Template

Step Template

Job Template

Best Practices

References

NEVER use `:latest` container images or `ubuntu-latest` blindly