GitLab CI/CD: Automated Pipelines at Scale

The .gitlab-ci.yml file at the repository root defines your entire CI/CD pipeline. It declares stages (ordered phases), jobs (units of work within stages), and global configuration (default image, variables, caching). GitLab parses this file on every push and creates a pipeline — a directed acyclic graph of jobs that execute on GitLab Runners.

Global keywords apply to all jobs: default sets the base image and timeout, variables defines environment variables, and workflow:rules controls when pipelines are created. Use include to import shared templates from other files or repositories, keeping the main CI file clean. The extends keyword lets jobs inherit from template jobs prefixed with a dot (e.g., .build-template).

A well-organized .gitlab-ci.yml separates concerns: global configuration at the top, include directives for shared templates, stage definitions, and then individual jobs grouped by stage. For large projects, split configuration across multiple files using include:local to reference files within the same repository.

Stages run sequentially (test before build before deploy), but jobs within the same stage run in parallel. This parallelism is key to fast pipelines: run unit tests, lint, and type-check simultaneously in the test stage. Use needs to create job dependencies that bypass stage ordering — a deploy job can start as soon as the build job finishes, even if other test-stage jobs are still running (DAG pipelines).

Artifacts are files produced by a job and passed to downstream jobs. Configure artifacts:paths to specify files and artifacts:expire_in to auto-delete them (e.g., 1 hour for build outputs, 30 days for test reports). Use artifacts:reports:junit to publish test results in the GitLab merge request UI. In production, test coverage reports are uploaded as artifacts and displayed directly in merge requests.

Caching persists files between pipeline runs on the same runner. Use cache:key to scope caches (per-branch, per-job) and cache:paths to specify directories. For Node.js, cache node_modules/ with a key based on package-lock.json hash: key: { files: ["package-lock.json"] }. This ensures the cache is invalidated when dependencies change, saving 30–90 seconds per job.

Job rules (rules) replace the older only/except syntax and provide fine-grained control over when jobs run. Rules evaluate conditions (branch name, file changes, variables) and set attributes (when: on_success, manual, delayed). A well-structured pipeline uses rules to skip unnecessary jobs — for example, skip deployment jobs on merge request pipelines, or skip frontend tests when only backend files changed.

The only/except keywords still work but are considered legacy. They support branch names, tags, and triggers but lack the composability of rules. A common migration pattern: replace only: [main] with rules: [{ if: '$CI_COMMIT_BRANCH == "main"' }]. Unlike only/except, rules entries can combine multiple conditions, set when and allow_failure, and use changes to detect file modifications.

workflow:rules controls pipeline creation at a global level — before any job runs. Use it to prevent duplicate pipelines when both a branch push and a merge request event fire for the same commit. The standard pattern creates pipelines for merge requests, tagged commits, and the default branch, skipping everything else. This eliminates redundant pipeline runs and saves runner capacity.

Building Docker images in CI requires either Docker-in-Docker (DinD) or Kaniko. DinD runs a Docker daemon inside the CI container, requiring --privileged mode — a security risk in shared runner environments. Kaniko builds images without a daemon, running entirely in userspace. For security and simplicity, Kaniko is the recommended approach for all new GitLab CI setups.

Kaniko in GitLab CI is configured as a job using the gcr.io/kaniko-project/executor image. Pass the build context, Dockerfile path, and destination registry. Enable layer caching with --cache=true --cache-repo=$REGISTRY/cache for dramatically faster rebuilds. Authentication to the registry is handled via a JSON key file mounted as a CI/CD variable.

In production, Kaniko eliminated the need for privileged runners entirely. Build times dropped from 4 minutes (DinD with cold cache) to 90 seconds (Kaniko with remote layer cache). The Kaniko job template is defined once in a shared CI library and included by all 26 microservice repositories, ensuring consistent build configuration across the organization.

GitLab environments track deployments to staging, production, and review environments. Define an environment in a deploy job with environment: { name: production, url: https://api.example.com }. GitLab tracks deployment history, enables rollback from the UI, and displays environment status in merge requests. Protected environments restrict who can deploy — only maintainers can deploy to production.

Review apps are dynamic environments created per merge request, giving developers a live preview of their changes. The deploy job creates the environment on merge request open and a stop job tears it down on merge. This is powerful for frontend changes where visual review matters. In production, review environments deploy to a shared GKE namespace with auto-cleanup after 48 hours of inactivity.

Environment scoping applies to CI/CD variables as well. Set DB_HOST to a staging database for the staging environment and a production database for production. Variables can be masked (hidden in job logs) and protected (only available on protected branches). This separation ensures that staging pipelines never accidentally touch production resources.

GitLab Runners are the agents that execute CI/CD jobs. They come in three scopes: shared runners are available to all projects in a GitLab instance (GitLab.com provides these by default on Linux), group runners serve all projects within a group, and specific runners are assigned to individual projects. Choose the scope based on security requirements, resource needs, and cost.

Runner tags control which runner picks up a job. Tag a macOS runner with macos and a GPU runner with gpu, then use tags: [macos] in the job definition. This is essential for mobile builds (macOS for iOS), ML workloads (GPU), and compliance (on-premise runners for sensitive code). In production, Fastlane iOS builds run on a dedicated macOS runner tagged macos-m1, while all other jobs use shared Linux runners.

For self-managed runners, the executor type determines how jobs are isolated. The Docker executor runs each job in a fresh container (most common). The Kubernetes executor spawns pods in a cluster, enabling autoscaling. The Shell executor runs directly on the host (useful for macOS). In production, group runners use the Kubernetes executor on GKE with autoscaling from 2 to 20 pods based on queue depth, keeping costs low during off-hours while handling peak loads.

CI/CD variables are defined at multiple levels: instance, group, project, and job. Lower levels override higher ones, so a project variable overrides a group variable with the same name. Define variables in the .gitlab-ci.yml for non-sensitive values (image tags, feature flags) and in the GitLab UI (Settings > CI/CD > Variables) for secrets like API keys, database passwords, and registry credentials.

Protect variables by marking them as protected (available only on protected branches and tags) and masked (redacted from job logs). For secrets that must never appear in YAML, use GitLab's integration with external secret managers like HashiCorp Vault via secrets:vault. Predefined variables like $CI_COMMIT_SHA, $CI_PIPELINE_ID, and $CI_REGISTRY_IMAGE are available in every job automatically.

In production, all secrets are stored as masked, protected CI/CD variables scoped to their respective environments. The 41-branded Android app pipeline uses group-level variables for shared signing configuration and project-level variables for brand-specific keys. A strict convention: no secrets in YAML, no unmasked passwords, and quarterly rotation enforced through pipeline validation jobs.

Shared CI/CD libraries are central repositories of reusable pipeline templates. Instead of duplicating .gitlab-ci.yml configuration across 20+ repositories, define templates once and include them everywhere. A template repository contains YAML files with hidden jobs (prefixed with .) that service repositories extend. Changes to the shared library propagate to all projects on their next pipeline run.

The include keyword supports four sources: local (same repo), file/project (another GitLab project), remote (any URL), and template (GitLab-maintained templates). Organize templates by concern: kaniko.yml for Docker builds, helm.yml for Kubernetes deployments, test-node.yml for Node.js testing, and security.yml for scanning. Service repositories include the template and extend the hidden job, overriding only what is service-specific.

In production, the your-org/ci-templates repository contains 8 shared templates used by all 26 microservices. When the Kaniko version needs updating or a new security scanning step is required, a single change to the shared library updates every project's pipeline. This centralized approach reduced CI/CD maintenance from hours per week to minutes per month.

Auto DevOps is GitLab's opinionated CI/CD pipeline that automatically detects the project language, builds, tests, scans, and deploys without any .gitlab-ci.yml configuration. It includes Auto Build (using Herokuish or Cloud Native Buildpacks), Auto Test, Auto Code Quality, Auto SAST, Auto Dependency Scanning, Auto Container Scanning, Auto Review Apps, Auto Deploy (to Kubernetes), and Auto Monitoring.

Auto DevOps is ideal for getting started quickly or for teams without deep CI/CD expertise. Enable it at the project or group level. It detects the language from the repository contents (e.g., a package.json signals Node.js) and applies the corresponding build and test steps. For Kubernetes deployments, set KUBE_CONTEXT or connect a cluster via the GitLab Kubernetes Agent.

For mature teams, Auto DevOps serves as a starting point rather than a final solution. In production, new microservices start with Auto DevOps enabled and gradually migrate to custom .gitlab-ci.yml files as their pipeline requirements become specific (custom Kaniko builds, Helm deployments, brand-specific mobile builds). The key benefit: no project ever ships without at least basic CI/CD, because Auto DevOps provides it by default.

GitLab includes built-in security scanning: SAST (Static Application Security Testing) analyzes source code for vulnerabilities, DAST (Dynamic Application Security Testing) scans running applications for runtime vulnerabilities, dependency scanning checks third-party libraries for known CVEs, and container scanning checks Docker images for OS-level vulnerabilities. Enable them by including GitLab's security templates.

SAST results appear directly in merge requests as code annotations, showing exactly which line has a vulnerability and its severity. Dependency scanning compares your lockfile (package-lock.json, Gemfile.lock, go.sum) against vulnerability databases. The security dashboard aggregates findings across all projects. Configure vulnerability allowlists to suppress false positives and severity thresholds to block merges on critical findings.

For custom scanning, Trivy integrates easily: add a job that runs trivy image --severity HIGH,CRITICAL --exit-code 1 $IMAGE in the scan stage. Trivy is faster than GitLab's built-in container scanner and produces fewer false positives. Combine with trivy fs for filesystem scanning and trivy config for IaC misconfiguration detection (Terraform, Kubernetes YAML, Dockerfiles). In production, SAST, dependency scanning, and container scanning run on every merge request, and critical findings block the merge until resolved.

Merge request pipelines run on the merge result — the code as it would look after merging, not just the branch. This catches integration issues that branch pipelines miss. Enable with workflow:rules that create pipelines for merge requests and protected branches. Use $CI_MERGE_REQUEST_IID to conditionally run or skip jobs.

Configure merge trains to automatically queue and test merge requests sequentially. Each MR in the train is tested with the combined changes of all MRs ahead of it, ensuring the main branch stays green. If an MR in the train breaks, it is removed and all subsequent MRs are retested. This eliminates the "merge and hope" pattern that causes main branch failures.

In production, merge request pipelines run test, lint, build, and security scan stages. Deployment stages are skipped (they only run on the main branch). Code coverage is displayed as merge request annotations. The merge is blocked if tests fail, coverage drops below 80%, or SAST finds critical vulnerabilities. This automated quality gate catches 90% of issues before human review begins.

Parent-child pipelines let a job in the main pipeline trigger a child pipeline defined in a separate YAML file. The parent pipeline generates or references the child configuration, and GitLab runs it as a linked pipeline. This is essential for monorepos: detect which services changed and trigger only their pipelines, avoiding a single massive pipeline that runs everything on every commit.

Downstream triggers (trigger keyword) start pipelines in other projects entirely. Use them for cross-project dependencies: when the shared library repo is updated, trigger pipelines in all consumer repos to validate compatibility. Combine with strategy: depend so the parent pipeline waits for the downstream pipeline to complete and reflects its status. Pass variables downstream with trigger:variables.

In production, the monorepo containing shared libraries uses parent-child pipelines: the parent detects changed packages, generates a dynamic child pipeline YAML with trigger:include:artifact, and each child pipeline tests only the affected package. For cross-project triggers, updating the your-org/ci-templates repo triggers validation pipelines in 5 critical consumer services to catch breaking changes before they propagate.

GitLab CI integrates with Fastlane for iOS and Android build automation. Fastlane handles code signing, building, testing, and publishing to the App Store and Google Play. In GitLab CI, Fastlane runs inside a macOS runner (for iOS) or a Linux runner with Android SDK (for Android). CI/CD variables store signing keys, provisioning profiles, and store credentials.

The mobile pipeline builds 41 branded Android APKs from a single codebase. Each brand has a different app name, icon, colors, and signing key. A matrix strategy in GitLab CI spawns parallel jobs for each brand, with Fastlane applying brand-specific configuration at build time. The entire pipeline — building, signing, and uploading 41 APKs to Google Play — completes in under 3 hours.

For iOS, Fastlane Match manages code signing certificates and provisioning profiles in a private Git repository. The CI runner fetches credentials via Match, builds the app with fastlane gym, and uploads to TestFlight with fastlane pilot. In production, iOS and Android builds trigger automatically on tagged releases, eliminating the manual 2-week release cycle that existed before CI/CD automation.

The platform runs GitLab CI/CD pipelines for 26 microservices and 41 branded mobile apps. The CI/CD infrastructure is the backbone of the entire development workflow — every code change goes through automated testing, security scanning, Docker building, and Kubernetes deployment without manual intervention.

CI/CD Components and Catalog (GA): CI/CD Components are reusable pipeline configuration units with typed input parameters, evolving beyond the traditional include mechanism. Components are versioned, published to the CI/CD Catalog, and discoverable across the organization. Each project can publish up to 100 components. The Catalog provides a searchable interface for finding and consuming components. Usage: include: component: gitlab.com/org/components/my-component@1.0. This shifts pipeline configuration from copy-paste YAML to proper software distribution with semantic versioning.

Fine-Grained Job Token Permissions (18.3): GitLab now supports least-privilege job tokens with fine-grained permission scoping. Instead of all-or-nothing CI_JOB_TOKEN access, you can define exactly which resources a pipeline token can access. Job tokens can now authenticate Git push operations, enabling pipelines that commit back to the repository (e.g., auto-versioning, changelog generation) without requiring personal access tokens. This eliminates a major security concern in CI/CD pipelines.

Secret Validity Checks (GA in 18.7): When GitLab detects leaked credentials in your repository (via Secret Detection), it now verifies whether those credentials are still active and exploitable. Instead of just flagging potential leaks, GitLab contacts the service provider (GitHub, AWS, GCP, Slack, etc.) to check if the secret is live. Active secrets are flagged as critical, while revoked secrets are marked as resolved. This dramatically reduces alert noise and focuses remediation on actual risks.

SLSA Level 1 for Reusable Components: CI/CD Components published to the Catalog now carry supply chain security attestations conforming to SLSA (Supply-chain Levels for Software Artifacts) Level 1. This provides provenance metadata about how components were built and published, establishing trust in reusable pipeline configurations. Combined with signed pipelines, this creates an auditable chain of trust from component source code to pipeline execution.

GitLab 18.11 (April 16, 2026): The latest release introduces the CI Expert Agent (beta), which examines repositories and proposes complete CI/CD pipeline configurations from natural language descriptions -- no manual YAML required. The Data Analyst Agent is now GA, enabling natural-language queries over project analytics and CI/CD metrics without writing SQL. Agentic SAST Vulnerability Resolution is now GA for Ultimate customers, automatically analyzing security scan results, generating code fixes with confidence scores, and creating merge requests. Incremental Advanced SAST scans analyze only changed code, dramatically reducing scan times. The self-hosted LLM platform list now includes Mistral AI, Azure OpenAI, and custom OpenAI-compatible endpoints alongside AWS Bedrock and Google Vertex AI. Kubernetes 1.35 is supported by the GitLab Agent for Kubernetes. Spending controls for GitLab Credits allow organizations to set subscription-level and per-user monthly caps on AI feature usage.