# Pipelines

Integrated GitOps pipelines in the Controller can comprise multiple [stages](https://docs.rafay.co/gitops/stages/overview/). The stages in the pipeline are then executed sequentially one after another.

In the example below, the pipeline has "four stages". It can either be manually run or triggered by updates to a Git repository which will send a webhook to the Controller. Once the manual approval stage is initiated and approval performed by an authorized user, three workloads are deployed in a sequence.

To add a stage in-between the two existing stages/at the end of pipeline/ at the beginning of the pipeline, use the **+** icon available next to/before each stage. User can either use the **+** icon or click **Add New Stage** to add a stage at the end of the pipeline. A map is available at the bottom right of the screen to show the spot of the existing stages.

The image below shows the job details for the multi-stage pipeline that was

- Approved by "[demos@rafay.co](mailto:demos@rafay.co)" on 17/Dec/2020 at 9:38 PM
- The first workload deployment took 12 seconds
- The second workload deployment took 14 seconds
- The third workload deployment took 12 seconds

> **Supported Interfaces**  
> This feature is supported via **UI**, **API**, **RCTL**, **Terraform**, and **GitOps**.

## **Parallel Stage Execution** [¶](https://docs.rafay.co/gitops/pipelines/#parallel-stage-execution "Permanent link")

GitOps pipelines support **parallel execution of multiple stages** within the same pipeline, enabling faster deployments and efficient utilization of resources. This flexibility allows users to configure stages to run **sequentially, in parallel, or in a combination of both**, based on workload dependencies.

With this capability, users can define whether stages should run one after another or simultaneously to optimize deployment time and streamline large-scale rollouts.

### 💡 **Defining Stage Execution Flow** [¶](https://docs.rafay.co/gitops/pipelines/#defining-stage-execution-flow "Permanent link")

To design how stages run, use the **black connector dots** on each stage card in the **Stages** tab of the pipeline.

Click and drag from one stage’s dot to another stage’s dot to define the execution order. The connection line represents a dependency, which means the source stage must complete before the target stage begins.

If stages are not connected, they will run **in parallel**. To remove a connection, click the **✕ icon** on the connector line. This provides full flexibility to mix sequential and parallel execution paths in a single pipeline.

> **Tip:** Use stage connections to control dependencies. If certain workloads can run independently, leave them unconnected for parallel execution. If one workload must complete before another begins, connect the stages accordingly.

### **Example 1: Mixed Sequential and Parallel Execution** [¶](https://docs.rafay.co/gitops/pipelines/#example-1-mixed-sequential-and-parallel-execution "Permanent link")

In this pipeline, there are four stages: **Stage 1**, **Stage 2**, **Stage 3**, and **Stage 4**.

**How it works:**  
Stage 1 runs first. After Stage 1 completes, **Stage 2** and **Stage 3** start simultaneously in parallel. Once both Stage 2 and Stage 3 finish, **Stage 4** begins. Stage 4 will wait until both preceding stages are completed.

**Key Takeaway:**  
Stages 2 and 3 run in parallel to save time. Stage 4 depends on the successful completion of both, providing an efficient mix of sequential and parallel orchestration.

### **Example 2: Fully Parallel Execution** [¶](https://docs.rafay.co/gitops/pipelines/#example-2-fully-parallel-execution "Permanent link")

In this example, the pipeline includes **Stage A**, **Stage B**, and **Stage C**, none of which are connected.

**How it works:**  
Because there are no connectors between the stages, all stages run at the same time. Each workload executes independently without any dependency on others.

**Key Takeaway:**  
Ideal for running unrelated workloads or deployments concurrently, this setup maximizes deployment speed and resource utilization.

### **Example 3: Hybrid Parallel Execution** [¶](https://docs.rafay.co/gitops/pipelines/#example-3-hybrid-parallel-execution "Permanent link")

This example demonstrates a mix of both sequential and parallel behavior.

**How it works:**  
**Stage A** and **Stage C** start in parallel. **Stage B** depends on **Stage A**, so it begins only after Stage A completes. This creates a combination of sequential and parallel paths within a single pipeline.

**Key Takeaway:**  
This configuration provides flexibility to design hybrid pipelines where dependent actions are properly sequenced while independent workloads still run concurrently.

### **Key Benefits** [¶](https://docs.rafay.co/gitops/pipelines/#key-benefits "Permanent link")

| Benefit | Description |
| --- | --- |
| **Faster Execution** | Multiple stages or workloads can run in parallel, significantly reducing total runtime. |
| **Optimized Resource Usage** | Independent actions can utilize resources simultaneously, increasing throughput. |
| **Flexibility** | Combine sequential and parallel execution to fit specific deployment strategies. |

✅ **Best Practice Tip:**  
Always verify the **connection lines** between stages before activation. These define whether your stages will run **in sequence or in parallel**, and a single connection can alter how the entire pipeline executes.

## Security Model [¶](https://docs.rafay.co/gitops/pipelines/#security-model "Permanent link")

Integrated GitOps pipelines in the Controller closely follow the [Projects](https://docs.rafay.co/security/tenancy/projects/) based hard security isolation model. This means that RBAC'ed users that belong to just "Project A" cannot access/view the pipelines in "Project B".

**Organization Admin** and **Project Admin** roles have the privilege to create, read, update and delete pipelines.

## Number of Pipelines [¶](https://docs.rafay.co/gitops/pipelines/#number-of-pipelines "Permanent link")

There are no constraints or limitations (both scale and licensing related) to the number of pipelines that can be used by an organization.

## Create Pipeline [¶](https://docs.rafay.co/gitops/pipelines/#create-pipeline "Permanent link")

To create a new pipeline,

- Navigate to your Project
- Select GitOps Pipelines
- Click on "New Pipeline", provide a friendly name and Save

### Add Stages [¶](https://docs.rafay.co/gitops/pipelines/#add-stages "Permanent link")

Every pipeline contains at least one stage. To add a new stage

- Click **Add New Stage** and provide a name

- Select the action based on the requirement. Refer [stages](https://docs.rafay.co/gitops/stages/overview/) for more info on each action.

**Important**  
User must select only **Deploy Workload Template** action to deploy applications in wavelength zone.

- Select the workload template, namespace, clusters, and other options as per the requirement.

> **Note:** To deploy an application on the wavelength zone, select **alpha.rafay.io/wl-zone** Node Grouping Keys from the **Advanced** drop-down.

- Click **Save**.

### Add Triggers [¶](https://docs.rafay.co/gitops/pipelines/#add-triggers "Permanent link")

Every pipeline contains at least one trigger. To add a new trigger

- Click on Add New Trigger
- Follow the directions for types of [triggers](https://docs.rafay.co/gitops/triggers/)

## View Pipelines [¶](https://docs.rafay.co/gitops/pipelines/#view-pipelines "Permanent link")

To view the list of all pipelines in a project

- Navigate to your Project
- Click on GitOps pipelines

In the example shown below, there are four active pipelines. The "nginx-gitops" pipeline has been triggered and run 87 times.

## Activate/Deactivate Pipeline [¶](https://docs.rafay.co/gitops/pipelines/#activatedeactivate-pipeline "Permanent link")

Administrators can deactivate a pipeline either temporarily or permanently. When a pipeline is deactivated, triggers received are ignored. However, it is still possible to manually run the pipeline.

Click on Activate to make the pipeline active. The current activation status of a pipeline can be viewed for every pipeline.

**Important**  
Pipelines triggered via the API (V2) using curl might result in errors. The issue arises when the **_Require console login before kubectl access_** option is enabled in user settings, which mandates API authentication before granting kubectl access. To resolve this, users should disable this option to ensure seamless pipeline execution.

## Manage Sharing Pipelines [¶](https://docs.rafay.co/gitops/pipelines/#manage-sharing-pipelines "Permanent link")

The existing pipeline can be shared to All/Specific/None projects. This helps to use the configured resource specs to infuse to the new pipeline if required or change the required resources.

## Edit Pipeline [¶](https://docs.rafay.co/gitops/pipelines/#edit-pipeline "Permanent link")

Authorized users can edit and update existing pipelines anytime.

- Click on the pipeline
- Click on Edit
- The user can now add/remove/update stages and triggers

## Delete Pipeline [¶](https://docs.rafay.co/gitops/pipelines/#delete-pipeline "Permanent link")

Click on Delete, acknowledge the prompt to permanently delete the pipeline. Note that this action is destructive and cannot be undone.

## Manual Jobs [¶](https://docs.rafay.co/gitops/pipelines/#manual-jobs "Permanent link")

If required, authorized users can launch a pipeline job "manually" by clicking on "Run".

### Trigger Events [¶](https://docs.rafay.co/gitops/pipelines/#trigger-events "Permanent link")

Trigger Events shows the pending events in queue and helps to debug the status of the event. Every time a job is triggered, initially, the events gets into pending status.

Users can filter the events by different statuses. **Failed**, **Skipped**, **Retry**, **Pending**, and **Success** are the five (5) statuses of Trigger events.

Once an event is moved to success state, it is converted to a job. Click the job id to view the job details. To view the reason for failed/skipped events, click the status icon.

Events that are triggered externally via webhook will have a red icon as shown below. Click the icon to view the changes committed in the repository and the **eye** icon to view the payload received from Git platform.

## View Pipeline Jobs [¶](https://docs.rafay.co/gitops/pipelines/#view-pipeline-jobs "Permanent link")

Every pipeline job's details (i.e. run) are visualized for users. Just select and click on your pipeline name. In the example below, the reverse chronological list of jobs, their status, total run time and start time are presented for quick comparison.

To view detailed information about a job, click on the "Job #". In the example below, clicking on "Job 20" provides the user detailed information about the pipeline.

Explore our blog for deeper insights on **Create Git Pipeline in 3 easy steps**, available [here](https://docs.rafay.co/blog/2023/06/14/create-git-pipeline-on-rafay-in-3-easy-steps/)!

### View Sequential and Parallel Stage Execution [¶](https://docs.rafay.co/gitops/pipelines/#view-sequential-and-parallel-stage-execution "Permanent link")

The Job Details page displays how stages within a pipeline were executed based on their configured sequence and dependencies.

In this example, the pipeline includes **Stage1**, **Stage2**, **Stage3**, and **Stage4**, each performing the **DeployWorkload** action for their respective workloads.

**Execution summary:**
1. **Stage1** executed first and deployed **Workload w1**. After completion, it triggered **Stage2** and **Stage3**.
2. **Stage2** and **Stage3** executed **in parallel**.
3. **Stage2** deployed **Workload w2**.
4. **Stage3** deployed **Workload w3**. 
Both stages ran independently without waiting on each other.
5. Once **Stage2** completed successfully, it triggered **Stage4**, which deployed **Workload w4**.
6. All stages completed successfully, and the overall job status is displayed as **Success**.