Recipe - Create OpenTelemetry based Blueprint - Rafay Product Documentation

Configure

In this section, you will create a standardized cluster blueprint with the OpenTelemetry components. You can then reuse this blueprint for all your clusters.

Assumptions

You have already provisioned or imported a Kubernetes cluster using the controller
Cluster has ingress setup
Cluster has cert-manager setup
Cluster has Prometheus setup with remote write receiver enabled
Elasticsearch to store the traces
RCTL is downloaded and setup

Note

Cert-manager is a requirement for OpenTelemetry. In this example, we will also use cert-manager to issue the certificate for ingress.

Step 1: Create Helm Repository

In this step, you will create Helm repository for Jaeger.

apiVersion: integrations.k8smgmt.io/v3
kind: Repository
metadata:
  name: jaegertracing
  project: defaultproject
spec:
  endpoint: https://jaegertracing.github.io/helm-charts
  type: Helm

Change the project to the name of the project that you will be working on
Save the above Repository spec into a file and apply using RCTL as shown below

rctl apply -f jaeger-repo.yaml

Step 2: Create Namespace

In this step, you will create namespaces for Jaeger and opentelemetry-operator-system.

Jaeger namespace

apiVersion: infra.k8smgmt.io/v3
kind: Namespace
metadata:
  name: jaeger
  project: defaultproject
spec: {}

opentelemetry-operator-system

apiVersion: infra.k8smgmt.io/v3
kind: Namespace
metadata:
  name: opentelemetry-operator-system
  project: defaultproject
spec: {}

Copy both the namespace spec into files and use RCTL apply as shown below

rctl apply -f jaeger-ns.yaml
rctl apply -f opentelemetry-ns.yaml

Step 3: Create Add-ons

In this step, you will create a custom add-on for the below components.

Jaeger
OpenTelemetry Operator
OpenTelemetry Collector

Step 3.1: Create Jaeger Addon

apiVersion: infra.k8smgmt.io/v3
kind: Addon
metadata:
  name: jaeger
  project: defaultproject
spec:
  artifact:
    artifact:
      chartName: jaeger
      repository: jaegertracing
      valuesPaths:
      - name: file://artifacts/jaeger/jaeger-custom-values.yaml
    type: Helm
  namespace: jaeger
  version: v1

Change the project to the name of the project that you will be working on
Save the above Add-on spec into a file

Jaeger Custom Values

storage:
  type: elasticsearch
  elasticsearch:
    host: <Elasticsearch endpoint>
    port: <Elasticsearch port>
    scheme: https
    anonymous: true
    usePassword: false
provisionDataStore:
  cassandra: false
  elasticsearch: false
collector:
  extraEnv:
    - name: METRICS_STORAGE_TYPE
      value: prometheus
    - name: PROMETHEUS_SERVER_URL
      value: "http://prometheus-server.monitoring.svc.cluster.local:80"
query:
  extraEnv:
    - name: METRICS_STORAGE_TYPE
      value: prometheus
    - name: PROMETHEUS_SERVER_URL
      value: "http://prometheus-server.monitoring.svc.cluster.local:80"
    - name: PROMETHEUS_QUERY_SUPPORT_SPANMETRICS_CONNECTOR
      value: "true"
  ingress:
    enabled: true
    hosts:
    - jaeger-demo.dev.rafay-edge.net
    annotations:
      kubernetes.io/ingress.class: nginx
      kubernetes.io/tls-acme: "true"
      cert-manager.io/cluster-issuer: "letsencrypt-demo"
    pathType: ImplementationSpecific
    tls:
    - secretName: jaeger-tls
      hosts:
      - jaeger-demo.dev.rafay-edge.net

Copy the custom values into a file and update the below values based on your environment
- storage.elasticsearch.host
- storage.elasticsearch.port
- storage.elasticsearch.user (Required if your elasticsearch instance is using authentication)
- storage.elasticsearch.password (Required if your elasticsearch instance is using authentication)
- collector.extraEnv (value for PROMETHEUS_SERVER_URL)
- query.extraEnv (value for PROMETHEUS_SERVER_URL)
- query.ingress.hosts
- query.ingress.annotations
- query.ingress.tls.hosts
After making sure all the values are updated, apply the addon spec as shown below

rctl apply -f jaeger-addon.yaml

Step 3.2: Create OpenTelemetry Operator Addon

Download the OpenTelemetry operator as shown below

curl -o opentelemetry-operator.yaml https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml

Create addon for OpenTelemetry Operator

apiVersion: infra.k8smgmt.io/v3
kind: Addon
metadata:
  name: opentelemetry-operator
  project: defaultproject
spec:
  artifact:
    artifact:
      paths:
      - name: file://artifacts/opentelemetry-operator/opentelemetry-operator.yaml
    type: Yaml
  namespace: opentelemetry-operator-system
  version: v1

Change the project to the name of the project that you will be working on
Save the above Add-on spec into a file and apply it using RCTL as shown below

rctl apply -f opentelemetry-operator-addon.yaml

Step 3.3: Create OpenTelemetry Collector Addon

apiVersion: infra.k8smgmt.io/v3
kind: Addon
metadata:
  name: opentelemetry-collector
  project: defaultproject
spec:
  artifact:
    artifact:
      paths:
      - name: file://artifacts/opentelemetry-collector/opentelemetry-collector.yaml
    type: Yaml
  namespace: opentelemetry-operator-system
  version: v1

OpenTelemetry Collector

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: simplest
spec:
  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.79.0
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: "0.0.0.0:4317"
          http:
            endpoint: "0.0.0.0:4318"
    processors:
      batch:
      spanmetrics:
         metrics_exporter: prometheusremotewrite

connectors:
          spanmetrics:
            namespace: span.metrics
            histogram:
              unit: s
              explicit:
                buckets: [10ms, 100ms, 200ms, 400ms, 800ms, 1s, 1200ms, 1400ms, 1600ms, 1800ms, 2s, 5s, 7s]
            dimensions:
              - name: http.method
              - name: http.status_code
              - name: http.target
              - name: http.url
              - name: http.route
    exporters:
      jaeger:
        endpoint: jaeger-collector.jaeger.svc.cluster.local:14250
        tls:
          insecure: true
      prometheusremotewrite:
         endpoint: "http://prometheus-server.monitoring.svc.cluster.local:80/api/v1/write"
    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch,spanmetrics]
          exporters: [jaeger,spanmetrics]
        metrics:
          receivers: [otlp,spanmetrics]
          processors: [batch]
          exporters: [prometheusremotewrite]

Copy the above spec into a file and update the prometheus endpoint based on your environment
After making sure all the values are updated, apply the add-on spec as shown below

rctl apply -f opentelemetry-collector-addon.yaml

Step 4: Create OpenTelemetry Blueprint

In this step, you will create a custom blueprint for OpenTelemetry

apiVersion: infra.k8smgmt.io/v3
kind: Blueprint
metadata:
  name: opentelemetry
  project: defaultproject
spec:
  base:
    name: default
    version: 1.25.0
  customAddons:
  - name: jaeger
    version: v1
  - name: opentelemetry-operator
    version: v1
  - name: opentelemetry-collector
    version: v1
  defaultAddons:
    enableMonitoring: true
  type: custom
  version: v1

Change the project to the name of the project that you will be working on.
Save the above Blueprint spec into a file and apply it using RCTL as shown below

rctl apply -f opentelemetry-blueprint.yaml

Step 5: Apply Blueprint

In this step, you will apply the previously created cluster blueprint to an existing cluster. The blueprint will deploy the OpenTelemetry add-ons to the cluster.

Select Infrastructure -> Clusters
Click the gear icon on the cluster card -> Update Blueprint
Select the previously created OpenTelemetry blueprint and version
Click Save and Publish

Next Step

At this point, you have done everything required to get OpenTelemetry configured and operational on your cluster. In the next step, we will see the traces and metrics in Jaeger UI.