## [NVIDIA NIM: Why It Matters—and How It Stacks Up](https://docs.rafay.co/blog/2025/10/20/nvidia-nim-why-it-mattersand-how-it-stacks-up/) Generative AI is moving from experiments to production, and the bottleneck is no longer training—it’s serving: getting high-quality model inference running reliably, efficiently, and securely across clouds, data centers, and the edge. NVIDIA’s answer is [NIM (NVIDIA Inference Microservices)](https://www.nvidia.com/en-us/ai-data-science/products/nim-microservices/). NIM is a set of prebuilt, performance-tuned containers that expose industry-standard APIs for popular model families (LLMs, vision, speech) and run anywhere there’s an NVIDIA GPU. Think of NIM as a “batteries-included” model-serving layer that blends TensorRT-LLM optimizations, Triton runtimes, security hardening, and OpenAI-compatible APIs into one deployable unit. ## [Deploy Workload using DRA ResourceClaim in Kubernetes](https://docs.rafay.co/blog/2025/09/16/deploy-workload-using-dra-resourceclaim-in-kubernetes/) In the [**first blog**](https://docs.rafay.co/blog/2025/08/23/introduction-to-dynamic-resource-allocation-dra-in-kubernetes/) in the DRA series, we introduced the concept of Dynamic Resource Allocation (DRA) that recently went GA in [Kubernetes v1.34](https://kubernetes.io/blog/2025/08/27/kubernetes-v1-34-release/) which was released end of August 2025. In the [**second blog**](https://docs.rafay.co/blog/2025/08/28/enable-dynamic-resource-allocation-dra-in-kubernetes/), we installed a Kubernetes v1.34 cluster and deployed an example DRA driver on it with "simulated GPUs". In **this blog**, we’ll deploy a few workloads on the DRA enabled Kubernetes cluster to understand how "Resource Claim" and "ResourceClaimTemplates" work. Info We have optimized the steps for users to experience this on their laptops in less than 5 minutes. The steps in this blog are optimized for macOS users. ## [GPU/Neo Cloud Billing using Rafay’s Usage Metering APIs](https://docs.rafay.co/blog/2025/09/13/gpuneo-cloud-billing-using-rafays-usage-metering-apis/) Cloud providers offering GPU or Neo Cloud services need accurate and automated mechanisms to track resource consumption. Usage data becomes the foundation for billing, showback, or chargeback models that customers expect. The Rafay Platform provides usage metering APIs that can be easily integrated into a provider’s billing system. In this blog, we’ll walk through how to use these APIs with a **sample Python script** to generate detailed usage reports. ## [Enable Dynamic Resource Allocation (DRA) in Kubernetes](https://docs.rafay.co/blog/2025/08/28/enable-dynamic-resource-allocation-dra-in-kubernetes/) In the previous [**blog**](https://docs.rafay.co/blog/2025/08/23/introduction-to-dynamic-resource-allocation-dra-in-kubernetes/), we introduced the concept of Dynamic Resource Allocation (DRA) that just went GA in [Kubernetes v1.34](https://kubernetes.io/blog/2025/08/27/kubernetes-v1-34-release/) which was released in August 2025. In this blog post, we’ll configure DRA on a Kubernetes 1.34 cluster. Info We have optimized the steps for users to experience this on their macOS or Windows laptops in less than 15 minutes. The steps in this blog are optimized for macOS users. ## [NVIDIA Performance Reference Architecture: An Introduction](https://docs.rafay.co/blog/2025/08/26/nvidia-performance-reference-architecture-an-introduction/) Artificial intelligence (AI) and high-performance computing (HPC) workloads are evolving at unprecedented speed. Enterprises today require infrastructure that can scale elastically, provide consistent performance, and ensure secure multi-tenant operation. NVIDIA’s Performance Reference Architecture (PRA), built on HGX platforms with Shared NVSwitch GPU Passthrough Virtualization, delivers precisely this capability. This is the introductory blog in a multi part series. In this blog, we explain **why PRA is critical** for modern enterprises and service providers, highlight the benefits of adoption, and outline the key steps required to successfully deploy and support the PRA design/architecture. ## [Deep Dive into `nvidia-smi`: Monitoring Your NVIDIA GPU with Real Examples](https://docs.rafay.co/blog/2025/08/24/deep-dive-into-nvidia-smi-monitoring-your-nvidia-gpu-with-real-examples/) Whether you're training deep learning models, running simulations, or just curious about your GPU's performance, **`nvidia-smi`** is your go-to command-line tool. Short for **NVIDIA System Management Interface**, this utility provides essential real-time information about your NVIDIA GPU’s health, workload, and performance. In this blog, we’ll explore what `nvidia-smi` is, how to use it, and walk through a **real output** from a system using an **NVIDIA T1000 8GB GPU**. * * * ### [What is `nvidia-smi`?](https://docs.rafay.co/blog/2025/08/24/deep-dive-into-nvidia-smi-monitoring-your-nvidia-gpu-with-real-examples/#what-is-nvidia-smi) `nvidia-smi` is a CLI utility bundled with the NVIDIA driver. It enables: - Real-time **GPU monitoring** - **Driver and CUDA version** discovery - Process visibility and control - GPU configuration and performance tuning You can execute it using: ``` nvidia-smi ``` ## [Introduction to Dynamic Resource Allocation (DRA) in Kubernetes](https://docs.rafay.co/blog/2025/08/23/introduction-to-dynamic-resource-allocation-dra-in-kubernetes/) In the previous [**blog**](https://docs.rafay.co/blog/2025/08/20/rethinking-gpu-allocation-in-kubernetes/), we reviewed the limitations of Kubernetes GPU scheduling. These often result in: 1. **Resource fragmentation** – large portions of GPU memory remain idle and unusable. 2. **Topology blindness** – multi-GPU workloads may be scheduled suboptimally. 3. **Cost explosion** – teams overprovision GPUs to work around scheduling inefficiencies. In this post, we’ll look at how a new GA feature in [Kubernetes v1.34](https://kubernetes.io/blog/2025/07/28/kubernetes-v1-34-sneak-peek/) — **Dynamic Resource Allocation (DRA)** — aims to solve these problems and transform GPU scheduling in Kubernetes. ## [Rethinking GPU Allocation in Kubernetes](https://docs.rafay.co/blog/2025/08/20/rethinking-gpu-allocation-in-kubernetes/) Kubernetes has cemented its position as the de-facto standard for orchestrating containerized workloads in the enterprise. In recent years, its role has expanded beyond web services and batch processing into one of the most demanding domains of all: **AI/ML workloads**. Organizations now run everything from lightweight inference services to massive, distributed training pipelines on Kubernetes clusters, relying heavily on GPU-accelerated infrastructure to fuel innovation. **But there’s a problem**. In this blog, we will explore why the current model falls short, what a more advanced GPU allocation approach looks like, and how it can unlock efficiency, performance, and cost savings at scale. ## [GitOps Without Borders: Running Argo CD Across Isolated Security Domains with Rafay’s Zero-Trust Kubectl](https://docs.rafay.co/blog/2025/08/06/gitops-without-borders-running-argo-cd-across-isolated-security-domains-with-rafays-zero-trust-kubectl/) Modern enterprises rarely run applications in a single cluster. A production fleet might include on-prem clusters in Singapore and London, a regulated environment in AWS us-east-1, and a developer sandbox in someone’s laptop. **GitOps with Argo CD** is the natural way to keep all those clusters in the desired state—but the moment clusters live in different _security domains_ (fire-walled data centers, private VPCs, or even air-gapped networks) the simple `argocd cluster add` story breaks down: - Bespoke bastion hosts or VPN tunnels for every hop - Long-lived bearer-token Secrets stashed in Argo’s namespace - High latency between the GitOps engine and far-flung clusters, turning reconciliations into a slog **Rafay’s Zero-Trust Kubectl Access (ZTKA)** solves all three problems in one stroke. By front-loading the connection with a hardened **Kube API Access Proxy**—and issuing _just-in-time_ (JIT), short-lived ServiceAccounts inside every cluster. > In this blog, we will describe how Rafay Zero Trust Kubectl Access Proxy gives Argo CD a secure path to every cluster in the fleet, even when those clusters sit deep behind corporate firewalls. ## [Turbo-charging kubectl: How Rafay’s Zero-Trust Access + Regional Proxies Deliver Lightning-Fast CLI Performance](https://docs.rafay.co/blog/2025/08/06/turbo-charging-kubectl-how-rafays-zero-trust-access--regional-proxies-deliver-lightning-fast-cli-performance/) When developers are halfway around the world from their clusters, every kubectl get pods can feel like it’s moving through molasses. Rafay’s Zero-Trust Kubectl (ZTKA) service fixes the security risks and the lag by adding a network of regional proxies between the user and the cluster. **Zero-Trust Kubectl in a Nutshell** Rafay ZTKA routes all CLI and web-terminal traffic through its Kube API Access Proxy. The key design goals are: 1. Friction-free for users (“vanilla kubectl”), 2. Zero infrastructure to manage for platform teams, 3. Centralized RBAC + audit, and “great performance” even for clusters behind firewalls. > Under the hood, users authenticate to Rafay; Rafay spins up just-in-time service accounts inside the target cluster and tears them down after idle timeouts, eliminating credential sprawl.