Load Modeling7 min readPublished 2026-11-05

Virtual Users (VU) vs. Requests Per Second (RPS): Choosing the Right Load Model

Understand closed-loop Virtual Users (VU) vs. open-loop Requests Per Second (RPS), Little’s Law concurrency math, think time, and how to avoid Coordinated Omission in API load tests.

Executive Summary
  • Virtual Users (VUs) model a closed queueing system where a new request only starts after the previous response finishes plus optional think time.
  • Requests Per Second (RPS) models an open arrival rate where traffic hits your endpoint independent of how fast the backend responds.
  • When backend p99 latency spikes during a fixed-VU test without rate targets, actual throughput drops automatically—masking saturation unless you monitor both RPS and percentiles.
1. Closed-Loop (Virtual Users) vs. Open-Loop (RPS) Execution

Every HTTP load test makes a fundamental queueing assumption about how traffic arrives at your server. In a closed-loop Virtual User (VU) model, each worker loop simulates a persistent client session: it dispatches an HTTP request, awaits the full response (or socket timeout), pauses for a configured think time, and only then initiates the next request.

In contrast, an open-loop Requests Per Second (RPS) model schedules request arrivals against wall-clock time. Whether your database responds in 12 ms or stalls for 1,400 ms due to lock contention, new requests continue arriving at the target rate until concurrency limits or socket timeouts are reached.

  • Use Virtual Users (VUs) when testing authenticated user journeys, connection-pool sizing, or stateful workflows where a single user cannot issue a second request until the first completes.
  • Use target RPS when benchmarking stateless REST/GraphQL APIs, webhook ingestion endpoints, or public edge routes where thousands of independent clients arrive simultaneously.
  • Combine VUs with a per-VU rate ceiling (maxRps) when you need predictable arrival rates bounded by a strict upper cap on active sockets.
2. Calculating Required Virtual Users with Little’s Law

A common mistake before launching a test is guessing the Virtual User count—for example, configuring 50 VUs and expecting 500 RPS against an endpoint that takes 250 ms to respond. Queueing theory defines the exact relationship between concurrency, throughput, and service time via Little’s Law.

In an HTTP load test, the time a Virtual User stays occupied per cycle is the sum of network + server response time (W) and client think time (Z). If each request takes 200 ms (0.2 s) with zero think time, a single VU can complete at most 5 requests per second. Reaching 500 RPS therefore requires at least 100 concurrent VUs.

VU = λ × (W + Z)

Little’s Law adapted for HTTP load generation with client think time

VU: Concurrent Virtual Users required to sustain target throughput
λ (RPS): Target arrival rate in requests per second
W (seconds): Mean or percentile HTTP response time (e.g., 0.18s for 180 ms)
Z (seconds): Think time pause between consecutive requests per VU
Engineering Takeaway: If your API latency doubles from 150 ms to 300 ms under load, the same 100 VUs will automatically drop from ~666 RPS to ~333 RPS unless additional VUs are available to absorb the inflight queue.
3. Avoiding the Coordinated Omission Trap

Coordinated Omission occurs when the load generator inadvertently coordinates with the system under test by backing off whenever the server slows down. Imagine a target server that freezes for 4 seconds out of every 10-second window due to a stop-the-world garbage collection pause or connection pool exhaustion.

In a purely closed-loop VU test, all active VUs get stuck waiting on the 4-second freeze. While they wait, zero new requests are sent. Once the freeze ends, fast 15 ms responses resume and quickly outnumber the few stalled samples—making your average latency look deceptively healthy while real production traffic would have piled up in an unbounded queue.

Example: LoadTest Engine Configuration (VU + Think Time + SLA Guardrails)
{
  "url": "https://api.example.com/v1/checkout",
  "method": "POST",
  "clients": 50,
  "duration": 120000,
  "thinkTimeMin": 100,
  "thinkTimeMax": 300,
  "maxRps": 10,
  "slaThresholds": {
    "maxP95Latency": 250,
    "maxP99Latency": 600,
    "maxErrorRate": 0.01
  }
}
4. Practical Sizing Workflow Before Production Releases

Instead of running a single massive spike test immediately, structure your validation in three repeatable stages inside your workspace:

  • Baseline Calibration (1–5 VUs, 30s): Measure unloaded p50 and p95 latency to establish W (service time) without queueing delay.
  • Steady-State Capacity Run (Target RPS via Little’s Law, 1–5 min): Verify that p95 and p99 remain flat over time rather than creeping upward due to memory leaks or pool exhaustion.
  • Headroom & Breakpoint Run (Step or Linear Ramp-Up): Gradually increase concurrency until p99 breaches your SLA threshold or HTTP 429/502/504 errors appear.