Scaling and tuning

This page collects the worker-level tuning knobs — the things you set once on the endpoint, not per request. For per-request options (backend selection, page range, output format), see API reference.

All values listed here are set as environment variables on the RunPod endpoint, not in the job payload. You'll find them under the endpoint's Environment Variables section in the RunPod dashboard or the RunPod Hub deploy form. The current deploy.py does not configure environment variables.

Concurrency

MINERU_MAX_CONCURRENCY is the in-worker concurrency knob — one of two concurrency levers. For how it interacts with workers_max and the request queue (and which to reach for), see Concurrency. It controls how many jobs a single worker handles in parallel. Default 1.

MinerU's own mineru-api server defaults this to 3, but that's a guidance value tuned for their full-fat hosted setup — for a generic serverless template across all of RunPod's GPU pool, 1 is the safe baseline.

Why not just always run higher concurrency? MinerU's VLM backend wraps vLLM, which pre-allocates a large KV cache at engine init. Two concurrent jobs share that cache; on a 16 GB GPU you OOM around 2 concurrent VLM parses. The pipeline backend is more frugal but still bound by per-page VRAM. See Choosing a GPU for the memory math.

Worker recycling

Long-lived workers accumulate VRAM fragmentation and Python heap growth from MinerU + vLLM. After hundreds of parses, the same job may run noticeably slower than on a fresh worker. Two thresholds opt into automatic recycling:

Both default to 0 (disabled). When a threshold crosses, the worker attaches refresh_worker: true to the response — RunPod's runtime then kills the worker after the response is delivered. The next request lands on a fresh worker (with a cold-start cost; pair with FlashBoot to keep that cheap).

Counter rules

• Both thresholds active: whichever trips first wins.
• Unbounded parses (end_page=-1, the default): contribute 1 to the jobs counter and 0 to the pages counter. Use the jobs threshold if you mostly do full-document parses.
• Errors and probes: don't increment either counter. Recycling is for memory hygiene, not error recovery.

Bounding the work per job

The endpoint's execution timeout is the backstop on a single job: a request for more pages than the GPU can chew through inside it fails after spending those GPU minutes. MINERU_MAX_PAGES_PER_JOB turns such a request away instead, before any work starts:

MINERU_MAX_PAGES_PER_JOB=200

A job asking for a larger range is rejected with the requested size and the ceiling in the message, so the caller can re-slice. Default 0 — no ceiling.

This applies to explicit ranges only. An open-ended request (end_page=-1, the default) has no page count until MinerU opens the document, so set this only where callers send ranges — batching a large PDF into slices is the pattern it fits, and the shape MineruClient callers already use for long documents.

FlashBoot

Our measurements indicate that FlashBoot restores process state, including the warm vLLM child process and GPU allocations, after scale-to-zero. RunPod does not document the exact snapshot implementation or scoping, so treat the host-and-image model below as an evidence-based operating model rather than a public platform guarantee. Enable FlashBoot in the endpoint configuration; it is on by default for Hub-deployed templates.

The boot-time warmup loads the MinerU model into VRAM before serverless.start() runs. In our test, the fast FlashBoot path restored that warm state: ~7-8 s wall-clock instead of ~110 s for a fresh boot.

The observed scope was worker host plus image SHA, not endpoint. In the four-request test, the first run on a host paid the full ~110 s warmup and a later run in the same host context took the fast path. Moving to another host repeated the warmup once. See Troubleshooting → observed FlashBoot behavior for the four-request investigation that pinned down this behavior.

Practical operator math: on endpoints with high enough traffic that the same hosts keep getting re-selected, you'll see almost all ~7-8 s cold starts. On quiet endpoints with long idle gaps, expect a mix — RunPod's scheduler may bounce between hosts and you'll see some ~110 s starts before each host warms up.

If FlashBoot is off, every cold start runs the ~110 s warmup, with no per-host amortization at all. Keep it on unless you have a specific reason to disable it.

To bypass the warmup entirely (e.g., for debugging cold-start ordering), set MINERU_SKIP_WARMUP=1 on the endpoint. The worker will fall back to lazy load, paying ~110-130 s on every cold start regardless of host history.

MinerU performance knobs

These pass straight through to MinerU's own configuration. Leave empty to use MinerU's defaults.

For the hybrid-http-client backend specifically, MinerU exposes one more knob: MINERU_HYBRID_BATCH_RATIO (defaults to ~4) for VRAM reduction. Only relevant if you're pointing the worker at an external vLLM server with limited VRAM.

Worker logs

LOG_FORMAT selects the worker's log output:

• json (default) — one JSON object per line, with a job_id field on every emission for cross-job correlation. Easier to filter in RunPod's log viewer, CloudWatch, Loki, or Axiom.
• text — human-readable, key=value pairs after the message. Useful for local development with runpod.serverless.start running on your laptop.

See Troubleshooting → reading worker logs for the field reference.

Two notes for production deployments:

• Throttling: RunPod drops logs from workers producing too much output. The worker's defaults (~4 lines per job) are well under any practical ceiling, but verbose forks may lose data.
• External shipping is opt-in: set OTEL_EXPORTER_OTLP_ENDPOINT to mirror structured logs and export traces and metrics to an OTLP/HTTP backend. The worker already includes the exporter; see OpenTelemetry for setup and verification.

Summary