name: coreweave-performance-tuning description: 'Optimize CoreWeave GPU inference latency and throughput.

Use when reducing inference latency, maximizing GPU utilization,

or tuning batch sizes and concurrency.

Trigger with phrases like "coreweave performance", "coreweave latency",

"coreweave throughput", "optimize coreweave inference".

' allowed-tools: Read, Write, Edit, Bash(kubectl:*) version: 1.0.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io tags:

• saas
• gpu-cloud
• kubernetes
• inference
• coreweave compatibility: Designed for Claude Code

CoreWeave Performance Tuning

GPU Selection by Workload

Inference Optimization

# Continuous batching with vLLM
containers:
  - name: vllm
    args:
      - "--model=meta-llama/Llama-3.1-8B-Instruct"
      - "--max-num-batched-tokens=8192"
      - "--max-num-seqs=256"
      - "--gpu-memory-utilization=0.90"
      - "--enable-prefix-caching"
      - "--dtype=float16"

Autoscaling Tuning

# HPA based on GPU utilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: inference-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: inference-server
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Pods
      pods:
        metric:
          name: DCGM_FI_DEV_GPU_UTIL
        target:
          type: AverageValue
          averageValue: "70"

Performance Benchmarks

Resources

• CoreWeave Inference
• vLLM Documentation

Next Steps

For cost optimization, see coreweave-cost-tuning.