← All posts

CMP 90HX for LLM: why a mining GPU slows down and what “six-seven” fixes

MoE tolerates CMP 90HX; dense models don't — compute paths are capped. Driver mod via V67 nearly doubles throughput but doesn't erase mining-card limits.

Tags
AINewsgpuhardwarellm
CMP 90HX for LLM: why a mining GPU slows down and what “six-seven” fixes
Contents

In brief

An author built an LLM server from CMP 90HX cards (NVIDIA Crypto Mining Processor on Ampere) expecting fast inference. Dense models were far slower than expected; MoE looked acceptable because only part of the network runs per token. The issue goes deeper than llama.cpp tuning: the card is selectively capped on compute paths. A driver modification (pearlfortune's V67 chain) yields nearly 2× gains but does not turn CMP into a full RTX.

What happened

CMP 90HX targets Ethereum-class mining — high memory bandwidth, not full FP32 / INT8 / tensor throughput for general workloads.

In practice:

  • Dense models run all layers every token — limits show immediately.
  • MoE models activate expert subsets — the problem is partly hidden.
  • Two inference phases behave differently: prefill (input processing) vs decode (token generation). Prefill hurts more when matrix paths are capped.

The author reproduced an unlock flow: at early GPU boot, the internal Booter briefly receives a substituted signature buffer (pSignatureMemdesc → V67). That opens FEAT_OVR_PLM, allowing full compute mode selectors (SS0, SS1). After FLR reset, the card starts without throttled paths.

Software workarounds (replacing DP4A with IMAD, shifting work to HFMA2) speed specific kernels but leave the root cap for other code.

Why it matters

Cheap used CMP cards tempt home LLM clusters: lots of VRAM, low price. This post is a blunt reminder: silicon ≠ suitability. NVIDIA's mining segmentation is product policy, not a firmware accident.

Engineers learn why prefill and decode must be measured separately, why MoE benchmarks mislead, and why driver patches are high risk (stability, updates, warranty, safety).

In practice

  1. Do not buy CMP "for LLM" without dense-model benchmarks at your target size.
  2. Split metrics: prefill (time to first token) vs decode (tokens/s).
  3. MoE may look "fine" where dense is already unacceptable — do not extrapolate.
  4. Driver mods are experimental — the author does not ship a production how-to.
  5. For stable inference, budget RTX / datacenter GPUs with normal CUDA stack support.

Takeaway

CMP 90HX + LLM is a story of uneven performance and product segmentation, not "cheap 10 GB VRAM." The six-seven / V67 method proves the cap lives in firmware/driver — with real but not magical gains. Most teams should skip fighting mining cards and buy inference-ready hardware upfront.