Params
unknown
Fastest decode
195.8 tok/s
RTX 5090 · IQ3_S · Strata
Refusal
Official
Setups
47
Best reported pair · not live serving speed ↗
Runs on
Quoted one-stream decode on real boxes.
- RTX 5090IQ3_S195.8 tok/s
- RTX 5090Q2_0191 tok/s
- 2× RTX 3090 24GBIQ3_S158.3 tok/s
- 2× RTX 3090 24GBIQ3_S153.4 tok/s
- RTX 5090 32GBIQ3_S142 tok/s
- M5 Max—126.5 tok/s
- 2× RTX 3090 24GBIQ3_S122.5 tok/s
- RTX 5090UD-Q4_K_XL114.3 tok/s
- RTX 5090 32GBIQ3_S104 tok/s
- RTX 3090 24GBIQ2_XS100.6 tok/s
- RTX 3090 24GBIQ3_S100 tok/s
- 2× RTX 5060 Ti 16GBIQ3_S99.9 tok/s
- RTX PRO 6000Q4_XL92.8 tok/s
- RTX 4090UD-IQ4_XS92 tok/s
- RTX 3090 24GBIQ2_XS87 tok/s
- RTX 5090 32GBNVFP480.9 tok/s
- DGX SparkEXL379.5 tok/s
- RTX 3080 Ti 12GBIQ2_XS79 tok/s
- DGX SparkNVFP474.8 tok/s
- RTX 3090 24GBIQ3_S72 tok/s
- RTX 3090 24GBIQ3_S70 tok/s
- RTX 5070 12GBIQ3_XXS66.7 tok/s
- DGX Spark4-bit QAD65 tok/s
- DGX SparkFP465 tok/s
- 4× Tesla V100 32GBQ4_K_XL-DN460 tok/s
- 2× DGX SparkFP858 tok/s
- Tesla V100 32GBUD-IQ4_XS57 tok/s
- RTX 4060 Ti 16GBIQ3_S55 tok/s
- RTX 4060 Ti 16GBIQ3_S55 tok/s
- Ryzen AI Max+ 395 128GBUD-IQ4_XS53.8 tok/s
- RTX 5070 Ti 16GBIQ3_S53.5 tok/s
- RTX 3080 10GBIQ3_S52.7 tok/s
- 2× AMD MI50 32GBIQ3_S52 tok/s
- DGX Spark4bit50.2 tok/s
- RTX 5080 16GBIQ3_S50 tok/s
- RTX 4060 TiIQ3_S50 tok/s
- 2× DGX SparkFP845 tok/s
- RTX 5090unspecified42.1 tok/s
- M5 Pro 64GB~2.6 bits41.4 tok/s
- 2× RTX 3060 12GBIQ3_S40 tok/s
- RTX 3090 + RTX 2070 SUPERIQ3_S34.2 tok/s
- Intel Arc B570 10GBQ2_032.4 tok/s
- DGX SparkNVFP432 tok/s
- RTX 3060 12GBIQ2_XS27.7 tok/s
- AMD MI50 32GBIQ3_S26.8 tok/s
- Intel Arc B570 10GBQ2_09.0 tok/s
- M5 Max 64GBQ4_K_M7.8 tok/s