GGUF. Separate from the 173 tok/s Q4_K_M print.
Params
26B-A4B
Fastest decode
—
Setups
0
Runs on
Quoted one-stream decode on real boxes.
No quoted pair yet.
Get it running
- 1. DownloadPuts the weights on this computer.
huggingface-cli download TrevorJS/gemma-4-26B-A4B-it-uncensored-GGUF - 2. Start itUse this only if the download is GGUF.
llama-server -hf TrevorJS/gemma-4-26B-A4B-it-uncensored-GGUF - Open the weightsHugging Face ↗