Ternary Bonsai 2 27B

i'm running a 27b uncensored model at 50 tok/s on a 12gb rtx 3060 ... the model is ternary bonsai 2 with the heretic uncensoring pass ... the speed is the head i grafted on ... you get 50 tok/s instead of 39 ... the whole thing, weights, head and a 128k context window, sits in 10.7 of the 12 gb. kv cache at q4.

First-person RTX 3060 12GB. Rank is the quoted 50 with the grafted draft head. 39 is the no-draft baseline, not a separate rank. Heretic uncensoring pass on ternary weights. Runtime inferred as llama.cpp from GGUF style.

UncensoredHereticclaimed

@sudoingXHF GGUF