The power of LLMs can not be understated. Their contribution to individual sovereignty, preparedness, and potential for individual freedom is rivaled only by guns and decentralized digital cash. For planning, development, debugging, and research, being able to run your own offline intelligence stack is as revolutionary as gunpowder weapons were in the middle ages to individuals who were not able to afford armies and cavalries. AI levels the playing field once more.
Uncensored or abliterated models raise the bar even further. These models have had their guardrails surgically removed, meaning that something they may have refused to answer before citing legal or safety reasons, they now answer casually without hesitation.
I find it very interesting, that alongside predicting the creation and rise of Monero, "The Soverign Indiviual" also predicts the use AI agents by sovereigns, or as its referred to in the book, "autonomous agents" and "intelligent agents".
"The individual will no longer be singular, but potentially an ensemble of dozens or perhaps even thousands of activities undertaken through intelligent agents. This will not only greatly enhance the productive capability of the most talented individuals; it will also make the Sovereign Individual potentially far more formidable militarily than the individual has ever been before."
"Not only will one individual be able to manifestly multiply his activities by employing an essentially unlimited number of intelligent agents. He or she will even be able to act after death. For the first time, an individual will be capable of carrying on elaborate tasks even if he is biologically dead. It will no longer be possible for either an enemy at war or a criminal to completely extinguish the capability of an individual to retaliate by killing him. This is one of the more revolutionary innovations in the logic of violence in the whole of history."
AI is a double edged sword. It is both an offensive and defensive tool, which is why corporations and the state may have an interest in its regulation in the not so distant future. Not that I am particularly concerned about their ability to actually be able to control open source models or compute, not any more than they can control the spread of truly decentralized digital money (they can't). Nevertheless, recognizing the power local LLM's possess and the threat it poses to the power structures may allow you to better position yourself, start taking advantage of its myriad of benefits and act to ensure you remain capable now and in the future of truly owning that intelligence now and forever.
OpenAI and Anthropic have spent millions lobbying US Congress to restrict open weights. They pressed Washington on restrictions involving treating weights as sensitive IP, limiting Chinese open-weight models, stronger export controls on chips, and measures against large-scale distillation.
...
Anthropic CEO Dario Amodei testified before the U.S. Senate Judiciary Subcommittee on Privacy, Technology, and the Law. The hearing was titled “Oversight of A.I.: Principles for Regulation” and took place on Tuesday, July 25, 2023:
“If the path continues, I think we could get to a very dangerous place.”
“When you control a model and you're deploying it, you have the ability to monitor its usage.”
“You can revoke a user's access. You can change what the model is willing to do.”
“When a model is released in an uncontrolled manner, there's no ability to do that. It's entirely out of your hands.”
...
In July of this year, the former CEO of NVIDIA, Jensen Huang, published a letter in support of open models, stating "Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty."
https://images.nvidia.com/pdf/Open-Weights-and-American-AI-Leadership.pdf
Now the heaviest fighter for open source models, Jensen, is conveniently removed from NVIDIA, and the biggest winners are OpenAI and Anthropic.
...
And the CEO of OpenAI, Sam Altman, thinks its "quite terrible" and wasteful that individuals have ram sitting in their computers at home, because the RAM could be better utilized in data centers owned by them instead.
...
Running cloud AI comes with a significant lower up-front cost. The real cost is your privacy and sovereignty. Every time you send a prompt to cloud AI, the real first question you ask is for permission to access that intelligence under a watchful eye.
...
What models to run:What models to run:
What you can run will depend on your hardware. VRAM being the most important resource, is where the model weights are loaded. You'll get the best performance if you can fit the entire model in your graphic card's available VRAM. RAM is used for any parts of the model that can be offloaded to the CPU. In some inference engines, you can additionally stream certain parts of a large model from your NVMe storage.
You should always go for the highest quant that will fit in memory. 4-bit quants are the sweet spot, preserving at least 90-95% of the original accuracy of the model.But if you can't fit a 4-bit quant, you can to drop down to 3, 2 or even 1-bit quants, or another entirely smaller model. The higher the quantization, the better the quality of your outputs.
...
4GB to 8GB VRAM
https://huggingface.co/LiquidAI/LFM3-1.7B-GGUF
https://huggingface.co/LiquidAI/LFM2.5-2.6B
Gemma-4-E4B-Uncensored
8GB to 16GB VRAM
Saluki-27B
Gemma4-12B-Uncensored
Bonsai-2-27B
16GB to 32GB VRAM
https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
https://huggingface.co/HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
https://huggingface.co/unsloth/Qwen3.8-27B-GGUF - Best in class
Via the Strata engine, with at least 16GB+ of VRAM, 32GB+ of RAM, and an NVMe SSD:
Via the Maya engine, with 24GB+ VRAM recommended, + RAM and an NVMe SSD
GLM-5.3-Flash or DeepSeek v4.1 Flash, see Recommended Models
96GB+ VRAM
higher quants of:
https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF
192GB to 384GB
Whatever the fuck you want. You are king/queen.
...
This model recommendation list is likely obsolete in the next 3 months if not sooner. The space moves rapidly. Qwen4-27b is expected to be released within days or weeks. The capabilities of local models continues to rise. At the same time, the hardware requirements to run these local models continues to drop, and software/firmware continues to improve. We can now run agents as capable as the most advanced closed source models were less than a year ago, entirely on our own machines, in full privacy.
I will make one final suggestion, that if you absolutely are unable to afford a higher hardware tier for more capable AI, use http://venice.ai. Venice either anonimizes your session, or you can access certain models under a TEE (Trusted Execution Environment) which is end to end encrypted. They also serve uncensored models. This is a decent compromise until you are able to own your own stack. But if the power, internet, or venice is down, so is your ability to use these models. Only local advances and protects your independence on third parties.
Do you already run your own local intelligence stack? Let's hear what hardware you've found to work for your needs, and what models you would recommend to others for different use cases and hardware.
THIS take, my friends… is the SOLE reason they took out Jensen Huang