I've been at this crossroads for months. On the one hand, the insulting convenience of opening ChatGPT or Claude and having the world's most intelligent model just a click away. On the other, the unease of sending all my data, drafts, and code to third-party servers.
The AI craze has divided us into two camps: those who pay the subscription without looking back and those who buy graphics cards with lots of VRAM to "own" their intelligence.
I've thoroughly explored both worlds. I've set up workflows with Ollama and LM Studio and I've burned through API credits OpenAI and Anthropic.
This is the reality, without unnecessary technicalities, of running LLMs locally versus in the cloud.
The Cloud: Raw power at the swipe of a credit card (SaaS)
When you use GPT-4o, Claude 3.5 Sonnet or Gemini 1.5 Pro, you're renting a giant brain.
The good
- Superior Intelligence: To this day, no programming language you can run on a consumer laptop comes close to the logical reasoning capabilities of Claude 3.5 Sonnet or GPT-4. For writing complex code or performing deep logical reasoning, the cloud still wins.
- Zero Friction: You don't install anything. You don't configure environments. You don't worry about whether your fan sounds like a plane taking off. It's pure productivity.
- Real Multimodality: Analyzing images and large files usually works much better and faster on their servers.
Bad
- Privacy: It's the elephant in the room. Everything you write, unless you use very specific Enterprise configurations, could To be used to train future models. Financial data? Proprietary code? Better not.
- Recurring cost: The €20/month or the API usage fee is cumulative.
- Censorship and Guardrails: Cloud-based models tend to be more "politically correct" and refuse to answer certain questions due to excessive security filters.
Local LLMsThe Privacy Rebellion
This is where players like Flame 3 (Goal), Mistral, Gemma (Google) o Phi (Microsoft). Tools such as Ollama They have democratized this to an absurd level.
The good
- Absolute Privacy: You can unplug the network cable and the AI will continue to function. Your data never leaves your machine. This is invaluable for legal or personal matters.
- Uncensored: You can download "uncensored" versions of the models if you need the AI to not lecture you on morality when writing a work of fiction.
- Zero Latency (almost): If you have the hardware, the response is instantaneous. There are no waiting queues or "the server is overloaded" messages.
- Free (once you have the hardware): Downloading and running Llama 3 is free.
The reality check: The Hardware
This is where many hit a wall. Local AI devours VRAM (Video Memory) and RAM.
My personal experience:
- With 8GB of RAM: Forget it. You'll only be able to run very small and "dumb" models (very aggressive quantizations).
- With 16GB of RAM/VRAM: The entry sweet spot. You can run 7B or 8B parameter models (like Llama 3 8B) at a decent speed.
- Mac with Apple Silicon (M1/M2/M3): This is where the game changes. Macs' unified memory allows for a significant amount of RAM to be allocated to the GPU. A Mac with 32GB or 64GB is a beast for local AI.
Technical note: If you try to run a model larger than your VRAM, the system will use regular RAM and the CPU. The result: your computer will generate 1 word per second. Unusable for work.
My verdict and hybrid workflow
You don't have to choose one. The key to productivity is knowing. when to use what.
After many tests, this is my current system:
1. Local Use (Ollama + Obsidian/VS Code)
I use it for repetitive, fast, or sensitive tasks.
- Summarize personal notes: I don't want my journals or business strategies in the cloud.
- Basic style correction: A local Llama 3 is excellent at rewriting emails or paragraphs.
- Simple code copilot: Fast autocomplete that doesn't require full project context.
2. Cloud Usage (Claude 3.5 Sonnet / GPT-4o)
I reserve it for "heavy work".
- Software Architecture: When I need to plan a complex database structure.
- Logical Reasoning: «"Where is the flaw in this argument?"»
- Complex code generation: When I need an entire Python script that works the first time.
Conclusion
If you have a computer with a good GPU (NVIDIA RTX 3060 or higher) or a Mac with Apple Silicon, Install Ollama today!. Having a local, private, and fast assistant for minor tasks changes your workflow and gives you an extra layer of security.
But let's not kid ourselves: if you need genius-level reasoning to solve a complex problem, Keep paying for your subscription. Local AI is advancing rapidly, but cloud giants still hold the crown of raw intelligence.
The question is: How much do you value your privacy versus convenience?