Loading request...
Users are interested in running local LLMs but are intimidated by the complexity and perceived effort required for setup. There's a need for tools or guides that make the process less daunting and clarify the cost/benefit compared to cloud services.
A question I'm getting more often: should I run AI locally on my own computer, or just use Claude or ChatGPT? Like most things, what you want to do guides what’s a good fit. 🥾 You probably don't need hiking boots to walk down a paved road. Are you vibe-coding? Doing research and analysis? Writing reports? Processing sensitive client data? The use case shapes the answer more than anything else. I run both for different things. I use Claude, Perplexity, ChatGPT, and occasionally Grok. I check out whatever's emerging and have my own local LLM. Here are a few high-level things to consider about if you're thinking about running your own LLM: 🎯 What are you trying to do? This is the starting point. Most everyday tasks like writing, research, brainstorming are handled nicely by cloud tools, with zero setup with constantly improving quality (and way better editing, if you're into that). 🔒 How sensitive is your data? If you're working with confidential info, your IP, or info you'd be uncomfortable uploading to a third-party server or having a company like Anthropic or OpenAI train their models on, a local LLM keeps everything on your machine. That said: local doesn't automatically mean risk-free. It's still murky with real unknowns governance-wise. My take: it's meaningfully safer but...it's a spectrum and I'm honestly still understanding what "safe" data looks like with AI. 💻 What hardware are you running? Local models need a reasonably modern machine. A recent Mac with an M-series chip handles it well (that's what I use). Older hardware can struggle, but...it all depends what you're doing. 🐇 How far down the rabbit hole do you want to go? Cloud tools work instantly, update automatically with no/minimal setup. Local models are powerful, but set up is real. I'm also learning that "just one more thing" at 11pm can be a necessary and very real rabbit hole. If you want to understand how AI works from the inside, learning firsthand (and sometimes breaking things within reason) is powerful and reshaping the way I think about engaging with AI. It also helped move me from being a spectator to plucking up, putting on the proverbial squirrel suit, and moving my toes towards the edge to start flying. What's driving your interest? Interested to hear what you're tinkering on and your thoughts on this👇 #futureofwork #rabbithole #AI #LLMs #localLLM