Post by @scottjenson@social.coop

2026-07-21 09:01 UTC

I've found LLM tech to be useful in limited, controlled conditions. Its current mass deployment is hard to support for ethical and environmental reasons. However, I want to keep exploring how best to use it, and of course, when NOT to use it. I'm not a freaking fanboi here. This tech has serious limits! I've now installed pi.dev so I can work with a local model for small things, an open model for medium things, and a frontier model (which I'd rather not use) for a limited number of high-level tasks. Is this perfect? Of course not; it's a process. As open and local models improve, I'll transition over. IMHO, it's inevitable that I'll eventually be 100% local, but it might take a few years. This setup allows me to keep working with and learning the pros and cons of the tech while steadily reducing my reliance on frontier models.

Replies (3)

@dahukanna@mastodon.social 2026-07-21 09:46

@scottjenson@social.coop I’ve taken a “local-first” approach to explore LLM models using ollama. Main difference to frontier & other models is application interface “glue & API” code that takes care of smoothing interactions, managing “so called memory/context window”, Retrieval Augmented Generation (RAG) pipeline, vector databases grounding in recent factual data, etc. For a technology touted as “intelligent”, it requires a huge amount of “tech assistance” to function semi-decently for a human to operate.🤷🏾‍♀️

Open ##3983930
@fazalmajid@social.vivaldi.net 2026-07-21 09:34

@scottjenson@social.coop Pi is an excellent choice. Sadly the limiting factor to your adoption of local models will be the price of RAM, not your learning curve... If you want to play around with semi-autonomous agents, you might also want to look at Hermes, just make sure you sandbox it heavily: https://hermes-agent.nousresearch.com/docs/

Open ##5320179
@tomasino@tilde.zone 2026-07-21 09:49

@scottjenson@social.coop I'm also doing local model stuff. I'd recommend building and expanding your library of mcps to cover regular topics you manage. It's a great boon for sturdier outputs and it drastically cuts context usage.

Open ##5320190