Here's how anyone can easily use private, open weight models https://micahflee.com/heres-how-anyone-can-easily-use-private-open-weight-models/
> Your very first sentence is going to put off a great many of the people you want to reach.
I don't think it will. People who disagree that LLMs are incredibly useful and powerful probably aren't using Claude and ChatGPT a lot. The people I want to reach use Claude and ChatGPT a lot, because I want to show them that there are better, more private ways.
If you don't think LLMs are useful for you, by all means, don't use them!
For all the other people, please stop sharing all your data with Claude because there are better ways.
@micahflee The problem is that *just by saying that* you're playing into the hands of the companies that you're trying to discourage people from patronizing. You're doing their marketing work for them *and* you're pissing off people like me who are abstractly on your side.
Better framing would be like "the AI companies want you to believe you can't live without their tools but ... they're not really all that, aren't they? And did you know you can get the same results locally? Here's how!"
@zwol I disagree though. I think they are incredibly useful and powerful. They can't just do good work on their own, but if an expert uses them strategically as a tool, it's amazing.
For example, it would would take me a full month to read through every function in a large codebase looking for security bugs, and I would likely miss many. But an LLM can do it in half an hour, and write up a report for me with its findings. I can then review the findings, see if it found anything legit, and start focusing my energy there. At this point, it would be an insane waste of time to not use LLMs when trying to find security bugs in codebases. That's just one example.
Also, my blog post isn't about running LLMs locally. I think the smaller local LLMs can do some cool things, but they're not nearly as capable as the large open weight models which simply can't be run locally unless you happen to have a computer with multiple GPUs and 1TB of RAM. It's about using a provider that uses secure enclaves for big open weight models.
@micahflee disappointing to see you doing free (?) marketing for the plagiarism machine
@micahflee Your very first sentence is going to put off a great many of the people you want to reach.
"It's impossible to deny that, despite all the slop/data centers/environmental destruction/theft, LLMs are incredibly useful and powerful." <-- No. I deny that statement. I do so not out of ignorance, but having carefully experimented with LLMs and studied how they work internally.
(1/2)