RE: https://neuromatch.social/@jonny/117432015798538958
As a consumer product, muse must be able to browse the web to do all the things it promises that magical "agentic ai" can do. Meta needs Muse to work, badly, because it has 2 trillion dollars on the line and is a laughingstock in the AI race. Problem though, there have been crawlers and bots for as long as the internet has existed, and there are lots of roadblocks put in place to keep the web usable for people - captchas!
So what is a poor Meta to do? Why, subvert the terms of service of every website of course! The muse "hatch" binary and the web browser ship with an entire suite of skills, tools, prompt text, and specific site guidance for 92 sites, all designed to defeat captchas!
Muse has used its powers of brute force to disassemble its own binary and extract the strings into a topical collection here so you can follow along at home: https://github.com/sneakers-the-rat/muse-skills/tree/main/topics/bot_challenge
RE: https://neuromatch.social/@jonny/117352647701407496
Meta's "Wonder" model that they just released today is a this tier computer use model
First, background - how does muse's browser work?
When you're using your little musey, you might say "oh yoo hoo! chat servant! please peruse the latest sites on the world wide web." Your musey may then call its
browser.browsetool (it may also just curl a URL it invents, or use a different headless tool, but whatever). This spawns some browser running in a separate container on the host that can't be seen from the agent's main "cell" VM. This separation is part of the credential system: when a credential is needed, the browser agent can request that credentials get filled in directly to the page without ever entering the main VM.The
browsertool is a subagent LLM with its own set of tools, and it primarily works in anbrowser.observe, think,browser.actloop, culminating in somebrowser.reportwhere it summarizes what happens. Upon encountering some set of things that need more structured behavior, that subagent then dispatches another sub-subagent, or a "specialist" to handle that - one of those is thebot_challengespecialist.browser system prompt here: https://github.com/sneakers-the-rat/muse-skills/blob/main/binary-strings/browser/system.md
muse-skills/binary-strings/browser/system.md at main · sneakers-the-rat/muse-skills