The article starts interestingly; I like the analogy between cold reading and LLM responses (well, the ones we were getting three years ago). Then it goes on to generalise to the tune of "This new era of tech seems to be built on superstition and pseudoscience" which I think was a conclusion better left to the reader.
We are used to articles, blogs that stayed relevant for years. It is hardly the case with AI, the speed of change is so high that an observation can become stale in a year. This makes it very important for articles to state that the ideas are based on the technology in year X, rather than generalising them as universal truths.
The title feels clickbaity. Direct API, CLI and MCP all have their uses.
Direct access (Curl/own small function): This requires agent to have full understanding of the API spec. Yes, context can be protected using progressive disclosure, but this essentially means agent needing to understand the API again and again, before every use. Also, a typical API spec may or may not be agent-friendly. If there are nuances when calling an endpoint, where do we put these? Into OAS description? Works, but clunky.
CLI: It works beautifully, especially when a 3rd party CLI already exists for a complex backend. Assumes a well documented, agent friendly CLI, most CLIs are designed for human or CI/CD consumption. Talking about MCP taking up too much context, think about agent starting with my_cli --help, and going down through the switches and parameters one at a time to figure out how the CLI should be called. Less of a problem when calling a well know CLI (e.g. aws), but anything more niche (or custom) requires multiple turns to compose the final CLI command.
MCP: Has its issues, but offers an agent-native solution. Everything agent needs to know about a tool becomes available at once. In an enterprise environment MCPs can be served through an MCP Gateway, providing governance and permission management, this is quite contrast against running a CLI that requires agent to have execute permissions in its shell.
The MCP use case is mainly another attempt at papering over the fact that effectively no API today is actually RESTful. We wouldn't need MCP at all if we still implemented REST APIs.
I wish this were a document that clearly described how mathematicians expect AI to behave, rather than how it is misaligned. I can kind of decipher the expectation from the declaration, but it still sounds a bit like "it would be better if AI didn't meddle with mathematics", instead of "we want AI to approach problems like this and that way".
They want the AI labs to stop solving "important" math problems so that humans can do it, on the theory that when AI solves "important" math problems, it robs humans of the opportunity to gain new insights into those problems.
The issue is that now that Navier-Stokes is "solved", the amount of resources put into researching it will be reduced, meaning that the insights and knowledge that would have come out of even failed attempts will never be.
An "unsolved problem" as a beacon to strive towards, gaining understanding along the way. With the beacon dead, noone will strive.
Funding agencies will have to adapt too, the community will have to adapt. Hiring committees will have to adapt. Everyone will have to adapt. They will have to rethink their criteria and adapt to reality. It's not at all guaranteed that this beacon interpretation will remain how people understand where to put resources. Yes, academia can be stiff, rigid, non-adaptive, set in its ways and navel gazing. But it's a wakeup call. Let's see.
Yes, quite a few uses in prod, main use case being abstracting API access for agents. In our case these are mostly in-house MCP servers, purpose built for the given agent.
Why MCP instead of CLI or agent accessing API directly?
Direct access (Curl/own small function): This requires agent to have full understanding of the API spec. Yes, context can be protected using progressive disclosure, but this essentially means agent needing to understand the API again and again, before every use in that context. Also, a typical API spec may or may not be agent-friendly. If there are nuances when calling an endpoint, where do we put these? Into OAS description? Works, but clunky.
CLI: It works beautifully, especially when a 3rd party CLI already exists for a complex backend. Assumes a well documented, agent friendly CLI, most CLIs are designed for human or CI/CD consumption. Talking about MCP taking up too much context, think about agent starting with my_cli --help, and going down through the switches and parameters one at a time to figure out how the CLI should be called. Less of a problem when calling a well know CLI (e.g. aws), but anything more niche (or custom) requires multiple turns to compose the final CLI command.
MCP: Has its issues, but offers an agent-native solution. Everything agent needs to know about a tool becomes available at once. In an enterprise environment MCPs can be served through an MCP Gateway, providing governance and permission management, this is quite contrast against running a CLI that requires agent to have execute permissions in its shell.
I must mention that we also utilise lazy-loading of MCPs. In use cases where 10s of tools needs to be loaded, only the most common ones are pre-loaded, then agent kernel connects the others as an when needed, and release them after a timeout (in case of long running sessions). This keeps the context lean.
We have taken to letting our agents use specific CLIs directly and we narrow down their access control with nono tool sandboxing via a customized Pi agent running headless. We have one agent that can use the aws CLI and nono feeds in a phantom SigV4 credential and then policy limits to just read-only http methods and certain L7 filtering on paths to limit its abilities - this means we can have it monitor and debug deployment failures without it doing any damage.
I'm really curious how you're dealing with lazy loading. Is the MCP then more like a tool-helper instead of the tools themselves?
I was trying to find out if the MCP has some sort of lazy loading feature/primitive in the works, but there seem to be a lot of disagreements about it.
Intuitively, lazy loading seems to be somewhere between a CLI and vanilla MCP. In the end, it sounds very similar to tiered/ progressive loading similar to images on low bandwidth connections.
Outages happen. What concerned me most was the time taken for the full resolution (almost 12 hours). I am hoping that root cause analysis promised will look into this as well and how it can be improved.
Size reduction is mostly based on Experts size. And it is limited by SSD speed.
Check for Colibri and Flash-Moe, they are doing similar things with bigger models, but tok/s is not high
TDD like this is a very good system for a lot of things, especially brownfield development - gotta make sure you understand the error space before you know you’ve fixed the error. :D
We are used to articles, blogs that stayed relevant for years. It is hardly the case with AI, the speed of change is so high that an observation can become stale in a year. This makes it very important for articles to state that the ideas are based on the technology in year X, rather than generalising them as universal truths.
reply