Let's build an Agentic Loop

With coding agents like Anthropic’s Claude Code and also OpenAI’s Codex CLI a genuine step change in how useful LLMs can be for producing working code - are thereby represented. Agents like these can now re-iterate the code they are producing, correct errors, work through existing implementation details, and also launch experiments to find the best possible solutions to problems.

But the usual conundrum regarding modern AI, there is a great deal of depth involved in achieving the full potential of the new functionalities.

An important new skill to now take under consideration is the development of agentic loops.

Think like this: They brute force solutions to coding problems, and you - as the operator - can now reduce your problem to a clear goal and a collection of tools that iterate towards that goal.

I preferably define an LLM agent as something that runs tools in a loop to achieve a goal. The best way of using them well is to take attention to the design of tools and loop for them to use.

• The YOLO mode


• Best tools for best loops


• The issue with tightly scoped credentials


• When to design an agentic loop


• A novel area of research

The YOLO mode #

Agents can be dangerous - they tend to make poor decisions and are prone to prompt injection attacks, either of which can result in harmful results from tool calls. Since the most powerful coding agent tool is “run this command in the shell” a rogue agent can do anything that you could do by running a command yourself.

Quoting Solomon Hykes: An AI agent is an LLM wrecking its environment in a loop.

Claude Code’s countermeasure is to always ask for approval - of almost every command about to run. Which is tedious, if you ask me, but more importantly: It dramatically reduces their effectiveness at solving problems through agentic loops.

Each tool provided supplies its own version of what I like to call YOLO mode, where everything gets approved by default. Yes, it’s dangerous, but it’s also key to getting the most productive results.

Consider these three key risks for unattended YOLO mode:

1. Bad shell commands deleting or side-effecting things you care about.


2. Exfiltration attacks where something steals files or data visible to the agent-source code or secrets held in environment variables are particularly vulnerable here.


3. Proxy Attacks that attack another target-for DDoS or disguise the source of other assets.

You want to run YOLO mode anyway? Well, you have options:

1. Run your agent in a sandbox with restricted permissions on files and secrets


2. Dedicate a computer for agentic loops alone. That way if your agent goes wild, there’s only so much damage they can do, including wasting CPU cycles that are dedicated to the agentic loop anyway.


3. Avoid exposing agentic loops to sources of malicious instructions and catch mistakes before they cause damage.

Despite the existence of container escapes I think option 1 using Docker or the new Apple container tool is decent enough for most people.

I like to use GitHub Codespaces for this. It provides a full container environment on-demand that’s accessible through the web-browser, with generous free tiers too. If anything goes wrong, keep in mind that it’s a Microsoft Azure machine somewhere that’s burning CPU. The worst that can happen is code you checked out into the environment might be exfiltrated by an attacker, or bad code might be pushed to the attached GitHub repository.

Plenty of other agent-like tools exist. They run code on dedicated computers. Code Interpreter mode in both ChatGPT and Claude take surprisingly long routes. Notingly, I had success at (ab)using OpenAI’s Codex Cloud.

Coding agents themselves implement various levels of sandboxing, though the documentation does not convince me of fully trusting them.

But it turns out Anthropic now have their own documentation on Safe YOLO mode for Claude Code, stating:

Letting Claude run arbitrary commands is risky and can result in data loss, system corruption, or even data exfiltration (e.g., via prompt injection attacks). To minimize these risks, use --dangerously-skip-permissions in a container without internet access. You can follow this reference implementation using Docker Dev Containers.

Locking internet access down to a list of trusted hosts is a great way to prevent exfiltration attacks from stealing your private source code.

Best tools for best loops #

Now that we’ve found a safe (enough) way to run in YOLO mode, the next step is to ellaborate on the tools we need to connect to the coding agent.

You can bring MCP into the toolset at this point, but I find that it’s usually more productive to think in terms in shell commands instead. Coding agents are amazing at running shell commands!

If your environment allows the necessary network access, they can also pull down additional packages from NPM, PyPI and similar. Just ensure your agent runs in an environment where random package installs don’t break things on your workstation.

Rather than leaning on MCP, I like to create AGENTS.md (or similar) files, detailing the requirements for tools to use.

Decent LLMs already know how to use ever-growing arrays of existing tools. If you state “use playwright python” or “use ffmpeg” most models will use those effectively. And since they’re running in a loop they oftentimes recover from mistakes they made and figure out the right approach without manual guidance.

The issue with tightly scoped credentials #

In addition to exposing the right commands, we also want to consider what credentials we want to expose.

Ideally we wouldn’t need any credentials at all - plenty of work can be done without signing into anything or providing an API key, but certain problems require a certain amount of authentification.

Yes, it’s a huge topic in itself. My two key recommendations are:

1. Provide credentials for testing and staging where any damage can be well contained.


2. If credentials spend money, set a tight budget limit.

I’ll use an example to illustrate. A while ago I was investigating slow cold start times for a scale-to-zero application I was running on Fly.io.

I realized I could work a lot faster if I gave Claude Code the ability to directly edit Dockerfiles, deploy them to a Fly account and measure how long they took to launch.

Fly allows you to create organizations, and you can set a budget limit for those organizations and issue a Fly API key that can only create or modify apps within that organization.

So I created a dedicated organization for just this one investigation, set a $5 budget, issued an API key and set Claude Code loose on it.

In that particular case the results weren’t useful enough to describe in more detail. But this was the project where I first realized that “designing an agentic loop” was an important skill to develop!

When to design an agentic loop #

THis pattern of working does not respons well to every kind of problem. It’s key to look out for problems with clear success criteria where finding a good solution involves minimizing or maximizing on given metrics.

A few examples:

• Debugging: The testing suite returns discouraging results, so you need to investigate the root cause. Consider coding agents to maximize for Tests Passed.


• Performance optimization: SQL query XYZ is going too slow. Would indexing improve performance? Have your agent benchmark the query and then add and drop indexes (in an isolated development environment) to measure their impact.


• Upgrading dependencies: You’ve fallen behind on a bunch of dependency upgrades? If your test suite is solid an agentic loop can upgrade them all for you and make any minor updates needed to reflect breaking changes. Make sure a copy of the relevant release notes remains available.


• Optimizing container sizes: Docker containers grow uncomfortably large? Have your agent try different base images and iterate on the Dockerfile to try shrinking it, while keeping the tests passing.

If you smell a general flavour of automated tests, then your smell is right. The value you can get from coding agents and other LLM coding tools is hugely improved by a good, cleanly passing test suite. Thankfully LLMs are great for accelerating the process of putting testing suites together. Therefore, a recommended first step in developing agentic loops is to start developing a solid testing suite that easily extends to a variety of use-cases.

A novel area of research #

Building agentic loops is a new skill. By labeling it, I hope to make the conversation about agentic loops moreproductive. There’s much more to figure out about how to use these tools as safely and effectively as possible.