AI(or narrow LLM version of it) is genuinely one of the most radical technologies to ever grace planet earth. There, I said it. Hopefully I don’t get dealt by pitchforks in the upcoming Butlerian Jihad.
A brief flashback
I have used AI in some form since OpenAI started with an API for its GPT-3 model(I have also used BeRT and other earlier language models). Remember Davinci, Curie, Ada etc? The earlier way of using involved understanding the temperature(randomness), top_p(top tokens in probability mass) etc. The use cases spanned from simpler html pages to a complex college essay applications. One of the most viral things at that era was a person who applied using AI written college essay and got into couple of prestigious programs. Like any person part of the herd, I too used davinci, curie, babbage etc for writing blogs, style check etc.
The next iteration came with chatgpt which let people actually chat with the llm and ask questions and let llm answer those. It was easy to have code pasted to chatgpt and getting it corrected. At software layer this was the fancy autocomplete era and was the time when people used AI as an autcomplete and used it for debugging issues in code. I used AI mainly for writing tests and validating the implementation etc. Aider chat was one of my favorite tools during this time.
The next era in AI tools was the agentic era where an agent with its harness was able to handle semi long horizon tasks without independent supervision. This is the era that I finally had to move away from my existing ways of working with software. AI was much faster than me with code generation and validation. The only thing I needed to do was guide it correctly with architecture, data modelling, and product guidance. AGENTS.md, CLAUDE.md etc helped me a lot as well lot of reading of people who were actually doing interesting work. Some of the people I followed included Shrivu Shanker and his way of working. Simon Willison and his blogs became one of my favorite reads.
Now we are probably in post harness era with RLM and custom harnesses for various use cases. I am also increasingly of the opinion that AI have reached a point where we should be able to see a lot of AI engineering efforts reach its own fruition. I dabble mainly with meta harnesses and orchestration efforts to parallelize my work. Most of the people I follow these days are from AI labs. Working with meta harnesses like omnigent shows what future of work looks like in this era.
So how do I work with AI now?
This is probably the only part in this blog that I wanted to avoid. There are a million posts out here with the same old wine rehashed in a new bottle of harness. I would recommend reading Shrivu’s and Simon’s blog to have a deeper understanding. My way of working is deeply influenced by my environment and experiences. The above image is also probably true for most productive developers I have known. So lets get on with how I work with AI.
Test and evals: TDD specifically helps a lot in steering models. If you are building on top of AI, evals provide a bit of determinism for outputs of your AI application. If TDD helps you steer outputs of AI generated code, evals adds determinism to the outputs if the product involves any sort of LLMs.
Context management: Efficient working with LLMs involve active context management, pruning compaction etc. Context can be thought of as a well organized closet. Getting a cloth suited for an occasion is an easy task if your closet is well organized. Similarly context for a task should only involve relevant discussions for that particular task. Context pollution and affects compaction as well. Claude code is especially prone to compaction affecting its performance over codex.
Verification of work: I mainly use ai agents in parallel and verify the results parallel and merge them. The real important question for me have been to decide where human in the loop is needed. There is a concept called side effect based testing which involves using side effects to verify. Due to this most of my application is well instrumented and there are logs. While using and running end to end suites with human in loop gives me the confidence I need. The log verification reduces the chances of the issues appearing.
Orchestration: How do I orchestrate? Orchestration of work across multiple agents is a hard problem. Each individual parallelizable unit of work needs an implementer and a verifier loop. My experience with project management including chopping down into epics, stories etc have helped me extensively here. Each unit of work has an acceptance criteria that the verifier evaluates the work against. Each implementer-verifier loop runs over its own set of git branches(using git workflows) and is eventually merged. Some times stacked PRs are used and some times normal PRs.
Tools I use: I use terminal based tools and a common GUI for reading code when I work. codex, claude, opencode, pi etc are used interchangeably within omnigent or sometimes with hermes. I have still not managed to find any reason to use cursor or similar agentic ide over intellij or zed.
Memory: This is useful for a cheapskate like me when I need to shift from codex to claude or from paid plans to opencode free plans etc. I have used hindsight but is in the process of writing my own memory system which I can integrate with omnigent.
Sandboxes: I am a big fan of sandboxed environments in local machines as well as in cloud(fly.io etc). It helps me to run tools extensively and sometimes to interface running sandboxed executions in fly.io containers which I can take a look at in morning. This used to be my way of working when I had the enterprise tokens to work. Now that I am working on my own, I have a shoestring budget and have largely avoided this in favor of using apple containers or docker container locally.
Local LLMs: I use local llms like qwen for some tasks when my weekly limits of codex, claude etc overflows. This is where a good memory system comes into play as local llms seems to be extremely sensitive to harness and memory and can spiral into thought loops figuring out things.
Things I don’t use
Factories: I have dabbled in this but have found it to be a token black hole with no measurable gain over a well maintained orchestration system. Gas towns, mayors, factorio etc seems to be mutations that might result in valuable feedbacks and I might have to reconsider my aversion towards “dark” software factories.
Agentic Ides: I am a strong believer that there are parts of codebase that needs to be read and parts which can be vibecoded. Ides like intellij idea, zed etc help me do this without clubbing the code generation activity. I like to read the code that have been generated to ensure that its manageable and well written rather than slop coded. My experience with cursor and windsurf(before it got “acqui-hired” by google) largely interrupts the reading process and does not let me compose my thought process.
Things I have not used but want to
Work or many iterations of it: chatGPT work, claude work etc seems to be quite interesting way of working but I have not got around in using them.
Agent Swarms: Similar concept to factories but the coordination and grounds up hierarchical structures for organizing work etc seems to be quite interesting in comparison to factories. Could be like an anarchist commune or kibbitz for agents to organize and plan the world domination.
One of the things with any adoption of new technology is the inversion of the question of what I use this technology for.
So what have I decided to not use AI for?
Blog posts or any kind of original thinking I do: I used to edit my blog posts extensively with AI tools and I have decided not to do that any more. This is a very contrarian take. I have been increasingly realizing that there are parts of my work which AI handles well. Blogging, reading, and picking up a new skill are few avenues which I want to keep on doing and not offload to AI.
Personal messages or love letters: One of the earliest use cases of mine was to offload parts of my personal message to the AI. This is quickly tiring as the connections that needs to be authentic quickly turns robotic and loses its touch. I have realized that this is crucial and important for me as a person.
Architecture: I have increasingly realized that architecture of the codebase how code gets organized matters lot more for me. There are times at which AI acts extremely knowledgable and will bluff you. This often results in bad architecture which becomes extremely difficult to change.
Data Modelling: This is one part where AI can augment and is quick to find holes in your design. Completely offloading this decision on an AI system seems to be a bit of problem. RL and other methods of post training have not improved this for some reason in my experience.