The Wild West of AI

Another example of the divergent convergent evolution happening as we all use our programming brains on the blank canvas of coding agents came across the wire today. The AI-Native SDLC playbook – Anthropic

Many of the principles described are similar to the work I’ve put into Shmorch (pronounced “shmork”), my own “solo” SDLC system focusing mostly on the standalone dev.

One Big Skill

Shmorch has one skill with many sub commands instead of what appears to be on the fly creation of many skills. I was hoping to avoid an explosion of random names, but it does make the skill extra beefy.

The Paper Trail

Agreed that the documents and gitlog are a great place for the source of truth versus a cumbersome external Confluence and Jira. There are now many ways of pulling and rendering the information and documentation directly from an already version controlled system.

Tracks and Merging

I am not clear where all the different tracks (folders of intent, plan, spec, etc.) are kept over a long project or how merging multiple worktrees works, especially when documentation and approvals may result in many open worktrees across dev machines (always a problem, right?).  

My solution for multiple tracks was to force serial PR merges using merge, not rebase, rebasing main under the next PR in the sequence.  This provides a nice visual punctuated commit chain.  

Documentation

As for the documents that go in the repo, I’ve concluded there are three kinds: the creator docs, the project docs, and the user docs.  For me all of this is built up and captured as the project grows and changes all planned in a docs folder.  The creator docs define the intent, ADRs, research, designs, decisions, specs, as the institutional knowledge artifacts. The project docs handle managing the ongoing efforts (tracks, sprints, sessions), and user docs are compiled for humans somewhat following the concepts found in Diátaxis.

Each of these have their own growth patterns as the repo/product goes from inception to mature. And there are a whole lot of them.

Context

Context, of course, can explode, bringing your model into the dumb zone. One critical trick not mentioned, is to create front matter for all docs in the tree so that the agent can grab the head to decide if it needs it. An agent task can traverse from the root all the way down to the leaves to read only the necessary aspects.  The drift that happens without layers of contextual directives narrowly sharded creates the balance needed between context growth and good decisions, especially if you’re on a context budget!

In addition, small markdown files with references to subfiles also helps.  When a file gets too large, Shmorch breaks it out into a subfolder with its own index.md and docs inside.  It makes it harder for humans to traverse, but much better for agents who have no problems expanding a deep tree small docs. And see Treeclusion for more on how bring back the readability.  

Testing/Success/Loops/Verification

I had told Claude to not modify the tests when running testing, but gating it makes a lot of sense. Testing, verification, and prompt/response evaluations are what makes code reviews a nice-to-have. Nothing gets through without thorough tests!  And since code is so easy to write, TDD, functional, integration, e2e are all part of making sure the implementation does what it should. And BDD with Gherkin, provides tests in English where it makes sense, so why not build that too, Claude.

I’ve found PostHog for evaluations making a breeze. Though for observability, I’m CloudWatch.

Containing the non-deterministic outputs of models within a deterministic wrapper, bounded by deterministic scripts further makes sure problems are caught early. Shmorch is at the point where there’s a lot of soft gates where it’s possible that steps “required” are somehow invisible to the LLM. The stages I have still help me avoid catastrophe, but Shmorch needs to have more hard gates to loop upon.

Living Diagrams

Totally missing from both, but hoping for a solo license: Living C4 diagrams integrated with IcePanel or your software architecture modeling tool of choice.

And there’s so much more under the hood. 

Conclusion

The parallels between my own thinking and Anthropic’s engineers demonstrates another example of a “solution” that has probably been implemented in some similar form by thousands in the wild west of the convergence and divergence of this undulating AI-lien in our town right now. Another take on it (with diagrams) was also recently posted by IcePanel.

But will all these frameworks and pipelines and processes coalesce like the unicorns of the past or will “custom”, or “in-house”, become the terms for enterprise systems? Compare life during the era of homogeneous typewriter output versus laser printed unique graphic design layouts of word processors and Pagemaker.

We shall see whether this new technology also expands our creativity to new heights.