In 2007 Intel adopted what’s known as their tick-tock processor development cycle. From Wikipedia: “Under this model, every new process technology was first used to manufacture a die shrink of a proven microarchitecture (tick), followed by a new microarchitecture on the now-proven process (tock)”. This isn’t so much a choice made by management as it is a formalisation of what already is going on with technologies that develop in lockstep. The die shrink comes first, and once you know what you are working with you can improve the microarchitecture.
AI development is following a very similar pattern. The models themselves are the tick, and the harnesses, frameworks, loops, pipelines, and agent definitions are the tock. It’s hard to work on the latter without the former, so all development is beginning to align within this pattern.
What that ends up meaning, beyond just being a neat observation, is that it’s not that useful to observe only one of these two components (like claiming a new model isn’t noticeably different from the last). Instead, you have to wait also for the tock to understand the real implications of a model beyond it’s performance on a benchmark and it’s feel from a prompt.