This week’s episode of “The Odd Couple” is simply Caitlin and Hannah as I needed to go to Georgetown to speak about Claude Code at a school retreat. However earlier than we get going with an outline, Hannah talked about at the beginning in the course of the ice breaker in regards to the opening theme music to the podcast, and for those who do not acknowledge the lyrics, that is Mac Miller’s “Small Worlds” sung by my two nephews.
So what is that this episode about? One of many themes I’ve been emphasizing in my talks on AI Brokers and my substack is that AI Brokers have triggered a separation between the historic bundling of the manufacturing of analysis and the verification of the outcomes. Since AI Brokers are actually in a position to produce so many features of the analysis mission autonomously — that’s with out a lot path from the human researcher — one of many new duties of the researcher is to confirm them.
Should you bear in mind from just a few weeks in the past, Claude Code had almost immediately labored up the county-level marriage knowledge right into a county panel of marriage charges and marriage counts by yr. We introduced Hannah Sayre, a current school graduate and present financial marketing consultant, into the mission to assist us work by the latter process of “human verification”. Had Claude finished it appropriately? How will we confirm that it’s appropriate? And if it’s not appropriate, why was it not appropriate, and the way generalizable is that inaccuracy? Hannah was our eyes and ears, our boots on the bottom, as she independently investigated the identical query, the identical process we gave Claude, to on the again find yourself assist us decide whether or not Claude had certainly discovered the identical irregularities within the unique marriage dataset, and if that’s the case, what autonomous choices had he made. And so on this episode, Hannah walks us by it, and he or she and Caitlin talk about each these findings, in addition to start the work of conceptualizing the method of verification in a world of AI Brokers. Whereas not definitive, it is a likelihood for others to listen to extra particularly about this. I a minimum of anticipate that every one of us must wrestle with verification going ahead in methods we weren’t anticipating, and possibly even are usually not ready for, a minimum of not universally, and undoubtedly not essentially if the truth is AI Brokers shrink the scale of the mission crew members on account of automation, and the way greatest to reply to that smaller scale, and subsequently, fewer individuals accessible to do the precise verification itself.
Thanks once more for tuning in! We hope you might be having as a lot enjoyable with this as we’re!
