Building AI Systems for Capital Markets

Episode Summary

Executive Summary: Goldman Sachs hosts discussed how AI product building has shifted from flashy demos to reliable, grounded systems. Chris Churchman described Marquee AI as an institutional tool that must answer with provenance, integrate research/data/workflows, and preserve human judgment. The conversation stressed trust, grounding, changing model capabilities, and the need to design for the next-generation user and institutional mandates.

Main Topics: Building AI products for institutional decision-making: Marquee AI is framed as a concierge for the investment process, helping clients make decisions under uncertainty by surfacing research, commentary, analytics, and calculations in one workflow. Grounding, provenance, and hallucination control: A major challenge is ensuring every answer is auditable and sourced, because plausible-sounding model outputs can be wrong and institutional users cannot afford unchecked hallucinations. Rapidly shifting model capabilities and product strategy: Churchman argues teams should build for the model at launch, not the model at the start of development, because context windows, tool use, and reasoning scaffolds evolve too quickly. From automation to reconception: The discussion distinguishes incremental AI automation from rethinking workflows entirely, with the latter seen as the path to order-of-magnitude improvements. Trust, human judgment, and apprenticeship: AI should empower rather than replace human reasoning; the speakers warn of cognitive atrophy and losing tacit knowledge if junior staff learn only through AI-mediated shortcuts. Scaling laws, world models, and the future of AI: Churchman expresses optimism that next-token prediction and scaling laws will continue to produce better systems, including synthetic world models and AI that can help automate AI research itself. Open gaps: self-learning, memory, and native form factors: He says key unresolved issues remain, including persistent self-learning, catastrophic forgetting, and finding the right native user interface for AI.

Key Arguments: Start with the business problem and user workflow, not the AI technology itself; Marquee AI is built around the investment process, not generic model use. Institutional AI requires grounding every sentence in a source or auditable calculation because users must be able to stand behind the output. A model that is 90% right is still operationally problematic in high-stakes settings unless the user can identify the 10% that are wrong. Many early technical workarounds, such as chunking, vector databases, and reasoning scaffolds, can become obsolete as model capabilities improve. Product teams should build for the model that will exist at launch, not the one they have during prototyping, because capabilities shift quickly. The next frontier is environment engineering and mandate engineering: secure tools, entitlement-aware access, and clear authority for agent actions. The most valuable AI applications may come from reconceiving workflows entirely, not merely automating existing ones faster. AI should augment human judgment rather than cause reasoning atrophy; firms need to preserve apprenticeship and tacit knowledge transfer. Scaling laws and transformer architectures support a belief that capability gains can continue and that AI can eventually help automate its own research and experimentation.

Data Points: Episode recording date: August 11, 2026 - Closing disclaimer states the episode was recorded on Tuesday, August 11th, 2026. ChatGPT launch reference: November 2022 - Used as the benchmark for how quickly model capabilities evolved and early assumptions became outdated. Early context window size: 4,000 tokens - Churchman referenced this as the original limitation that forced chunking and vector database approaches. Later context window size: 128,000 tokens - He cited this as a later jump that made earlier chunking strategies less necessary. Google context window size: 1 million tokens - Referenced as a breakthrough that helped solve retrieval issues like the needle-in-the-haystack problem. Model confidence rate example: 90% right / 10% hallucination rate - Used to illustrate why even a relatively accurate system is risky if users cannot identify which facts are fabricated. Scaling improvement estimate: Up to 10 orders of magnitude - Mentioned in the discussion of scaling laws and why small experiments can sometimes extrapolate to much larger systems.

Pivotal Quotes: "you've got to start with the problem to be solved, not just, oh, let's use AI on that and that" — Chris Churchman: On how Marquee AI is designed around client workflow and investment decision-making rather than generic model adoption. "a demo is judged on its best day, whereas a product is judged on its worst day" — Chris Churchman: Explaining why real products require much more reliability, grounding, and edge-case handling than flashy demos. "I think there's a real danger of that" — Chris Churchman: Referring to the risk that AI leads to cognitive atrophy by outsourcing reasoning and weakening human judgment.

Implications: AI product winners will be those that combine model capability with trust, grounding, and workflow redesign. Firms should protect human reasoning, preserve apprenticeship, and build for evolving models rather than current limitations.