Supper Club × Matt Rothenberg and Idan Gazit on Github Next

Episode Summary

Executive Summary: The episode explores GitHub Next’s origin as a risk-taking innovation lab, how it prototyped and shipped GitHub Copilot, and why fast, context-aware AI is crucial for developer workflows. The guests discuss prompt engineering, model tradeoffs, Copilot X, pull-request automation, future AI interfaces, and the organizational trust required to turn experiments into products.

Main Topics: GitHub Next’s origin and mission: Matt and Idan explain that GitHub Next began as the Office of the CTO to pursue high-risk, future-facing bets that normal engineering orgs tend to avoid. Its role is to prototype ideas, prove value, and hand them off to product teams. How Copilot emerged and scaled: The team describes Copilot as a small experimental project that grew rapidly once the UX and workflow fit became clear. Early skepticism gave way to confidence as they solved latency, trust, and integration problems. Prompting, context, and model engineering: The conversation dives into the “below-the-waterline” work: prompt crafting, context injection, AST-based code restructuring, and reformatting prompts so models can complete code in the middle of files. Speed, latency, and reliability as product requirements: Guests stress that AI features must feel instantaneous to fit developer flow. They note that latency directly affects acceptance rates and that smaller models can be preferable when speed and cost matter. Copilot X and expanding AI across the dev lifecycle: The guests outline GitHub’s move beyond editor completions toward chat, pull requests, issues, and other parts of the software lifecycle, while noting that some newer features are still exploratory and not yet reliable enough to ship broadly. Future of developers and AI augmentation: Rather than replacing developers immediately, the speakers frame AI as a productivity multiplier that lowers the floor for juniors and raises output for experienced engineers, similar to giving developers a mech suit. Team culture, trust, and product handoff: They emphasize that Next earns organizational trust by shipping real value and then handing products off to other teams. Success is measured by whether an experiment becomes something durable and useful in the broader GitHub ecosystem.

Key Arguments: GitHub Next exists to take risky, high-upside bets that standard engineering orgs are incentivized to avoid. Copilot succeeded because the team solved workflow integration, not just model access; the UX had to fit inside the editor. Latency is not a nice-to-have in AI code completion; if suggestions arrive too late, users keep typing and never see them. Prompt engineering is iterative and partly art: the team tests, refines, and rebuilds prompts until results are reliable enough. Some tasks are better served by smaller, faster, cheaper models than by the newest large model, especially when responsiveness matters. Copilot for Pull Requests shows AI can help beyond writing code by summarizing diffs and drafting PR descriptions. Many ambitious AI ideas are not yet reliable enough to ship, especially those that try to synthesize entire PRs or workflows end-to-end. AI will likely augment developers rather than replace them in the near term, especially by automating tedious scaffolding and setup work. GitHub Next’s broader mission is to improve software developers’ lives, not just to ship flashy demos. Shipping Copilot bought the team organizational trust, but also raised the bar for every future project.

Data Points: GitHub Next team size: about 20 people - Idan describes the group as small but diverse and ongoing since the early days. Original team size: about 5 people - The Office of the CTO started as a small group focused on risky future bets. Copilot first public debut: June 2021 - Idan cites June 2021 as the first time the world saw Copilot. Completion acceptance in some regions: way lower outside the US - Latency and lack of regional GPU availability reduced acceptance rates because suggestions arrived too late. Pull requests without descriptions: 34% - Matt cites research suggesting a large share of open-source pull requests are submitted without descriptions. Model cost difference: 3.5 Turbo is 16x cheaper - The speakers compare GPT-4 vs GPT-3.5 Turbo for different tasks, noting cost-speed tradeoffs. Prompting scale: 200 bucks worth of prompts in one day - Wes mentions quickly spending money while iterating on wording for AI outputs on the Syntax site.

Pivotal Quotes: "What if we had an AI that could write code that doesn't suck?" — Idan Gazik: Describing the original Horizon 3 bet that helped motivate GitHub Next’s work on Copilot. "The magic of Copilot is underpinned by all this work to make it that simple, that magic." — Matt Rothenberg: Explaining that the apparent simplicity of Copilot hides a large amount of engineering and UX work. "It feels like a mech suit that they step into that allows them to like jump over buildings and do stuff they couldn't otherwise do." — Idan Gazik: Describing how AI tools may augment developers rather than replace them.

Implications: AI development tools will likely move deeper into workflows, not just editor autocomplete. Speed, trust, and UX will decide winners, and teams that can turn experiments into reliable products will shape the next generation of developer tooling.