AI-powered Search for the Enterprise — with Deedy Das of Glean

Episode Summary

Executive Summary: Didi Das of Glean discusses how enterprise search evolved from a practical internal pain point into an AI-native workplace product. He argues that search works best as part of a broader employee portal, not a standalone tool, and that hybrid retrieval plus ranking still matters more than buzzword-driven LLM demos. The conversation also covers Google vs. ChatGPT, the economics of AI, open source models, and the future of multimodal assistants.

Main Topics: Glean’s origin and product thesis (Priority: 5/5): Glean was built to solve the real workplace problem of finding internal information across many SaaS tools, inspired by Google’s internal search experience. The company was not founded as a technical search demo but as a solution to a persistent enterprise pain point. Why enterprise search must become an employee portal (Priority: 5/5): Das argues that search alone is not sticky enough for retention. Glean adds feeds, collections, GoLinks, and mentions so users return regularly, making it more like an employee portal than a pure search engine. How modern enterprise search actually works (Priority: 5/5): He explains that effective search is still a hybrid system: classic information retrieval, synonym and acronym expansion, personalization, document quality signals, and vector search all matter. LLMs help, but they are not the whole solution. Google vs. ChatGPT and the future of search (Priority: 4/5): Das says ChatGPT is now his default for many queries, but Google remains superior for freshness, navigation, and certain structured information needs. He sees chat as powerful but not a universal replacement for search. Economics, latency, and the limits of LLM hype (Priority: 4/5): He emphasizes that cost and latency are central to product viability, especially in startups. He is skeptical of solution-first AI startups and argues that many LLM companies lack a durable moat or clear problem fit. Open source models and democratization (Priority: 4/5): Das views open source LLMs as broadly positive for innovation because they let individuals and smaller teams build useful systems, though he also worries about misuse and the power of text generation at scale. AI beyond text: multimodal and assistant futures (Priority: 3/5): He is most excited by image, voice, video, and multimodal systems, especially a future assistant that combines speech, vision, and language into a lifelike interactive presence.

Key Arguments: Enterprise search became viable now because SaaS apps expose robust APIs, distributed search infrastructure exists, and companies use far more tools than before. Search is not compelling enough on its own to retain users; products need adjacent workflows like feeds, mentions, and shortcuts to become daily-use portals. Modern search quality depends on careful ranking and retrieval engineering, not just adding embeddings or LLMs. ChatGPT is excellent for many tail queries and coding help, but Google still wins on freshness, navigational queries, and click-through-based exploration. LLMs are currently best used as part of retrieval-augmented systems, not as standalone replacements for search indexes. AI startups should be problem-first, not solution-first; many current companies start with “LLMs are cool” and then search for a use case. Cost transparency matters because infrastructure economics determine whether AI products can scale profitably. Open source models accelerate innovation by enabling local, cheaper, and more accessible experimentation, even if they are not frontier-grade. The most exciting near-term AI opportunities are in multimodal systems, voice, video, and practical automation rather than generic chat. Watermarking generated text is useful both for model hygiene and for identifying machine-generated content in education and other settings.

Data Points: Years at Glean: Almost 4 years - Das says he has been at Glean for nearly four years and was on the founding team. Books read per year: About 10 - He says he usually gets through roughly 10 books a year, mostly fiction for pleasure. Google internal search tool: MoMA - He references Google’s internal search system as the inspiration for workplace search. Typical company software stack: 10 to 100 apps - He cites the explosion of SaaS tools as a major reason enterprise search is needed. Natural language queries share: Still a small fraction - He says natural language queries exist in Glean but remain a minority of total queries. Google search ad revenue: $11 billion - He uses Bing’s ad revenue as an example of how search monetization remains enormous. Bing ad revenue: $11 billion - Used to argue that subscriptions alone cannot replace search advertising economics. Latency effect: More latency = less engagement - He describes a near-linear relationship between latency and user engagement in products. Llama training cost estimate: $4 million - His estimate for the final training step of Meta’s Llama model. PaLM training cost estimate: $27 million - His estimate for the final training step of Google’s PaLM model. Bard training cost estimate: $4 million - He says his estimate for Bard is also around $4 million. Indian board exam scale: 1.1 million students - He notes the number of students affected by the exam-board grading system he analyzed. Grace mark threshold: 33 - He found that exam scores below 33 were rounded up to passing marks. Missing score values: 30 out of 61 numbers between 33 and 93 - He observed many absent score values, suggesting manipulation or non-transparent grading. Open source model example: Llama 7B - He mentions the leaked and cloned 7B version as part of the open-source ecosystem.

Pivotal Quotes: "Search in itself is not a compelling enough use case to keep people drawn to your product." — Didi Das: Explaining why Glean expanded beyond search into feeds, mentions, and employee-portal features. "It’s not sort of, hey, drop in LLMs and embeddings and we become amazing at search. That’s not how we think it works." — Didi Das: Describing Glean’s hybrid ranking and retrieval approach. "My sense is that the people who focus on problem first usually get much further than the people who focus solution first." — Didi Das: His critique of many AI startups that begin with technology rather than a user problem.

Implications: Enterprise AI winners will likely combine strong retrieval, workflow integration, and clear ROI rather than rely on chat alone. The next wave of innovation may come from multimodal assistants, cheaper models, and practical automation that solves real user pain.