The TWIML AI Podcast
The TWIML AI Podcast

Multimodal, Multi-Lingual NLP at Hugging Face with John Bohannon and Douwe Kiela - #589

In this extra special episode of the TWIML AI Podcast, a friend of the show John Bohannon leads a jam-packed conversation with Hugging Face’s recently appointed head of research Douwe Kiela. In our conversation with Douwe, we explore his role at the company, how his perception of Hugging Face has ch

Topics Discussed

Episode Summary

Executive Summary: The conversation explores Hugging Face’s evolution from an NLP-focused open-source company into a broader AI organization, and the research agenda shaping that shift. Topics include distributed research culture, multilingual and multimodal models, grounding and intentionality in language, evaluation/data-centric AI, retrieval-augmented systems, and the limits of hype around AGI and sentience.

Main Topics: Hugging Face’s identity and research culture (Priority: 5/5): The guest describes Hugging Face as an open, transparent, community-driven AI company that occupies a middle ground between bottom-up academic research and top-down lab direction. Multimodality and grounded meaning (Priority: 5/5): A major theme is that future AI should integrate text, images, audio, video, and potentially embodied interaction to better represent meaning the way humans do. Bloom and large-scale multilingual modeling (Priority: 5/5): The discussion covers Big Science’s Bloom model, a highly collaborative multilingual language model trained in the open with curated data across many languages. Evaluation, data measurement, and data-centric AI (Priority: 4/5): The guest argues that better measurement of datasets and models is essential, and that future progress depends on evaluation tools, active learning, and smarter data curation. Retrieval and semi-parametric systems (Priority: 4/5): The conversation highlights retrieval-augmented generation and hybrid parametric/non-parametric models as a way for models to access fresh knowledge beyond fixed pretraining. Limits of current language models and skepticism toward hype (Priority: 4/5): The guest cautions against over-attributing understanding, sentience, or AGI to current models, emphasizing evaluation crises, anthropomorphism, and real deployment risks.

Key Arguments: Hugging Face plays a unique ecosystem role because it democratizes models, datasets, and tools in a way large proprietary labs cannot. Research at Hugging Face needs a hybrid structure: enough top-down coordination to support ambitious projects, but still community-connected and open. Current language models are insufficient because they lack multimodal grounding and the intentional, interactive nature of human communication. Meaning is not just referential; it is representational across modalities such as sight, sound, touch, and action. Bloom is important not only as a multilingual model but also as a community-built, globally curated scientific artifact. Evaluation should move beyond static benchmark scores toward real human interaction, dataset inspection, and model/data measurement. Data selection and active learning can improve both labeling efficiency and pretraining by choosing data more intelligently for downstream goals. Retrieval-augmented and semi-parametric systems can help models stay current and mimic human semantic/episodic memory. The field should avoid both over-hyping and under-hyping AI: progress is real, but claims about understanding or sentience should be treated carefully.

Data Points: Hugging Face headcount: 30–35 people - Estimated size of the research team discussed by the guest Hugging Face company size: 100–200 people - Approximate overall company size mentioned from Crunchbase-style stats Big Science participants: around 1,000 signed up - Scale of the open collaborative effort around Bloom Active Big Science participants: hundreds active daily - Estimate of day-to-day active contributors Countries represented in the company: more than 25 countries - Geographic distribution of Hugging Face employees Bloom language coverage: 45 languages - Guest cites the curated multilingual dataset feeding the model Training data size for Bloom: 800 gigabytes - Amount of data mentioned by the interviewer from Thomas Wolf’s comments Time since joining Hugging Face: about 6 months - Guest reflects on settling into the role after half a year FAIR tenure: 5 years - Previous role at Meta/Facebook AI Research Twitter update on Bloom training: 87% - Progress snapshot mentioned during training Dynamic adversarial data collection gain: about 10% better - Improvement claimed from iterative model-in-the-loop data collection Big Science working groups: 40–50 working groups - Structure of the one-year collaborative workshop Podcast reference: 3 months earlier - Prior interview with Thomas Wolf referenced in the discussion Research paper citation rank: most cited paper from 2017 - Guest’s cited paper on sentence representations referenced by interviewer

Pivotal Quotes: "Hugging Face is no longer an NLP company per se... I like to think of Hugging Face as an AI company." — Dawa: Describing the organization’s expansion beyond language into broader AI areas "The two main things missing in our current paradigm are multimodal understanding of concepts and the intentionality with the T of language." — Dawa: Explaining why current language models still fall short of human-like understanding "We should be careful to not over-interpret what we're seeing." — Dawa: Summarizing a cautious stance on model capability, sentience, and benchmark progress

Implications: The industry is moving from pure text modeling toward multimodal, retrieval-based, and data-centric systems. Open evaluation and open science may become as important as model scale, while hype about sentience or AGI should not distract from current deployment and trust issues.

🔓 Sign Up for Unlimited Episode Search

About The TWIML AI Podcast

View all episodes from The TWIML AI Podcast