Episode Summary
Executive Summary: Chris Benson and Daniel Whitenack look back at notable AI developments in 2019 and forecast trends for 2020. They highlight transformer language models, robotics, deepfakes, TensorFlow 2, and growing concern about AI’s environmental costs. They also assess progress in practical tooling, ethics, and global inclusion, predicting more focus on responsible deployment, multimodal and on-device AI, and research beyond deep learning.
Main Topics: Transformers and large language models: The hosts identify BERT and GPT-2 as defining developments in 2019. Large pretrained transformer models learned from massive text collections and transferred to tasks including translation, reading comprehension, named-entity recognition, and classification. Hugging Face helped make them accessible through Write with Transformers and a standardized library. AI milestones: robotic dexterity and deepfakes: OpenAI’s reinforcement-learning work on a robotic hand manipulating a Rubik’s Cube demonstrated improved dexterity and robustness to variation. Deepfake videos showed how generative models can blur the distinction between authentic and fabricated media, with both beneficial applications and serious risks. Frameworks and practical AI tooling: TensorFlow 2’s Keras-centered, more approachable interface was praised as a major usability improvement. More broadly, the hosts observe that AI workflows are maturing through better versioning, monitoring, infrastructure, libraries, and tutorials, helping teams build and deploy complete systems rather than just models. AI’s environmental and computational costs: The hosts discuss estimates that training a large model can have substantial carbon emissions. They call for efficiency through model distillation and optimization, alternative architectures, and more efficient data centers powered by sustainable energy. Ethics, safety, and authenticity: AI ethics moved from a peripheral concern toward a mainstream industry discussion, with major organizations publishing principles. The hosts argue that those principles need practical tools and standards, especially as surveillance, election interference, deepfakes, and other harmful uses become more visible. Inclusion and the limits of deep learning: Daniel argues that addressing global challenges and supporting low-resource languages requires researchers and communities from the regions concerned. The hosts also caution that deep learning can be data-hungry and narrow, and that the field should remain open to biological insights and alternative approaches. Predictions for 2020: Chris predicts growth in ethical-AI tools, integration of neural-network development into software lifecycles, more accessible tooling, and renewed exploration beyond deep learning. Daniel forecasts increased emphasis on multimodal learning, AI on mobile devices, and federated learning to improve privacy and reduce data movement.
Key Arguments: Transformers were a major turning point for NLP because large pretrained models could be adapted to many tasks, while products and libraries from Hugging Face lowered the barrier for both nontechnical users and developers. Robotic dexterity advances were notable not simply for solving a Rubik’s Cube, but for showing that reinforcement learning and domain randomization could help systems handle perturbations and unfamiliar situations. Deepfakes make media authenticity harder to establish quickly, creating risks for personal fraud, public trust, and national security; however, generative techniques can also support useful applications such as creating synthetic medical imagery. The AI field is becoming more practically mature as software-engineering practices and supporting infrastructure improve, but successful deployment still requires integrating models into production workflows. AI’s environmental footprint warrants action through more efficient computation, smaller optimized models, alternative architectures, and sustainable data-center energy—not merely awareness of the problem. Ethical principles need to be translated into usable tools and development practices so that responsibility is actionable for engineers who are not ethics specialists. AI research must include local expertise and representation, particularly for low-resource languages and other globally relevant problems; treating English as a stand-in for linguistic diversity misses important cultural, script, and domain issues. The hosts expect continued progress in multimodal and on-device AI, while warning against assuming that larger deep-learning models alone will solve problems requiring broader generalization or human-like flexibility.
Data Points: Podcast episode number: Approximately episode 71 - Chris estimates the show has passed 70 episodes since it began in 2018. GPT-2 training corpus: Over 8 million documents; 40 gigabytes of text - Chris describes the corpus used for the larger GPT-2 model, gathered from web pages linked in Reddit submissions with at least three upvotes. Hugging Face funding: $15 million - Daniel notes Hugging Face’s funding announcement at the end of 2019 to support its work, including the Transformers library. Estimated emissions from training one AI model: As much carbon as five cars over their lifetimes - Daniel cites a 2019 article’s comparison to illustrate the potential climate impact of training a large model; the transcript presents this as an estimate. TensorFlow 2.0 release: November 9, 2019 - Daniel gives this as the date of the final official release, adding humorously that he checked it with Google Search. Project Voice conference dates: January 13–16, 2020 - The hosts say they planned to attend the conference in Chattanooga and record episodes and give a keynote.
Pivotal Quotes: "we need to better involve researchers and developers from all over the world." — Daniel Whitenack: He argues that solving global problems and building useful AI for underserved communities requires local representation and expertise. "we really need to get to biological roots of natural intelligence to understand what our next steps are going to be in the AI space." — Chris Benson: He summarizes a growing view that progress may require looking beyond current deep-learning methods. "we need to keep our flexibility intact" — Daniel Whitenack: He cautions against overcommitting to one fashionable architecture and encourages drawing on other ideas and approaches.
Implications: The next phase of AI depends not only on stronger models, but on trustworthy deployment, efficient computation, global participation, and practical safeguards. Teams should prepare for multimodal and on-device systems while remaining open to approaches beyond deep learning.