293: True Personalization Through Reinforcement Learning

Episode Summary

Executive Summary: This episode features data scientist Peyman Hesami discussing his move from Qualcomm to a startup, his work in reinforcement learning, and how RL enables adaptive, personalized systems without labeled data. The conversation explores RL vs supervised learning, practical applications in devices, advertising, and energy, plus the importance of choosing the right problem and starting with simpler solutions.

Main Topics: Career transition from large company to startup: Peyman explains leaving Qualcomm after seven years to gain closer product ownership, faster feedback, and greater direct impact at a startup. Reinforcement learning fundamentals: RL is explained intuitively as training a puppy: actions are rewarded or punished so an agent learns to maximize reward through interaction with its environment. Personalization through reinforcement learning: The discussion centers on using RL for implicit, automatic personalization in devices such as speakers, phones, alarms, and TVs rather than just recommendation lists. Why RL matters vs supervised learning: The speakers contrast RL with classification/supervised learning, emphasizing less dependence on labeled data, adaptability, and goal-oriented optimization. Real-world and future applications: Examples include Alibaba advertising, energy optimization for hot water, and self-driving cars, illustrating RL’s potential for business ROI and efficiency gains. Practical learning and implementation advice: Peyman advises learning RL theory first, then building toy projects, and stresses that the hardest part is correctly mapping a business problem to an RL environment and reward structure. Common mistake: using RL where it does not fit: Peyman shares that he once tried to force RL onto a problem better solved by simpler methods, highlighting the importance of choosing the simplest effective approach.

Key Arguments: Reinforcement learning is powerful because it can learn from interaction with an environment rather than requiring large labeled datasets. RL is especially useful for personalization when the desired behavior is implicit and context-dependent, such as device settings adapting to user environment. Compared with supervised learning, RL can reduce model maintenance because it can keep adapting after deployment instead of repeatedly retraining from scratch. RL can drive innovation by discovering solutions humans may not explicitly label or anticipate, such as optimized physical designs. Bias in machine learning often comes from the data; RL avoids some of that inherited bias, though it can still be limited by an overly narrow environment. The hardest part of applying RL is not the algorithm itself but accurately defining the environment, rewards, and punishments for the business problem. The best way to start with RL is to learn the theory and math enough to understand optimization, then apply it in small projects. Not every problem should be solved with RL; simpler methods should be considered first before using more complex machine learning approaches.

Data Points: Episode number: 293 - Super Data Science Podcast episode featuring Peyman Hesami Years at Qualcomm: 7 years - Peyman worked at Qualcomm before moving to a startup RL return on investment uplift at Alibaba: 240% - Example cited for reinforcement learning in advertising without increasing budget Energy reduction in water heating study: 20% - Reinforcement learning deployed across 32 houses with no reported loss of comfort Number of houses in energy study: 32 houses - RL applied to optimize hot water energy use Data Science Go attendance estimate: 600-800 attendees - Conference audience size mentioned in the episode outro Conference speakers: 40 speakers - Data Science Go 2019 speaker count promoted in the intro Personal RL study duration: Close to 3 years - Peyman’s experience learning and applying reinforcement learning Conference date: 27th, 28th, 29th of September 2019 - Data Science Go 2019 event dates in San Diego

Pivotal Quotes: "In layman terms, I can describe it as training a puppy." — Peyman Hesami: Explaining reinforcement learning intuitively "Once you see the impact that you have made clearly and very closely, that's going to make a big feedback loop that drives you further." — Peyman Hesami: Describing why he wanted to move from Qualcomm to a startup "The main thing is the intuition behind it and how the model works." — Peyman Hesami: Advising listeners how to learn reinforcement learning effectively

Implications: RL is a strong fit for adaptive, goal-driven, personalized systems across devices, ads, and operations. For practitioners, the key is choosing the right problem and defining rewards well; for industry, RL promises efficiency, innovation, and less reliance on labeled data.