Ep. 70: Who We Are At 2 A.M.

Episode Summary

Executive Summary: The episode argues that anonymous digital traces—especially Google searches, streaming behavior, and A/B tests—reveal what people really think, want, and feel, often more accurately than surveys or self-reports. Through examples on sex, race, politics, romance, marketing, and health, it shows how big data can expose hidden bias, predict behavior, and even flag disease, while raising serious ethical questions about privacy, manipulation, and whether companies should act on what they learn.

Main Topics: Search data as a window into private life (Priority: 5/5): Shankar Vedantam and Seth Stephens-Davidowitz explain that Google searches are valuable because people use them like anonymous confessions, revealing embarrassing, intimate, or hidden concerns they would not share aloud. Surprising correlations in big data (Priority: 5/5): The conversation highlights how search correlations can uncover unexpected patterns, such as links between unemployment and porn/solitaire searches, demonstrating that digital behavior can proxy for real-world conditions. Sexual orientation, pornography, and the limits of self-reporting (Priority: 4/5): Search and viewing data suggest that surveys understate sexual interests and pornography consumption, showing a gap between what people say and what they do online. Implicit bias and racial prejudice in search behavior (Priority: 5/5): Google search patterns reveal gendered assumptions in parenting and racist attitudes across regions, including searches involving racist slurs and their relationship to political behavior. Politics and voting behavior (Priority: 5/5): Racist search volumes correlate strongly with poor Obama performance and with support for Donald Trump, suggesting that hidden racial animus can help explain electoral outcomes. Algorithms outperform intuition in entertainment and relationships (Priority: 4/5): Examples from Netflix, Facebook, and the host’s grandmother show that data often predicts preferences and relationship outcomes better than people’s own beliefs or advice from personal experience. Ethics of predictive data and health forecasting (Priority: 5/5): The episode closes by asking whether companies should use search histories to infer serious illnesses like pancreatic cancer and proactively warn users, balancing potential benefit against creepiness and privacy concerns.

Key Arguments: People often lie to surveys and sometimes to themselves; aggregated digital behavior can reveal truer patterns than direct questioning. Search terms function like anonymous self-disclosure, which is why Google data can expose sensitive topics such as sexuality, insecurity, prejudice, and health concerns. Big data can identify macro-level conditions, such as unemployment, through seemingly unrelated online activity like pornography or solitaire searches. Survey data on gay male attraction likely understates true behavior; search data suggests roughly 5% of male pornography searches are for gay porn. Google search patterns show that parents disproportionately associate sons with intelligence and daughters with appearance/weight, implying persistent gender bias. Racist search volumes are strongly associated with electoral support patterns, including poorer Obama performance and stronger Trump support in certain areas. Data can challenge folk wisdom: Facebook analysis suggested that separate friend groups may predict more durable relationships, contrary to the grandmother’s advice. Companies use A/B testing because intuition about what works is frequently wrong; behavioral experiments can outperform expert judgment. Predictive text and search behavior can forecast loan default risk, but some correlations raise fairness and legal concerns when unrelated traits, like mentioning God, are used as proxies. Health-search sequences can predict disease risk; if a pattern suggests a likely serious illness, there may be an ethical case for warning users early.

Data Points: Correlation with unemployment rate: Slutload (pornography site) was the single search most highly correlated with unemployment in the cited period - Used as an example of a surprising proxy for labor-market conditions Another unemployment-linked search: Solitaire - Illustrates that boredom/leisure searches can track joblessness Self-reported male attraction to men: About 2.5% to 3% - Survey estimate of men primarily attracted to men in the United States Gay male pornography search share: About 5% of male pornography searches - Search behavior suggests higher prevalence than surveys Pornhub viewing time in 2015: 2.5 billion hours - Used to illustrate how much pornography is actually consumed Search bias in parenting: Parents searching 'is my son' are much more likely to use words like 'gifted' or 'genius' - Indicates stronger intellectual expectations for sons Search bias in parenting: Parents searching 'is my daughter' are more likely to use 'overweight' or 'ugly' - Indicates greater appearance-focused concern for daughters Racist search measurement: Percent of Google searches including a racial slur - Used as a regional indicator of racist sentiment Regional racism pattern: Much higher in eastern U.S. than western U.S. - The episode says the divide is East vs. West rather than South vs. North Political correlation: High racist-search regions were much more likely to support Donald Trump - Described as stronger than other tested variables in some analyses Loan default risk: Mentioning 'God' made borrowers 2.2 times more likely to default - From peer-to-peer lending text analysis Loan prediction example: 2.2 times less likely to pay back - Same lending study, expressed in the transcript's wording Click-through uplift: Headline B got 108% more clicks - Boston Globe headline test on a rape-trial story Click-through uplift: Headline A got 33% more clicks - Boston Globe headline test about the first Boston subway Click-through uplift: Headline A got 38% more clicks - Boston Globe headline test about a rare baseball card Disease prediction timing: Four weeks earlier diagnosis could improve survival chances - Used in discussion of whether search platforms should warn users of possible pancreatic cancer Search-symptom pattern: Indigestion followed by abdominal pain - In Bing data, this sequence was associated with later pancreatic cancer diagnosis

Pivotal Quotes: "Google knows everything. I agree to that." — Shankar Vedantam: Opening framing of the episode on the power of search data "You do learn a lot about people that's very, very different from what they say, and kind of the weirdness at the heart of the human psyche that doesn't really reveal itself in everyday life or at lunch tables, but does reveal itself at 2 a.m. on Pornhub." — Seth Stevens-Davidowitz: On why online behavior can reveal hidden desires and impulses "The legal system is really not set up for a world in which companies potentially can mine correlations over just about everything anybody does in their life." — Seth Stevens-Davidowitz: On the policy and fairness challenges posed by predictive analytics

Implications: Big data can uncover hidden bias, preference, and risk earlier than surveys or intuition, but it also enables manipulation and unfair profiling. Listeners should expect more algorithmic prediction—and more urgent debates over consent, transparency, and when companies should intervene.