061: Discovering Data Science Workflows and The Importance of Mentorship

Episode Summary

Executive Summary: Daniel Whitenack discusses Pachyderm’s role in production-grade data science workflows, emphasizing reproducibility, simplicity, and ownership of end-to-end pipelines. He also explains when Go is useful for data science, shares his career path from physics to data science, and stresses mentorship and community involvement as key accelerators for growth.

Main Topics: Pachyderm and production data science workflows: Daniel explains Pachyderm as a platform for data pipelining, versioning, and reproducible end-to-end ML workflows, letting data scientists own deployment and pipeline logic in familiar languages. Best practices for data science workflows: He argues for simple, interpretable solutions, reproducibility, and understanding how outputs will be used before building models, especially as data science moves toward production. Go for data science and infrastructure: Daniel describes Go’s strengths—simplicity, static typing, easy deployment, and concurrency—and notes that the Go data science ecosystem is still developing but growing quickly. Career path from physics to data science: He recounts transitioning from physics academia to industry, learning data science jargon and workflows, then moving through startup, consulting, and platform-focused roles. Mentorship and community learning: He highlights mentorship as mutually beneficial and recommends local meetups and peer-to-peer programs as effective ways to enter and grow in data science. The future of data science: Daniel sees convergence between data engineering and data science, with more responsibility shifting to data scientists for infrastructure and production ownership.

Key Arguments: Data science workflows are becoming essential because modern data science is multi-stage, production-oriented, and increasingly tied to business value. Production challenges are often not about model sophistication but about making the workflow reproducible, understandable, and maintainable. Pachyderm helps teams avoid reimplementation across teams by allowing Python, R, Bash, TensorFlow, and other tools to live in one connected pipeline. Simple models are often preferable when they solve the problem, because they are easier to maintain, deploy, and explain. Reproducibility is non-negotiable; even non-deterministic systems should have known expected ranges or distributions of outcomes. Go is attractive for production data science and infrastructure because it is simple, fast to deploy, and strong at concurrency. New data scientists should first build broad, end-to-end competence in smaller, hands-on teams before specializing or moving into more diverse consulting roles. Mentorship and community participation accelerate learning by exposing people to different perspectives, jargon, and real-world practices.

Data Points: Episode number: 61 - Super Data Science Podcast episode identifier Conference speaking frequency: dozens per year - Daniel describes how often he presents at data science conferences Lafayette location: about two hours from Chicago and an hour from Indianapolis - Where Daniel is based Gopher Slack community size: 16,220 people - Public Gopher Slack membership mentioned during the interview Career timeline in industry: 6 or 7 years - Daniel estimates how long ago he transitioned from academia to industry

Pivotal Quotes: "I think the biggest challenge for me has been to kind of rein in that mindset and really focus when a new problem comes up on asking the right questions." — Daniel Whitenack: On learning to ensure data science work creates actionable business value "I think in general, and this was displayed at ODSC because everywhere you turn, there was another data platform solution or a solution to help data scientists deal with infrastructure... there is a pressure to push these two sides together." — Daniel Whitenack: On the trend toward unifying data engineering and data science "I really recommend to people that outside of everything else, we shouldn't be making excuses around reproducibility." — Daniel Whitenack: On the importance of reproducible data science workflows

Implications: Listeners should prepare for a future where data science is more production-focused, workflow-driven, and integrated with engineering. Building reproducible, simple, end-to-end systems—and learning tools like Go and platforms like Pachyderm—will become increasingly valuable.