841: Andrew Ng on AI Vision, Agents and Business Value

Episode Summary

Executive Summary: Andrew Ng argued that most companies should prioritize building agentic AI applications over training frontier models, since API costs are falling rapidly and effective workflows can beat larger models on practical tasks. He highlighted the rise of multi-step vision systems, the role of code-generating agents in visual AI, and the growing opportunity to apply generative AI to vast stores of unstructured data across industries.

Main Topics: Agentic workflows vs. larger foundation models: Ng said most businesses should focus on building value with agentic AI workflows rather than attempting to train or compete with frontier LLMs, which requires massive capital and is only practical for a few giants. Falling cost of generative AI: He emphasized that model/API costs have dropped sharply, making cost optimization less urgent at the start and reducing the barrier to adoption for enterprises. Society of mind and modern AI: Ng connected historic ideas about many simple agents working together with the transformer era, suggesting that both the underlying algorithm and the richness of data drive intelligent behavior. Vision agents and visual AI: He described Landing AI's VisionAgent approach as using planning, tool use, and code generation to handle complex image tasks more accurately than zero-shot prompting. Unstructured data as the next frontier: Ng argued that most data is unstructured and that multimodal AI, especially vision systems, will unlock major new applications beyond classic manufacturing and healthcare. Safety, hallucinations, and user guardrails: He discussed how probabilistic AI outputs require product design safeguards, user training, and confirmation flows rather than expecting deterministic behavior.

Key Arguments: Most companies should build applications with agentic workflows instead of spending billions to train frontier models. The cost of using generative AI APIs is falling quickly, so teams should optimize for value creation first and cost reduction later. The hardest problem is still making something useful work at all; premature optimization can distract from product-market fit. A smaller or cheaper model can outperform a larger one when paired with better agentic orchestration and task decomposition. Modern AI echoes both the society-of-mind idea and the single-algorithm/deep-learning view: one strong algorithm plus rich data can produce broad capabilities. Vision tasks often need stepwise plans and code-executed tool use, not just a single image prompt, to achieve reliable results. Agentic systems can generate code, choose libraries/models, and test pipelines, lowering the barrier to building practical visual AI applications. AI safety should rely on guardrails such as confirmations, UI design, and training, because even traditional ML and search are not perfectly deterministic. Hallucination risk has decreased relative to earlier AI systems, though it remains important in high-stakes settings. Visual AI has broad potential in media indexing, security, robotics, self-driving systems, and healthcare, not just manufacturing.

Data Points: GenAI API cost decline: ~80% year over year - Ng said the cost of using generative AI APIs has fallen rapidly over the last year and a half. Frontier-model budget: a few billion dollars - He estimated this as the scale of investment needed to compete with major frontier model labs. Weekend coding example: about a $5 OpenAI bill - Ng described spending a day experimenting and ending up with a very small API bill. Possible higher agentic spend: tens of dollars to low hundreds of dollars - He noted some agentic workloads can become more expensive, but still less than many expect. Coursera scale: 300 universities to over 100 million students - Referenced in the episode introduction to highlight Ng's background and impact.

Pivotal Quotes: "I think almost all companies, with the exception of a few giants, should be focusing on building applications using agentic workloads." — Andrew Ng: He answered whether companies should build better workflows or pursue larger models. "The hardest thing is just building something that works. That's still pretty hard." — Andrew Ng: He explained why teams should prioritize usefulness before cost optimization. "I think there'll be a lot. And I find it, I'm going to give that unsatisfying answer. It's a little bit like where will electricity be used?" — Andrew Ng: He was asked where visual AI will be most transformative beyond traditional industries.

Implications: Teams should build practical agentic systems now, using the best available models and adding guardrails, rather than chasing frontier-model parity. The biggest near-term opportunity is in multimodal and vision applications that convert unstructured data into operational value.