S9 Ep31: How well does patent screening work?

Episode Summary

Executive Summary: Mark Shankerman argues that patent offices do a reasonably good job screening claims, but the process still produces meaningful type 1 and type 2 errors. Using AI and structural modeling on millions of USPTO claims, his research finds heavy applicant “padding,” substantial abandonment of valid claims, and major social costs driven mostly by legal and prosecution costs rather than the office itself.

Main Topics: How patent screening works (Priority: 5/5): Patent examiners compare each claim against prior art in a multi-round negotiation with applicants, granting protection only when claims are sufficiently distinct from what already exists. Type 1 vs type 2 patent errors (Priority: 5/5): The discussion distinguishes granting invalid claims (type 1) from refusing valid ones (type 2), emphasizing that type 2 errors are often overlooked because they are invisible. AI and structural modeling of the patent process (Priority: 5/5): Shankerman explains how machine learning measures claim similarity to prior art, enabling estimation of thresholds, errors, strategic behavior, and counterfactual policy effects. Evidence on USPTO performance (Priority: 4/5): The paper finds that while many initial claims are too close to prior art, the final grant process reduces errors substantially, supporting the view that the USPTO is doing a reasonable job. Costs of screening and litigation (Priority: 4/5): Most social costs come from prosecution and litigation, not just examiner decisions; in the U.S., litigation is especially dominant when bad patents are granted. Policy reforms and incentives (Priority: 4/5): Restricting the number of examination rounds appears to reduce costs and strategic behavior, while examiner incentives matter less than intrinsic motivation unless that motivation is weakened. Broader institutional comparisons and standards (Priority: 3/5): Shankerman says future work should compare the USPTO with the EPO, China, Japan, and Korea, and also test whether the underlying patentability standards themselves are set too high or too low.

Key Arguments: Patent screening is fundamentally about deciding who deserves temporary property rights over inventions, not simply rewarding all valuable ideas. The economically correct standard is not whether an invention is valuable, but whether it would have been developed without patent protection and generates social welfare. Granting an invalid patent causes higher prices and, in the U.S., expensive litigation; these are type 1 errors. Refusing a valid patent reduces incentives to innovate and is a type 2 error, which is often underestimated because it is not visible. AI-based similarity measures can approximate the examiner’s prior-art comparison and quantify whether claims are too close to prior inventions. The USPTO appears to reduce initial error-heavy applications substantially through negotiation; final grants are much cleaner than initial filings. A large share of applicants strategically “pad” claims to maximize scope while staying above the rejection threshold. Type 2 errors may be more frequent than type 1 errors in this dataset, even if their social costs need separate evaluation. Most social costs of screening come from legal/prosecution processes, not the patent office’s internal operations. Limiting the number of response rounds could cut social costs by reducing strategic back-and-forth and litigation-related expense. Examiners appear to have strong intrinsic motivation; extrinsic credit incentives matter less unless intrinsic motivation is removed. The current analysis evaluates performance against legal standards, but the standards themselves may still be too strict or too loose.

Data Points: Patent term: 20 years - Property rights over inventions typically last two decades. Initial claims below threshold: About 81% - Share of independent claims in first applications that are too close to prior art. Granted patents with at least one invalid claim: About 13% - Share of granted patents containing at least one claim below the threshold. Granted claims below threshold: About 6% - Overall share of granted claims that should not have been granted. Abandoned applications with at least one valid claim: 32% - Share of abandoned applications containing at least one claim that should have been granted. Abandoned claims that are valid: 16% - Overall share of abandoned claims that were above the threshold and thus valid. Cohorts studied: 2011, 2012, 2013 - USPTO application cohorts used in the paper. Applications studied: About 1 million - Total patent applications in the sample. Claims studied: About 20 million - Number of claims across the sampled applications. Examiner decisions: About 55 million - Decisions by examiners across rounds of negotiation. Historical prior-art comparison base: Back to 1976; about 100 million claims - Database used to compute similarity to previous claims. Negotiation rounds: About 2.5 on average; most done after 6 rounds - Typical examination process length. Claim narrowing per round: About 35% - Estimated reduction in claims over each round of negotiation. Social cost of screening: About $15.4–$15.5 billion - Estimated annual social cost of screening per cohort of applications. Share of screening costs from errors: About 25% - Portion of social cost attributable to type 1 and type 2 errors. Share of screening costs from prosecution/litigation: About 75% - Majority of screening cost due to office process, legal fees, and U.S. litigation. Share of private-sector R&D: About 5% to 6% - Screening social cost relative to private-sector performed R&D.

Pivotal Quotes: "I think the answer must be yes." — Tim Phillips: Opening framing question about whether the Patent Office is doing a reasonable job. "The key economic objective ... is that you want to give property rights to inventions that would not otherwise be developed if you didn't give them property rights." — Mark Shankerman: Explaining the economic rationale for patents and screening. "About 32%, so about a third, of all abandoned applications have at least one valid claim." — Mark Shankerman: Key result showing type 2 errors are common.

Implications: Patent offices appear better than anecdote suggests, but system design still matters: reducing rounds, managing legal costs, and improving standards could cut waste. Future research may redefine best practice across major patent systems.