patent public advisory committee

Patent Public Advisory Committee Quarterly Meeting

Clarity of the Record Pilot

Robin Evans Director, Technology Center 2800

November 3, 2016

This program is to develop best Examiner practices for enhancing the clarity of various aspects of the prosecution record and then to study the impact on the examination process of implementing these best practices.

Purpose

3

Goals

• Identify best practices

• Find the correct balance for appropriate recordation

• Use data/feedback to assist other programs, such as:– Refinement of the Master Review Form (MRF)– The reevaluation of examination time

4

Areas of Focus• Enhanced documentation of claim interpretation

– Special definitions of claim terms– Optional language– Functional language– Intended use or result (preamble and body of claim)– Non-functional descriptive material– "Means-plus-function" (35 U.S.C. §112(f))– Computer-implemented functions that invoke 35 U.S.C. §112(f) ("specialized" or

"non-specialized")

• More precise reasons for allowance

• More detailed interview summaries

• Pre-search interview – Examiner’s option5

Pilot Participants

• Examiners– 125 participants (2/3 primary and 1/3 junior)– GS 11-15, with at least two years of experience– Randomly selected individuals, who met the requirements

for participation, were invited to volunteer for the Pilot• Supervisors (SPEs)

– 45 participants

6

Examiner Participant Duties

• Enhance clarity of Office actions for applications in the pilot

• Attend Pilot-specific training and quality enhancement meetings (QEMs)

• Record amount of time spent on Pilot activities

7

Supervisor Participant Duties

• Manage Pilot-specific QEMs and group training

• Review Office actions using the Master Review Form (MRF)

• Provide individual feedback and assistance

8

Evaluation• Approximately 2,600 applications were reviewed:

– Pilot Reviews (Examiners participating in the Pilot)• Pre-Pilot cases – office actions completed prior to Pilot start date• Pilot treated cases – actions completed using pilot guidelines/training• Pilot non-treated cases – actions not assigned/no requirement to follow

Pilot guidelines– Control Group Reviews (Examiners closely matched with Pilot Examiners

(e.g. same technology and GS Level))• Examiner Best Practices were gathered during:

– Initial Pilot Training – Pilot Specific Quality Enhancement Meetings (QEMs) with

Examiners• QEMs were held at least monthly

– Focus Sessions held with Pilot SPES9

Results and Recommendations –Interview Summaries• Identified best practices (also key drivers) for improving

the clarity of interview summaries included:– Adding the substance of the Examiner’s position – Providing the details of an agreement, if reached – Including a description of the next steps that will follow the interview

• Recommendations: – Provide corps-wide training on enhancing the clarity of interview summaries that

focuses on the identified best practices/key drivers– Consider whether to require examiners to complete more comprehensive

interview summaries – Continue to evaluate Pilot cases to see whether improved interview summary

clarity has a long-term impact on prosecution10

Results and Recommendations –112(f) Limitations• Identified best practices (also key drivers) for improving

the clarity of 112(f) presumptions included:– Explaining 112(f) presumptions and how the presumptions were

overcome (when applicable)– Using the appropriate form paragraphs – Identifying in the specification the structure that performs the function

• Recommendation:– Consider whether to require examiners to use the form paragraph which

will result in a more comprehensive explanation of means-plus-function limitations

11

Results and Recommendations –Claim Interpretation: Prior Art Rejections• Identified best practices (also key drivers) for

improving overall clarity of prior art rejections included: – Clearly addressing all limitations in 35 USC 102

rejections when claims were group together – Explaining the treatment of intended use and non-

functional descriptive material limitations in 35 USC 103 rejections

12

Results and Recommendations –Claim Interpretation: Prior Art Rejections (cont.)• Some identified best practices both added to and

detracted from overall clarity– Providing, in 35 USC 102 rejections, an explanation for

limitations that have been identified as inherent– Providing, in 35 USC 103 rejections, annotations to pin-point

where each claim limitation is met by the references• Recommendation:

– Assess how to use the identified best practice of recording claim interpretation to improve the clarity of Office actions withoutdetracting from clarity

13

Results and Recommendations –Reasons for Allowance• Identified best practices for improving the clarity:

– Identifying the allowable subject matter rather than merely reciting the entire claim – Specifying applicant’s persuasive arguments– If reasons for allowance previously identified during prosecution, providing the

Office action where reasons for allowance can be found– Addressing each independent claim separately

• Recommendations:– Provide corps-wide training on enhancing the clarity of reasons for allowance that

focuses on the identified best practices/key drivers– Consider whether to require Examiners to complete more comprehensive reasons

for allowance14

Results – Additional Practices that did not Significantly Impact Overall Clarity• Providing an explanation regarding the patentable weight

given to a preamble• Providing an explanation of how relative terminology in a

claim is being interpreted• Providing an explanation for how a claim limitation that was

subject to a rejection under 35 USC 112(b) has been interpreted for purposes of applying a prior art rejection

15

Pilot Implementation - Hours• Examiners were not limited on the amount of non-

production time they could claim while working on Pilot-treated cases

• In total, examiners logged approximately 5400 hours on their Pilot cases

• On average, examiners used less than 4 hours per biweek of non-production time

• There was no difference in the number of hours claimed by primary examiners compared to junior examiners

16

Next Steps • Surveys

– Internal surveys sent to Pilot Examiners– Data currently being collected

• Additional Quality Chats on Clarity– Gather information/thoughts on any differences seen during Pilot time period– Share data results of Pilot– Discuss/share best practices

• Focus Sessions with Pilot Examiners– Are best practices still being used?– Discuss amended cases resulting from Pilot

17

Next Steps (Cont.)

• Monitor Pilot Treated Cases– Are applicant’s arguments more focused?– Average time to disposal compared to pre-pilot cases?

• Recommendations for Implementation– Discuss implementation of training and best practices in all

Technology Centers– Discuss further efforts to enhance claim interpretation

including key drivers that didn’t significantly impact clarity– Extend & Expand Pilot to gather additional data analysis

18

Additional Information

19

http://www.uspto.gov/patent/initiatives/clarity-record-pilot

http://www.uspto.gov/patent/initiatives/clarity-record-pilot

Questions?


Post-Prosecution Pilot (P3)

Jerry LorengoDirector, Technology Center 1600

November 3, 2016

Post-Prosecution Pilot (P3)

• Advanced as a program of the Enhanced Patent Quality Initiative (EPQI) under Pillar 3, Excellence in Customer Service

• Developed to test its impact on enhancing patent practice during the period subsequent to final rejection and prior to the filing of a notice of appeal

3

Overview• Retains popular features of the Pre-appeal Brief Conference

Pilot and After Final Consideration Pilot (AFCP) 2.0 programs: Consideration of 5-pages of arguments after final Consideration of non-broadening claim amendments after

final• Adds requested features: Presentation of arguments to a panel of examiners Explanation of the panel’s recommendation in a written

decision after the panel confers

4

P3 Timeline• Pilot began July 11, 2016• Runs six (6) months or upon receipt of 1,600 compliant

requests, whichever occurs first– 200 per Technology Center

• Formal comments about P3 will be received through November 14, 2016 at [email protected]

5

mailto:[email protected]

P3 Statistics

6

P3 Statistics

7

P3 Statistics

8

Reopen+ Allow = 37.5%

P3 Statistics

9

P3 Statistics

10

Outcomes

11

P3 – Looking Ahead Consider:

– Internal and external survey results– Formal Comments from FR Notice– Stakeholder feedback about the program from other

sources Decide:

– Whether or not to continue the program, optionally with modifications

12

For More Information on P3• Visit our website:

http://www.uspto.gov/patent/initiatives/post-prosecution-pilot– Program details and forms– Examiner training materials– FAQs

• Contact us by email: [email protected]

13

http://www.uspto.gov/patent/initiatives/post-prosecution-pilot

mailto:[email protected]

Thank You!


Quality Metrics

Greg Vidovich Associate Commissioner for Patent Quality

November 3, 2016

Historical Perspectiveon Measuring Patent Quality

3

Review of Allowances

in the Office of Patent Quality

Assurance (OPQA)

Add review of In-Process

Office actions by OPQA

External Quality Survey

administered by OPQA

Quality Index

Reports (QIR) begin

Quality Composite

Score begins

Where are we today?

FY 1983 FY 2005 FY 2011FY 2008FY 2007 FY 2017

FY 2011 – FY 2015Product IndicatorsMaster Review FormCapturing both correctness and clarity of examiners’ final work product using uniform criteria gathered in a single database

Process IndicatorsTransactional QIRTracking the efficiency and consistency of our processes (for example, to identify “churning”)

Perception IndicatorsSurvey ResultsContinuing to internally and externally poll perceptions of patent quality

Composite Score

Moving ForwardFinal Disposition Compliance

In-Process ComplianceFirst Action (FAOM) Review

Search ReviewQuality Index Reporting (QIR)

External Quality SurveyInternal Quality SurveyComposite Score

Quality Metrics Redefined

4

Key Product Indicators

Correctness

Clarity

5

Product IndicatorsMaster Review Form

Capturing both correctness and clarity of examiners’ final work product using uniform criteria gathered in a single database

Process IndicatorsTransactional QIR

Tracking the efficiency and consistency of our processes (for example, to identify “churning”)

Perception IndicatorsSurvey Results

Continuing to internally and externally pollperceptions of patent quality

6

Key Product Indicators – MRF Based• Metrics are derived from reviews done using the new

Master Review Form (MRF) • The MRF:

– Is a single, comprehensive tool that can be used by all areas of the Office to consistently review finished work product

– Collects information on clarity and correctness– Collects review results in a single data warehouse for more

robust analysis

OPQA Reviews of Finished Office Actions

FY 2015• Completed 7,900 reviews

FY 2016• Completed 12,000 MRF reviews

FY 2017• Projected to complete 18,500 MRF reviews • Data captured at TC Level

7

8

Key Product Indicators – Correctness

• Correctness metrics will show compliance rate by statute

• Compliance Rate = Total Reviews – Non-Compliant ReviewsTotal Reviews

• Non-Compliant Reviews = Omitted + Improper Rejections

• The total number of reviews will remain constant for all statutes and includes those reviews that USPTO’s Office of Patent Quality Assurance conducts on randomly-sampled Office actions

9

Key Product Indicators – Clarity

• The USPTO is working on developing clarity metrics

• The Office is continuing to work on ensuring that the MRFcaptures clarity data as accurately as possible

• The USPTO is analyzing the MRF’s clarity data for purposes of identifying quality trends

Clarity Standards• Clarity questions are assessed as follows:

– Average– Below– Above average

• Definition of AVERAGE drives the definition for above and below– Average clarity is the level of clarity expected for

Office Actions from the great majority of examiners that is sufficient to allow anyone reviewing the Office Action to readily understand the position taken.

Key Process Indicators

11

Reopening Prevention

Consistency of Decision-Making

Rework Reduction







12

Key Process Indicators – Approach

• Focus on three process indicators from our Quality Index Report (QIR)– Reopening Prevention– Rework Reduction– Consistency of Decision Making

• Use data to identify outliers for each indicator for further root-cause analysis

• Based on root-cause analysis, work to either capture any identified best-practices or train examiners, as appropriate

Example – Rework Reduction

Metric is sum of transactional QIR data points including consecutive finals, consecutive restrictions, and 2nd+ non-finals

Note: Instances of rework impacted by Alice Corp. v. CLS Bank decision 12

Key Perception Indicators

14

Root Cause Analysis

Validation/Verification







15

Key Perception Indicators

• USPTO has conducted internal and external perception surveys semi-annually since 2006– External survey is of 3,000 frequent-filing

customers– Internal survey is of 750 randomly selected

patent examiners• The survey results will be used to validate other

quality metrics

Perception Survey Results -Example

16

Frequency of Technically, Legally, and Logically Sound Rejections (Percent reporting “most” or “all” of the time)

Next Steps

• Publish correctness targets• Develop clarity metrics• Use indicators to identify areas for

improvement

17

Questions?

patent public advisory committee

Documents