The COGPED Completed Cycle Audit

Tim Swanwick

Director of Postgraduate General Practice Education

London Deanery

what are your issues?

in pairs (5 minutes)

what are my issues?

really useful assessment very clear marking schedule no one should fail I don’t want to have to advise on so many

resubmissions!

marking schedule

1. Reason for choice of audit

2. Criterion/criteria chosen

3. Standards set

4. Preparation and Planning

5. Data collection 1

6. Changes to be evaluated

7. Data collection 2

8. Conclusions

Potential for change & relevant to practice

Relevant to audit subject and justifiable e.g. current literature

Target towards a standard with a suitable timetable

Evidence of teamwork and adequate discussion where

appropriate

Results compared against standard

Actual example described

Comparison with Data collection (1)

Summary of main issues learned

exercise

in threes “I’m going to send this in, what do you think?”

reason for choice of audit

potential for change & relevant to practice

• potential for change - can do something about problems

• relevant to practice – hospital OK – e.g. emergency admissions

common errors

• over-complicated

• too ambitious

• no potential for change

• irrelevant to practice

criterion/criteria chosen

relevant to audit subject and justifiable

• relevant – is it about the chosen subject?

• justifiable – is it a suitable thing to measure?

common errors

• no understanding of a criterion

• not “inclusion criteria”

• too many criteria

• nested criteria

• criteria confused with standards

what is a criterion?

a criterion is a statement of quality a criterion is meaningful and measurable “all” and “should” rule a criterion requires a measurable aspect of care , a

quality marker and a “population”

The last recorded BP of all patients with diabetes should be130/80 or less

All telephone calls into the surgery should be answered within five rings

examples of criteria (exercise)

In threes, give an example of a criterion in each of the following categories– chronic disease– prescribing– management of acute conditions– practice administration– screening

standards set

target towards a standard + suitable timeframe evidence that might be used

• NHS or local targets

• guidelines

• expert advice

• focus group

• benchmarking

• practice data on other audits

common errors

• confused with criteria

• not justified

• 100% usually over-zealous

what is a standard?

a figure, usually a % every criterion must have one must be justified

preparation & planning

evidence of teamwork and adequate discussion where appropriate

• teamwork is an absolute !!

• reasonable description of process

common errors

• “I did it my way…”

data collection 1

result compared with standard

• a grid makes life easier common errors

• “simply, too many notes”

• data not related to criteria

• data not compared to standard

• discrepancies unexplained

changes to be evaluated

actual examples described

• short term and..

• … long term: “what will happen in the practice when I’m gone?”

• group work

• systems, practice procedures etc

common errors

• no changes described at all

• no sustainable changes: “I wrote a letter to all the patients…”

data collection 2

comparison with data collection 1

• again a clear grid is a big help

• results may be better, worse or the same

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Standard 1st Data 2nd Data

Audit results

data collection 2 (continued)

common errors data 2 not collected in same

way as data 1 only the patients subjected to

short term intervention counted

results not compared comparison not adequately

discussed

conclusions

summary of main issues learned

• specific points about the audit

• general points about the process

common errors

• rare to fail just this

• inadequate discussion

• unjustified assertions

results

During the the period 1.4.02-31.3.03 214 GPRs passed SA in London 12 through NPMS (5.6%) 202 through audit 22 of these were resubmissions (c.10%) 2 failed during this period (c.1%)

referral

there’s no failure, only feedback

the vast majority can be rewritten

“how to do it” guide on deanery web site www.londondeanery.ac.uk

guidance on resubmission (only) from TS if required

exercise

“my audit’s been referred – help!”

questions and answers?

National Project Marking Schedule

National Project Marking Schedule (NPMS)

Approved by the JCPTGP from April 2000 A project:

– addresses a defined problem– is related to previous work– presents qualitative or quantitative findings– interprets these findings– draws conclusions from the evidence presented

NPMS

competencies tested:

– the ability to construct a logical argument– the ability to communicate in written English– the ability to plan and sustain activity over time

NPMS

range of acceptable work includes:– a small research study e.g.questionnaire, notes

review, interview study– a literature review– a case study– a proposal for a new service– a discussion paper

NPMS marking schedule

six criteria for the project each criterion marked from 0 – 5 a pass will score 18 or more each criterion must score 2 or more

NPMS marking schedule (continued)

aims/question/problem clearly stated relevant literature cited method appropriate relevant findings presented discussion appropriate conclusions appropriate

NPMS 2002/3

7% of all registrars April 2002- March 2003

– 126 audits– 87% first and second level passes– 13% referred

all marked by the Yorkshire Deanery

NPMS - The Future

National Project Marking Schedule to become “National”

Audit may become subsumed into trainer’s report

Marking panels in all deaneries Timescale … 2-3 years ?A role within the MRCGP