public service broadcasting: the bbc’s performance...

Public service broadcasting: the BBC’s performance measurement framework NAO review presented to the BBC Governors’ Audit Committee,May 2005, by the Comptroller and Auditor General, and a response to the review from the BBC

British Broadcasting CorporationBroadcasting HouseLondon W1A 1AA

This report is available online atbbcgovernors.co.uk

for blank pages 17/6/05 11:53 am Page 5

In Building public value the BBC committeditself to introducing a new system ofperformance monitoring based onobjectivity, rigour and transparency.The objective was to strengthen the Board of Governors’ ability, on behalf oflicence fee payers, to ensure that BBCservices deliver against their wider publicpurpose.The performance measurementframework is the first of a number ofrelated initiatives being introduced as partof this commitment.

This external Value for Money review,commissioned by the Governors’ AuditCommittee, considered public servicebroadcasting in terms of the BBC’sperformance measurement framework.It was undertaken by the National AuditOffice, following the agreement in 2003between the Government and the BBC to evolve the BBC Governors’ overview of value for money into a programme ofreviews.This review focused on the keydrivers of public value – reach, quality,impact and value for money.

Governors welcome the independentreview’s confirmation of the framework as a rigorous and appropriate method toassess the delivery of public value, with theBBC appearing to be at the “forefront ofcurrent thinking on the topic”. However,we recognise there is further work to bedone and endorse BBC management’sresponse to the report’s recommendations.

The NAO report highlights evidence ofthe framework beginning to “influencethinking and decision-making about publicservice broadcasting within the BBC”.Thisgoes to the heart of the framework’spurpose, fully aligning the BBC’s decision-making with the public interest. Governorsbelieve that the framework provides botha consistent and coherent framework

within which decisions can be made andmeasures against which the BBC’s deliveryof public value can be assessed.Wesupport the NAO’s recommendation thatperformance reports should be tailored tothe needs of different audiences. Governorswill work with management both tofacilitate assessment of key performanceindicators and, as custodians of the publicinterest in the BBC, to ensure the deliveryof public value to licence fee payers.

Governors accept that there is a need toreflect the performance framework in theBBC’s overall objectives for public servicebroadcasting and that the link betweenperformance monitoring and theCorporation’s public service broadcastingobjectives can be made more explicit.Thereformed governance system, as set out inthe Green Paper, has implications for thesystem and structure of performancemonitoring, which will in turn influence theways in which the proposed BBC Trustaccounts for the BBC’s performanceagainst key measures in the Annual Reportand to Parliament.The details of thesechanges are currently being worked outand Governors will outline their plans indue course.

In summary, the Board of Governors ispleased to note the NAO’s conclusionthat “The BBC has made good progress indeveloping the performance measurementframework [with] important buildingblocks in place…”We have consideredand approved the attached response fromBBC management.

Board of GovernorsJune 2005

Public service broadcasting: the BBC’sperformance measurement frameworkBBC response to the National Audit Office Value for Money study

The performance measurementframework, introduced in autumn 2004, isevidence-based and designed to provideBBC Governors and management, licencefee payers and other stakeholders withassurance that BBC services are deliveringpublic value.

The key focus of the framework reviewedby the NAO is on the four drivers ofpublic value – reach, quality, impact andvalue for money.The NAO has recognisedthat these drivers share much commonground with the characteristics of publicservice broadcasting set out by Ofcom inits review of public service broadcasting.

Response to the report’s key findings

The NAO has acknowledged within itsreport the BBC’s achievements to date in introducing the framework. In particularit notes that the new performancemeasurement framework arrangementshave the potential to work for the BBC on two levels. In the broadest terms, thedrivers of reach, quality, impact and valuefor money give the BBC a commonframework and language for thinking aboutpublic service broadcasting. At a detailedlevel, the technical measurement ofperformance against these four driversprovides an evidence base for decision-making, performance monitoring and accountability.

The study also recognised that the BBC’sdrivers of public value is consistent withthe Government’s Green Paper on thereview of the BBC’s Royal Charter. Inaddition, the BBC’s continuing commitmentto developing the framework is reflectedthrough the acknowledgment that theBBC appears to be at the forefront ofcurrent thinking on the topic andcontinues to liaise with other publicservice broadcasters in Europe and beyond.

The NAO does, however, also note thatthere is further work required indeveloping and refining the current system,which the BBC should address over thenext 12 months.The BBC thereforewelcomes the input of the NAO into theframework at this stage of developmentand sets out below how theserecommendations will be progressed.

Responding to therecommendations and conclusions

Since the review fieldwork betweenJanuary and March 2005, the BBC hasundertaken a number of actions.

•The Quarter 1 2005 performancereport has been published, with a moreuser-friendly opening section (TalkingPoints) which highlights the key issuesfor the period.

• Detailed objectives have been set for alldivisions in the BBC, with clear targetsagreed by the Director-General. Outputcommissioning divisions have been set,Reach, Quality, Impact and Volume(RQIV) targets (for example, increasereach by X%, improve ‘channel best for’scores from X to Y.) Output divisionswithout commissioning responsibilitieshave been given relevant quality andvalue for money targets.These will be monitored via the quarterlyreporting process.

• An internal reporting group has beenset up to look at more effective waysof reporting financial indicators withinthe RQIV framework.

The BBC welcomes the NAO’s input intothe continuing development andrefinement of the performancemeasurement framework.The reportdraws attention to the following areaswhere further work could be undertakenby the BBC over the next 12 months.

• Establish clear definitions of reach, quality,impact and value for money and thesupporting performance measures to helpachieve consistency across the organisation.

The BBC accepts this recommendationand aims to integrate the formaldivisional targets and supportingmeasures into the reporting frameworkover the summer. Once this frameworkhas been produced a performancemeasurement ‘manual’ will be producedfor users across the organisation, to beimplemented by the September 2005quarterly performance report.

• Assess the extent to which theperformance measurement framework is embedded at all levels of the BBC.

The BBC recognises that theframework may take some time tobecome embedded throughout thewhole organisation and proposes toconduct regular reviews with therelevant divisions in order to identifyany issues and recommend ways ofembedding the framework moreeffectively.These reviews will be carriedout at least once a year.

• Ensure that there is an explicit focus onproviding assurance about the accuracyand reliability of the performancemeasurement framework.

The BBC is confident that appropriateassurance provisions are in place withits main research suppliers (BARB,RAJAR and TNS). All performance dataprovided for external publication, forexample in the BBC’s Annual Reportand Accounts, are currently subject toan internal assurance process. However,the BBC recognises the lack of a formalassurance programme across the wholeperformance reporting process to date.The BBC’s internal audit function willundertake an operational review ofperformance reporting by the end of

Public service broadcasting: the BBC’sperformance measurement frameworkBBC management response

2005, when the new system will havebeen through a full year’s reportingcycle.This review is likely to berepeated on an annual basis.

• Make the performance reports more user-friendly, principally by tailoring the reportsto meet the needs of different audiencesand by differentiating between priorityand less important information.

Different versions of the reports arealready being produced for board leveland general staff consumption.We willclosely monitor user satisfaction of thereports as part of the ongoing reviewprocess. Individual performance targetshave been agreed with BBC outputdivisions for 2005/2006 and these willbe highlighted in the September reportto draw the distinction between formallyagreed targets and other measures.

In addition to the points raised above, twofurther areas for the BBC to address arehighlighted by the NAO.These areaddressed in the Governors’ response.

• Reflect reach, quality, impact and value formoney in the BBC’s overall objectives forpublic service broadcasting.

• Set a timetable for reporting on reach,quality, impact and value for money toParliament and the public.

Public service broadcasting: the BBC’s performance measurement framework

REVIEW BY THE COMPTROLLER AND AUDITOR GENERALPRESENTED TO THE BBC GOVERNORS’ AUDIT COMMITTEE | 27 May 2005

5617VC_Book.indb I5617VC_Book.indb I 15/06/2005 18:34:3015/06/2005 18:34:30

This report has been prepared under Clause 10B of the amendment to theAgreement between the Secretary of Statefor Culture, Media and Sport and the BBC,dated 4 December 2003.

John BournComptroller and Auditor GeneralNational Audit Office

The National Audit Office study team consisted of:

Laura Brackwell, Keith Hawkswell,Jonathan Mackay and Ailbhe McNabola

A link to this report can be found on theNational Audit Office web site at www.nao.org.uk

For further information about theNational Audit Office please contact:

National Audit OfficePress Office157-197 Buckingham Palace RoadVictoriaLondonSW1W 9SP

Tel: 020 7798 7400

Email: [email protected]

© National Audit Office 2005

CONTENTS

EXECUTIVE SUMMARY 1

PART 1

Introduction to the performance 4measurement framework

Where does the performance measurement 5framework sit within the BBC?

How do the BBC’s drivers of public value link 6with current thinking on the characteristics ofpublic service broadcasting?

How does the BBC’s approach to measuring 7public service broadcasting compare with overseas?

5617VC_Book.indb II5617VC_Book.indb II 15/06/2005 18:34:5215/06/2005 18:34:52

PART 2

The design of the performance 8measurement framework

Has the BBC identified performance measures for 9the four drivers of public value?

Does the performance measurement framework 12focus on the BBC’s objectives?

Does the performance measurement framework 12cover all the BBC’s public service broadcasting?

Do the performance measures allow the BBC to 13benchmark its performance with others?

PART 3

The data systems supporting the 14performance measurement framework

Are systems in place to generate comprehensive 15and timely data?

Does the BBC have adequate quality assurance 17arrangements?

PART 4

How performance information is being 18used and reported

Is the performance measurement framework being 19used to infl uence thinking in the BBC?

Are the arrangements for reporting 20performance effective?

Does the performance measurement framework 20provide a basis for the BBC to be externallyaccountable?

APPENDIXStudy methods 22

Photographs courtesy of the BBC

5617VC_Book.indb III5617VC_Book.indb III 15/06/2005 18:34:5215/06/2005 18:34:52

executive summary

EXECUTIVE SUMMARY

5617VC_Book.indb IV5617VC_Book.indb IV 15/06/2005 18:34:5415/06/2005 18:34:54

executive summary

PUBLIC SERVICE BROADCASTING: THE BBC’S PERFORMANCE MEASUREMENT FRAMEWORK 1

1 This report is about the new performancemeasurement framework which the BBC began to introducein Autumn 2004. The framework is a new evidence-based system of performance measurement designed to giveBBC Governors and Management, licence fee payers and other stakeholders assurance that BBC services aredelivering public value. The framework is part of thearrangements set out in ‘Building PublicValue’, which theBBC published in June 2004 to contribute to the debate on the review of its Royal Charter which expires in 2006.

2 At the heart of the performance measurement framework are four key drivers of public value identifiedby the BBC – reach, quality, impact and value formoney. These drivers share much common ground with the characteristics of public service broadcasting set out by Ofcom1 in its review of public service televisionbroadcasting. The BBC’s drivers are also consistent with the Government’s recent Green Paper on the Review of the BBC’s Royal Charter. The BBC’s framework is similar to approaches being developed by public servicebroadcasters overseas and the BBC appears to be at theforefront of current thinking on this topic.

3 The BBC is continuing to develop and refine theframework and to an extent is learning as it goes. So our aim has been to review progress and contribute to theBBC’s thinking on the further work to be done. In doingour work, we were guided by the good practice principles set out in the publication ‘Choosing the right FABRIC - aframework for performance information’2, which identifies

the key components of a good performance measurement system (set out in the Appendix to this report). Against this background, we considered:

p the design of the performance measurement framework;

p the data systems supporting the performancemeasurement framework;

p how performance information is used and reported.

4 Our work was not about the BBC’s editorial andprogramming judgements or whether BBC services fulfilits public service broadcasting remit, but was about thedesign and implementation of the new framework. As it is still in its early days, we considered it too soon to conduct a detailed test of how well the BBC matches up to the‘FABRIC’ guidance. Instead we were concerned with thegeneral direction the BBC is going in with the framework.Our approach to the exercise is outlined in the Appendix.

Our main findings5 The BBC has made good progress in developing the performance measurement framework. Important building blocks are in place in that the BBC has identifiedperformance measures, data collection is underway, andperformance is being reported to Senior Management and the Board of Governors. There is also evidence of theframework beginning to influence thinking and decisionmaking about public service broadcasting within the BBC.

1 Ofcom (the Office of Communications) was established by the Communications Act 2003 as the independent regulator and competition authority for theUnited Kingdom communications industries, with responsibilities across television, radio, telecommunications and wireless communications services.

2 Choosing the right FABRIC – a framework for performance information (HMTreasury, Cabinet Office, National Audit Office, Audit Commission and Office for National Statistics).

5617VC_Book.indb 15617VC_Book.indb 1 15/06/2005 18:35:0115/06/2005 18:35:01

executive summary

PUBLIC SERVICE BROADCASTING: THE BBC’S PERFORMANCE MEASUREMENT FRAMEWORK2

6 On the design of the performance measurementframework, the BBC has identified a package of measures which will support its assessment of performance for each of the four drivers of public value – reach, quality,impact and value for money. However, at present there areinstances of different BBC divisions using the same measure to assess different drivers – formalising definitions for useacross the organisation would help achieve consistency.

7 In designing the performance measurement framework, the BBC has identified a range of quantitativemeasures, which give it a clear basis for trackingperformance over time and benchmarking against other broadcasters. However, many of the measures are necessarily perception-based and, as the BBC recognises, the results in these areas tend to require moreinterpretation than in others where the data is harder.

8 To get the most out of the performance measurement framework, there should be a direct link between theframework and the BBC’s objectives for the delivery ofpublic service broadcasting. The BBC is in the process ofagreeing objectives for individual broadcasting divisions in reach, quality, impact and value for money terms but has not linked objectives for the organisation as a whole to theperformance measurement framework. The framework is being applied in full in six of the eight BBC broadcastingdivisions funded by the licence fee – the two exceptions aredivisions which supply programming to the other divisions and have less influence over the performance drivers.

9 On the data systems supporting the performancemeasurement framework, the systems the BBC has inplace are generating data for each of the four performancedrivers. Some of the systems are longstanding, using well established industry sources such as BARB (theBroadcasters’ Audience Research Board); and the BBC has introduced others to improve the quality and timeliness of the data, including a new Internet survey designed togenerate richer data on quality and impact. At present the BBC’s quality assurance arrangements do not focus explicitly on the performance measurement framework.

10 On the use and reporting of information generatedby the performance measurement framework, theperformance drivers – reach, quality, impact and valuefor money – are becoming common currency at senior levels in the BBC and work to embed the framework throughout the organisation is continuing. Performance reports are produced and considered quarterly at Boardlevel but there is scope to make them sharper andmore user-friendly, and to differentiate between priorityinformation and material that is of secondary importance.The reports are also not tailored to different audiences – at present the Board of Governors and Senior Management Boards receive the same report.

11 As regards external reporting, ‘Building PublicValue’ stated the Governors’ intention to use the measures of reach, quality, impact and value for money to set objectives for the BBC and its services each year, and that they would form the basis for the BBC’s Annual Report toParliament and the British public.


executive summary


12 The new performance measurement arrangements have the potential to work for the BBC on two levels. In the broadest terms, the drivers of reach,quality, impact and value for money give the BBC a common frameworkand language for thinking about public service broadcasting. At a detailedlevel, the technical measurement of performance against these four drivers provides an evidence base for decision making, performance monitoringand accountability.

13 The BBC is continuing to refine the performance measurement frameworkand the main text of this report identifies a number of areas where there is further work to be done. Over the next 12 months, it will be particularlyimportant for the BBC to:

p reflect reach, quality, impact and value for money in the BBC’s overallobjectives for public service broadcasting;

p establish clear definitions of reach, quality, impact and value for moneyand the supporting performance measures to help achieve consistencyacross the organisation;

p assess the extent to which the performance measurement framework is embedded at all levels of the BBC;

p ensure that there is an explicit focus on providing assurance about theaccuracy and reliability of the performance measurement framework;

p make the performance reports more user-friendly, principally by tailoring the reports to meet the needs of different users and by differentiatingbetween priority and less important information;

p set a timetable for reporting on reach, quality, impact and value for money to Parliament and the public.

OUR CONCLUDING COMMENTS AND RECOMMENDATIONS


PUBLIC SERVICE BROADCASTING: THE BBC’S PERFORMANCE MEASUREMENT FRAMEWORK

part one

4

PART ONEIntroduction to the performance measurement framework



part one

5

Where does the performancemeasurement framework sit within the BBC?1.1 The British Broadcasting Corporation (BBC) is theUnited Kingdom’s main public service broadcaster,receiving licence fee revenue (£2.8 billion in 2003-04)and free access to the spectrum in return for deliveringpublic service broadcasting. Figure 1 provides informationon the BBC’s organisational structure.

1.2 Under its Royal Charter, the BBC’s objects include“to provide, as public services, sound and televisionbroadcasting services…and to provide sound and television programmes of information, education andentertainment for general reception…”. To meet this requirement, across the United Kingdom the BBCcurrently provides:

p eight television channels – two analogue (BBC Oneand BBC Two) and six digital (BBC Three, BBC Four,CBBC, CBeebies, BBC News 24 and BBC Parliament);

p ten radio networks – five analogue (BBC Radios 1,2, 3, 4 and Five Live) and five digital (BBC Five LiveSports Extra, 1Xtra, BBC 6Music, BBC 7 and BBCAsian Network) – and local and national radio stations;

p new media – the online site, bbc.co.uk, and BBCi,the BBC’s interactive services.

In 2003-04 the BBC’s total spending on broadcasting(including overheads) was £3.0 billion.

1 The BBC’s organisational structure

p The BBC is a public corporation, established in 1927 by a renewable Royal Charter. Many of the BBC’s publicobligations are set out in an accompanying Agreement with the Secretary of State for Culture, Media and Sport.

p The BBC is overseen by a Board of 12 Governors,appointed by The Queen on advice from Ministers.The Board of Governors is responsible for ensuring that the BBC meets its statutory and other obligations and that it is managed in the public interest by, among other things, approving strategy and policy, setting objectives and monitoring performance. The Board of Governors is supported in its work by the recently establishedGovernance Unit.

p The Board of Governors appoints the BBC’s Director-General and the eight other members of the ExecutiveBoard, which is responsible for managing the BBC.The Executive Board is supported by three boards covering the BBC’s main activities – the Creative Board (also chairedby the Director-General), the Journalism Board and theCommercial Board.

p The BBC’s day to day operations are run by 16 divisions – nine broadcasting divisions which manage the BBCnetworks and their programme commissioning andproduction; five divisions which provide professional support; and two commercial divisions which sell goods and services around the world.

Source: National Audit Office



part one

6

1.3 The BBC’s current Royal Charter expires on31 December 2006 and is presently being reviewed by theDepartment for Culture, Media and Sport. To contribute to the debate on the Charter, in June 2004 the BBC published‘Building Public Value – renewing the BBC for a digital world’, which describes how the BBC adds value to life in the United Kingdom and sets out how it intends to buildon this in the future. The document sets out five ways the BBC creates public value, which it terms its publicpurposes (Figure 2). The Government took account of theBBC’s views in developing the public purposes for theBBC proposed in the Green Paper on the Review of theBBC’s Royal Charter, published in March 2005.

1.4 ‘Building PublicValue’ also outlines the BBC’s plans for a new system designed to demonstrate its delivery ofpublic value, comprising four components for assessingnew services and monitoring existing services (Figure 3).

1.5 In addition, under the Communications Act 2003the BBC along with other public service broadcasters is required to publish a ‘Statement of Programme Policy’,describing how in the coming year it will fulfil its public service remit and reporting performance against theprevious year’s proposals. The Statements set out minimum requirements for programming in areas such as news and current affairs, independent production, regionalprogramming, and access to services for deafand visually-impaired audiences.

How do the BBC’s drivers ofpublic value link with current thinking on the characteristics of public service broadcasting?1.6 Under the Communications Act 2003, Ofcom is required to carry out every five years a review of public service broadcasting television. Ofcom carried out thefirst of these reviews in three phases, culminating in thepublication of its final report in February 2005.

1.7 As part of its review, Ofcom defined thecharacteristics of public service broadcasting. ComparingOfcom’s characteristics with the BBC’s drivers of public value on which the performance measurement frameworkis based indicates that, while expressed differently, theyshare much common ground although in its definition the BBC has the additional driver of value for money(Figure 4). Ofcom told us that it considered the BBC’s drivers of public value were very much in line with its own characteristics, reflecting the extensive dialoguebetween the two bodies. Going forward, it will beimportant for the BBC to continue to liaise with Ofcomon these matters.

1.8 The BBC’s drivers of public service broadcasting arealso consistent with the Government’s recent Green Paper on the Review of the BBC’s Royal Charter, which proposed that the BBC should provide a wide range of content,across every programme genre, trying to reach thegreatest possible range of audiences. Programmes shouldaim to be excellent, distinctive and entertaining – more specifically they should be of high quality, challenging,original, innovative and engaging. As the Charter review process goes forward, it will be important for the BBC toensure that its performance measurement framework takes account of this wider context.

2 The BBC’s public purposes – how it creates public value

Source: ‘Building Public Value’ (BBC, June 2004)

p Democratic value: supporting informed citizenship

p Cultural and creative value: enriching the creative life of theUnited Kingdom

p Educational value: extending horizons

p Social value: connecting and uniting communities

p Global value: supporting the United Kingdom’s rolein the world



part one

7

How does the BBC’s approach to measuring public servicebroadcasting compare with overseas?

1.9 We looked at how public service broadcasters in other countries were approaching performancemeasurement. Our overall sense was that the BBC is at theforefront of current thinking on this topic and continues toliaise with other public service broadcasters in Europe andbeyond. And the European Broadcasting Union3, of which the BBC is a member, is currently considering approaches to the measurement of performance in public servicebroadcasting.

1.10 Points of particular interest are noted at relevant points in this report but in general terms other public service broadcasters, in particular in Scandinaviancountries, are implementing performance measurement structures similar to those being implemented by the BBC.The public service broadcaster in the Netherlands (PubliekeOmroep) uses a performance ‘quality card’, comprisingmeasures of reach and share, quality, impact and efficiency.

3 The BBC’s proposals for assessing and monitoring public value

Source: National Audit Office, drawing on ‘Building Public Value’ (BBC, June 2004)

Public value test

Service licences

Public value survey

Performance measurement framework

The public value test will be used to judge the contribution a service would potentially make to the delivery of public value by the BBC. The test will include an independent evaluationof a service’s market impact and audience research, and will be applied to all new serviceproposals and significant changes to existing services.

In March 2005 the Board of Governors commissioned Spectrum Strategy Consultants toassess whether the public value test is robust enough to be the main tool used in deciding whether to invest public money in markets in which there may be commercial players.

Service licences are to be granted to each BBC channel and service by the Board ofGovernors. A licence will set out a service’s remit, purpose and contribution to public value,including performance targets.

The licences are currently being developed and the BBC expects to introduce them in a shadow form in 2006 and in full in 2007. The licences will be published.

Every three to five years, the Board of Governors will commission an independent public value survey of 10,000 licence fee payers. The survey will be designed to provide adetailed, audience-based assessment of the impact, value and effectiveness of the BBC’s services. The results will be published.

The performance measurement framework is a new evidence-based system of performancemeasurement which aims to give BBC Governors, licence fee payers and other stakeholders greater assurance that existing BBC services are delivering public value.

This report focuses on the performance measurement framework which is centred on four keydrivers of public value identified by the BBC – reach, quality, impact and value for money.

4 Comparing the BBC’s drivers and Ofcom’s characteristics of public service broadcasting

The BBC’s drivers

Reach – providing universalavailability and access

Quality – providing high quality,distinctive programming

Impact – enriching the lives ofindividuals and citizens

Value for money – providing the best value for money for licence fee payers

Ofcom’s characteristics

Widely available

High quality

Original

Innovative

Challenging

Engaging

Source: National Audit Office

3 The European Broadcasting Union is a professional association of national broadcasters which, among other things, negotiates broadcasting rights, provides services and represents the interests of public service broadcasters.



part two

8

PART TWOThe design of the performance measurement framework



part two

9

2.1 This part of the report considers the design of the BBC’s new performance measurement framework.Informed by the ‘Choosing the right FABRIC’ guidance(see paragraph 3 of the Executive Summary), we focusedon a number of high-level questions which will be key as the BBC takes the new framework forward:

p whether the BBC has identified performancemeasures for the four drivers of public value;

p whether the performance measurement frameworkfocuses on the BBC’s objectives;

p whether the performance measurement frameworkcovers all the BBC’s public service broadcasting;

p whether the performance measures allow the BBC to benchmark its performance with others.

2.2 The embodiment of the performance measurement framework is the information presented in the BBC’s performance reports. So our analysis focused on theinformation set out in the summary reports, which cover the performance of the BBC as a whole; and we alsolooked at the reports produced at divisional level, whichfeed into and supplement the summary reports.

Has the BBC identified performancemeasures for the four drivers ofpublic value?2.3 The BBC has identified measures of performance for each of the four drivers in the performance measurement framework – reach, quality, impact and value for money.Some of these have been carried forward from previous performance systems and others are new measures developed specifically for the framework.

2.4 The performance measurement framework is still relatively new and developing, and going forward it willbe important for the BBC to:

p establish clear definitions of reach, quality, impact and value for money, and the supporting measures.The BBC staff most heavily involved in the frameworkappear to have broad working definitions but formalised definitions approved for use across the organisation would help achieve consistency.At present there are instances of different BBCdivisions using the same measure to assess different performance drivers. For example, approval ratings are used as a measure of quality by the Television divisionand as a measure of impact by several other divisions.



part two

10

p maintain a balance between reach, quality, impact and value for money so that no single performancedriver has an undue influence. A strength of the framework is that the package of drivers come together to provide an overall picture ofperformance. But focusing on any individual driverrisks unbalancing the picture as ‘good’ results in onearea may be achieved at the expense of ‘bad’ results in another. For example, high quality programmingcould be more expensive and increase the cost per viewer hour, and vice versa.

(a) Reach

2.5 Reach is the headline indicator of take-up of theBBC’s services and has the benefit of lending itself tofirmly quantifiable measures. The ways the BBC currentlymeasures reach are summarised in Figure 5.

2.6 Audience share (the percentage of the total audience watching or listening to a particular channel or service rather than to any of the others over a given period of time) together with audience volumes has traditionallybeen the key measure of performance in broadcasting.And share continues to be part of the BBC’s new performance measurement framework.

2.7 However, in line with the commitment it made in‘Building PublicValue’ (see paragraph 1.3), the BBC now places greater emphasis on overall weekly reach – thepercentage of the population who have viewed, listened toor used a BBC service for a consecutive period of 15minutes over the course of a week. Placing greater emphasis on weekly reach reflects the BBC’s aim to serve the wholepopulation whose licence fees help to fund its services.

2.8 Clearly though, reach and share are interrelatedand an increase in one may result in an increase in theother. In addition, audience share and volumes feed intoa number of other performance measures, including thecost per viewer hour, one of the key measures of valuefor money (see Figure 8). The BBC needs to ensure that these interrelationships do not distort the balance of theperformance measurement framework – for example, aparticular emphasis on reducing cost per viewer hour could have the effect of prioritising audience share overweekly reach.

2.9 In general terms, the aim of the shift to weeklyreach is to place more emphasis on attracting a widerrange of users and as part of this the BBC is aiming to reach under-served audiences. But overall there appears to have been a lack of consistency in what information iscollected and reported. The summary performance report covering the BBC as a whole provides information on reach among young people and some BBC divisions havedata on particular social and ethnic groups. However, theBBC is currently in the process of defining which particular groups it is seeking to reach, both across the board and at divisional level.

(b) Quality

2.10 The BBC seeks to provide high quality, distinctiveprogramming which it believes the public expects in return for the licence fee. Quality is inherentlyperception-based and the BBC is seeking to supplement audience perceptions by also recording its success in winning broadcasting awards and assessments of mediacoverage. The ways the BBC currently measures qualityare summarised in Figure 6.

2.11 Although many measures of quality are quantifiable,because of the interdependence of measures interpretationcan be complex. For example, audience views on anindividual programme may impact on wider perceptions of the channel concerned and of the BBC as a whole.Also, audience perceptions need to be treated with somecaution, in part because people tend not to watch or listen to programmes they think they will not like. The BBC recognises that the results for quality may be less clear cut and require more interpretation than for some other areas where the data is harder, and it is seeking to generatemore sophisticated data to help with assessing qualitythrough its new Internet survey (see paragraph 3.7).

5 The ways the BBC measures reach

p Overall weekly reach (see paragraph 2.7)

p Weekly reach among under-served audiences

p The weekly reach achieved overall and by individual television channels (for all homes and for homes with digital television) and radio services

p The average number of hours consumed per week by each viewer/listener/user

p Audience share achieved overall and by individual television channels (for all homes and for homes with digital television) and radio services (see paragraph 2.6)

Source: BBC



part two

11

(c) Impact

2.12 The BBC describes impact as ‘enriching the livesof individuals and citizens’ and the ways it is currentlymeasured are summarised in Figure 7.

2.13 The strength of impact as a driver of performance is that it seeks to assess whether the BBC is delivering against its public purposes (as set out in Figure 2). For example, tohelp assess whether it is ‘supporting informed citizenship’,the BBC asks audiences for their views on statements such as “the BBC helps people understand and make up their minds” and “the BBC helps me understand what is happening in the United Kingdom and the world”.

2.14 Impact is, however, the driver of public value that is most difficult to measure and a particular challenge is thequestion of attribution. For example, if the BBC concludes that public awareness of particular issues covered inBBC broadcasts has increased, it also needs evidence ofits contribution to that change. As with quality, the BBCexpects its new Internet survey (see paragraph 3.7) togenerate improved data to help with assessing impact.

(d) Value for money

2.15 Clearly the need to measure the value for moneyprovided to licence fee payers is not a new concept andit is a quantifiable measure of the BBC’s performance.The ways value for money is currently measured are summarised in Figure 8. The indicators are largelycost-based, focusing on inputs rather than wider concepts of value for money, although the BBC also measures audiences’ perceptions of the value for money it provides.

6 The ways the BBC measures quality

p Audiences’ approval ratings for the BBC as a whole,assessed on a scale of one to ten

p Audiences’ perceptions of the BBC as a whole basedon their responses to statements such as “the BBC is independent and impartial” and “I think the BBC makes great programmes to watch and listen to”

p Audiences’ perceptions of individual television channels based on their responses when asked which channel is ‘best for’ particular types of programme, such as soapoperas or children’s programming

p Audiences’ approval ratings for individual radio services,assessed on a scale of one to ten

p Appreciation indices by television channel based on viewers’ responses to individual programmes, assessed ona scale of one to ten

p The number and percentage of broadcasting awards won

p The number of press articles, split between those that arepositive about the BBC and those that are negative

p A shortlist of programmes that BBC channel and networkcontrollers consider to be especially distinctive or innovative

Source: BBC

7 The ways the BBC measures impact

p Audiences’ views on the memorability of programmes,based on the number and proportion of respondents saying that a programme ‘stood out’

p The ‘impact factor’ – the percentage share of memorableprogrammes on a particular channel compared with its audience share

p Audiences’ perceptions of the BBC and its impact on them,based on their responses to statements such as “my localinterests and concerns are well reflected by the BBC’s coverage” and “I’ve learnt new skills or developedexisting ones”

p The volume and favourability of press articles

p Assessment of the impact of the BBC’s social andeducational campaigns

Source: BBC

8 The ways the BBC measures value for money

p Cost per viewer, listener or user

p Cost per viewer or listener hour (or cost per user reachedin the case of online services) – calculated by dividing the total programming cost by the number of viewing or listening hours

p Audiences’ views on: the value for money of the licencefee; how wisely the BBC spends the licence fee; whether the BBC is well run; and whether the BBC is accountable tolicence fee payers

Source: BBC



part two

12

2.16 A measure of value for money, which is not currentlycovered by the performance measurement framework,is the proportion of its resources that the BBC spends onprogramming. As shifting resources away from overheads into programming is a priority for the BBC, reflecting this in the measures of value for money would be a logical step. It would also be consistent with the approachadopted by some public service broadcasters overseas.For example, the Australian broadcaster (ABC) reports onoverhead expenditure as a percentage of gross expenditureas one of a number of measures of efficiency.

2.17 To date performance against the value for moneydriver has been assessed less consistently than the other drivers, with cost information not presented each quarter and in some divisions only at the end of the financial year,making it more difficult to track performance.

Does the performance measurement framework focus on the BBC’s objectives?2.18 The performance measurement framework shouldbe firmly linked to the BBC’s objectives relating to thedelivery of public service broadcasting. In ‘BuildingPublicValue’ (see paragraph 1.3), the BBC stated that theBoard of Governors would draw heavily on the drivers of reach, quality, impact and value for money to set annualobjectives for the BBC and its services, and would requireBBC Management to report against these areas.

2.19 At April 2005 the BBC had not explicitly linked its public service broadcasting objectives to the performancemeasurement framework, although at divisional levelobjectives for 2005-06 were being framed in these terms.Clearly the BBC will need to ensure that the detailedmeasures within the framework keep step with overall anddivisional objectives as they are developed.

2.20 Linked to objectives is the need to consider the setting of performance targets. At present the BBC sees reach as the area most susceptible to target settinggiven its generally accepted goal for the organisationas a whole of achieving weekly reach of 95 per cent across the United Kingdom population. Some public service broadcasters overseas – for example the Finnishbroadcaster, Yleisradio Oy – already have targets for reach.However, the BBC is setting annual targets across all four performance drivers as divisions frame their objectives for 2005-06 in this way.

Does the performance measurement framework cover all the BBC’s public service broadcasting?2.21 The BBC is applying the performance measurement framework in full in six of its eight broadcasting divisions which are funded by the television licence fee to providepublic service broadcasting4 (Figure 9). These six divisions are responsible for commissioning and scheduling theBBC’s programming and are able to influence all of theperformance drivers of reach, quality, impact and valuefor money.

4 The ninth broadcasting division, BBCWorld Service and Global News, is not funded by the television licence fee. The BBCWorld Service is funded by theForeign and Commonwealth Office and is subject to separate arrangements for performance measurement; and Global News is the broadcasting arm ofBBCWorld, which is a subscription-based service.

9 The BBC divisions which provide public service broadcasting

p Television

p Radio and Music

p New Media

p News

p Sport

p Nations and Regions

p Factual and Learning

p Drama, Entertainment and CBBC

Source: BBC



part two

13

2.22 The remaining two broadcasting divisions – Factualand Learning, and Drama, Entertainment and CBBC– are by and large non-commissioning divisions, which supply programming to the other divisions and have less influence over the performance drivers. These divisions have their own performance measurement arrangements focusing mainly on the quality of their output, and inaddition relevant aspects of their performance are coveredby the framework by being included in the reports for theTelevision and New Media divisions.

Do the performance measures allow the BBC to benchmark its performance with others?2.23 A useful way for organisations to assess their performance is by comparing it with the performanceof similar organisations, and in line with good practice the BBC compares aspects of its performance with that achieved by other broadcasters. For example:

p in reporting the weekly reach and audience shareachieved by its television channels, the BBC alsopresents data for other United Kingdom channels;

p on quality, the ‘best for’ assessments (whereaudiences are asked which television channel is best for particular types of programme) are by their naturecomparative and the BBC presents data for all the relevant channels;

p for reach and quality in radio, the BBC compares theperformance of its services with that of commercial radio stations.

2.24 Although value for money also lends itself to performance benchmarking, the BBC and other broadcasters do not share data on costs due tocompetition constraints and concerns about the difficultyof achieving like for like comparisons.



PART THREEThe data systems supporting the performancemeasurement framework

part three



3.1 Good quality and reliable data is essential to aneffective performance measurement framework. This part of the report considers the BBC’s arrangements for generating performance data, in particular whether:

p systems are in place to generate comprehensiveand timely data;

p the BBC has adequate quality assurance arrangements.

Are systems in place to generatecomprehensive and timely data?3.2 The BBC has systems for generating data for eachof the four drivers in the performance measurement framework. Some of these systems are longstandingand others are being put in place to improve the qualityand timeliness of the data. And increasing mediafragmentation, with more people accessing BBC services by new means, will present further challenges in terms ofaudience research in the future.

(a) Reach

3.3 At the core of the data systems for measuring reachare two well established industry-wide systems whichgenerate audience data – the Broadcasters’ AudienceResearch Board for television and Radio Joint AudienceResearch Limited for radio (Figure 10 overleaf). The BBCis a founder member and major shareholder of these two systems, and has representatives on the boards of both.

3.4 In addition to collecting data on television and radio reach, the BBC has commissioned an Internet researchcompany (Nielsen//NetRatings) to provide information on the volumes of people accessing its online services. Usinga panel of some 5,000 people, the company measures usage of both the BBC’s online site (bbc.co.uk) and other websites, allowing comparisons to be made. The datacovers only Internet usage at home and the BBC has alsocommissioned the British Market Research Bureau (BMRB)to track Internet usage at home, work and school. BMRBconducts two surveys – one of 8,000 people aged 15 andover every month and a second of 500 children agedbetween seven and 14 every quarter. The BBC also now collects data on the number of people using BBCi,its interactive services.

part three



part three

(b) Quality and impact

3.5 The BBC uses a large number of measures to assess the quality and impact of its public service broadcasting.The key data systems for these performance drivers are two surveys which generate data on audience perceptions.

3.6 The ‘pan-BBC tracking study’ is conducted on behalfof the BBC by the market research company, Taylor NelsonSofres, and measures audience approval and attitudes towards the BBC and its services overall. It is a continuous quantitative survey, consisting of 700 face-to-faceinterviews each month amongst a sample of people aged15 and over across the United Kingdom. A different sample of people is interviewed each month. Childrenaged under 15 are not part of the survey and the approval ratings for children’s channels (CBeebies and CBBC) aregiven by parents. The data is available to the BBC around three weeks after the end of each survey month.

3.7 To strengthen its data collection, in April 2005 theBBC introduced a new Internet survey carried out by themarket research company, the GfK Group, to generate dataon audience perceptions of individual programmes. Theapproach is based on experience in the Netherlands where the GfK Group has been running an Internet survey for thepublic service broadcaster, Publieke Omroep, since 2002.

3.8 The survey, which replaced a previous diary postal survey, asks respondents to rate the previous day’s programmes. It uses a panel of 15,000 people (up from the sample of 4,000 previously used), of whom an averageof 5,000 complete the survey every day, and there is a separate panel of 1,500 children. The survey provides more timely data with the results for each programmeavailable within 36 hours of transmission rather than16 days as was previously the case.

10 How data on television and radio reach is generated

Television

p The Broadcasters’ Audience Research Board (BARB) was established in 1981. It is a non-profit making limitedcompany owned by the BBC, ITV, Channel 4, five, BSkyBand the Institute of Practitioners in Advertising. BARBcommissions contractors to produce audience ratings on its behalf.

p Reach (in-home viewing, not out-of-home viewing) is measured electronically using a nationally representativepanel of 5,100 homes (representing around 11,500 viewers). The television sets and video recorders withineach home are electronically monitored by a meter, whichautomatically collects information about which programmes and channels residents and guests are viewing. Theaudience data is detailed in that it is produced on aminute-by-minute basis for all channels received within the United Kingdom, and timely in that viewing figures are available to the BBC the morning after programmes are transmitted. Consolidated data including figures for videocassette recorder playback is available a week later.

Radio

p Radio Joint Audience Research Limited (RAJAR) was established in 1992. It is owned by the BBC and theCommercial Radio Companies Association. RAJAR employs a contractor to produce audience ratings on its behalf andgenerates data via a nationally representative sample ofpeople keeping a written diary of their radio listeningfor seven days. Approximately 130,000 diaries a year are completed and people are asked to record which stations they listened to at what times, and where they were listening.

p Every three months RAJAR releases audience figures for each station. In January 2005 RAJAR published the results of an industry-wide consultation on the issues facing radio measurement, and one of the conclusions was that electronic measurement should be introduced at somepoint in the future. This could involve the use of audiometers (also called radiometers), which are small portable devices designed to ‘hear’ any broadcasts that the wearers areexposed to, and would result in more timely availability of data.

National Audit Office, drawing on information from BARB and RAJAR



part three

3.9 In implementing this new approach to measuringaudience perceptions, the BBC is giving particular attention to ensuring that there are enough respondents – it is looking to sustain the current daily sample of5,000 respondents – and that the sample is representativegiven that the Internet is less accessible to some sections of the population. The BBC is also considering broadening the scope of the survey to cover daytime as well as peakevening programmes and more radio stations (at present the survey covers only Radio 4).

3.10 In addition to its ongoing audience surveys, theBBC commissions bespoke research to evaluate theimpact of its periodic social and educational campaigns.This may involve tracking, for example, the number ofpeople who accessed the programme’s website or joineda discussion forum.

(c) Value for money

3.11 The data used to assess performance against thecost-based measures of value for money comes from theBBC’s established financial reporting systems. Each BBCdivision prepares monthly reports and accounts, including this cost data, which are analysed by the central Financedivision. Data for the audience perception measures of the value for money of the BBC is generated by the pan-BBC tracking study (see paragraph 3.6).

Does the BBC have adequate qualityassurance arrangements?3.12 Much of the data the BBC uses is generated byestablished external sources (including BARB, RAJARand market research companies) and it is then subject to processing within the BBC. Users of the performancemeasurement framework need to be confident that the source data is reliable and that it is reported accurately,and responsibility for providing this assurance rests with the BBC.

3.13 The BBC relies on external data providers’ systems of quality assurance and its Internal Audit team carries out spot checks on the non-financial figures reported in theBBC’s annual report. At present, however, the BBC does not have quality assurance arrangements which focus explicitly on the performance measurement framework to provide assurance that source data is reliable, usedappropriately, reported accurately and interpreted fairly.



PART FOURHow performance information is being usedand reported

part four



4.1 Once performance information has been gatheredand validated, the overall success of a performancemeasurement framework lies in how it is used. This part of the report looks at how the BBC is using and reporting theinformation generated by its performance measurement framework. In particular we considered whether:

p the performance measurement framework is being used to influence thinking in the BBC;

p the reporting arrangements are effective;

p the performance measurement framework provides abasis for the BBC to be externally accountable.

Is the performance measurement framework being used to influence thinking in the BBC?4.2 It is early days in the life of the new performancemeasurement framework but from our discussions withBBC Senior Managers and other staff it was apparent that reach, quality, impact and value for money – the drivers of the framework – are becoming common currency in theBBC’s thinking about public service broadcasting.And as highlighted in part 2 of this report, the performancemeasurement framework is being incorporated into theBBC’s internal objective setting process and in this wayreach, quality, impact and value for money are beingcascaded throughout the organisation.

4.3 Embedding reach, quality, impact and value for money throughout the organisation is however, as the BBC recognises, a longer term challenge. To help with this,

the reports reflecting the information gathered under theperformance measurement framework are made available to staff via the BBC intranet. The BBC’s aim of embedding the framework across the organisation underlines theimportance of clear definitions to help achieve consistent use and understanding (see paragraph 2.4).

4.4 The BBC divisions we spoke to found the reports produced under the performance measurement framework a useful tool to inform their ongoing business management. And although it is early days, there was evidence of the BBC using the framework to influencedecisions. For example:

p the Executive Board is using the performancemeasurement framework to inform future fundingallocations for the broadcasting divisions, withprogramming plans being prepared and thenassessed in terms of their contribution to reach,quality, impact and value for money;

p the Television division has used the performancemeasurement framework to review its programmingand inform decisions on the allocation of funding.Resource bids have been presented in reach, quality,impact and value for money terms, making it easier to compare them;

p the News division has used the performancemeasurement framework to help decide whether particular programmes should be rescheduled,repeated or re-commissioned. It has found theobjectivity and evidence-based approach of theframework to be a useful support tool in making such decisions.

part four



part four

Are the arrangements for reportingperformance effective?4.5 Performance reports, reflecting the informationgathered under the performance measurement framework,are produced quarterly for the BBC Board of Governors,the Executive Board, the Creative Board and theJournalism Board, which oversee the BBC’s public servicebroadcasting (see Figure 1). All these Boards receivea summary report covering the BBC as a whole and adetailed report on the performance of individual divisions,including a divisional summary. In line with goodpractice, the reports present performance over time, withdata for previous quarters and performance comparedyear-on-year.

4.6 The BBC previously reported performance everymonth but, in the light of concern that this frequencyrisked diminishing the impact of the reports andproducing ‘knee-jerk’ reactions to short term fluctuations in performance, it has now moved to quarterly reporting.The staff we spoke to felt this change had led to moremeaningful discussion, and there was still theopportunity to raise pressing performance issues at monthly Board meetings.

4.7 The BBC is seeking to improve performance reporting in the light of feedback from users and weidentified a number of ways in which the reports (inparticular the summary report) could be sharpened andpresentationally made more user-friendly. For example:

p there is a need to differentiate between priorityinformation and material that is of secondaryimportance. At present the performance reports include a large number of measures, which arenot prioritised for the reader (for example, theFebruary 2005 summary report included over 80individual measures presented graphically plus other qualitative information). It would be usefulfor the reports to highlight a smaller number of keymeasures across the four drivers (which may varyfor different audiences), with the less important information presented separately;

p there is scope for providing users with clear definitions of the performance measures and themethods by which they are calculated;

p the reports could highlight more clearly key issues for discussion by supplementing the graphicalpresentation of data with more explanation. For example, the News division has introduced a ‘talkingpoints’ section at the front of its performance reports,flagging up areas for discussion;

p the reports could set out more clearly expectedlevels of performance to help readers interpret theinformation. Although the reports use a ‘traffic light’ system to signal whether performance is average, or above or below average, the parameters for thesejudgements are not made clear and therefore there is a risk of inconsistency.

4.8 At present the performance reports are not tailored to different audiences – the Board of Governors and theSenior Management Boards receive the same reports.In the case of the Governors, the Governance Unit is considering how the summary report could be better tailored to the specific needs of the Governors and supplemented by briefing papers to highlight key issues.The Governance Unit is also reviewing its arrangements for systematic follow-up of issues raised by the Governors to ensure the arrangements are as effective as possible.

Does the performance measurement framework provide a basis for theBBC to be externally accountable?4.9 ‘Building PublicValue’ (see paragraph 1.3) envisaged the performance measurement framework enabling theBoard of Governors, BBC Management and the public to understand better how well the BBC’s services are doing.The document stated the Governors’ intention to use themeasures of reach, quality, impact and value for moneyto set objectives for the BBC and its services each year,and that they would form the basis for the BBC’s AnnualReport to Parliament and the British public.



4.10 In taking forward the external reporting dimensionof the performance measurement framework, a keyquestion for the BBC will be how to present theinformation in an understandable and manageableform given the volume of data involved. There are also technical challenges since external reporting may involve the BBC aggregating and interpreting the results of a wide range of performance measures. At the same time the BBC will need to ensure that the integrity of the measurement framework is maintained.

4.11 The Finnish broadcaster (Yleisradio Oy) publishes an annual audience report, alongside its annual report,which sets out the extent to which it has succeededin providing universal television, radio and online services, and “discusses Finnish people’s perceptions andexpectations of public service broadcasting as well as their satisfaction with the service”. The report provides both narrative and statistical information on areas suchas reach and share, satisfaction with programming, andperceptions of value for money. The Netherlands public service broadcaster (Publieke Omroep) plans to use its performance quality card (see paragraph 1.10) as aninstrument for external accountability.

part four



1 Our study focused on the new performancemeasurement framework which the BBC began tointroduce in Autumn 2004. The BBC is continuing todevelop and refine the framework so our aim was to review progress and contribute to the BBC’s thinking on the further work to be done.

2 In doing our work, we were guided by the goodpractice principles set in ‘Choosing the Right FABRIC – aframework for performance information’, produced jointlyby HMTreasury, the Cabinet Office, the National Audit Office, the Audit Commission and the Office for NationalStatistics. The publication identifies the key components of a good performance measurement system.

APPENDIXStudy methods

appendix

Components ofperformance measurement

MonitoringComparing performanceover time or with other

organisations

Evaluationof the performance

information system itself

StrategyAims, objectives and

action plans

MeasuresAligned with

strategic objectives

TargetsSetting intended levels

of performance

VerificationThrough internal orexternal means

ResultsActual performance

achieved



Work at the BBC

Interviews with BBC staff

3 We met the Director-General and Deputy Director-General of the BBC to get their views on the performancemeasurement framework.

4 We carried out interviews with key staff from theBBC divisions involved in implementing and supporting the performance measurement framework, including theStrategy and Distribution division and the Marketing,Communications and Audiences division. The interviews covered the design of the framework, the systems for datacollection, and the performance reporting arrangements.

5 We interviewed staff from three of the BBC’s broadcasting divisions – Television, News and Sport – which we used as case studies of how the performancemeasurement framework is being implemented in practiceand how it is being used to influence planning andmanagement.We also sought their views on how theframework might be improved from the perspective of thebroadcasting divisions.

6 We interviewed key staff from the BBC’s GovernanceUnit, which supports the Board of Governors in its work,to get their views on how the performance measurement framework is being used and how it might be improvedfrom the perspective of the Governors.

Review of performance reports andother material

7 We reviewed the quarterly reports that are produced under the performance measurement framework – both the summary reports covering the performance of the BBCas a whole and the more detailed divisional reports whichfeed into and supplement the summary reports.

8 We also examined a range of other published and unpublished material, including ‘Building PublicValue’,the BBC’s Annual Report, minutes of meetings of the BBC’s Board of Governors and Management Boards, divisionalobjectives and audience research material.

Interviews with stakeholders9 We met officials from the Department for Culture,Media and Sport with responsibility for broadcastingpolicy to discuss the BBC’s approach to performancemeasurement, the Government’s Green Paper on theReview of the BBC’s Royal Charter, and wider matters relating to public service broadcasting.

10 We interviewed staff from Ofcom (the independent regulator of the United Kingdom communications industries) to discuss Ofcom’s review of public service television broadcasting, the BBC’s approach toperformance measurement, and the alignment between the two.

11 We met the team from Spectrum StrategyConsultants, commissioned by the Board of Governors to assess the BBC’s proposed ‘public value test’ (seeFigure 3), to discuss our and their work.

12 We met a number of other parties with experience or expertise in public service broadcasting to get their views on the BBC’s approach to performance measurement and wider matters.We interviewed: Patrick Barwise, Professor of Management and Marketing at the London Business School; Philip Booth, Editorial Director at the Institute ofEconomic Affairs; Jamie Cowling, then Media ResearchFellow at the Institute of Public Policy Research; andClaire Grimmond, Head of Research at Channel 4.

Research on approaches tomeasuring public servicebroadcasting overseas13 We looked at how public service broadcasters inother countries are approaching performance measurement.Although comparing public service broadcasting indifferent countries is problematic because structures,funding systems and remits vary from one country to thenext, we wanted to see in general terms how the BBC’s performance measurement framework compared witharrangements being implemented overseas.

14 We carried out research and liaised with representatives from the European Broadcasting Union and with parties with knowledge or experience of public servicebroadcasting in other countries, including broadcasters,industry organisations and academics.We also met theBBC’s Head of European Policy and representatives from the Nordic public service broadcasters.

15 Through these contacts, we gathered informationon the approaches to performance measurement in thefollowing countries: Australia, Belgium, Denmark, Finland,Germany, the Netherlands, Norway and Sweden.

appendix


for blank pages 17/6/05 11:53 am Page 5

Public service broadcasting: the BBC’s performance measurement framework NAO review presented to the BBC Governors’ Audit Committee,May 2005, by the Comptroller and Auditor General, and a response to the review from the BBC

British Broadcasting CorporationBroadcasting HouseLondon W1A 1AA

This report is available online atbbcgovernors.co.uk

public service broadcasting: the bbc’s performance...

Documents