uk energy lab final report - annex c - methods & design v2 ... ·...

!!!!

UK!Energy!Lab:!!Feasibility!Study!Final!Report!

August&2014&&

Annex&C&/&Methods&&&Design&&!!!!!!!!!!!!!!!!!!!!!!!!!!Authors!Alun&Humphrey&Adam&Cooper&David&Shipworth&!& !

UK&Energy&Lab&Feasibility&Study:&Annex&C&–&Methods&and&Design&

! 2&

Contents!

1.!Introduction!.......................................................................................................!3&a.& Classes&and&levels&of&data&..................................................................................................&3&

2.!Sample!design!....................................................................................................!9&a.& Coverage:&domestic&vs&domestic&and&commercial&............................................................&9&

b.& Sample&frames&...................................................................................................................&9&

c.& Unit&of&analysis&................................................................................................................&11&

d.& Boosting&sub/groups&........................................................................................................&12&

e.& Geographic&clustering&......................................................................................................&13&

f.& Sample&Size&.......................................................................................................................&14&

g.& Stratification&....................................................................................................................&19&

3.!Survey!design!...................................................................................................!20&a.& Longitudinal&or&cross/sectional&design&............................................................................&20&

b.& Data&collection&................................................................................................................&22&

c.& Mixed&mode&.....................................................................................................................&22&

d.& Response&and&incentives&.................................................................................................&23&

4.!A!proposed!approach!.......................................................................................!24&a.& Broad&objectives&of&the&design&........................................................................................&24&

b.& Maintaining&the&longitudinal&sample&at&80%&of&total&sample&.........................................&25&

c.& The&design&........................................................................................................................&27&

5.!Other!considerations!........................................................................................!29&a.& A&scalable&approach&.........................................................................................................&29&

b.& An&Omnibus&element&.......................................................................................................&29&

c.& An&Innovation&panel&.........................................................................................................&31&

&

!

!!!!!!!!&

! !


! 3&

1.!Introduction!&

This&document&outlines&the&key&design&considerations&relating&to&the&

development&of&a&nationally&representative,&repeat/wave&socio/technical&UK&

energy&survey,&referred&to&here&for&reasons&of&brevity&as&‘UK&Energy&Lab’.&It&

draws&on&work&undertaken&as&part&of&other&work&packages&in&this&feasibility&

study.&&&

&

In&this&first&section,&we&set&out&the&broad&elements&of&the&overall&research&

programme,&the&various&types&of&data&to&be&collected,&and&different&means&of&

data&collection.&

&

The&following&section&describes&sample&design&considerations&with&the&aim&of&

identifying&the&likely&scale&in&terms&of&overall&sample&size&that&would&be&

required.&Section&3&describes&survey&design&issues&such&as&approaches&to&data&

collection&and&issues&for&with&respect&to&a&longitudinal&element.&&

&

Section&4&then&brings&together&the&previous&3&sections&to&outline&a&possible&

broad&design&for&a&UK&Energy&Lab.&The&final&section&outlines&some&additional&

considerations&for&how&the&study&could&be&modified.&

a. Classes!and!levels!of!data!&

As&outlined&in&the&main&synthesis&report,&the&survey&will&seek&to&collect&a&

range&of&different&types&of&data.&We&have&identified&four&classes&of&data&that&

the&survey&will&aim&to&collect:&

• social&or&human&measures&such&as&social&status,&income,&beliefs&and&

expectations&and&physiological&measures&such&as&weight&and&heart&

rate.&&

• data&relating&to&buildings&such&as&the&type,&size,&age&etc.&&• energy&data&such&as&overall&consumption&fuel&type,&etc.&

• environmental&data&such&as&temperature&or&humidity&

&

The&extent&of&data&collected&across&these&different&classes&will&vary.&We&have&

identified&three&different&levels&of&data&collection&with&higher&levels&representing&a&greater&amount&of&data&collected.&&

&

These&three&levels&correspond&with&different&data&collection&activities.&&

&

• At&level!1,&data&will&be&collected&mainly&from&a!household!interview!administered&in&respondents’&homes&by&an&interviewer.&It&would&also&

be&possible&to&gather&some&basic&building&and&energy&data&for&

example&by&matching&to&administrative&data&such&as&EPC&and&energy&

use&data.&In&addition,&respondents&could&be&provided&with&low/cost&

post/back&monitoring&equipment&or&interviewers&could&carry&devices&

that&collect&technical&data&at&the&address.&However,&the&main&element&

for&costing&purposes&is&a&household&interview.&


! 4&

• Level!2&will&involve&a&visit!from!a!surveyor&(similar&to&the&approach&

adopted&in&the&English&Housing&Survey&(EHS))&to&collect&data&relating&

to&the&accommodation.&In&addition,&some&monitoring&of&energy&or&

environmental&data&will&be&collected&with&devices&installed&by&the&

surveyor&that&report&back&remotely,&including&the&use&of&smart&meter&

data.&

• Level!3&would&see&the&most&detailed&data&collection&and&involve&the&

installation&of&much&more&extensive!monitoring&equipment.&This&

would&require&separate&installation&and&later&decommissioning&i.e.&

would&entail&separate&visits&by&a&technician.&It&may&also&involve&

respondents&carrying&devices&for&a&short&period&of&time&(e.g.&to&record&

movement&or&activity).&The&general&aim&here&is&to&collect&detailed&data&

from&a&small&sample&(ca.&100&homes)&that&enables&the&better&

understanding&of&uncertainties&around&the&level&2&data.&We&anticipate&

that&the&homes&would&only&participate&in&this&level&of&monitoring&for&a&

year&at&most,&with&some&data&collection&being&much&shorter&

(social/human&monitoring&is&likely&to&last&a&week&at&most,&buildings&

data&collection&may&be&done&on&one&day).&

&

These&three&data&collection&activities&(interview,&surveyor&visit&and&extensive&

monitoring)&are&increasingly&complex&to&collect&with&the&interview&being&the&

least&expensive&and&monitoring&data&likely&to&be&the&most&expensive.&A&design&

is&envisaged&whereby&smaller&sub/samples&of&the&overall&survey&population&

are&subject&to&the&more&expensive&data&collection&activities.&In&other&words,&a&

model&is&envisaged&whereby&the&full&sample&has&the&household&interview&

(level&1),&a&smaller&sample&additionally&has&a&building&survey&with&some&

monitoring&equipment&installed&(level&2).&Level&3&(the&more&extensive&

monitoring&data)&would&be&very&expensive&and&thus&would&only&be&feasible&

for&a&much&smaller&sub/sample.&This&would&mean&that&the&level&3&sample&

could&not&be&considered&nationally&representative.&Instead&the&objective&here&

would&be&more&qualitative,&to&understand&in&much&more&detail&about&what&

influences&certain&types&of&behaviours&or&to&confirm&or&explain&data&collected&

at&level&1&or&2&for&these&cases.&

&

Table&1&shows&the&likely&types&of&data&collected&at&each&level&across&the&four&

classes&of&data&identified&above.&&

&

&

&

&

&

&

&

&

&

&

&

UK#Energy#Lab#Feasibility#Study:#Annex#C#–#Methods#and#Design#

!! Level!1!(Full!sample)! Level!2!(Large!sub3sample)! Level!3!(small!sub3sample)!Energy# 1.#CAPI#questions#on#use#of#nonDgrid#fuels:#

Bottled#gas;#oil;#solid#fuels#(coal;#coke;#wood;#etc.)#Also#presence#of#renewable#energy#technologies##2.#Permission#to#access#DECC#Annualised#Quantity#Data;###3.#Quarterly#meter#reads#by#participants#following#the#CaRB#Method#(Note,#this#method#is#discontinued#on#installation#of#SMs#in#homes)#Questions#on#heating#system#–#main#vs#secondary.#Possibly#an#image#of#thermostat.#The#costs#of#these#additional#readings#would#need#to#be#traded#off#against#overall#survey#cost.##Note:#Progressive#installation#of#SMETS#2#duelDfuel#smart#meters#into#this#group#as#a#condition/incentive#to#participate.#Done#in#conjunction#with#suppliers.#UKEL#arranges#access#(addressing#a#major#issue#for#suppliers)#and#suppliers#fit#meters#at#their#own#expense#(as#per#standard#practice#in#the#rollout).##Note:#Level#1#is#subject#to#targeted#monitoring#in#addition#to#the#above#done#as#part#of#inference#testing#and#validation#of#models#constructed#using#Level#3#&#2#data.#

Level#1#+##1.#SmartDmeter#Consumer#Access#Device#(CAD)#taking#halfDhourly#gas,#and#~7#secondly#electricity#data#from#the#Home#Area#Network#(HAN)#and#reporting#back#through#a#home#ethernet#or#GPRS#link#to#the#UKEL#data#centre.##Note:#Level#2#is#subject#to#targeted#monitoring#in#addition#to#the#above#done#in#order#to#test#hypothesis#derived#from#model#inferences#constructed#from#Level#3.#E.g.#Statistical#models#built#on#Level#3#data#may#infer#appliance#behaviour#at#Level#2,#and#these#may#be#subject#to#tests#done#by#sending#Level#2#households#individual#plugDin#appliance#monitors#to#test#the#accuracy/precision#of#the#inferences.#

Level#2#+##1.#SmartDmeter#Consumer#Access#Device#(CAD)#taking#halfDhourly#gas,#and#~7#secondly#electricity#data#from#the#Home#Area#Network#(HAN)#and#reporting#back#through#a#home#ethernet#or#GPRS#link#to#the#UKEL#data#centre.##2.#WithinDhome#gas#and#electricity#end#use#disaggregation#using#single#point#monitoring#technologies.##D#Electricity:#e.g.#(http://www.informetis.com/en/)#or#(https://www.bidgely.com).##D#Gas:#domestic#hot#water#and#central#heating#gas#demand#disaggregation#through#sensing#at#the#boiler###

UK'Energy'Lab'Feasibility'Study:'Annex'C'–'Methods'and'Design'

! 6'

'Social/'Human'

1.'CAPI'interview'of'around'40550'minutes'length'with'the'household'reference'person'covering'topics'comparable'to'CaRB.'Includes'questions'about'income'and'quality'of'life'to'assess'fuel'poverty''Note:'Level'1'is'subject'to'targeted'monitoring'in'addition'to'the'above'done'as'part'of'inference'testing'and'validation'of'models'constructed'using'Level'3'&'2'data.'

Level'1'+''

1.'Possibly'a'self5completion'survey'gathering'data'from'each'individual'household'member,'as'well'as'psychological'and'personal'data'inappropriate'for'an'interviewer'to'gather.'(Done'in'a'manner'similar'to'CaRB.)'2.'Also'(or'instead)'monitoring'devices'(e.g.'recording'room'occupancy,'window/door'opening'etc.)'may'be'installed.'Note:'Level'2'is'subject'to'targeted'monitoring'in'addition'to'the'above'done'in'order'to'test'hypothesis'derived'from'model'inferences'constructed'from'Level'3.'

Level'2'+''1.'Life5logging/quantified5self'technologies.'These'technologies'will'range'from'wearable'sensing'such'as'accelerometers'and'magnetometers,'through'to'environmental'activity'monitors'such'as'movement'sensors'to'determine'participants’'locations'in'their'homes.'E.g.'http://www.ukbiobank.ac.uk/physical5activity5monitor/'as'a'precedent'for'this'in'which'an'activity'monitor'is'sent'to'participants'for'a'week'to'measure'activity'levels'and'therefore'personal'energy'expenditure.''

Building' 1.'CAPI'questions'on'building'demographics'comparable'to'EHS'2.'Social'surveyor'building'metric'data'captured'with'the'http://www.ikegps.com/spike/'system'(~£375'each,'one'per'social'surveyor'needed)''3.'Permissions,'if'needed,'to'access'EPC'data'where'available'4.'Imputation'of'building'performance'metrics'from'analysis'of'smart'meter'data.'Note:'Level'1'is'subject'to'targeted'monitoring'in'addition'to'the'above'done'as'part'of'inference'testing'and'validation'of'models'constructed'using'Level'2'&'3'data.'

Level'1'+''1.'Building'surveys'by'trained'surveyors'done'in'a'manner'similar'to'EHS'2.'Broader'use'of'building'metric'data'captured'with'the'http://www.ikegps.com/spike/'system'(~£375'each,'one'per'building'surveyor'needed)''3.'Infrared'imaging'done'with'http://www.flir.com/flirone/'by'building'surveyors'(~£300'each,'one'per'building'surveyor'needed)'Note:'Level'2'is'subject'to'targeted'monitoring'in'addition'to'the'above'done'in'order'to'test'hypothesis'derived'from'model'inferences'

Level'2'+'''1.'Full'laser'scanning'of'the'property'2.'Blower'door'pressure'test'3.'Boiler'combustion'efficiency'test'

UK'Energy'Lab'Feasibility'Study:'Annex'C'–'Methods'and'Design'

! 7'

constructed'from'Level'3.'Environ5mental'

1.'CAPI'questions'on'thermostat'settings,'TRV'use,'timer'use,'ventilation'practices,'etc'covering'topics'comparable'to'CaRB.'2.'Interpolation'of'local'environmental'variables'from'existing'meteorological'data'3.'Deployment'of'post5back,'short'logging'duration'temperature'sensors'comparable'to'MyJoulo'(http://myjoulo.wordpress.com)'or'remote'data'logging'sensors.''Note:'Level'1'is'subject'to'targeted'monitoring'in'addition'to'the'above'done'as'part'of'inference'testing'and'validation'of'models'constructed'using'Level'3'&'2'data.'

Level'1'+''1.'Temperature'and'relative'humidity'measurements'in'a'range'of'rooms'done'in'a'manner'similar'to'the'EFUS.''Note:'Level'2'is'subject'to'targeted'monitoring'in'addition'to'the'above'done'in'order'to'test'hypothesis'derived'from'model'inferences'constructed'from'Level'3.'

Level'2'+''High'spatial'and'temporal'monitoring'of'data'including'occupancy'sensing'and'operation'of'doors'and'windows.'This'is'broadly'comparable'to'the'’30'Homes’'trial'of'ETI'CRaB'project.'Environmental'parameters'recommend'to'be'monitored'include:''5'Comfort'(ambient'temperature;'relative'humidity,'air'movement,'light'level'and'spectra).'Living'room;'occupied'bedrooms'5'Location'(occupancy'sensing).'Living'room,'occupied'bedrooms;'kitchen,'bathroom,'hall'5'Activity'(relative'humidity;'volatile'organic'compounds)'Living'room,'occupied'bedrooms;'kitchen,'bathroom.'5'Fabric'interactions'(door'opening;'window'opening;'heating'system'controls'adjustments).'All.'5'Health'(carbon'dioxide'levels,'volatile'organic'compounds).'Living'room,'occupied'bedrooms;'kitchen.'

!Table!1:!The'likely'variables'for'collection'at'different'three'different'levels'and'for'each'class'of'variable.'The'suggested'variables'here'are'subject'to'change'but'give'a'good'indication'of'the'difference'in'depth'of'data'collection'and'help'illustrate'the'difference'between'the'different'classes'of'variable.!


At#this#stage#details#of#what#monitoring#equipment#data#are#to#be#used#is#necessarily#uncertain.#It#is#inevitable#that#technological#advances#will#take#place#and#the#costs#and#what#is#possible#to#monitor#will#develop#and#change.#Cost%estimates%and%sample%sizes%for%this%level%in%particular%are%therefore%indicative.#%# %


! 9#

2.%Sample%design%

a. Coverage:%domestic%vs%domestic%and%commercial%#

The#requirement#is#primarily#for#a#survey#of#domestic#energy#usage.#We#have#assessed#the#possibility#of#extending#the#survey#to#cover#businesses/commercial#premises#(see#Annex%G#for#a#more#detailed#feasibility#analysis#of#nonMdomestic#surveying)#in#addition#to#private#households.#However,#it#is#strongly#recommended#that#the#survey#concentrates#solely#on#domestic#energy#use#for#a#number#of#reasons.#Firstly,#and#most#obviously,#any#survey#including#businesses#in#addition#to#private#households#will#for#any#given#overall#sample#size#provide#smaller#numbers#of#domestic#energy#users#and#therefore#less#precise#estimates#and#less#potential#for#subMgroup#analysis.#Secondly,#surveys#of#households#and#businesses#tend#to#use#different#sample#frames:#the#Postcode#Address#File#(PAF)#is#usually#used#for#household#surveys#whilst#databases#such#as#the#InterMdepartmental#Business#Register#(IDBR)#and#NBD#are#used#for#surveys#of#businesses.#The#approach#to#sampling#from#PAF#and,#say,#the#IDBR#would#also#be#fundamentally#different,#with#the#samples#stratified#and#clustered#in#very#different#ways.#Furthermore,#survey#protocols#for#the#two#types#of#survey#are#necessarily#distinct;#for#example,#sampling#an#individual#(at#an#address)#to#respond#to#a#household#survey#requires#a#different#process#from#finding#an#appropriate#person#(at#a#business#premises)#to#respond#to#a#survey#of#businesses.#These#points#illustrate#that#a#survey#that#included#businesses#would#to#all#intents#and#purposes#be#two#surveys#rather#than#one.#

b. Sample%frames%%

The%Postcode%Address%File%The#Postcode#Address#File#(PAF)#is#widely#recognised#as#the#best#sample#frame#for#household#surveys.#It#is#the#only#available#sampling#frame#in#UK#that#covers#almost#all#of#the#general#household#population.#The#percentage#of#ineligible#addresses#on#the#PAF#is#reasonably#small#(at#about#8%)#which#means#that#relatively#little#interviewer#time#is#needed#for#identifying#and#excluding#ineligible#(‘deadwood’)#addresses.#It#also#has#full,#and#accurate,#postcode#information#for#each#address#which#means#that#matching#on#geoMdemographic#data,#such#as#aggregated#census#characteristics#or#the#index#of#multiple#deprivation,#is#extremely#straightforward.##

#A#potential#alternative#to#using#PAF#would#be#to#sample#from#the#central#address#database#for#either#the#electricity#or#the#gas#market.#In#principle,#this#would#be#a#logical#choice#if#it#were#decided#that#the#unit#of#analysis#was#an#electricity#or#gas#meter#(see#below#for#more#on#this)#and#would#have#the#natural#advantage#of#consumption#data#being#available#for#each#sampled#meter.####


! 10#

Alternative%sample%frames%The#XoServe#database#records#the#address#of#each#gas#meter#(uniquely#identified#by#the#Meter#Point#Reference#Number#or#MPRN)#whilst#the#ECOES#(Electricity#Online#Central#Enquiry#Service)#database1,#administered,#by#Gemserv#is#the#equivalent#for#the#electricity#market#and#records#the#address#of#each#electricity#meter#(uniquely#identified#by#the#Meter#Point#Administration#Number#or#MPAN).#Neither#database#covers#Northern#Ireland,#however,#so#coverage#would#be#limited#to#Great#Britain.#Furthermore,#a#significant#minority#of#households#(at#least#11%#in#Great#Britain#according#to#DECC#estimates)#are#not#connected#to#the#gas#network#which#means#that#the#XoServe#database#alone#would#not#provide#full#coverage#of#Great#Britain.###As#for#ECOES,#coverage#of#private#households#should#be#very#good#as#it#is#likely#that#every#dwelling#will#have#an#electricity#meter.#In#fact#there#are#more#meters#than#properties:#for#example,#an#apartment#building#may#have#a#meter#for#the#building#as#a#whole#in#addition#to#each#individual#apartment;#also#some#households,#mainly#in#Scotland,#have#2#or#3Mrate#meter#systems#with#one#meter#to#measure#peak#rate#consumption#and#another#to#measure#offMpeak#consumption.#However,#the#quality#of#address#data#is#poor#in#some#cases:#recent#figures#from#Ofgem#suggest#that#errors#in#both#XoServe#and#ECOES#have#led#to#tens#of#thousands#of#customers#being#incorrectly#transferred#when#attempting#to#switch#energy#suppliers#(managing#switching#was#the#main#reason#for#setting#up#these#databases#in#the#first#place).#Also,#a#small#proportion#of#addresses#are#known#to#be#incomplete.##A#further#issue#with#using#either#of#these#databases#is#the#issue#of#defining#what#constitutes#a#domestic#meter.#Electricity#data#is#divided#between#domestic#and#nonMdomestic#categories#according#to#the#meter's#profile#type:#there#are#eight#profile#types#for#‘nonMhalf#hourly’#meters#with#profiles#1#and#2#representing#domestic#customers#and#3#to#8#being#nonMdomestic.#However,#profile#1#and#2#meters#are#reMallocated#to#the#commercial#sector#if#annual#consumption#is#greater#than#100,000kWh.#Also,#some#meters#with#annual#consumption#between#50,000M100,000kWh#are#reMallocated#if#address#information#suggests#that#nonMdomestic#consumption#is#taking#place#(e.g.#if#an#address#contains#plc#or#ltd).#This#could#be#seen#as#analogous#to#the#smallMuser#PAF,#which#is#generally#textMfiltered#to#remove#business#addresses#before#it#is#used#for#sampling#for#household#surveys.#So#although#there#is#no#definitive#delineation#between#domestic#and#nonMdomestic#meters,#the#PAF#(effectively)#suffers#from#the#same#issue.##Combining%PAF%with%other%databases%One#potential#way#to#combine#the#advantages#of#the#PAF#and#the#XoServe#and#ECOES#databases#would#be#to#match#them#together.#In#fact#this#has#already#been#done#by#DECC#in#producing#the#NEED#data#framework#which#

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1#See:#https://www.ecoes.co.uk#


! 11#

combined#address#data#from#the#National#Land#and#Property#Gazetteer#with#electricity#and#gas#consumption#data#(plus#various#other#datasets).#However,#the#NEED#data#is#an#analytic#dataset,#not#a#sampling#frame#as#such#and#is#now#some#four#years#out#of#date.#Matching#the#latest#versions#of#the#relevant#databases#together#to#create#one#sampling#frame#would#be#a#costly#and#timeMconsuming#exercise.#A#better#option#would#be#to#sample#from#the#PAF#and#then#match#on#meter#point#data#from#the#two#databases#either#prior#to#fieldwork#(to#provide#information#to#interviewers#on#meters#that#they#should#find#at#the#address)#or#after#fieldwork#to#append#historical#consumption#data.###Given#the#issues#with#coverage#and#the#quality#of#address#information#on#the#XoServe#and#ECOES#databases,#PAF#would#seem#to#be#the#safer#option#and#is#our#recommended#sample#frame.##

c. Unit%of%analysis%#

The#proposed#survey#is#mainly#intended#to#capture#data#about#people#(living#in#and#within#particular#physical,#technical#and#environmental#contexts)#in#relation#to#their#energy#usage#(i.e.#this#will#cover#social,#psychological#and#physiological#measures).#To#this#end,#there#are#a#number#of#possible#units#of#analysis:#an#address,#a#dwelling,#a#household,#or#(people#sharing)#an#electricity#or#gas#meter.#The#respondent#to#the#main#interview#would#be#required#to#answer#questions#on#behalf#of#the#unit#to#which#they#belong.#For#the#majority#of#residential#properties,#all#of#these#units#will#be#synonymous:#a#family#living#in#a#house#with#one#electricity#meter#and#one#gas#meter#for#example.#However,#for#properties#split#into#flats#or#bedsits#the#situation#can#be#much#more#complex#and#it#is#important#to#establish#an#appropriate#unit#of#reference#for#the#main#interview#which#then#becomes#the#primary#unit#of#analysis.#

#For#addresses#which#contain#multiple#households,#it#could#prove#difficult#for#a#respondent#in#one#household#to#answer#questions#about#energy#usage#in#other#households#at#the#same#address.#Evidence#from#the#EHS#suggests#that#error#can#be#introduced#when#this#is#a#requirement.#Asking#questions#only#about#the#respondent’s#household#would#ensure#that#the#information#provided#was#as#accurate#as#possible.#If#information#were#required#about#the#dwelling#or#address#then#an#attempt#could#be#made#to#interview#at#all#households#at#each#sampled#dwelling#or#address.#The%household%therefore%would%seem%to%be%the%most%appropriate%unit%of%analysis.##An#alternative#would#be#to#base#the#survey#on#(people#sharing)#one#electricity#or#gas#meter.#In#the#case#of#a#house#converted#into#bedsits#with#only#one#of#each#type#of#meter#this#would#present#the#same#problem#as#discussed#above#i.e.#a#respondent#might#have#difficulty#answering#questions#about#the#behaviour#of#those#in#other#bedsits.#In#this#case,#attempting#to#interview#all#inhabitants#of#the#house#would#be#problematic#for#obvious#reasons.#The#scale#of#this#issue#is#likely#to#be#very#small.#Nevertheless,#basing#the#survey#on#


! 12#

electricity#or#gas#meters#would#require#the#use#of#XoServe#and/or#ECOES#databases#with#the#associated#disadvantages#discussed#above.##

d. Boosting%subEgroups%#

There#may#be#certain#subgroups#such#as#particular#housing#types#or#types#of#energy#user#that#are#considered#to#be#interesting#for#analysis#purposes#but#whose#natural#prevalence#in#the#population#is#small.#In#these#cases,#a#representative#sample#may#provide#insufficient#numbers#for#analysis#and#boosting#might#be#appropriate.#The#feasibility#of#boosting#would#depend#on#the#ability#to#identify#these#groups#at#the#sampling#stage.#Screening#for#small#prevalence#groups#would#be#likely#to#be#extremely#costly.##The#primary#subMgroup#of#interest#that#might#require#boosting#is#those#households#that#are#“offMgas”#i.e.#households#without#a#gas#meter.#This#constitutes#at#least#11%#of#households#in#Great#Britain#according#to#DECC#estimates.#Clearly#at#the#UK#level#a#representative#sample#would#provide#sufficient#numbers#but#if#analysis#of#offMgas#households#within#individual#countries#was#desired,#boosting#would#be#needed.##There#are#effectively#two#types#of#offMgas#household:#those#who#live#in#postcodes#not#connected#to#the#gas#grid#and#those#who#live#in#postcodes#connected#to#the#grid#but#with#no#gas#connection.#Two#sources#of#information#can#help#us#with#identifying#these#households:#

#i)#DECC#subMnational#estimates#of#the#proportion#of#households#without#a#gas#meter#(published#December#2013)#which#provides#estimates#of#numbers#of#offMgas#households#at#LSOA#level.##ii)#Xoserve#list#of#offMgas#grid#postcodes.##

Both#of#these#datasets#are#freely#available.#The#first#would#enable#us#to#identify#areas#that#are#more#likely#to#contain#offMgas#households#but#not#to#sample#them#directly.#The#second#would#enable#addresses#that#are#not#connected#to#the#gas#grid#to#be#sampled#directly.##The#best#way#to#boost#offMgas#households#might#be#to#combine#these#two#datasets.#The#DECC#subMnational#estimates#could#be#used#to#overMsample#areas#with#a#high#concentration#of#offMgas#households.#The#Xoserve#list#could#then#be#used#to#sample#individual#postcodes#within#these#LSOAs#and#then#to#sample#addresses#within#each#of#these#postcodes#in#order#to#secure#interviews#with#those#in#offMgas#grid#households.#At#the#same#time,#screening#could#be#undertaken#for#addresses#in#gas#postcodes#but#with#no#gas#connection.#However,#a#considerable#level#of#further#exploration#and#analysis#would#be#required#to#determine#the#feasibility#and#likely#cost#of#such#an#approach.#

#


! 13#

e. Geographic%clustering%Some#social#surveys,#such#as#the#Welsh#Health#Survey,#are#based#on#a#completely#unMclustered#sample#of#addresses#from#the#Post#Office#Address#File.#A#systematic#sample#(with#a#random#start#and#fixed#interval)#is#drawn#from#a#list#of#addresses#that#are#stratified#(sorted)#by#geography#(e.g.#local#authority).#This#results#in#a#set#of#addresses#roughly#evenly#spread#across#the#country.#For#practical#reasons,#however,#most#faceMtoMface#social#surveys#are#based#on#samples#of#addresses#that#are#drawn#in#clusters.#A#twoMstage#process#takes#place#where#areas#–#for#example#postcode#sectors#–#are#selected#first#and#then,#within#each#area,#a#(systematic)#sample#of#addresses#is#drawn.#This#results#in#a#set#of#“assignments”,#each#one#a#cluster#of#addresses#in#a#distinct#geographic#area#that#can#be#given#to#an#interviewer.#This#is#a#more#practical#approach#as#it#means#interviewers#have#to#travel#much#less#between#the#selected#addresses,#thereby#reducing#the#cost#of#fieldwork.!#However,#survey#measures#often#naturally#cluster.#For#example#age#and#type#of#property#are#not#likely#to#vary#very#much#in#areas#that#were#all#built#around#a#similar#time.#This#natural#clustering#reduces#the#precision#of#survey#estimates#in#clustered#samples.#The#more#individuals#within#each#cluster#resemble#one#another#on#a#survey#measure,#the#lower#the#precision#and#the#larger#the#margin#of#error#around#the#observed#measure.#For#a#given#sample#size#therefore,#the#larger#the#geographic#clusters,#the#lower#the#precision.##The#precision#of#survey#estimates#will#obviously#vary#between#measures,#depending#on#how#clustered#they#are.##In#the#following#section,#we#have#provided#estimates#of#margins#of#error#for#a#range#of#sample#sizes.#We#have#not#allowed#for#the#impact#of#clustering#because#it#will#depend#on#which#type#of#measure#is#being#considered#(i.e.#whether#it#is#a#measure#with#a#high#or#low#degree#of#clustering).#However,#we#have#attempted#to#provide#some#very#broad#guidance#of#the#likely#impact#on#the#precision#of#the#estimates.#

# #


! 14#

f. Sample%Size%%

Introduction%As#already#established,#a#total#achieved#sample#in#the#area#of#10,000#to#20,000#is#recommended.#However,#this#section#will#go#into#a#bit#more#detail#in#terms#of#quantifying#the#effects#of#sample#size#on#measurement#precision#within#this#range.#Particularly#crucial#when#determining#sample#sizes#are#the#precision#of#estimates#for#important#subgroups.#In#this#case,#the#two#main#types#of#subgroup#are#related#to#geography,#and#to#housing#type.#It#is#understood#that#country#within#the#UK#is#DECC’s#primary#geographical#unit#of#interest.#Thus,#as#detailed#below,#a#strategy#of#boosting#samples#in#Scotland,#Wales#and#Northern#Ireland#is#suggested.#Given#the#relative#size#of#England,#ONS#English#regions#should#also#be#taken#into#account#in#the#sampling#strategy,#and#provide#the#optimum#unit#of#analysis,#though#potentially#other#geographical#divisions#could#be#looked#at,#including#climatic#region#or#electricity#and#gas#distribution#regions.#Local#authorities,#of#which#there#are#433#in#the#UK,#would#form#too#small#a#geographical#unit#of#analysis#to#obtain#sensible#estimates#for,#given#sample#sizes#of#this#order.#A#sample#size#of#around#10,000,#with#approximately#25#cases#in#each#local#authority#would#produce#extremely#large#margins#of#error#for#any#local#estimates#(+/M#19.6%).#

#Geographic%boosting%Country#has#been#indicated#as#the#most#important#geographical#unit#of#interest.#As#shown#in#table#1,#if#sampled#proportionally,#sample#sizes#in#Wales#and#Northern#Ireland#in#particular#would#be#very#small.#Therefore,#in#order#to#achieve#the#targeted#margins#of#error#of#2#to#3%,#it#is#suggested#that#Scotland,#Wales#and#Northern#Ireland#be#boosted#to#a#minimum#level#of#1,000#cases#in#each.#Table#2#gives#estimates#of#margins#of#error,#taking#into#account#sample#sizes.#Boosting#results#in#unequal#selection#probabilities,#introducing#a#‘design#effect’,#which#reduces#effective#sample#sizes#slightly.#The#design#effect#due#to#weighting#(DEFT)#in#this#case#would#be#1.05.#Thus#the#NEFF#(effective#sample#size)#for#a#sample#of#10,000#would#be#9,073,#and#for#20,000#would#be#18,146.#It#should#be#noted#that#there#would%also%be%a%design%effect%due%to%the%clustered#nature%of%the%sample,#which%would%reduce%the%effective%sample%size%further%slightly.%It#is#difficult#to#estimate#exactly#how#much#at#this#juncture.#However,#as#a#very#rough#guide,#it#would#be#reasonable#to#assume#that#the#margins#of#error#shown#in#the#remainder#of#the#report#could#be#a#third%larger%than%shown.#For#example,#in#Table#2,#the#overall#margin#of#error#for#the#UK#as#a#whole#is#+/M#1.0%,#this#might#be#+/M#1.3%#once#clustering#is#taken#into#account.#It#is#worth#noting#that#all#the#margins#of#error#shown#are#based#on#an#observed#percentage#of#50%;#they#would#of#course#be#lower#for#different#percentages.##For%the%reasons%stated%above,%the%margins%of%error%throughout%the%remainder%of%this%report%take%into%account%the%level%of%geographic%boosting%outlined%above%but%do%not%allow%for%any%clustering%of%the%sample.%These%


! 15#

points%should%be%borne%in%mind%when%considering%the%data%in%the%tables%below%%%

% Number%(proportionate)%

Number%(boosted)%

Margin%of%error%

10,000%total%sample#England# 8,310# 7,000# +/M1.2%#Wales# 500# 1,000# +/M3.1%#Scotland# 910# 1,000# +/M3.1%#Northern#Ireland# 280# 1,000# +/M3.1%#Total% 10,000% 10,000% +/E1.0%%Effective(sample(size( ( 9,073( (# # # #20,000%total%sample## # # #England# 16,620# 14,000# +/M0.8%#Wales# 1,000# 2,000# +/M2.2%#Scotland# 1,820# 2,000# +/M2.2%#Northern#Ireland# 560# 2,000# +/M2.2%#Total% 20,000% 20,000% +/E0.7%%Effective(sample(size( ( 18,146( (%Table%2:%Margins#of#error#(excluding#clustering)#by#country#for#two#different#overall#sample#size#scenarios#

#As#can#be#seen#from#the#table,#margins#of#error#in#England#for#an#estimate#around#50%#in#a#total#survey#sample#of#10,000#would#be#+/M#1.2%,#dropping#to#+/M#0.8%#for#a#sample#of#20,000.#In#Wales,#Scotland#and#Northern#Ireland,#estimates#would#come#with#margins#of#error#of#+/M#3.1%#in#a#10,000#sample,#and#+/M#2.2%#in#a#20,000#sample.#It#is#important#to#note,#that#for#smaller#geographies,#the#gains#to#precision#of#larger#sample#sizes#are#greater.#

#This#is#even#clearer#if#one#was#to#look#at#ONS#regions#within#England.#For#the#North#East,#the#smallest#of#the#regions#with#the#lowest#level#of#precision,#margins#of#error#would#be#+/M5.2%#for#a#10,000#sample,#whereas#this#drops#to#+/M3.7%#for#a#sample#of#20,000.#

#Housing%types%An#additional#crucial#consideration#in#the#sample#design#is#the#breakdown#of#housing#types.#Table#3#estimates#the#margins#of#error#for#a#range#of#housing#type.#These#calculations#are#based#on#prevalences#estimated#from#the#English#Housing#Survey,#so#are#to#be#taken#as#indicative#only.#However#they#give#a#broad#idea#of#the#level#of#precision#of#analysis#conducted#at#the#level#of#housing#type.#As#the#table#shows,#margins#of#error#range#between#+/M#1.9%#and#4.9%#for#a#sample#of#10,000;#and#between#+/M1.3%#and#3.5%#for#a#sample#of#20,000.#Converted#flats#have#quite#a#low#prevalence,#and#thus#have#


! 16#

considerably#the#largest#margins#of#error.#All#other#types#have#similar#levels#of#precision.#%


Margin%of%error%

10,000%Sample% % %Detached#house/bungalow# 2,240# +/M2.2%#SemiMdetached# 3,010# +/M1.9%#Terrace# 2,720# +/M2.0%#PurposeMbuilt#flat# 1,560# +/M2.6%#Flat#conversion/rooms# 440# +/M4.9%## # #20,000%Sample% % %# # #Detached#house/bungalow# 4,480# +/M1.5%#SemiMdetached# 6,020# +/M1.3%#Terrace# 5,440# +/M1.4%#PurposeMbuilt#flat# 3,120# +/M1.8%#Flat#conversion/rooms# 880# +/M3.5%#

%Table%3:%Margins#of#error#(excluding#clustering)#by#housing#type#for#two#different#overall#sample#size#scenarios#

#As#described#above,#it#is#understood#that#consideration#of#subMsamples#within#England#is#of#particular#interest.#Table#4#provides#a#more#detailed#breakdown#of#differing#subMsamples.#It#shows#estimates#of#margins#of#error#for#different#types#of#household,#within#housing#type.###This#table#can#also#be#used#as#a#guide#of#margins#of#error#for#other#sample#groups.#For#example,#if#it#might#be#of#interest#to#understand#the#precision#of#estimates#from#a#sample#of#10%#of#the#households#in#England#(i.e.#around#700#cases).#This#would#be#expected#to#be#somewhere#between#the#3.5#for#805#cases#in#the#first#row#and#4.4.#for#the#506#further#down#the#table.#

##% %


! 17#

#%

%Table%4:%Margins#of#error#(excluding#clustering)#by#household#type#and#housing#type#M#England#

##


Margin%of%error%

10,000%Sample% 7,000% %Detached#house/bungalow# # #

couple,#no#dependent#child(ren)# 805# 3.5#couple#with#dependent#child(ren)# 357# 5.2#lone#parent#with#dependent#child(ren)# 40# 15.5#other#multiMperson#households# 85# 10.6#one#male# 105# 9.6#one#female# 174# 7.4#

SemiMdetached# # #couple,#no#dependent#child(ren)# 813# 3.4#couple#with#dependent#child(ren)# 506# 4.4#lone#parent#with#dependent#child(ren)# 144# 8.2#other#multiMperson#households# 169# 7.5#one#male# 185# 7.2#one#female# 291# 5.7#

Terrace# # #couple,#no#dependent#child(ren)# 562# 4.1#couple#with#dependent#child(ren)# 454# 4.6#lone#parent#other#with#dependent#child(ren)#

191# 7.1#

multiMperson#households# 196# 7.0#one#male# 226# 6.5#one#female# 274# 5.9#

PurposeMbuilt#flat# # #couple,#no#dependent#child(ren)# 234# 6.4#couple#with#dependent#child(ren)# 109# 9.4#lone#parent#with#dependent#child(ren)# 98# 9.9#other#multiMperson#households# 92# 10.2#one#male# 282# 5.8#one#female# 275# 5.9#

Flat#conversion/rooms# # #couple,#no#dependent#child(ren)# 62# 12.4#couple#with#dependent#child(ren)# 32# 17.4#lone#parent#with#dependent#child(ren)# 16# 24.3#other#multiMperson#households# 38# 16.9#one#male# 89# 10.4#one#female# 71# 11.6#

Total# 7,000# 1.2#


! 18#

#Physical%assessment%and%Monitoring%(at%levels%2%and%3)%Table#5#gives#margins#of#error#for#the#subMsamples#undergoing#physical#assessment#and#monitoring.#Table#5#assumes#60%#of#the#full#sample#is#followedMup.#Margins#of#error#for#physical#assessment#households#in#England#would#range#between#+/M#1.5%#and#+/M#1.1%.#While#for#other#countries#this#will#range#from#+/M#4%#to#+/M#2.8%.###If#just#10%#of#the#sample#were#followedMup,#in#England,#estimates#will#remain#close#to#the#2M3%#target,#but#in#Wales,#Scotland#and#Northern#Ireland,#margins#of#error#could#be#as#much#as#+/M9.8%.#In#this#situation#breakdowns#by#country#would#not#be#recommended.#However,#margins#of#error#for#the#smaller#subMsample#sample#would#nonetheless#be#+/M3.3%#for#the#10,000#sample#and#+/M2.3%#for#the#20,000#sample.##%

% Number%(boosted)%

Margin%of%error%

10,000%total%sample# # #England# 4,200# +/M1.5%#Wales# 600# +/M4.0%#Scotland# 600# +/M4.0%#Northern#Ireland# 600# +/M4.0%#Total% 6,000% +/E1.3%%# # #20,000%total%sample# # #England# 8,400# +/M1.1%#Wales# 1,200# +/M2.8%#Scotland# 1,200# +/M2.8%#Northern#Ireland# 1,200# +/M2.8%#Total% 12,000% +/E0.9%%

%Table%5:%Margins#of#error#(excluding#clustering)#for#physical#assessment#–#60%#of#sample#for#two#different#overall#sample#size#scenarios#%

#Longitudinal%sample%Estimating#error#for#the#longitudinal#section#of#the#sample#is#somewhat#more#complex.#What#is#of#interest#in#longitudinal#analysis#is#measuring#change.#To#calculate#margins#of#error#around#this#change#involves#several#assumptions,#so#in#order#to#provide#some#indicative#results,#very#simple#assumptions#are#used.#Assuming#that#the#change#in#proportions#of#a#binary#variable#is#being#measured,#in#year#one,#the#distribution#is#50M50.#In#year#two#this#distribution#changes#to#55M45.#The#question#then#becomes#what#are#the#margins#of#error#for#that#change,#given#various#sample#sizes.#Table#6#displays#the#margins#of#error#around#that#change#for#10,000#and#20,000#samples,#using#both#the#option#of#an#80%#longitudinal#sample#and#a#50%#longitudinal#sample.#The#achieved#longitudinal#sample#size#accounts#for#deadwood,#households#who#


! 19#

have#moved#and#cannot#be#traced#and#a#wave#2#nonMresponse#rate#of#20%.#If#50%#of#a#total#10,000#sample#were#sampled#for#reMinterview#in#a#second#year,#the#margins#of#error#around#the#above#assumed#change#would#be#+/M#1.5%.#This#falls#to#+/M#1%#for#a#total#sample#of#20,000.#With#a#larger#proportion#of#households#reMinterviewed#(80%),#margins#of#error#in#the#10,000#case#would#be#1.2%,#and#in#the#20,000#case#falling#to#0.8%.#While#these#estimates#are#made#under#a#very#specific#and#limited#set#of#assumptions,#they#are#nonetheless#indicative#of#the#level#of#effect#of#sample#size#on#potential#longitudinal#analysis.###

Total%sample% Longitudinal% Margin%of%error%

10,000# 5,000# +/M#1.5%#10,000# 8,000# +/M#1.2%#20,000# 10,000# +/M#1.0%#20,000# 16,000# +/M#0.8%#

%Table%6:#Margins#of#error#(excluding#clustering)#for#longitudinal#sample#for#four#different#scenarios:#two#different#overall#sample#sizes#by#two#different#proportions#of#the#sample#being#longitudinal.#

g. Stratification%Stratification#should#take#into#account#both#geographic#and#socioMdemographic#variables#considered#likely#to#be#related#to#key#survey#outcomes.#A#stratified#sample#ensures#that#random#sampling#doesn’t#produce#samples,#purely#by#chance,#that#happen#to#be#biased#in#a#certain#characteristic.#With#Wales,#Scotland#and#Northern#Ireland#being#boosted,#the#sample#is#effectively#being#stratified#by#country,#ensuring#representativeness#by#country.#However,#due#to#its#size,#it#is#also#suggested#that#the#sample#be#stratified#by#ONS#region#within#England,#of#which#there#are#nine.#Region#is#one#of#the#most#commonly#used#stratifiers#in#household#surveys#to#ensure#an#even#geographic#spread#across#the#country.#

#Within#these#countries#and#regions,#the#sample#would#be#stratified#by#population#density,#in#order#to#ensure#the#correct#mix#of#urban#and#rural#locations,#which#are#likely#to#have#very#different#housing#profiles.#Finally,#as#socioMdemographic#indicators#are#frequently#highly#associated#with#all#kinds#of#behavioural#and#social#phenomena,#an#additional#postcodeMsector#level#stratifier#of#either#the#proportion#of#household#reference#persons#engaged#in#nonMmanual#occupations,#or#the#percentage#of#owner#occupiers#should#be#used.#Both#of#these#figures#come#from#census#data.#While#both#measures#approximate#similar#area#characteristics,#the#latter#may#be#more#useful,#as#tenure#status#is#likely#to#be#associated#with#survey#outcomes.#This#combination#of#geographic#and#socioMdemographic#stratifiers#will#greatly#assist#the#selection#of#a#representative#sample#of#types#of#household#and#dwelling.#


! 20#

3.%Survey%design%

a. Longitudinal%or%crossEsectional%design%!

A#key#feature#of#the#study#design#will#be#whether#it#is#crossMsectional#or#longitudinal.#We#envisage#the#survey#being#repeated#over#a#number#(we#assume#5#in#the#proposed#approach)#of#waves#(each#wave#probably#being#carried#out#on#an#annual#basis).#Collecting#data#across#a#number#of#waves#will#mean#the#study#can#deliver#a#basic#but#important#objective#of#measuring#change#over#time.#If#certain#questions#or#measures#are#fielded#in#a#consistent#manner#at#regular#intervals,#then#changes#in#those#measures#(be#they#social#or#technical)#can#be#tracked#over#time.#This#can#help#to#understand#the#impact#of#certain#policies.#%Cross%sectional%%With#crossMsectional#research,#each#wave#of#data#collection#is#undertaken#using#a#fresh#sample#of#respondents.#This#approach#is#widely#used#for#many#largeMscale#government#surveys,#a#key#advantage#being#that#provided#a#consistent#approach#is#used,#each#sample#of#respondents#should#be#similar#in#profile,#meaning#observed#changes#in#data#are#likely#to#reflect#real#changes#in#the#population.##Longitudinal%Longitudinal#research#involves#returning#to#the#same#respondents#at#each#successive#wave.#Respondents#can#be#asked#the#same#or#similar#questions#or#at#least#checked#to#see#if#their#circumstances#have#changed.#The#advantage#here#is#that#specific#respondents#can#be#tracked#over#time,#thus#facilitating#analysis#of#longerMterm#impacts#of#specific#events#or#changes#in#policies.#This#can#be#particularly#useful#if#a#key#research#question#relates#to#behaviour#change,#as#it#is#likely#to#with#an#energy#survey.#For#example,#over#time,#a#longitudinal#survey#would#generate#data#that#could#increase#understanding#of#the#impact#on#energy#use#of#changes#in#the#building#such#as#a#new#boiler,#other#energy#efficiency#measures,#as#well#as#the#impact#of#changes#related#to#life#course#such#as#the#birth#of#children#or#children#leaving#home.##Furthermore,#the#installation#of#some#basic#monitoring#equipment#for#different#subMsamples#at#successive#waves#would#mean#that#over#several#waves,#a#larger#sample#of#monitored#households#would#be#builtMup.#For#these#reasons,#it#is#envisaged#that#a#UK#Energy#Lab#would#be#mainly#longitudinal.###Panel%conditioning%However,#the#survey#will#also#be#required#to#provide#indications#of#social,#psychological#and#(to#an#extent)#physiological#changes#over#time.#While#weighting#(nonMresponse#weighting)#is#routinely#used#in#longitudinal#studies#to#account#for#the#effects#of#attrition#on#the#survey#measures,#there#is#some#evidence#that#the#actual#fact#of#participating#in#a#longitudinal#study#can#impact#people’s#behaviours#and/or#the#way#in#which#they#answer#questions#–#


! 21#

a#phenomenon#referred#to#as#‘panel#conditioning’.#For#example,#if#a#survey#participant#knows#that#an#interviewer#will#soon#come#and#ask#them#again#whether#they#turn#on#the#heating#or#put#on#another#jumper#when#it#gets#cold,#they#may#change#their#behaviour#as#a#result#of#their#participation#in#the#panel#(or#at#least#report#that#they#have#changed#their#behaviour).#For#this#reason,#there#may#be#an#argument#for#also#regularly#collecting#the#same#measures#from#a#fresh,#nationally#representative#sample,#which#has#not#been#subject#to#prior#participation#in#the#study.#The#design#could,#therefore,#incorporate#an#element#of#a#crossMsectional#(fresh)#sample#each#data#wave.##%Refreshing%the%sample%A#further#advantage#of#including#a#fresh#sample#element#is#that#it#could#be#used#as#a#source#of#refreshing#the#core#longitudinal#sample.#This#could#be#done#in#such#a#way#as#to#retain#the#representativeness#of#the#longitudinal#sample.#In#other#words,#the#longitudinal#sample#will#be#subject#to#attrition#each#wave#as#respondents#drop#out#and#it#is#likely#that#this#might#be#disproportionate#across#different#sample#groups#(e.g.#age#or#economic#status#for#example).#Certain#subMgroups#from#the#crossMsectional#element#could#be#targeted#for#recontact#in#order#to#attempt#to#keep#the#longitudinal#sample#balanced.##There#would#be#some#practical#complexities#and#risks#related#to#data#and#analysis,#however,#with#having#both#longitudinal#and#crossMsectional#sample#components.#If#this#design#were#adopted,#with#similar#questions#included#on#both#surveys,#the#measures#obtained#from#the#two#survey#types#would#be#very#unlikely#to#match#perfectly#(although#it#may#be#possible#to#adjust#for#certain#key#measures#in#the#calculation#of#weights#for#the#longitudinal#sample).#There#would#need#to#be#careful#thought#given#to#how#the#data#is#presented,#and#clear#communications#and#training#around#the#purpose#and#most#appropriate#use#of#both#types#of#data.#There#may#otherwise#be#a#risk#that#tracking#aggregate#measures#in#the#easierMtoManalyse#crossMsectional#data#only#become#the#key#focus,#with#users#(and#funders)#overlooking#the#analytic#potential#of#the#more#complicated#longitudinal#data.##A#rotational#panel#model,#similar#the#Labour#Force#Survey#where#each#quarter’s#sample#is#retained#in#the#study#for#five#waves,#with#a#fifth#of#the#sample#fresh#at#each#wave,#may#also#be#possible#as#a#way#to#provide#some#‘fresh’#sample#each#wave.#This#would#also#make#the#communications#for#the#survey#easier#as#compared#with#having#some#people#recruited#for#a#crossMsection#and#others#for#a#longitudinal#survey,#with#each#new#and#existing#panel#member#being#presented#with#the#same#request#to#take#part#for#a#given#number#of#waves.##Data%weighting%A#further#consideration#is#the#data#weighting#that#would#be#necessitated#by#longitudinal#data.#Even#with#a#design#incorporating#a#purely#crossMsectional#sample#but#where#a#subMsample#is#followedMup#for#a#building#survey,#multiple#weighting#schemes#would#be#required#(for#the#interview#and#building#survey#samples).#A#longitudinal#sample#introduces#further#complication.#Attrition#at#


! 22#

each#wave#would#need#to#be#controlled#for#using#weighting#and#there#would#be#separate#weighting#schemes#for#the#core#panel#(those#that#had#been#included#since#wave#1)#and#the#annual#samples.#

b. Data%collection%!

A#basic#assumption#of#this#feasibility#study#is#that#the#initial#social#survey#will#be#based#on#an#interviewerMadministered#faceMtoMface#interview.#This#report#does#not#provide#a#detailed#discussion#of#different#data#collection#methodologies.#Instead#a#brief#overview#of#the#reasons#for#these#assumptions#is#provided#below.##To#date,#no#approach#for#sampling#a#representative#sample#of#the#British#population#either#online#or#by#telephone#has#been#successfully#demonstrated.#An#increasing#proportion#of#the#population#do#not#have#a#landline,#instead#relying#on#mobile#phones.#Furthermore,#internet#penetration#is#also#far#from#universal#and#even#if#it#were,#no#single#list#of#household#email#addresses#that#could#be#used#as#a#sample#frame#currently#exists,#nor#is#likely#to#in#the#near#future.###The#nature#of#the#required#data#is#such#that#it#will#include#a#significant#element#of#factual#data#relating#to#the#physical#attributes#of#the#household’s#accommodation#such#as#property#characteristics,#heating#and#insulation#facilities#etc.#It#is#envisaged#that#much#of#this#might#be#collected#via#a#visit#from#a#surveyor#but#the#nature#of#these#data#lend#themselves#to#a#personal#visit#from#an#interviewer.###FaceMtoMface#interviewing#is#also#associated#with#much#higher#response#rates#than#other#approaches.#It#is#not#unreasonable#to#anticipate#a#household#response#rate#of#in#the#region#of#50%#to#60%#using#faceMtoMface#interviewing,#this#compares#with#10%#to#30%#using#random#digit#dialing#telephone#approaches.###Finally,#the#likely#household#survey#interview#length#(in#the#region#of#40#to#60#minutes)#pushes#the#boundaries#of#what#is#likely#to#be#considered#acceptable#over#the#telephone#or#online.##For#these#reasons,#it#is#recommended#that#the#main#social#survey#mode#of#data#collection#is#a#faceMtoMface#visit#by#an#interviewer.#

c. Mixed%mode%!

The#recommendation#above#does#not#preclude#a#mixed#mode#design#being#introduced#after#the#first#wave.#Once#a#longitudinal#panel#is#established,#it#would#be#feasible#to#undertake#further#stages#of#analysis#via#means#other#than#faceMtoMface#interviewing.#The#disadvantages#outlined#above#would#largely#disappear#or#reduce#significantly.#Contact#details#of#panel#members#


! 23#

should#be#collected#such#that#biases#due#to#landline#or#broadband#penetration#could#at#least#be#known#about#and#hence#controlled#for.##Understanding#Society#has#introduced#a#mixed#mode#approach#combining#faceMtoMface,#web#and#telephone#data#collection.#This#has#a#more#complex#design#in#that#a#unit#of#analysis#is#individual#and#hence#separate#data#collection#techniques#can#be#employed#within#households.#Since#the#unit#of#analysis#in#an#energy#study#is#likely#to#be#address#or#household,#this#would#not#be#an#issue.###The#introduction#of#mixed#modes#for#data#collection#is#an#option#for#later#waves#of#the#study#and#can#be#expected#to#introduce#cost#savings.#However#for#the#purposes#of#this#feasibility#study,#it#is#assumed#that#all#social#survey#data#is#collected#through#a#faceMtoMface#interview#in#the#respondent’s#home.#

d. Response%and%incentives%!

This#study#has#made#some#basic#assumptions#with#regard#to#likely#response#rates#based#on#other#similar#surveys.#In#practice,#response#will#vary#due#to#a#number#of#factors#including#how#the#survey#is#presented#to#respondents,#the#training#and#experience#of#the#interviewers#and#how#they#are#incentivized#to#take#part.#The#household#survey#costs#assume#a#relatively#small#level#of#incentives#(e.g.#£5#per#issued#case).#There#may#be#other#methods#of#encouraging#participation.#For#example,#households#could#be#promised#feedback#on#the#condition#of#their#property.#This#would#need#to#be#carefully#planned.#This#is#not#possible#on#the#English#Housing#Survey#because#the#data#collected#as#part#of#the#physical#survey#are#not#in#a#format#that#is#readily#communicable#to#respondents.#A#whole#series#of#processing#and#modelling#is#required#before#–#for#example#–#an#energy#efficiency#rating#is#produced#for#an#address.#This#only#takes#place#at#the#end#of#fieldwork#and#so#is#too#late#to#offer#as#an#incentive#for#participation.#However,#it#is#possible#that#specific#measurements#could#be#built#into#the#study#that#would#facilitate#such#feedback#to#respondents.##An#additional#consideration#here#would#be#the#conditioning#of#panel#members.#Providing#feedback#to#panellists#could#potentially#influence#their#behaviour#and#hence#skew#data#from#later#rounds.##Finally,#there#are#longerMterm#implications#of#offering#incentives#as#part#of#a#longitudinal#survey.#Specifically,#providing#an#incentive#at#one#wave#creates#an#expectation#that#it#or#something#similar#might#be#provided#at#a#later#wave.#If#incentives#aren’t#then#offered#at#later#waves,#where#there#is#an#expectation,#then#this#could#harm#response#rates.# #


! 24#

4.%A%proposed%approach%!

Previous#sections#of#this#report#and#other#work#packages#have#considered#the#issues#related#to#sampling,#types#of#data#to#be#collected#and#dataMcollection#approaches.#In#this#section,#we#bring#these#together#to#develop#a#possible#design#for#a#UK#Energy#Lab.#It#covers#sample#sizes#for#the#various#elements#and#the#frequency#of#contact.#This#is#one#possible#design.#It#is#likely#that#this#design#could#be#revised#and#refined#as#more#is#known#about#the#specific#data#requirements#and#for#example#as#technological#advances#occur#in#technical#data#collection.#

a. Broad%objectives%of%the%design%%Sample%size%As#stated#previously,#the#broad#objective#is#to#develop#a#survey#that#will#provide#a#level#of#precision#of#survey#estimates#broadly#in#the#region#of#+/M#3%#at#the#95%#confidence#level#(the#minimum#scientificallyMaccepted#level#of#confidence).#This#of#course#must#be#a#very#general#objective#because#this#depends#heavily#on#the#actual#survey#estimate#(the#closer#the#observed#survey#measure#is#to#50%,#the#larger#the#margin#of#error#around#it).###Margins#of#error#for#percentages#were#provided#in#a#previous#section#of#this#annex.#The#margin#of#error#also#depends#on#the#sample#size#of#the#population#of#interest.#For#example,#if#there#is#interest#in#a#measure#among#only#a#small#subMsample#of#respondents,#then#the#margin#of#error#will#be#much#greater.#The#proposed#design#is#therefore#based#on#some#very#general#ideas#about#likely#analysis#requirements#but#nonetheless#demonstrates#the#scale#of#survey#that#would#be#required#to#meet#this#broad#objective#in#terms#of#precision#of#estimates.#%BuildEup%of%activity%The#survey#programme#will#comprise#a#number#of#different#interMrelated#elements#(social#survey,#building#inspection,#monitoring).#These#three#are#sequential#elements#of#the#design#i.e.#starting#with#an#interview,#followed#by#a#building#inspection,#followed#by#monitoring.#In#addition,#in#some#respects,#the#three#elements#have#increasing#complexity.#Undertaking#largeMscale#probabilityMbased#social#survey#is#a#longMestablished#activity,#conducting#building#surveys#on#such#a#scale#has#been#done,#for#example#on#the#EHS#but#would#be#likely#to#require#more#development#work.#The#monitoring#aspect#is#the#least#wellMdeveloped#and#technological#advances#are#likely#to#mean#what#is#possible#could#change#quite#rapidly.##

#The#development#of#the#survey#is#considered#in#much#more#detail#in#Work#Package#6#(Pragmatics#and#Piloting).#However,#in#order#to#enable#the#survey#to#become#operational#as#soon#as#possible,#one#option#is#to#proceed#with#the#theoretically#less#complex#social#survey#element#first#before#introducing#the#building#survey#and#monitoring#later#on.#There#is#an#inherent#risk#in#this#


! 25#

approach,#which#is#that#due#to#this#being#a#longitudinal#survey,#it%is%imperative%that%it%is%setEup%with%a%clear%notion%of%what%later%data%collection%activities%need%to%take%place.#A#simple#example#is#ensuring#that#there#is#clarity#about#participation#being#in#a#panel#rather#than#a#oneMoff#survey#but#there#are#also#considerations#with#respect#to#the#structure#of#the#data,#commitments#in#relation#to#incentives#etc.#Nonetheless,#the#proposed#design#incorporates#the#first#year#comprising#just#the#social#survey#element,#allowing#additional#time#for#developing#the#other#activities,#which#would#be#incorporated#into#year#2.#

b. Maintaining%the%longitudinal%sample%at%80%%of%total%sample%#

For#the#first#wave,#the#initial#sample#size#will#need#to#consider#the#requirements#for#the#number#of#longitudinal#interviews#required#at#future#waves.#The#proposed#design#is#based#on#an#annual#sample#size#of#around#10,000#comprising#approximately#80%#longitudinal#cases.#When#returning#to#longitudinal#cases#at#a#second#wave,#there#will#be#a#proportion#of#cases#who#have#moved.#A#longitudinal#survey#would#normally#attempt#to#trace#such#movers#but#this#would#not#be#possible#in#all#cases.#Furthermore,#as#identified#separately,#one#possible#amendment#to#the#longitudinal#design#would#be#to#interview#new#households#moving#into#the#vacated#houses#of#moving#panel#members.#This#opens#up#the#potential#for#analysis#of#how#a#change#to#the#household#occupying#an#address#impacts#energy#use#or#behaviour#(subject#to#sample#size#and#ethical#considerations).#

#The#table#below#illustrates#how#a#sample#of#approximately#8,000#longitudinal#cases#might#be#generated#considering#all#the#factors#above.#The#proportion#of#truly#longitudinal#cases#at#wave#2#would#probably#be#slightly#less#than#80%.##It#is#necessary#to#gain#respondents’#consent#to#reMcontact#whenever#further#followMup#surveys#are#planned.#Depending#on#how#these#questions#are#phrased,#we#would#anticipate#approximately#80%#agreeing#to#further#contact#based#on#experience#on#other#surveys.#However,#if#the#initial#contact#with#respondents#clearly#states#that#we#are#seeking#their#participation#in#a#panel,#with#annual#(say)#interviews,#then#it#is#not#necessary#to#request#such#permission.#Obviously,#this#intention#must#be#clearly#communicated#to#respondents#in#written#material#and#via#the#interviewer.#Instead,#details#of#a#stable#contact#address#are#collected#(to#facilitate#tracing#of#movers#at#later#stages).#At#that#point#in#the#questionnaire#if#refusal#to#participate#in#further#waves#is#spontaneously#offered,#then#it#is#recorded#and#respondents#are#not#reMcontacted#for#later#waves.#This#approach#was#undertaken#on#the#first#wave#of#Understanding#Society.##A#small#proportion#(we#have#assumed#7%,#based#on#data#from#the#English#Housing#Survey)#might#have#moved#by#the#time#the#second#wave#takes#place.#Based#on#experience#from#Understanding#Society,#we#have#assumed#that#it#would#be#possible#to#trace#approximately#70%#of#these#and#once#traced#


! 26#

interview#approximately#80%,#meaning#that#around#55%#of#those#movers#could#be#reMinterviewed#again#at#wave#2.##Furthermore,#one#possible#innovation#for#this#study#would#be#to#attempt#to#interview#the#new#occupants#of#vacated#addresses#of#movers#from#earlier#waves.#This#would#provide#a#means#of#toppingMup#the#sample#and#potentially#(given#ethical#considerations)#analysing#the#impact#of#changes#in#occupation#on#a#fixed#address.#It#would#seem#reasonable#to#assume#that#a#similar#response#rate#is#achieved#at#these#addresses#as#would#be#if#they#were#freshly#sampled#(58%).#

#Of#those#who#remain#at#the#address#interviewed#at#wave#1,#we#have#assumed#a#response#rate#of#75%#M#80%#is#achieved#at#wave#2#(subject#to#appropriate#financial#incentivisation).#These#cases#have#already#taken#part#and#are#expectant#of#further#contact.#Longitudinal#surveys#would#anticipate#achieving#a#response#rate#of#this#order#at#a#second#wave#(for#comparison,#e.g.#wave#2#of#Understanding#Society#achieved#a#response#rate#of#77%).#Thus,#this#would#deliver#a#total#of#8,230#longitudinal#interviews#at#wave#2,#though#of#course,#the#new#occupants#at#vacated#addresses#are#longitudinal#only#in#the#sense#that#the#address#at#which#an#interview#has#been#undertaken#before.#Excluding#these#cases,#the#longitudinal#sample#would#be#in#the#region#of#7,800#cases.##

#Element%of%sample% Number%# #Total#interviews#at#Wave#1# 10,000## #Moved#at#wave#2#(7%)# 700#Movers%traced%and%interviewed%at%wave%2%(55%)% 385%# #Interviews#with#new#occupants#at#vacated#addresses#(58%)#

405#

# #Remaining#nonMmoving#households## 9,300#Responding%households%at%wave%2%(80%)% 7,440%# #Total%longitudinal%sample%at%wave%2% 7,825%%Table%6:%Wave#2#sample#breakdown#for#the#proposed#approach%

#The#wave#2#sample#would#be#bolstered#by#a#fresh#sample#of#in#the#region#of#2,000#cases.#This#sample#would#perform#two#functions:##• Provide#a#sample#for#timeMseries#monitoring#of#general#attitudinal#and#

behavioural#measures#unaffected#by#participation#at#earlier#waves#


! 27#

• Provide#a#source#of#refreshing#the#core#longitudinal#sample#(i.e.#the#wave#3#longitudinal#sample#would#include#the#remaining#panellists#from#wave#1#as#well#as#additional#sample#from#those#who#took#part#as#part#of#the#crossMsectional#sample#at#wave#2.#

#It#should#be#noted#that#successive#waves#of#a#longitudinal#sample#would#be#likely#to#deliver#increasing#response#rates#among#the#longitudinal#sample.#As#the#panel#continues#to#be#subject#to#attrition,#the#remaining#cases#become#an#increasingly#committed#‘core’#and#so#the#response#rate#among#the#original#sample#can#be#expected#to#increase#until#stabilising#at#around#90%#to#95%#after#five#or#so#waves.#

#Of#course,#the#panel#will#also#be#refreshed#at#each#wave#by#cases#that#were#part#of#the#previous#wave’s#fresh#sample.#This#means#that#the#response#rate#among#all#longitudinal#cases#will#not#quite#increase#as#fast#as#among#those#from#the#very#first#wave.#Further#complications#are#introduced#with#the#fact#that#additional#stages#of#physical#surveying#and/or#monitoring#are#introduced.#This#might#be#expected#to#have#some#effect#on#response#rates#and#could#serve#to#increase#levels#of#attrition#over#time.#The#overall#design#then#becomes#very#complex#to#model.#We#have#therefore#restricted#the#proposed#design#to#some#simplifying#assumptions#(we#have#not#allowed#for#varying#levels#of#attrition#over#time)#in#order#to#develop#a#basic#model.#

c. The%design%!

The#overall#design#of#the#different#elements#is#summarised#below.#‘Waves’#refer#to#continuous#episodes#of#data#collection#in#the#field#(i.e.#actual#visits#to#homes,#as#opposed#to#automated#data#collection#from#sensors#which#is#likely#to#be#even#and#continuous#across#the#year),#which#are#likely#take#several#months,#and#probably#focused#around#the#October#to#February#Winter#heating#season.#‘Levels’#refer#to#the#level#of#data#collection#across#the#4#classes#of#variables#set#out#on#pages#3M7#above.##Wave%1%Level#1#M#a#total#achieved#sample#of#10,000#household#interviews#No#level#2#or#3#data#collection##Wave%2%Level#1#M#a#total#achieved#sample#of#10,000#comprising#approximately#2,000#

crossMsectional#cases#and#around#8,000#cases#interviewed#at#wave#1#(level#1)#

Level#2#M#approximately#2,750#buildings#surveys#completed#by#a#surveyor#and#energy#smart#metering#and#around#5#devices#installed##

Level#3#M#approximately#100#households#with#the#full#range#of#monitoring#(20M30#devices#installed)#

#Wave%3%


! 28#

Level#1#M#a#total#sample#of#10,000#comprising#approximately#2,000#crossMsectional#cases#and#8,000#cases#interviewed#at#wave#2#

Level#2#–#a#further#ca.#2,750#buildings#surveys#completed#by#a#surveyor#comprising#2,500#cases#not#surveyed#at#wave#2#plus#250#who#were#surveyed#at#wave#2#to#deal#with#changes/issues#

Level#3#M#approximately#100#households#with#the#full#range#of#monitoring#(20M30#or#so#devices#installed)#–#all#cases#had#not#had#the#buildings#survey#or#monitoring#at#previous#waves.#

#Wave%4%Level#1#M#a#total#sample#of#10,000#comprising#approximately#2,000#crossM

sectional#cases#and#8,000#cases#interviewed#at#wave#3#Level#2#–#a#further#ca.#2,750#buildings#surveys#completed#by#a#surveyor#

comprising#2,500#cases#not#surveyed#at#wave#2#plus#250#who#were#surveyed#at#wave#2#

Level#3#M#approximately#100#households#with#the#full#range#of#monitoring#(20M30#or#so#devices#installed)#–#all#cases#had#not#had#the#buildings#survey#or#monitoring#at#previous#waves.#

#Wave%5%onwards%At#this#point#level#2#data#would#have#been#collected#(social#and#buildings#data)#and#would#be#continuing#to#be#delivered#(energy#and#environment#data)#from#approximately#7,750#households.#This#would#generate#an#average#95%#confidence#level#margin#of#error#of#approximately#+/M#3%#across#the#five#different#housing#type#groupings#(higher#for#flats#and#conversions#but#lower#for#all#other#categories).####Appendix%B#to#this#Annex#represents#this#design#graphically.###At#this#point,#the#number#of#panellists#who#had#taken#part#in#all#five#waves#would#have#reduced#to#around#3M4,000.#One#option#would#be#for#them#to#drop#out#of#the#panel#altogether.#If#this#was#undertaken,#then#it#would#be#necessary#to#increase#the#scale#of#the#crossMsectional#sample#from#the#annual#2,000#or#so#to#3,000#to#4,000#to#maintain#the#overall#size#of#the#panel.#Such#decisions#could#be#undertaken#at#a#later#stage#once#the#longerMterm#aspirations#of#the#survey#and#more#information#about#achieved#attrition#and#response#rates#from#initial#waves#were#known.##

# %


! 29#

5.%Other%considerations%

a. A%scalable%approach%!

A#further#consideration#is#attempting#to#build#in#some#kind#of#resilience#into#the#survey#design.#The#likelihood#that#the#survey#would#be#funded#from#multiple#sources#in#addition#to#general#changing#economic#circumstances#mean#that#future#changes#in#the#availability#of#funding#cannot#be#ruled#out.#The#survey#should#therefore#be#designed#with#this#in#mind.#The#proposed#design#would#facilitate#a#scalingMup#or#down#of#activity#to#meet#changing#financial#circumstances#in#each#of#the#following#ways:##• change%the%sample%size#–#in#year#1#this#could#be#achieved#by#increasing#

the#initial#sample#(e.g.#between#12,500#and#20,000).#Clearly#the#sample#size#in#the#first#year#has#consequences#for#the#number#of#longitudinal#respondents#at#later#years.#In#years#2#onwards,#it#would#be#advisable#to#maintain#the#size#of#the#longitudinal#sample#at#the#level#of#all#panellists.#Although#it#would#be#possible#to#for#them#to#miss#out#a#year,#this#would#be#detrimental#to#response#in#the#year#they#return#to#the#survey.##

• change%or%remove#the%crossEsectional%element%in#certain#years#if#necessary.#Again#there#would#be#consequences#for#the#continued#topMup#and#refresh#of#the#longitudinal#panel#sample#but#a#small#reduction#could#be#recovered#by#boosting#the#crossMsectional#element#in#subsequent#years.#

• change%or%remove%the%building%and%monitoring%elements.#These#would#not#have#longMterm#impacts#in#the#sense#that#the#panel#would#be#able#to#continue#and#the#data#may#be#collected#at#later#point,#though#this#would#reduce#the#efficacy#and#value#for#money#overall.#The#lowest#risk#element#that#could#be#removed#temporarily#is#level#3#data#collection.#

• change%the%questionnaire%length.%This#is#likely#to#be#less#efficient#than#removing#cases#but#would#deliver#reductions#in#costs.#A#common#approach#would#be#to#develop#a#set#of#‘core’#question#modules#which#are#included#every#year#alongside#‘rotated’#modules#which#can#be#included#or#excluded#depending#on#policy#requirement#or#funding#availability#(see#Omnibus#section).#

#Other#changes#to#the#sample#design#M#such#as#increasing#or#decreasing#the#level#of#sample#clustering#M#would#not#be#an#option.#Once#the#longitudinal#panel#is#established,#the#location#of#panel#members#defines#where#interviewers#and#surveyors#or#monitors#have#to#travel#to#and#hence#there#is#no#scope#for#adjusting#this.#

b. An%Omnibus%element%From#time#to#time,#it#may#be#of#interest#to#generate#feedback#on#particular#areas#of#energy#policy#as#they#arise.#A#regularly#fielded,#largeMscale#nationally#representative#survey#could#provide#a#facility#for#generating#feedback#on#such#issues.#As#already#described,#a#common#practice#is#to#develop#selfMcontained#


! 30#

modules#of#questions.#These#can#either#be#rotated#on#or#off#the#main#questionnaire#on#an#annual#or#regular#basis#or#alternatively#separate#modules#can#be#run#on#separate#subMsamples#(e.g.#a#random#half#of#the#total#sample).#This#means#more#topics#can#be#covered#without#increasing#the#overall#questionnaire#length#and#hence#respondent#burden.##A#similar#approach#could#be#taken#to#provide#an#‘omnibus’#or#‘policy#barometer’#facility.#A#certain#portion#of#the#questionnaire#could#be#ringMfenced#for#us#as#an#omnibus#facility#with#questions#developed#and#fielded#as#they#are#required.##A#key#advantage#of#such#an#approach#is#that#responses#to#the#particular#omnibus#questions#could#be#crossManalysed#by#the#variables#routinely#created#as#part#of#the#main#survey.#It#would#not#be#necessary#to#ask#routine#socioMdemographic#questions#as#they#would#be#part#of#the#main#questionnaire.#Furthermore,#all#the#substantive#question#variables#as#well#as#the#building#characteristics#or#monitoring#data#would#also#be#available#and#would#likely#to#be#interesting#analysis#variables.#

#If#this#approach#were#to#be#taken,#there#would#need#to#be#some#consideration#of#which#respondents#were#to#be#included#in#the#omnibus#sample.#It#is#likely#that#this#would#need#to#be#the#fresh#sample#element#each#wave,#in#order#to#avoid#having#responses#affected#by#prior#involvement#in#the#study.#However,#it#may#be#that#feedback#from#the#longitudinal#panel#may#also#be#of#interest,#depending#on#the#nature#of#the#topics#or#questions#asked.#

#The#feasibility#of#building#in#an#omnibus#element#to#the#study#will#ultimately#depend#on:#

#• the%availability%of%space%on%the%questionnaire#–#there#may#be#

pressure#on#what#needs#to#be#covered#on#the#questionnaire#meaning#that#there#is#insufficient#space#to#include#an#omnibus#element.##

• the%timeframe%for%how%quickly%feedback%is%required#–#such#policy#feedback#requirements#often#require#a#relatively#short#turnaround,#perhaps#just#a#few#months.#The#nature#of#large#surveys#is#that#they#tend#to#be#run#to#an#annual#cycle.#Seasonality#means#that#this#would#almost#certainly#be#a#feature#of#a#largeMscale#energy#survey.#This#means#that#there#is#normally#a#delay#of#up#to#two#years#between#questions#being#developed,#fielded#and#data#delivered.#For#example,#if#a#particular#issue#became#apparent#just#after#the#annual#questionnaire#had#been#designed,#it#would#be#necessary#to#wait#a#year#before#the#next#survey#was#launched,#a#further#year#for#data#collection#and#then#the#subsequent#data#processing#and#analysis#to#take#place.##

#For#these#reasons,#it#is#unlikely#that#such#a#survey#would#be#able#to#provide#a#quick#‘policy#barometer’#function.#


! 31#

c. An%Innovation%panel%Understanding#Society#incorporates#an#innovation#panel.#This#is#a#separate#sample#of#respondents#that#are#used#as#a#testMbed,#often#for#methodological#experiments#for#example.#On#the#energy#study,#this#could#operate#as#a#means#of#testing#new#monitoring#equipment#prior#to#its#inclusion#on#the#main#sample.###To#generate#the#sample#for#a#panel,#a#small#number#of#cases#(say#100#or#so)#who#are#willing#to#have#new#equipment#could#be#taken#from#the#main#sample#after#having#had#level#2#monitoring#–#say#at#year#3.#Funding#for#such#cases#would#need#to#be#separately#funded#or#subsidised#from#the#main#survey#costs.###

# #


! 32#

Appendices%%Appendix%A:%Graphic#representation#of#the#proposed#approach#to#the#UK#Energy#Lab#design#Appendix%B:#UKEL#5#year#cost#calculator%(available#only#to#DECC#in#a#separate#workbook)% %


! 33#

Appendix(A:(Graphic#representation#of#the#proposed#approach#to#the#UK#Energy#Lab#design##

Sample'100%'

0%'

Longitudinal+

80%'

Social/human'

Time'spent'collec6ng''at'each'home'

Building' Energy'Monitoring'

Wave'1:'Year1B2'

Individual''households'

Wave'2:'Year'2B3'

Environmental'Monitoring'

Total'6me'the'same'as'Wave'I'for'X'sec6onal'

Addi6onal'repeat'visits'of'varying'lengths'will'be'needed'for'a'subsample'due'to'fabric'changes'or'monitoring'equipment'issues'

Smart'metering'starts'appearing'more'frequently'through'roll'out'

Wave'3:'Year'3B4'

X-sec1onal+

150%'

Individual''households'

Ini6al'sample'150%'of'ongoing'sample'in'order'to'hit'target'longitudinal'level'due'to'expected'drop'out'rates'

Level'1:'Straight'social'survey'

Level'2:'1'+'self'complete'for'all'h/h'

Level'3:'2'+'1B2'physical'device(s)'

Level'1:'Building'self'report'as'in'EHS'

Level'2:'Surveyor'visit'for'SAP'

Level'3:'2'+'more'measures'

Level'1:'Self'report'and'meter'reading'

Level'2:'Smart'meter' Level'3:'2'+'2B3'more'monitors'

Level'1:'Self'report'of'condi6ons'

Level'2:'short'version'of'1'+'3'monitors'

Level'3:'2'+'3B5'monitors'(6B8'total)'

Data'collected'in'that'wave'

Data'collected'in'previous'waves'that'is's6ll'relevant'

Time'(with'varying'scales)'

Key$

UK$Energy$Lab/Longitudinal$Panel$baseline$design$

uk energy lab final report - annex c - methods & design v2 ... ·...

Documents