1

Doing Policy Research

UA702: Research Methods for Urban & Public Policy

Danilo Yanich

Center for Community Research & ServiceSchool of Public Policy & AdministrationUniversity of DelawareNewark, DE 19716

File: 800\2004\Mtg15\DoingPolicyResearch

2

Definition of policy research

• A mixture of science, craftlore, and art (Peter Rossi)

– science is the body of theory, concepts, and methodological principles

– craftlore, the set of workable techniques, rules of thumb, and standard operating procedures

– art, the pace, style, and the manner in which one works

3

Definition of policy research, p.2

• The process of conducting research on, or analysis of, a fundamental social problem…

• …in order to provide policymakers with pragmatic, action-oriented recommendations for alleviating the problem

• Although there are several types of research processes to alleviate social problems, policy research is…

– unique in focusing on…– action-oriented recommendation to…– fundamental social problems

4

Defining characteristics of policy research

• Originates in the world of action

• Results are destined for the world of action

5

Properties of the world of action

• Time constraints

• Language and concepts are different from those of discipline

6

Properties of the world of action, p.2

• Interest, control of resources, conflict

– Set of parties who control resources in an on-going system that involves conflict

– Researcher is outside of system...but...

– Research problem comes from the system...and...

– Research results are injected back into the system...

– Results in adding to the resources of certain parties & diminishing those of others...

– In short, a political system---power structure will change based on the research results

– While research results will have been arrived at by a procedure that is neutral to these interests, their results are not neutral in their impact on the world of action

7

Properties of the world of action, p.3

• Economy vs redundancy

– Discipline research---uses theory to substitute as economy of information---makes predictions

– Action world---economy of information is unimportant

– What is important is the usefulness of the research results for policy purposes...

• A criterion which often dictates redundancy

8

Context of the Policy Arena

• Research findings are only one of many inputs to policy decision

• Policy is not made, it accumulates...

– Policy research must be able to provide the empirical evidence to support the series of successive approximations that constitute policy

• Process of making policies is as complex as the social problem itself

9

Characteristics of policy research studies

• Are multidimensional in focus...as is social problem

• Use empirico-inductive research orientation...

– Not traditional scientific hypothesis-testing approach

• Focus on malleable variables...

– Those attributes of problem amenable to intervention by policy.

• Responsive to study users

• Explicitly incorporate values

10

Principles governing policy research---given character of world of action

• Decision and action will be taken within a certain time, so…

– Results must be offered then, not later...so…

– Partial accurate information at the time of action is better than complete information after the time for action has passed

– Implications for research design

• Must be fitted to time sequencing of social actions

• Must provide partial results at various points

• Must have steady accumulation of research results that can aid decisions, rather than a single research result at the end

11

Principles governing policy research, p.2

• Value of the research is based on…

– the high probability of giving approximately right guides to action…

– rather than correspondence to good theory

12

Principles governing policy research, p.3

• Different classes of variables

– Discipline has independent and dependant variables

– Policy research:

• outcomes of policy…intended or not

• policy variables…amenable to social control

• situational variables…play part in causal structure and must be controlled in analysis, but not subject to policy control

– Therefore, it is necessary to treat differently policy variables which are subject to manipulation and situational variables which are not

– Important because must give information on variables which provide a handle for action

13

Principles governing policy research, p.4

• Canons of scientific method and values implied in those canons govern

the execution of the research

– Values of world of action govern the formulation of policy research problems

– Transmission of results back to world of action can be governed be either set of values.

14

Stages of Policy Research

1. Negotiation Stage: Period from initial discussion with client to the beginning of analysisProduct: Signed contract & blueprint for research

2. Analysis Stage: Period between beginning of research tasks & preparation of final reportProduct: Final report

3--Communication Stage: Begins when final report is presented & ends when the action the research was to inform is takenProduct: Active dissemination

15

Policy Research, Stage 1, Negotiation

• Questions– What must be accomplished?

– Context of problem?

– Who came to whom?...critical question

• If you went to client, you are seller of skills...method, specialized knowledge, experience, analysis

• Difficult position because it suggests a lack of same capacities on part of client...

– I try to pitch toward the value of objective outside source of information

• Point: must make skills that you have relevant to concerns of client

16

Policy Research, Stage 1, p.2

• If client came to you, questions of relevance also apply

– Must be careful...not skeptical...of agendas

• May want policy research to confirm already held position/strategy

• May want policy research to be change agent for change that is already thought out...needs hammer, better yet, screwdriver

• May want appearance of activity by using research as a substitute...use it reluctantly because must do something. NOTE: Woody Hays notion of the value of the pass...OR...

• May really want unbiased information...often couch the request that way, but the way the question is asked would skew the results

– REMEMBER, underlying this notion is the concept of interested parties who control resources to one extent or another...Results will likely change the balance

– Must say what research can and cannot do

17

Policy Research, Stage 1, p.3

• Research Issue and Research Question

– Getting to agree on these matters should take place over several meetings & discussions...with preliminary proposals

– Result of these discussions (in which information flows in both directions) is…

• A research question (in scientific definition of term) which, when examined, will yield information that world of action can use

• Research issue is negotiable, research question is not

• Separation of policy outcome, policy control and situational variables should be basis for discussion between client and researcher

18

Policy Research, Stage 1, p.4

• Research question must be one that is relevant to client’s information needs and…

– It must meet the rigors of the scientific method

• Contract that is signed must specify what is expected of both parties...

– From client, access to data, other support

– From researcher, set of deliverables and schedule

19

Policy Research, Stage 2, Analysis

• Communication begun in 1st stage must continue so that any changes in the work that conditions warrant do not surprise client

• Canons of scientific method must govern research

– That is why implied difference between basic and applied research in terms of method is bogus...research is research with same rigor, same grounding in theory

– POINT: Any change in method must first satisfy scientific rigor...if can not...then no change...

• And must explain to client...if discussions good during first stage, no problem, if not, big problem

• Partial reports on progress should continue throughout 2nd stage

20

Policy Research, Stage 3, Communication

• Communication carried through first two stages pays off here

• Speaking truth to power is a process...must advocate for positions to which the research brought us...

• Not advocate a particular course of action...but advocate for the relevance of the results

• Too often, we drop off the final report on decision-maker’s desk and walk away feeling that the merit of the information is not only pre-eminent, but also self-evident

– Many reports on back shelves of decision-makers’ offices which attest to the inaccuracy of that position

– “Social-sciencese” is not the language of the world of action

21

Policy Research, Stage 3, p.2

• Many valid sources of information to decision makers beyond research results

– All of which have advocates...so, too, should policy research

– Can not argue that advocacy is political and therefore, would sully our objective reputation...

– Do not advocate that policy research results should be the only info. that dec-mkrs see, but it must be seen as part of many others...

– That does not violate the objectivity of the research...

– The world of action is POLITICAL in the best sense of the word...reconciling differences

22

Policy Research, Stage 3, p.3

• This speaking-truth-to-power process does not occur in a political vacuum...

– If we are to be successful, to be heard, we must engage the world of action when that engagement is critical to the use of policy research to solve problems

23

Guidelines for communicatingpolicy research results to policy makers

• Communicate throughout the study...two way communication...may use pilot project

• Communicate to different users

• Effective presentation creates the basis for good communication.

• Oral communication is usually more effective than written