the real time delphi oct 2007

The Millennium ProjectOctober, 2007

Conference room problems

Now the “classic” means for combining expert opinions

Anonymity

Feedback

The 1964 S&T Study

Developed in a DARPA study Web based Does not use “rounds” as in conventional

Delphi Permits revisits during the study Some applications to date

Energy MP Priorities Education and learning Decision making

Asynchronous, round based Pencil and paper: mail or fax back On line sequential questionnaires

Synchronous, round based Consensor Specialized machines, laptops

Real Time Delphi, roundless

An on-screen form presents a series of questions to each participant, and for each: The average (or median) response of the group

so far (and possibly the distribution of responses) The number of responses made so far A button that opens a window showing reasons

that others have given for their responses. A button that opens a window that allows the

respondent to type in justifications for their own answer.

A space for the new numerical estimate, answering the question.

If an answer is beyond a pre-specified distance from the average an indicator flags the question

When the flag is “up” reasons become particularly important

There is no explicit second round When the respondent comes back to the

study in a minute or a day, the input form is presented with prior answers

By then others may have contributed judgments, the averages or medians may have changed and other questions may be flagged

The respondents are now not shown the group average until after they had inserted an answer

The answers can be submitted cell by cell rather than the entire document at once.

To avoid crowding, the reasons can appear on a separate page.

A notation appears after the respondent submits an answer indicating that the response has been received and entered into the database.

One, two, or four columns of questions

Inclusion of extensive hyperlink reference material permits the respondents to retrieve data on line while completing the questionnaire.

Reference material can appear directly on the questionnaire

No Questions Your Responses Group Reasons

1 When will Israel be recognized as a nation by the Palestinian Authority?

Link go

Your estimate: 2008

Num Responding: 16

Average: 2010

Your reasons: Conciliation is mandatory

Things will happen fast when the suicide bombing stops

Conciliation is mandatory

2 When will cell phones replace 75% of standard phones?

Link go

Your estimate: 2010

Num Responding: 15

Average:2010

Your reasons: Cells phones leap frog infrastructure

Cells phones leap frog infrastructure

Too many towers required

3 When will a method be found to denature spent uranium rods?

Link go

Your estimate: 2050

Num Responding:5

Average:2100

Your reasons

DIFFERENT THAN AVERAGE

Immensely valuable

Tough to do

Number

Possible Project Importance to the MP

.

1

Promotion of 2007 SOF (translations, launchings, reviews, Foreign Affairs ad, mailings, etc.) Are there any new techniques that we have not tried that may prove effective?Should the Nodes have responsibility for selling in their countries?Should we print free copies for the General Assembly?

Your answer has been entered, and you may change it at any time. The average group answer 6.1 Respondents: 8 Submit only this cell Reasons click here

2

Complete and issue a new CD ROM: Futures Research Methodology 3.0 (Jerry) We have the raw materials already in hand but not the time to complete.What else can we do to finish this project?

The average group answer will appear after you have entered a value above Respondents: 5 Submit only this cell Reasons click here

go

go

Number Possible Development Likelihood by 2030 (%)

1

National programs for improving collective intelligence

Some richer as well as lower income countries have (by this year of 2030) made improving collective intelligence a national goal; this includes improving individual as well as intelligence for their nations-as-whole. Click here to see references

Likelihood by 2030 The average group answer: 61,7Respondents: 195

Comments on this possibility (page 2) click here

2

Just in time knowledge and learning

Rote learning has diminished in importance. With ubiquitous computing and education for life-long learning, 'just in time knowledge' has become the norm. Reasoning, problem solving, and learning strategies form the core focus of public educational systems. Click here to see references

Likelihood by 2030 The average group answer: 71.1 Respondents: 185

Comments on this possibility (page 2) click here

72

65

Submit

Submit

Variables

What is the best plausible value this

variable can achieve in 2016?

What is the worst plausible value this

variable can achieve in 2016?

How important is this variable to the future of the world over the next

10 years (10= essential)

How important is this variable to the future of your country over the

next 10 years (10= essential)

1

Infant mortality (deaths per 1,000 births)World (1983)= 89.1World (2005)= 50.0Singapore (2005)= 2.29Angola (2005)= 187.5Click here to see references

Best plausible global value in 2016? 38The average group answer: 40.5Respondents: 17 Reasons click here (after submitting a numerical answer above.)

Worst plausible global value in 2016? 90.0The average group answer: 100Respondents: 11 Reasons click here (after submitting a numerical answer above.)

Its importance to the future of the world? 7.0The average group answer: 8.0 Respondents: 6 Reasons click here (after submitting a numerical answer above.)

Its importance to the future of your country? 5.0Respondents: 4 Reasons click here (after submitting a numerical answer above.)

2

Food availability (Cal/cap)World (1980)= 2549World (2002)= 2798Egypt (2002)= 3300Eritrea (2002)= 1500Click here to see reference

Best plausible global value in 2016? 3000The average group answer: 3,166.5 Respondents: 21Reasons click here (after submitting a numerical answer above.)

Worst plausible global value in 2016? 2300The average group answer: 2,400.0 Respondents: 11Reasons click here (after submitting a numerical answer above.)

Its importance to the future of the world? 6The average group answer: 4.5 Respondents: 21Reasons click here (after submitting a numerical answer above.)

Its importance to the future of your country? 7Respondents: 11Reasons click here (after submitting a numerical answer above.)

Go GoGo Go

Go Go Go Go

.No serious negative effects

Likely to be effective

Implement quickly

PlausibleProvides

useful feedback

Reasonable cost

Weights

Avg.: 4Responses:

5Reasons.Input 4Justify

Avg.: 6Responses:

5Reasons.Input 5 Justify

Avg.: 7Responses:


Avg.: 8 Responses:


Avg.: 3 Responses:


Avg.: 6Responses: 5Reasons.Input 5Justify

Governments modify school curricula to remove cultural biases

Avg.: 10Responses:


Avg.: 6Responses:


Avg.: 4Responses:


Avg.: 5 Responses:


Avg.: 9 Responses:



The world implements a vastly improved disease early warning system

Avg.: 2Responses:


Avg.: 3Responses:

5Reasons.Input 2 Justify

Avg.: 1Responses:


Avg.: 5 Responses:


Avg.: 9 Responses:



URL: www.realtimedelphi.com Just sign in and use one of the

following codes: mp08 edu xsofix

Average, median, standard deviation

Distribution of opinions

Summaries by region

Reasons could be identified and associated with a particular numerical response,

Analysis techniques had also progressed. In the case of the later priorities study the analysis included not only computation of average scores, but also the distribution of responses, so that it was possible to identify distribution of responses, and thus proposed projects about which there was high agreement and those which may have shown similar average scores, but were in fact bi-modal.

Client PriceNode initiated studies; limit 1 study per year per node, total 5 node initiated studies max per year.

Free; but if external funding is available for the node’s study then, $1,500 or 10%

whichever is greaterConsulting firms for internal use, limit two studies per year for evaluation and familiarity

$15,000 per year/first year

Sponsors for internal use, limit two studies per year for evaluation and familiarity

$10,000 per year/first year

Consulting firms for external use with clients, limit 500 participants per external study.

$50,000 per study

Universities for teaching purposes $8,000/yearCorporations, NGO’s, government agencies, limit 500 participants per study. $35,000 per study

Consulting firms for external use with clients, limit 500 participants per external study.

$50,000 per study

the real time delphi oct 2007

Business