whole30® program evaluation rationale vika thayer walden ... · metabolism, control sugar...

Program Evaluation Rationale 1

Whole30® Program Evaluation Rationale

Vika Thayer

Walden University

Dr. Michael Burke

Program Evaluation (EIDT 6130)

January 22, 2017


Whole30® Program Evaluation Rationale

The Whole 30 ® Program is a 30-day elimination diet that results in better overall health

(n.d.). It makes that claim through reducing food groups, where users consume none of the

following in 30 days: legumes, grains, dairy, sugar, and alcohol. It is based on the premise that

people are at odds with how to eat for health, based on misguided attempts at guidelines by the

USDA. The rules are very rigid and clear so that there are not excuses to fail. The creators and

consumers alike can reset all of their body markers in just a month, and create a more holistic

relationship with food as fuel.

The United States Department of Agriculture (USDA) is at odds with the Whole 30®

Program, because it claims to recommend building blocks for better health that includes: 2.5

cups a day of vegetables (legumes), 6 ounces per day of grains, 3 cups per day of dairy, and to

eat less sugar (Sisson, 2015).

Big Agriculture, consisting of Soybean farmers, Corn farmers, Dairy farmers, and Cattle farmers,

is at odds with the Whole 30® Program too. Soybean farmers, who produce soy sauce, tofu, and

miso, do not get purchased for 30 days. Corn is considered a grain on the program, dairy is not

allowed, and cattle farmers also lose out, as Whole 30® promotes grass-fed meat, which is a

product that traditional cattle farmers in Big Agriculture do not produce.

Big Pharmaceutical (Big Pharm) is at odds with the Whole 30® Program because the

food restrictions often result in healing conditions that Big Pharm also claims to fix: healing

aches and pains, achieving ideal weight, balance immune system, balance hormones, and reduce

hunger.


References

Whole 30. (n.d.). Rules. Retrieved from http://whole30.com/whole30-program-rules/

Sisson, M. (2015, October 22). A brief history of U.S. dietary guidelines [Blog post]. Retrieved

from http://www.marksdailyapple.com/a-brief-history-of-u-s-dietary-guidelines/

http://whole30.com/whole30-program-rules/http://www.marksdailyapple.com/a-brief-history-of-u-s-dietary-guidelines/

Program Analysis 1

Whole30® Program Analysis for Evaluation

Vika Thayer

Walden University

Dr. Michael Burke


January 29, 2017

Program Analysis 2

Whole30® Program Analysis for Evaluation

The Whole 30® Program was created by Dallas Hartwig (MS, PT) and Melissa Hartwig

(CISSN) in 2009. It is a short term, 30-day diet based on the Paleo diet where users eliminate

foods that commonly irritate the gut, namely sugar, grains, legumes, dairy, and alcohol, to reset

metabolism, control sugar cravings, minimize unhealthy habits, balance immune system, and for

people to “figure out the way the food they are eating is actually affecting them” (Whole9life,

n.d.).” It “could lead to weight loss, higher energy levels, better sleep, improved focus and

mental clarity and even ‘a sunnier disposition’” (Heid, 2016). Jackson and many consumers

have “experienced a significant reduction (if not complete remission) of their autoimmune

disorder by eating the diet (2014). Other notable Paleo diet stakeholders include Dr. Loren

Cordain, a nutritionist and exercise physiologist who coined the term; Robb Wolf, a research

biochemist and promoter of the Paleo diet; Arthur De Vany, a former economics professor and

Paleo content publisher; and Mark Sisson, a former elite endurance athlete and Paleo content

promoter.

Consumers of the paleo diet and Whole30® program specifically are stakeholders, which

include gym owners, medical/wellness clinics, small businesses, and other community groups

(Whole30, 2014). They already believe the process does all the aforementioned and their money,

or lack thereof, can speak volumes to producers of the products that are eliminated.

Besides the paleo diet advocates and consumers, other notable stakeholders include the

United States Department of Agriculture, Big Agriculture or “Big Ag” for short, and Big

Pharmaceutical, or “Big Pharm” for short. USDA, supported by many nutritionists and

dieticians, voice concern over eliminating entire food groups, especially for long term health and

because it may worsen underlying conditions (Heid, 2016; Miller, 2014). Big Ag is a group

Program Analysis 3

loosely made of farmers of soybean, corn, and dairy, all of which are eliminated on the diet; Big

Pharma are companies that develop and market drugs for medication treating autoimmune

disorders, the same disorders Whole 30® coincidentally seems to cure.

Whole 30® uses steps in its operations: Step 1 is learning the basics, Step 2 is learning

the rules, Step 3 is committing, Step 4 is building support, Step 5 is purging your food stores and

meal preparation, Step 6 is planning for conflicts, Step 7 is hiding the scale, and Step 8 is start

the 30 days of elimination (Whole30.com, n.d.). The operation is rigidly detailed, which is what

seems to make it successful and off-putting to many.

Whole 30® has a large testimonial section touting performance success, although it is not

always clear if these are healthy individuals or individuals looking to fix a disorder. CrossFit

Potential and CrossFit Orillia members produced large strength and endurance gains on Whole

30 (Crossfit7220, 2015). Many dieticians seem unable to stick to 30 days for a true evaluation

(prettylittlegrub, 2016; Toups, 2015), and outside dietician evaluations seem negative (Miller,

2014). The Whole30® program has not been formally evaluated at this time. My evaluation

would have evaluated the implementation of Whole30®, because the outcomes at this time point

to success: a 2013 survey of more than 2300 Whole30’ers, 87% started and completed program

as reported by Melissa Hartwig (Ballantyne, 2014).

Political context is an important factor for Whole 30®, especially as the USDA and

dieticians in general, Big Ag, and Big Pharma all oppose the program. Whole 30® is readily

available to consumers via a website and multiple books, as well as a burgeoning support

community of social media. It is a direct contrast to the Standard American Diet (SAD) that is

endorsed by the USDA, notably “avoid too much fat, saturated fat, and cholesterol” and “eat

plenty of…grains” (Sisson, 2015). Big Ag has put effort into paid advertising, pro-agriculture

Program Analysis 4

campaigns, and agriculture-friendly foundations to combat bad publicity, and Big Pharma has a

well-established track record with advertising (Morris, 2011).

The biggest potential ethical challenges in the rapid rise in popularity. “Fad diets and diet

cults appeal to emotion, not to evidence-based reasoning. They give their followers a sense of

belonging, a sense of control over their health, and a virtuous feeling of doing something that is

difficult but good for them…true believers can always find a few cherry-picked scientific studies

that tend to support their beliefs” (Hall, p. 15, 2014). All paleo-based clinical trials to date have

been with small sample groups, even if they all promote the lifestyle (Ballantyne, 2016).

Ultimately, Whole 30® is a very restrictive diet that either cleans up an already healthy

consumer, or affects them in the long term. It may help someone or it may bring to light serious

medical conditions. Miller correctly identifies with Fitzgerald in that “science has not identified

the healthiest way to eat” (2014).

Program Analysis 5

References

Ballantyne, S. (2014, March 24). Who should do the Whole30: an interview with Melissa

Hartwig [Blog post]. The Paleo Mom. Retrieved from

https://www.thepaleomom.com/whole30-interview-melissa-hartwig/

Crossfit7220. (2015, April 26). Whole30/MEBB study: Crossfit Potential and Crossfit Orillia

[Blog post]. Retrieved from https://cf7220.wordpress.com/2015/04/26/whole30-results-

on-performance/

Hall, H. (2014). Food myths: what science knows (and does not know) about diet and nutrition.

Skeptic Magazine, 19(4), p. 10-19

Heid, M. (2016, October 12). You asked: should I try the Whole30 diet? Time. Retrieved from

http://time.com/4525768/whole30-diet-nutrition-food/

Jackson, K. (2014, October 8). How I ate away my RA (Rheumatoid Arthritis) with the paleo

diet. Retrieved from http://www.paleoplan.com/2014/10-08/ra-rheumatoid-arthritis-

paleo-diet/

Miller, A. M. (2014, December 15). Should you try the Whole 30 diet? U.S. News and World

Report. Retrieved from http://health.usnews.com/health-news/health-

wellness/articles/2014/12/15/should-you-try-the-whole30-diet

Morris, F. (2011, March 4). Agriculture industry seeks to restore its image. Retrieved from

http://www.npr.org/2011/03/09/134263421/agriculture-industry-seeks-to-restore-its-

image

Prettylittlegrub.com. (2016, April 28). My Whole 30 experience and why I quit after 8 days

[Blog post]. Retrieved from http://www.prettylittlegrub.com/2016/04/28/my-whole-30-

experience/

https://www.thepaleomom.com/whole30-interview-melissa-hartwig/https://cf7220.wordpress.com/2015/04/26/whole30-results-on-performance/https://cf7220.wordpress.com/2015/04/26/whole30-results-on-performance/http://time.com/4525768/whole30-diet-nutrition-food/http://www.paleoplan.com/2014/10-08/ra-rheumatoid-arthritis-paleo-diet/http://www.paleoplan.com/2014/10-08/ra-rheumatoid-arthritis-paleo-diet/http://health.usnews.com/health-news/health-wellness/articles/2014/12/15/should-you-try-the-whole30-diethttp://health.usnews.com/health-news/health-wellness/articles/2014/12/15/should-you-try-the-whole30-diethttp://www.npr.org/2011/03/09/134263421/agriculture-industry-seeks-to-restore-its-imagehttp://www.npr.org/2011/03/09/134263421/agriculture-industry-seeks-to-restore-its-imagehttp://www.prettylittlegrub.com/2016/04/28/my-whole-30-experience/http://www.prettylittlegrub.com/2016/04/28/my-whole-30-experience/

Program Analysis 6

Sisson, M. (2015, October 22). A brief history of U.S. dietary guidelines [Blog post]. Retrieved

from http://www.marksdailyapple.com/a-brief-history-of-u-s-dietary-guidelines/

Toups, K. (2015, May 25). Whole 30: a wholly misguided approach to healthy eating [Blog

post]. Retrieved from https://kellytoups.com/2015/05/25/whole30-a-wholly-misguided-

approach-to-healthy-eating/

Whole9life.com. (n.d.). You ask, we answer: “do you ever cheat?” [Blog post]. Retrieved from

http://whole9life.com/2014/02/you-ask-we-answer-do-you-ever-cheat/

Whole30.com. (n.d.). Rules. Retrieved from http://whole30.com/whole30-program-rules/

Whole30. (2014, December 18). 9 steps to a successful group Whole30 “challenge” [Blog post].

Retrieved from http://whole30.com/2014/12/group-whole30-challenge/

http://www.marksdailyapple.com/a-brief-history-of-u-s-dietary-guidelines/https://kellytoups.com/2015/05/25/whole30-a-wholly-misguided-approach-to-healthy-eating/https://kellytoups.com/2015/05/25/whole30-a-wholly-misguided-approach-to-healthy-eating/http://whole9life.com/2014/02/you-ask-we-answer-do-you-ever-cheat/http://whole30.com/whole30-program-rules/http://whole30.com/2014/12/group-whole30-challenge/

Program: Whole30® diet Logic Model Situation: Americans are continuing to grow obese at an alarming rate, despite government food guidance. Whole30® program is a diet is set to make losing weight effortless, breaks food addiction, and change the food lifestyle forever.

• Whole30®

rules

• Whole30® online support

• Whole30® book

• Paleo blogs

Make people knowledgeable on the effects of

certain foods • Food addicts

• Sugar addicts

• Carbohydrate addicts

• People with chronic

conditions

Awareness of psychological food issues (bad habits, comfort)

• Obese people

• Athletes

• Curious people

• Type “A” people

Participants heal aches and pains

Inputs Outputs Activities Participation

Outcomes Short Medium Long

Promote clean eating lifestyle

Improved sleep quality and self-esteem

Participants balance hormones

Participants reduce hunger

Participants balance immune system

• Improved energy levels, mood

• Stress-relief

Assumptions

• Everyone suffers from a “hangover” detox in first few days

• Everyone suffers from food in some way

• Everyone has tried and failed at a diet before

External Factors

• Social factors affecting participants (competition for time, including family time)

• USDA campaign for standard American diet vs. Paleo diet

Effortless weight loss

Dramatically decrease or eliminate symptoms of medical condition

• Combat the obesity epidemic and health issues of Americans

• Change in lifestyle

• Change relationships with

food

Rev. 7/09

Program Evaluation Approaches Comparison and Questions

A Comparison of Advantages and Disadvantages of Various Program Evaluation

Approaches and Process Evaluation Questions

Vika Thayer

Walden University

Dr. Michael Burke


February 15, 2017


A Comparison of Advantages and Disadvantages of Various Program Evaluation

Approaches and Process Evaluation Questions

Of the four general types of evaluation approaches studied, including expertise and

consumer-oriented approaches, program-oriented approaches, decision-oriented approaches, and

participant-oriented approaches, the obvious choice for a program evaluation for the Whole30®

is a program-oriented approach (See Table 1). An expertise-oriented approach involves many

biases, either entirely for paleo-style eating by someone who has adopted that lifestyle, or

entirely against paleo by someone who has a strong bias towards another diet, with diet meaning

way of eating rather than a food restriction. Consumer-oriented approaches follow suite, most

notably U.S. News & World Report ranking Whole30® as the #38 of 38 diets studied.

Decision-oriented approaches are “meant to serve decision makers,” which has nothing to

do with the Whole30® and therefore, the evaluation (Fitzpatrick, Sanders, and Worthen, p. 172,

2010). A participant-oriented approach is an interesting approach that could work for

Whole30®, except that stakeholders have vastly opposing interests, and an inexperienced

evaluator like myself would be woefully inept at facilitating an agreement on evaluation

questions and everything that followed. On the other hand, a program-oriented approach is

feasible, practical, and effective for this program evaluation.

The Program Evaluation Standards speak of feasibility as practical procedures (F2) and

contextual viability (F3) for this program evaluation. The evaluator must not stick to

preordained standards but be “practical and responsive to the way the program operates”

(Fitzpatrick, et. al., p. 500, 2010). Theories and ideas start the process, but the actuality of the

program should drive the evaluation. “The point is to understand the program from beginning to


end and, then, to choose the appropriate links or components to evaluate given the stage of the

program and the information needs of the stakeholders” (Fitzpatrick, et. al., p. 167, 2010). Only

by thoroughly coming to know the program and the objectives can the evaluator effectively

proceed.

The Whole30® program is best evaluated by producing a logic model, using the Duignan

Visual Evaluation Approach to form the questions and answers, and displaying it in a poster

version (Laureate Education, n.d.). This approach produces a product that walks the stakeholder

step-by-step through the intended outcomes, questions to evaluate how the program might reach

them, and then the answers to those questions. Using Duignan’s approach overcomes the usual

limitation of unintentionally ignoring “program actions, links, outputs, or outcomes that merit

attention” (Fitzpatrick, et. al., p. 167, 2010).

The Whole30® program has proven successful outcomes. A quick online search for

“Whole30 success” brings up over 400,000 results. This evaluation will evaluate processes, to

analyze if the implementation is sound, and if there is any room for improvement. Five

evaluation questions (1-5) that determine what the Whole30® evaluation will answer are based

on recommendations from Fitzpatrick, Sanders and Worthen (p. 327, 2010). The questions and

associated standards are:

1. Have the appropriate personnel been selected to implement the program?

Standard: Appropriate personnel are those who have successfully completed

one Whole30®, can identify 3 Whole30® resources (recipe sites, online

forums, motivators, etc.), are willing to provide daily support (either

personally or via Whole30® website), and use other motivators than weight

loss (Whole 30, 2014).

2. What portion of participants complete the program?


Standard: 75% of participants from Day 1 will successfully complete through

Day 30 of Whole30® program. Success is 100% rule compliance for 30 days.

The rules are basically to eat real foods, including seafood, poultry, red meat,

eggs, oils, vegetables, fruits, nuts and seeds; and to avoid dairy, grains,

legumes (including peanut butter), alcohol, added sugar, and additives like

MSG, sulfites, and carrageenans.

3. Is the implementation of Whole30® at the necessary level of intensity and duration?

Standard: Implementation of the Whole30® is at the necessary level of

intensity and duration to break food cravings and emotional eating, where

intensity is 100% compliance and duration is 30 days

4. Do participants take part in the intended manner?

Standard: 75% of participants take part in the intended manner, which is

100% rule compliance in 30 days.

5. Is the Whole30® program encouraged outside of the website and affiliated links?

Standard: Whole30® is encouraged worldwide, with workout partners, office

colleagues, and bloggers.

These questions will impact the program evaluation by highlighting program implementation to

identify if there might be better procedures for conducting the program.

The evaluation will not cover context, needs, or outcomes or the Whole30®. The validity

of the Whole30® has everyone from researchers, archeologists, doctors and dieticians

conflicting, so context will not be evaluated. Needs of the Whole30® are completely qualitative

per person, regardless on the long-term outcome of societal effect, so that will also not be

evaluated. Lastly, the outcomes of Whole30® are overwhelmingly positive for those that stick

through the 30 days, so that evaluation would not be useful to most stakeholders. The group of


stakeholders involved in determining evaluation questions would be those interested in

improving Whole30® implementation and those that had perceived to fail because of

implementation issues.


Table 1: Evaluation Approach Comparison

Evaluation Model Advantages Disadvantages

EXPERTISE AND CONSUMER-ORIENTED

APPROACHES

Expertise-Oriented Approach (Fitzpatrick, et.

al., p. 142, 2010)

- “Emphasized central role of expert

judgment, experience, and human

wisdom”

- “Articulated standards help to introduce

consistency across experts”

- “Experts should look beyond the

standards and use their connoisseurship to

describe, interpret, and judge” important

qualities

Consumer-Oriented Approach (Fitzpatrick, et.

al., p. 147-149, 2010)

- “does not rely on content experts, but

rather evaluation experts”

- “programs with “randomized control trials

(RCTs) or regression discontinuity

designs…receive the highest

ratings…studies using quasi-experimental

designs may be endorsed with

reservations”

- Scriven’s “process emphasized a

comprehensive appraisal of the product,

including need, side effects, process,

support for users, and cost, as well as

several criteria concerning outcomes or

performance”

- ”Prompt the potential user to consider the

effectiveness of the program in achieving

its outcomes and to provide a central

location for accessing comparable

information of educational programs and

products”

Expertise-Oriented Approach (Fitzpatrick, et.

al., p. 142-143, 2010)

- “May permit evaluators to make

judgments that reflect little more than

personal biases”

- “The presumed expertise of the experts is

a potential weakness…too often, the team

contains only content expert...but may lack

experts in the evaluation process itself”

- “Independence or neutrality of experts are

questioned”

- “Lack of transparency of the process”

- “Expert judgement cannot be easily

dismissed” (like it would for an art critic)

Consumer-Oriented Approach (Fitzpatrick, et.

al., p. 147-149, 2010)

- “success [is] confined to program

outcomes” only


PROGRAM-ORIENTED EVALUATION

APPROACHES

(Fitzpatrick, et. al., p. 166-169, 2010)

- Simple (“easily understood, easy to follow

and implement, and produces information

that program directors…agree is relevant

to their mission”)

- “Caused program directors to reflect about

their intentions and to clarify formerly

ambiguous generalities about intended

outcomes”

- Face validity – “discussion of appropriate

objectives with the community being

served”

- “Logic models overcome many

disadvantages by requiring dialogue,

achieving an understanding and a clear

articulation of the reasoning behind the

program”


- “Single minded focus on objectives and their

measurement at the cost of evaluators ignoring

other important outcomes…beneficial and

detrimental, causing judgements to be

incomplete”

- Neglects program description, “the need to

gain an understanding of the context in which

the program operations and the effects of that

context on program success or failure”

- “Evaluators may neglect their role in

considering the value of the objectives

themselves”

- “Likely to focus on the theory and may ignore

unintended program actions, links outputs, or

outcomes that merit attention; desire to test the

theory as a whole may prompt them to neglect

values or information needs of stakeholders”

- “Oversimplifying the complexity of program

delivery and context; can lead to a failure to

understand how difficult, and costly, it is for

programs…to achieve stated goals”

DECISION-ORIENTED EVALUATION

APPROACHES


- “Focus on decision makers to evaluation

results will be used”

- “Among oldest approaches and still in

frequent use...their longevity speaks to their

success”

- “Succeed because they provide information

that helps people, typically managers or

policymakers, make decisions”

- “Evaluator in in charge of the evaluation and,

although focusing on the decisions of primary

stakeholders, will often use advisory groups or


- “Too focused on decisions; may neglect

stakeholders with less power”

-“Focus on managers could restrict the

information that evaluators seek, the types of

data they collect, and the dissemination of the

results”

- “Evaluators must take care to not come

across as a ‘hired gun’”

- “Programs that lack decisive leadership are

not likely to benefit from this approach to

evaluation”


seek information from other stakeholders to

supplement decisions about the evaluation”

- “Assume important decision and info to make

them have been clearly identified in advance,

and that they will remain stable during the

evaluation…evaluator should be prepared to

frequently reassess and make adjustments for

change”

PARTICIPANT-ORIENTED EVALUATION

APPROACHES


- “Awareness of value of stakeholder inputs

throughout study”

- “Stakeholders can provide more valid data

than evaluators”

- “Involving stakeholders [allows them to] gain

trust in evaluation, begin to understand it, and

consider how they might use it”

- “Policymakers know what other programs

they are considering, the budgetary and time

constraints for their decisions, and the factors,

political and otherwise, that may influence their

decisions”

- “Many evaluations make use of advisory

groups of stakeholders to pursue this

partnership – to exchange information on the

program, the clients, and the evaluation”

- “Deep participatory involvement of a few

managers and staff…is probably most

successful…in formative evaluations – those

concerned with program improvement.

Summative decisions are often made by

policymakers who are higher in the

organization…not likely to have the time to

become closely involved in the evaluation of a

program”


- “Two broad categories of drawbacks are (a)

the feasibility, or manageability, of

implementing a successful participative study;

and (b) the credibility of the results to those

who do not participate in the process”

- “Can be increased time and costs”

- “Difficult to obtain participation for the most

disadvantaged stakeholders”

- “Few evaluators have extensive training or

skills in facilitation, particularly with groups that

may have political differences”

- “Outside audience wonder if stakeholders are

able to be objective; there is potential from

bias; it is difficult to judge ones’ own work

objectively”

- “Stakeholders may not be competent to

perform the tasks that some evaluation

approaches call for; may even be true for

some evaluators” who can then call on a

colleague


References

Fitzpatrick, J., Sanders, J., & Worthen, B. (2010). Program evaluation: Alternative

approaches and practical guidelines (4th ed.). Boston, MA: Pearson

Laureate Education (Producer). (n.d.). Evaluating programs in instructional design:

Developing logic models (outcomes models) [Video file]. Baltimore, MD: Author.

Whole 30. (2014, December 18). 9 steps to a successful group Whole20 “challenge”

[Blog post]. Retrieved from http://whole30.com/2014/12/group-whole30-challenge/

http://whole30.com/2014/12/group-whole30-challenge/

Reporting Strategy 1

Whole30® Process Evaluation Reporting Strategy

Vika Thayer

Walden University

Dr. Michael Burke


February 26, 2017


Whole30® Process Evaluation Reporting Strategy

Stakeholder Reporting Strategy Implications Stakeholder Involvement

Whole30® creators and

promoters

- Interim and final reports, to reach many audiences

- Executive summary - Verbal presentation - Infographics or

Webgraphics

- News media communications

All types of reports should go

to the creators and promoters.

“General program managers

and staff are interested in the

most detail on program

operations, outputs, and

outcomes. They know the

program well and want to

improve it” (Fitzpatrick, p. 459,

2010).

Melissa Hartwig reported that

“In a 2013 survey of more than

2,300 Whole30’ers, 87% who

started the program were able

to successfully complete all 30

days” (Ballantyne, 2014).

Positive implementation results

would back up the existing

process and further validate the

results. Negative

implementation results could

become a factor for

overhauling the implementation

strategy, or provide feedback as

a need for more resources. I

believe that may have

happened a few times, now that

Whole30® is on version 5.0

and recommending guidelines

for challenges (Whole30, 2011;

2014).

These stakeholders should be

involved in weekly updates

because one iteration of the

program is only 30 days long.

Any unforeseen results should

be discussed within 48 hours.

Whole30® consumers

- Executive summaries - Posters - Infographics or

Webgraphics


Consumers and potential

consumers will be looking for


will have consumers practicing

Whole30® as usual. Negative

implementation implications

will likely not affect current

consumers much because it is a

short program. I would expect

a negative evaluation to change

how the creators implement the

These stakeholders may be

involved during the evaluation

for information collection, both

initial and follow-up questions.


brief summaries of the

evaluation, because most will

be looking for positive

reinforcement of their choices.

The saavy looking for more

information will visit the

Whole30® website or promoter

websites.

program, thereby effecting

consumers at that time.

Under Secretary for Farm and

Foreign Agricultural Services

as a representative of the Farm

Service Agency (FSA) under

the United States Department

of Agriculture (USDA) – for

soybean, corn, dairy farmers

- Executive summaries - News media

communications

The Under Secretary and policy

makers “external to the

organization who may vote on

issues relevant to the

program…are typically more

interested in information about

outcomes and impact”

(Fitzpatrick, p. 459, 2010).

A positive or negative

implementation evaluation will

not have much implication for

the Under Secretary, unless the

implementation influenced the

outcome. A decline or negative

outcome would influence

advertising and sales.


involved in choosing evaluation

questions and deliberations

over content.

Under Secretary for Food,

Nutrition, and Consumer

Services as a representative of

the Center for Nutrition Policy

and Promotion (CNPP) – for

dieticians and nutritionists

- Newsletters, bulletins brochures, posters

- Executive summaries - Infographics or

Webgraphics


The Under Secretary and policy

makers “external to the

organization who may vote on

issues relevant to the

program…are typically more


implementation evaluation will

not have much implication for

the Under Secretary, unless the

implementation influenced the

outcome. A decline or negative

outcome might influence

statistics, studies, and

advertising.




over content.


interested in information about

outcomes and impact”

(Fitzpatrick, p. 459, 2010).

Pharmaceutical Research and

Manufacturers of America

(PhRMA), Biotechnology

Industry Organization, and the

top 20 pharmaceutical

companies

- Executive summaries - News media

communications

The pharmaceutical lobby will

be “more interested in

information about outcomes

and impact” (Fitzpatrick, p.

459, 2010).


implementation evaluation

again will not have much

implication for Big Pharma,

unless the implementation

influenced the outcome. I

believe a decline or negative

evaluation would result in

either a little lest spent in

advertising because Whole30®

would not be as big of a

competitor, or that advertising

would rise dramatically to end

Whole30® completely.




over content.

Gyms, medical/wellness

clinics, small businesses, and

other community groups”

(Whole30, 2014).

- Infographics or Webgraphics

- Executive summary - News media

communications

“Those less familiar with the

program or its totality…are

also interested in brief program

descriptions” (Fitzpatrick, p.

459, 2010).


will have implementers

continuing their programs as

they are. Negative

implementation implications

would still not affect current

implementers much because it

is a short program. I would

expect a negative evaluation to

change how the creators

implement the program,

thereby effecting these groups

when a new version would be

produced.


involved during the evaluation

for information collection, both

initial and follow-up questions.


• Values, Standards, and Criteria:

Fitzpatrick, Sanders, and Worthen state that “evaluation is often valued by stakeholders because they perceive it to be objective,” and

that “the evaluator’s strength lies in the integrity of the study, the use of appropriate methods, honest and balanced interpretation of

results, and judgements and recommendations based on those results” (p. 69, 73, 2010). These authors also elaborate on Chelimsky’s

recommendations for “evaluative independence and the political requirements of a democratic society,” which are (1) “expand the

design phase” to have time to learn about political context; (2) “include public groups in evaluations,” where an “advisory or planning

task force” for collecting data increases validity and credibility by collecting data from many different perspectives; (3) “lean heavily

on negotiation” to find room for compromise and change; and (4) “never stop thinking about credibility” because that is an evaluator’s

strength (Fitzpatrick, et. al., p. 72-73, 2010). A few Program Evaluation Standards I will use to ensure openness and credibility are:

(U1) evaluator credibility by using qualified evaluators; (U2) attention to the full range of stakeholders; (U3) evaluation purposes that

are negotiated based on the needs of the stakeholders; (U4) clarification and specification of “individual and cultural values

underpinning purposes, processes, and judgments;” (F3) “recognize, monitor and balance cultural and political interest and needs of

individuals and groups”…through (P2) formal agreements; (P5) transparency and disclosure; and (P6) conflicts of interest”

(Fitzpatrick, et. al., p 499-500, 2010).

I have biases towards Whole30® being a success because I believe that it does enable self-control, weight loss, great sleep, and relief

from ailments. I have completed two Whole30® challenges approximately 5-8 years ago. I also believe that the USDA, in both the

FSA and CNPP capacities, might have an unspoken agreement to promote soy, corn, and dairy. I am aware that individual farmers are

likely separate from this effort. These biases must be reported so that they can be mitigated throughout the final evaluation report.

That being stated, this process evaluation, rather than summative program evaluation, helps to ensure those biases do not come into

play, because the evaluation is not about outcomes, but rather about a successful process.

• Potential ethical issues:

A potential purpose for the results is to promote understanding of issues, changing attitudes, changing the nature of dialogue or

interaction among groups, influencing policy, and introducing those involved to new ways of thinking through evaluation (Fitzpatrick,

p. 455, 2010). Whole30®, which requires elimination of soy, corn, wheat, and dairy, can be thought to promote the destruction of

some family’s livelihood. Understandably, some dialogue and negotiation will always be required, and likely through the entire

evaluation. Whole30® has never been formally evaluated, and fledgling scientific papers are scarce, with the first one being

published in 1984. Additionally, the USDA FSA and CNPP are not likely to be open to receive input to change their

recommendations. “Clients and other – the public – look to evaluators to provide objective, independent judgments about the quality


of programs, products, or policies. We use analytic methods to judge the product and assume that the transparency and validity of

those methods will substantiate our findings” (Fitzpatrick, et. al., p. 79, 2010). Morris and Cohn via Fitzpatrick, Sanders, and

Worthen, cite “possible ethical problems as challenges in the contracting phase, ethical concerns regarding confidentiality or

disclosure agreements, challenges in presenting findings, and ethical concerns after the report is complete concerning

misinterpretation or misuse…almost two-thirds of the evaluators reported being pressured by stakeholders to alter results” (p. 79-80,

2010).

.


References

Ballantyne, S. (2014, March 24). Who should do the Whole30: an interview with Melissa

Hartwig [Blog post]. The Paleo Mom. Retrieved from

https://www.thepaleomom.com/whole30-interview-melissa-hartwig/

Fitzpatrick, J., Sanders, J., & Worthen, B. (2010). Program evaluation: alternative approaches

and practical guidelines (4th ed). Boston, MA: Pearson.

Gifford, J. (20123, January 29). Infographics are dead. Long live webgraphics. Venture Beat.

Retrieved from http://venturebeat.com/2013/01/29/infographics-webgraphics/

Whole30. (2014, December 18). 9 steps to a successful group Whole30 “challenge” [Blog post].

Retrieved from http://whole30.com/2014/12/group-whole30-challenge/

Whole30. (2011, October 17). Better butter on the Whole30 version 5.0 [Blog post]. Retrieved

from http://whole30.com/2011/10/better-butter-on-the-whole30-version-5-0/

https://www.thepaleomom.com/whole30-interview-melissa-hartwig/http://venturebeat.com/2013/01/29/infographics-webgraphics/http://whole30.com/2014/12/group-whole30-challenge/http://whole30.com/2011/10/better-butter-on-the-whole30-version-5-0/

WK2AssgnThayerV_updatedWK3AssgnThayerV_updatedWK4AssgnThayerV_updatedWK5AssgnThayerV_updateReporting Strategy_FinalWhole30

whole30® program evaluation rationale vika thayer walden ... · metabolism, control sugar...

Documents