university of groningen hownutsarethedutch (hoegekisnl) van … · revised 10 july 2015; accepted...

University of Groningen

HowNutsAreTheDutch (HoeGekIsNL)van der Krieke, Lian; Jeronimus, Bertus F.; Blaauw, Frank; Wanders, Rob; Emerencia,Armando Celino; Schenk, Hendrika; de Vos, Stijn; Snippe, Evelien; Wichers, Maria; Wigman,JohannaPublished in:International Journal of Methods in Psychiatric Research

DOI:10.1002/mpr.1495

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.

Document VersionEarly version, also known as pre-print

Publication date:2016

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):van der Krieke, L., Jeronimus, B. F., Blaauw, F., Wanders, R., Emerencia, A. C., Schenk, H., de Vos, S.,Snippe, E., Wichers, M., Wigman, J., Bos, E., Wardenaar, K., & de Jonge, P. (2016).HowNutsAreTheDutch (HoeGekIsNL): A crowdsourcing study of mental symptoms and strengths.International Journal of Methods in Psychiatric Research, 25(2), 123-144. https://doi.org/10.1002/mpr.1495

CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.

Download date: 02-06-2021

https://doi.org/10.1002/mpr.1495https://research.rug.nl/en/publications/hownutsarethedutch-hoegekisnl(060326b0-0c6a-4df3-94cf-3468f2b2dbd6).htmlhttps://doi.org/10.1002/mpr.1495

HowNutsAreTheDutch (HoeGekIsNL): acrowdsourcing study of mentalsymptoms and strengths

Q1Q2 LIAN VAN DER KRIEKE,1† BERTUS F. JERONIMUS,1† FRANK J. BLAAUW,1,2 ROB B.K. WANDERS,1

ANDO C. EMERENCIA,1 HENDRIEKA M. SCHENK,1 STIJN DE VOS,1 EVELIEN SNIPPE,1

MARIEKE WICHERS,1 JOHANNA T.W. WIGMAN,1 ELISABETH H. BOS,1 KLAAS J. WARDENAAR1 &PETER DE JONGE1

1 Department of Psychiatry, Interdisciplinary Centre Psychopathology and Emotion Regulation (ICPE),University of Groningen, University Medical Centre Groningen, Groningen, The Netherlands

2 Johann Bernoulli Institute for Mathematics and Computer Science, Distributed Systems Group, University ofGroningen, Groningen, The Netherlands

Key wordsmental symptoms, mentalstrengths and resources,longitudinal, ecologicalmomentary assessment,crowdsourcing

CorrespondenceBertus F. Jeronimus, POBox 30.001, 9700 RB Groningen,The Netherlands.Telephone (+33) 050-3615655Email: [email protected]

†Joint first authors.

Received 10 March 2015;revised 10 July 2015;accepted 17 August 2015

Abstract

HowNutsAreTheDutch (Dutch: HoeGekIsNL) is a national crowdsourcingstudy designed to investigate multiple continuous mental health dimensionsin a sample from the general population (n=12,503). Its main objective is tocreate an empirically based representation of mental strengths and vulnerabil-ities, accounting for (i) dimensionality and heterogeneity, (ii) interactivity be-tween symptoms and strengths, and (iii) intra-individual variability. To do so,HowNutsAreTheDutch (HND) makes use of an internet platform that allowsparticipants to (a) compare themselves to other participants via cross-sectional questionnaires and (b) to monitor themselves three times a day for30 days with an intensive longitudinal diary study via their smartphone. Thesedata enable for personalized feedback to participants, a study of profiles of men-tal strengths and weaknesses, and zooming into the fine-grained level of dy-namic relationships between variables over time. Measuring both psychiatricsymptomatology and mental strengths and resources enables for an investiga-tion of their interactions, which may underlie the wide variety of observed men-tal states in the population. The present paper describes the applied methodsand technology, and presents the sample characteristics. Copyright © 2015 JohnWiley & Sons, Ltd.

Introduction

The debate about the optimal way to define and classifymental health problems has intensified over the past three

decades (Kapur et al., 2012; Kendler and First, 2010;Kendler et al., 2011; Wakefield, 1992). The Diagnosticand Statistical Manual of Mental Disorders (DSM) broughtstandardization to a field that used to be heavily

Copyright © 2015 John Wiley & Sons, Ltd. 1

International Journal of Methods in Psychiatric ResearchInt. J. Methods Psychiatr. Res. (2015)Published online in Wiley Online Library(wileyonlinelibrary.com) DOI: 10.1002/mpr.1495

Journal Code Article ID Dispatch: 05.09.15 CE:M P R 1 4 9 5 No. of Pages: 21 ME:

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

fragmented, and allowed for a shared clinical language.Nonetheless, DSM categories have also been criticized fortheir lack of empirical support and the absence of an un-derlying theoretical framework (Kapur et al., 2012;Kendler et al., 2011; Wardenaar and de Jonge, 2013;Whooley, 2014). Although the DSM system is mandatoryin psychiatric practice, scientists raised concerns about itsuse, and argued that the current classification system ham-pers our understanding of psychiatric disorders and canlead to scientific stagnation (Dehue, 2014; Insel, 2013;Kapur et al., 2012; Whooley, 2014). In the following wepropose three ways in which research can help to improvethe DSM system into an empirically based etiologicalsystem.

First, the descriptive consensus-based DSM categoriesimply a dichotomy of disordered versus healthy people,namely subjects fulfill a sufficient number of polytheticdiagnostic disorder classification criteria or they do not(Kendler and Parnas, 2015; Krueger and Markon,2006). Research suggests, however, that mental strengthsand symptoms are generally continuously distributed inthe population, without any evident “zone of rarity”,and that existing cutoffs are arbitrary and inconsistent(e.g. Gutiérrez et al., 2008; Kendell and Jablensky, 2003;Kendler, 2012; Ormel et al., 2013; Widiger and Sankis,2000). Mental health problems that might require carecan be located at the extreme ends of continuously distrib-uted mental state dimensions (Clark and Watson, 1991;Durbin and Hicks, 2014; Krueger, 1999; Mineka et al.,1998). Although the dimensional approach to psychopa-thology regains influence in psychiatry (Dumont, 2010;Kendler, 2012; Kendler and Parnas, 2015), research intoan empirical foundation remains imperative.

Second, the DSM defines mental health in terms of thepresence or absence of symptoms, which is a rather nar-row (and negative) view on the numerous psychologicalprocesses that define a person’s mental state (Duckworthet al., 2005; Horwitz and Wakefield, 2007; Huber et al.,2011; Kendler, 2012; Sheldon et al., 2011; Whooley,2014). Focusing solely on mental illness can, at best, re-duce mental illness, but does not suffice for mental health(Keyes, 2007; Westerhof and Keyes, 2010). Mental healthrequires the absence of disease and disability (negativestates), social and psychological well-being (positivestates), and abilities to adapt to one’s environment andself-manage (Q4 World Health Organization, 1948Q5 ; Huberet al., 2011; Keyes, 2007; Solomon, 2014). More funda-mentally, it is unlikely that we learn to understand themechanisms underlying psychiatric symptomatologywhen we fail to take into account the role of mentalstrengths, resources, and contextual factors in the eventual

(non)expression of mental health problems, such as hu-mour, self-acceptance, hope, social participation, and so-cial support (Duckworth et al., 2005; Keyes, 2007;Larson, 1999; Luthar et al., 2000; Seery et al., 2010;Seligman and Csikszentmihalyi, 2000; Sheldon et al.,2011; Westerhof and Keyes, 2010).

Third, while some DSM criteria already specify vari-ability between and within individuals, such as havingsymptoms “most of the day, nearly every day” versus“most days”, these specifications lack a solid empiricalfoundation, and do not allow for the identification ofcourse fluctuations (Horwitz and Wakefield, 2007;Hyman, 2007; Kapur et al., 2012; Kupfer et al., 2002;Wardenaar and de Jonge, 2013; Widiger and Samuel,2005), or for sequential expressions, such as a shift fromsadness to anxiety over time (Doré et al., 2015; Kessleret al., 2005; Stossel, 2013). While DSM categories are pre-sented as homogenous disease entities, combinations ofsymptoms prevail (co-morbidity), while the boundariesbetween diagnostic categories are necessarily fuzzy (Clarket al., 1995; Kendler, 2012, Krueger and Markon, 2006;Ormel et al., 2013; Widiger and Samuel, 2005; 2008). Ad-ditionally, treatment effects tend to be rather non-specific(Roest et al., 2015) Q6, and even genetic predispositions defyDSM syndrome boundaries in twin (Kendler, 1996), fam-ily (Dean et al., 2010), and genome-wide association stud-ies (Psychiatric Genomics Consortium, 2015).

Taken together, as long as the DSM is unable to takethe dynamic interactions between mental symptoms,strengths, and contextual factors into account, and lacksan empirically based identification of between and withinindividual variation, our research on the classification ofmental disorders remains unrealistic. This may help ex-plain why research into disease mechanisms is stagnantand difficult to replicate.

The current project

The research project HowNutsAreTheDutch (Dutch:HoeGekIsNL) is designed to allow for the investigationof mental health as a dimensional and dynamic phe-nomenon, characterized by both vulnerabilities andstrengths. HowNutsAreTheDutch (henceforth HND) isa widely broadcasted national crowdsourcing study in theNetherlands collecting self-report data on mental healthin a general population sample. The project uses an inter-net platform to recruit participants and invite them to as-sess themselves on multiple mental health dimensions.The combination of a cross-sectional and intensive longi-tudinal diary design allows for a data-driven empirical ap-proach that may help us generate new ideas about how key

HowNutsAreTheDutch Crowdsourcing Study Krieke et al.

Int. J. Methods Psychiatr. Res. (2015). DOI: 10.1002/mprCopyright © 2015 John Wiley & Sons, Ltd.2

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

variables interact in a multidimensional representation ofindividuals’ mental condition. The primary purpose ofHND is to explore the associations and dynamic interac-tions between mental strengths and vulnerabilities, bothbetween and within participants. This paper presents theobjectives, methods, and technology of both the cross-sectional and longitudinal part of the HND study, and pro-vides an overview of the sample’s characteristics in terms ofdemographics, psychometrics, and missing data.

Methods

Crowdsourcing procedure

For HND we applied a crowdsourcing procedure, an im-portant model for doing psychological research in whicha task is outsourced to a group of people, often online,in an open call (Brabham, 2008; Howe, 2006). Previoushealth research studies have used crowdsourcing as amethod to collect new information, and showed that itcan be a useful method to obtain information that other-wise tends to be overlooked by researchers (e.g.Bevelander et al., 2014), or to collect big datasets on mul-tiple outcomes that could not be realized without the par-ticipation of a large crowd (e.g. Revelle et al., 2010).

The crowdsourcing method enables for the develop-ment of “citizen science”, in which the general public vol-unteers to assist scientists in their research activities andcontribute with their intellectual effort, knowledge, ortools and resources to answer real-world questions (Hand,2010). We hope that the HND crowdsourcing approach(a) engages Dutch inhabitants with the debate about psy-chiatric classification (Dehue, 2014) and (b) results in asizeable sample of participants that allows for data-drivenanalyses of the relations between mental symptoms andstrengths both within and between individuals.

With HND we launched an open call to inhabitants ofthe Netherlands to join our research, and invited them tovisit the Dutch website www.HoeGekIs.nl (also www.HowNutsAreTheDutch.com), which has been online since19 December 2013. The open call was announced on localand national radio broadcasts, television, during local po-dium discussions, in newspapers, and in magazines. Thenews about the HND research project was picked up andfurther disseminated via online blogs, twitter, and othersocial media.

To join the project, participants had to register and cre-ate an account. Participants filled out their email addressand a password on www.HoeGekIs.nl and received anemail with a hyperlink to confirm their account. Beforestarting the actual research, participants were asked toprovide information about their gender, birth year and

month, their postal code area, and country of residence(the Netherlands/Belgium/Other). Although HND istargeted on Dutch citizens, we added a question aboutcountry of residence after news about HND was pickedup by the Belgium media and Dutch speaking participantsfrom Belgium started to join the website.

Two studies

The HND website comprises a cross-sectional study and alongitudinal study with intensive repeated assessments indaily life, namely a diary study with ecological momentaryassessments (EMA, see Bolger et al., 2003). Participantscould complete either one of these studies, or both.

Cross-sectional study

Procedure and questionnaires. The cross-sectional studywas launched together with the website on 19 December2013. In this study participants were invited to completevarious questionnaire “modules”, that is, a questionnaireor a set of combined questionnaires covering a specificdomain (see Table T11). The order in which the modulescould be completed was partly fixed. All questionnairemodules were visible from the start, but initially only partof them was activated. The first mandatory module wasthe “Start” module, assessing participants’ socio-demographic profile. Subsequently, participants got accessto three key modules; (i) (an extensive assessment ofone’s) Living situation; (ii) Affect/mood; (iii) Well-being,which could be completed in any order. After theAffect/mood and Well-being modules had been com-pleted all other modules became available and could becompleted in any order. These latter modules were notyet available at the launch of the HND website, but wereadded to the website one at a time over the year followingthe launch. Every three months participants were in-formed about these additions via an email newsletter.All implemented are outlined in Table 1. Eligible partici-pants were aged 18 or older and consented to their databeing used for research.

Feedback generation. After completing a questionnairemodule participants received instant and automated feed-back on the website. This feedback consisted of bars show-ing their scores relative to the maximum possible score ona given questionnaire and bars or spider plots reflectingtheir personal scores relative to the average scores of theHND participants. Examples are presented in FiguresS1–S3 in the Supplementary Material.

Krieke et al. HowNutsAreTheDutch Crowdsourcing Study

Int. J. Methods Psychiatr. Res. (2015). DOI: 10.1002/mprCopyright © 2015 John Wiley & Sons, Ltd. 3

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Tab

le1.

Mod

ules

,instrumen

ts,an

dco

nten

tsof

thecros

s-se

ctiona

lstudy

Mod

ule

Instrumen

tDes

criptio

nIte

ms

Res

pons

erang

eAutho

rs

Start

Gen

der,birthye

ar,po

stal

code

area

,relatio

nshipstatus

,nu

mbe

rof

children,

educ

ationleve

l,an

doc

cupa

tiona

lsitu

ation.

8

Living

situation

(soc

io-dem

ograph

y)Cou

ntry

oforigin

(also

forbo

thpa

rents),family

inco

me,

living

arrang

emen

t,family

mem

bers,pe

ts,religion,

time

spen

ton

television

/internet/spo

rts,

leng

than

dweigh

t,an

dha

ndpreferen

ce.

17

Affe

ct/m

ood

PANAS

PANAS

Flemishve

rsion,

asse

sses

10po

sitivean

d10

nega

tive

emotions

over

thepa

stwee

k.20

1–5

Pee

ters

etal.,19

96;

Rae

set

al.,20

09QIDS

The

QIDS

asse

sses

andclas

sifies

DSM

major

depres

sion

with

nine

domains

,sa

dmoo

d,co

ncen

tration,

self-criticism

,su

icidal

idea

tion,

interest,

energy

/fatig

ue,

slee

p,ch

ange

inap

petite/

weigh

t,ps

ycho

motor,ov

erthepa

stwee

k.

160–

3Rus

het

al.,20

03,20

06

DASS

The

DASSmea

suresmoo

dov

erthepa

stwee

kan

disse

nsitive

tosu

bthres

hold

symptom

s.42

0–3

Lovibo

ndan

dLo

vibo

nd,

1995

a,19

95b

Q7

DeBeu

rset

al.,20

01Well-b

eing

MANSA

The

MANSAmea

suresqu

ality

oflifeon

multip

ledo

mains

,with

12ite

msforasu

mscore,

andfour

additio

naly

es/noite

ms.

161–

7Prie

beet

al.,19

99Q8

Prie

beet

al.,20

10Hap

pine

ssinde

xThe

Hap

pine

ssinde

xas

sesses

thede

gree

towhich

onejudg

esthe

quality

ofon

e’slife

ina

sing

leite

m:Do

you

feel

happ

yin

gene

ral?

10–

10Fordy

ce,19

88;

Vee

nhov

en,19

94;

Abd

el-Kha

lek,

2006

SPF-IL

The

SPF-ILmea

suresthefive

universa

lprim

arygo

alsaffection,

beha

viou

ralc

onfirm

ation,

status

,co

mfort,an

dstim

ulation,

which

,ac

cordingto

SPFtheo

ry,un

derla

yindividu

alwell-b

eing

.

150–

3Niebo

eret

al.,20

05

Ryff

The

Ryff

scales

ofps

ycho

logica

lwell-b

eing

mea

sure

self-

acce

ptan

ce,

positive

relatio

nswith

othe

rs,

autono

my,

environm

entalm

astery,p

urpo

sein

life,

andpe

rson

algrow

th.

391–

6Van

Dierend

onck,

2004

Perso

nality

NEO-FFI-3

The

NEO-FFI-3

person

ality

inve

ntory

asse

sses

the

Big

Five

person

ality

domains

Neu

roticism,

Extrave

rsion,

Ope

nnes

sto

expe

rienc

e,Agree

ablene

ss,

and

Con

scientious

ness

with

60ite

ms.

Thisinstrumen

twas

extend

edwith

36ne

uroticism

items

from

the

NEO-PI-3

tode

rive

allface

ttraits

forthe

neuroticism

domain.

961–

5DeFruyt

andHoe

kstra,

2014

DarkTria

dThe

Dark

Tria

das

sesses

tend

encies

towards

Narcissism,

Mac

hiav

ellianism

,an

dPsych

opathy

.12

1–9

Q9

Pau

lhus

etal.,20

02Q10

,Klim

stra

etal.,20

14

(Continu

es)



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Tab

le1.

(Con

tinue

d)

Mod

ule

Instrumen

tDes

criptio

nIte

ms

Res

pons

erang

eAutho

rs

Doing

,Fee

ling,

Think

ing

Doing

,Fee

ling,

Think

ing

asse

sses

tend

encies

towards

the

beha

viou

ralstyles

Doing

(practically-orie

nted

),Fee

ling(relation-

oriented

),an

dThink

ing(con

tent-lo

gic-oriented

).

91–

3DeKlerk

et?a

l.,20

03

Som

atic

Sym

ptom

sPHQ-15

The

PHQ-15is

ascreen

ingan

ddiag

nostic

tool

toas

sess

somatic

symptom

sas

sociated

with

men

talh

ealth

diso

rders.

151–

3Kroen

keet?a

l.,20

02

RoS

iA

compo

site

offive

itemsfrom

theSCLso

matic

scalean

deigh

tite

ms

deriv

edfrom

anex

pert

committee

supe

rvised

byJ.

Ros

malen

.

130–

2Arrinde

llan

dEtte

ma,

1986

(SCL)

WhiteleyInde

xThe

Whiteleyinde

xmea

surestend

encies

towards

hypo

chon

dria.

140–

1Spe

cken

set?a

l.,19

96Psych

otic

Exp

erienc

esCAPE

The

CAPE

mea

sures

positive

and

nega

tive

symptom

sof

psycho

siswith

42ite

ms.

We

only

selected

the

34ite

msab

out

psycho

ticsymptom

san

dskippe

dthede

pres

sion

items.

340–

3Kon

ings

et?a

l.,20

06

Hum

our

HSQ

The

HSQ

asse

sses

affiliative,

self-en

hanc

ing,

aggres

sive

,an

dse

lf-de

featinghu

mou

rstyles

.32

1–7

Martin

et?a

l.,20

03

Optim

ism

LOT-R

The

LOT-R

asse

sses

disp

osition

alop

timism

(and

pessim

ism).

100–

4Glaes

mer

et?a

l.,20

12Empa

thy

EQ

The

EQ

ques

tionn

aire

mea

sures

both

affective

empa

thy

via

shared

emotions

andco

gnitive

empa

thyor

theo

ryof

mind.

400–

2Baron

-Coh

enan

dWhe

elwrig

ht,20

04Childho

odad

versity

CTQ-SF

The

CTQ-SFis

aretros

pectivese

lf-repo

rtqu

estio

nnaire

design

edto

asse

ssfive

dimen

sion

sof

childho

odmaltrea

tmen

t:ph

ysical

abus

e,em

otiona

lab

use,

sexu

alab

use,

physical

neglec

t,an

dem

otiona

lneg

lect.

281–

5Tho

mbs

et?a

l.,20

09

Intellige

nce

ICAR

Wese

lected

11ite

msmea

surin

gindu

ctivereas

oningan

d24

items

mea

surin

gthree-dimen

sion

al(3D)rotatio

nab

ilitie

sfrom

theICAR

cogn

itive

item

pool.

351–

61–

8Con

donan

dRev

elle,

2014

Eva

luation

Eva

luation

The

evalua

tionqu

estio

nnaire

was

design

edto

evalua

tetheHND

web

site,thecros

s-se

ctiona

lque

stionn

aire

mod

ules

,thefeed

back

upon

completed

mod

ules

,an

dtheim

pact

ofpa

rticipation.

71–

10

Note:

CAPE=Com

mun

ityAsses

smen

tofP

sych

icExp

erienc

es;C

TQ-SF=Childho

odTraum

aQue

stionn

aire

–Sho

rtForm;D

ASS=Dep

ressionAnx

iety

Stres

sSca

le;

DSM

=Diagn

ostic

andStatistical

Man

ualo

fMen

talD

isorde

rs;E

Q=Empa

thyQuo

tient;H

ND=How

NutsA

reThe

Dutch

;HSQ=Hum

ourS

tylesQue

stionn

aire;ICAR=In-

ternationa

lCog

nitiveAbilityRes

ourceBas

e;LO

T-R

=Life

Orie

ntationTes

t–Rev

ised

;MANSA=The

Man

ches

terS

hortAsses

smen

tofq

ualityof

life;

NEO-FFI-3=The

Neu

roticism–Extrave

rsion–

Ope

nnes

sFive-Fac

torInve

ntoryup

datedan

drevise

dve

rsion;

PANAS=Pos

itive

And

Neg

ativeAffe

ctSch

edule;

PHQ-15=The

Patient

Hea

lthQue

stionn

aire

15ite

mve

rsion;

SCL=Sym

ptom

Che

cklist;SPF=Soc

ialProdu

ctionFun

ctions

;SPF-IL=The

SPF

Instrumen

tfortheLe

velof

well-b

eing

;QIDS=The

Quick

Inve

ntoryof

Dep

ress

iveSym

ptom

atolog

y;RoS

i=Ros

malen

Som

atic

itemsscale.



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Diary study

Procedure. The diary study was launched on 22May 2014. Inthis study participants were intensively monitored in theirnatural environments by means of electronic diaries, threetimes a day for 30 days, resulting in a maximum of 90 assess-ments per individual. Assessments were prompted at equi-distant time points with a six-hour interval in between,with the exact time points depending on participants’sleep–wake schedule. Participants received a text messageon their mobile phone with a link to a questionnaire. Theywere asked to fill out the questionnaire immediately afterthe alert, or, if impossible, within one hour, after whichthe questionnaire could no longer be accessed. Participantswere informed about the requirements and procedure ofthe diary study by means of an information page and a shortanimatedmovie clip on the HNDwebsite. Additionally, theycould download an information booklet with details on thestudy procedure, diary items, and their reward, being a dig-ital report containing personalized feedback.

The study requirements were: age 18 or above, having amobile phone with internet connection, not engaged inshift work, not anticipating a major disruption of dailyroutines (e.g. a planned trip abroad, an anticipated surgi-cal operation), being aware that participation would beuseless in case too many assessments would be missed,and approving of one’s anonymous data being used forscientific research. Participants had to check a box for eachof these requirements before they could proceed. Subse-quently, they had to complete a baseline assessmentconsisting of the items of the positive and negative affectschedule (PANAS; Peeters et al., 1996; Raes et al., 2009),the Quick Inventory of Depressive Symptoms (QIDS;Rush et al., 2003, 2006), and two extra items retrievedfrom the Inventory of Depressive Symptomatology (IDS;Rush et al., 1996) to assess anxiety/panic symptoms. Fi-nally, participants configured their personal settings forthe daily assessments, i.e. their telephone number, theirpreferred start date (within five days after completion ofthe baseline assessment), and the sampling schedule. Par-ticipants were instructed to pick a sampling schedule thatfitted their daily rhythm, with the evening measurementpreferably half an hour before their regular bedtime.

After completion of their diary study, participants weresent a short evaluation questionnaire. Participants whoquit the study prematurely were asked for their reasonsto quit by means of a short questionnaire.

Questionnaire items. The diary questionnaire contained 43items. It combined items from existing and validated ques-tionnaires and a few newly created items. We assessed

subjective well-being, sleep, mood, anxiety, depression,physical activity, physical discomfort, self-esteem, worry-ing, loneliness, mindfulness, context (location, social com-pany, activities), and the appraisal of this context, stressfulevents, time pressure, the feeling one makes a difference,laughing, and being outdoors. All questionnaire itemsand literature references are presented in Table T22. Addi-tionally, participants could define a personal item that theyfelt relevant to their situation. This item could be chosenfrom a list of options or could be self-created during theconfiguration of personal settings. Examples of personalitems were: “I worry a lot” or “I smoked a lot since the lastassessment”. All items except categorical ones were ratedon a visual analogue scale (VAS) ranging from 0 to 100,with appropriate labels at the extremes and middle of thescale, and the middle as default positive. To answer a ques-tion the slider had to be moved.

Feedback generation. After completion of the study partic-ipants received instant and automated feedback on thewebsite. Participants who completed at least 65% (t≥ 59)of the assessments received basic personalized feedbackconsisting of graphs and explanatory text (see FiguresS4–S7 in the Supplementary Material). Participants whocompleted at least 75% (t ≥ 68) of the assessments also re-ceived personal network models showing the interrela-tionships between variables assessed in the diary study,see Figure F11 for an example network. The first networkmodel depicts the concurrent relationships between vari-ables. This model shows how a participant’s affect, cogni-tions, and behaviours are related to each other at the samemoment in time. The second network model shows thedynamic, directed relationships between variables, indicat-ing how they affected each other over time.

The network models were estimated for each partici-pant separately using vector autoregressive (VAR) model-ling (Brandt and Williams, 2007; Lütkepohl, 2006). Toautomate this procedure, we used Autovar, an open sourceR package that reads raw data and automatically fits andevaluates VAR models (Blaauw et al., 2014; Emerenciaet al., 2015, Van der Krieke et al., submitted for publica-tion a, submitted for publication b Q11) Q12. The number of vari-ables included in the network models was limited to six,for the sake of comprehension and in order to focus onthe most informative results. These six variables were se-lected based on highest moment-to-moment variability(as indicated by the mean squared successive difference,MSSD) and lowest skewness. Both variables usually per-ceived as “positive” (e.g. laughing, relaxation) and vari-ables perceived as “negative” (e.g. rumination, feelingdown) were selected, as well as the participant’s personal



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Tab

le2.

Itemsof

thediarystud

y

Que

stion

Dutch

Trans

latio

nRes

pons

erang

eRan

geDes

criptio

n

1.Hoe

gaat

het

opdit

mom

entmet

u?How

areyo

udo

ingrig

htno

w?

“Veryba

d”to

“Verygo

od”

0–10

0Mom

ent-to-m

omen

tqu

ality

oflife

2.Hee

ftusind

she

tvo

rige

mee

tmom

entg

eslape

n?Did

youslee

psinc

ethe

last

mea

suremen

t?(1)No

1–2

Sleep

(che

ckbo

xes).If

yes,

goto

3–4.

(2)Yes

3.Hee

ftugo

edge

slap

en?

Did

youslee

pwell?

“Not

atall”to

“Verywell”

0–10

0Qua

lityof

slee

p4.

Hee

ftu

lang

geno

egge

slap

en?

Did

you

slee

plong

enou

gh?

“Too

short”to

“Too

long

”0–

100

Durationof

slee

p

5.Ik

voel

meon

tspa

nnen

Ifeel

relaxe

d“N

otat

all”to

“Verymuc

h”0–

100

Pos

itive

affect

Dea

ctivation

6.Ik

voel

meso

mbe

rIfeel

gloo

my

“Not

atall”to

“Verymuc

h”0–

100

Neg

ativeaffect

Dea

ctivation

7.Ik

voel

meen

ergiek

Ifeel

energe

tic“N

otat

all”to

“Verymuc

h”0–

100

Pos

itive

affect

Activation

8.Ik

voel

mean

gstig

Ifeel

anxiou

s“N

otat

all”to

“Verymuc

h”0–

100

Neg

ativeaffect

Activation

9.Ik

voel

meen

thou

sias

tIfeel

enthus

iastic

“Not

atall”to

“Verymuc

h”0–

100

Pos

itive

affect

Activation

10.

Ikvo

elmeon

rustig

Ifeel

nervou

s“N

otat

all”to

“Verymuc

h”0–

100

Neg

ativeaffect

Activation

11.

Ikvo

elmetevred

enIfeel

conten

t“N

otat

all”to

“Verymuc

h”0–

100

Pos

itive

affect

Dea

ctivation

12.

Ikvo

elmeprikke

lbaa

rIfeel

irrita

ble

“Not

atall”to

“Verymuc

h”0–

100

Neg

ativeaffect

Activation

13.

Ikvo

elmeka

lmIfeel

calm

“Not

atall”to

“Verymuc

h”0–

100

Pos

itive

affect

Dea

ctivation

14.

Ikvo

elmelusteloo

sIfeel

dull

“Not

atall”to

“Verymuc

h”0–

100

Neg

ativeaffect

Dea

ctivation

15.

Ikvo

elmeop

gewek

tIfeel

chee

rful

“Not

atall”to

“Verymuc

h”0–

100

Pos

itive

affect

Activation

16.

Ikvo

elmemoe

Ifeel

tired

“Not

atall”to

“Verymuc

h”0–

100

Neg

ativeaffect

Dea

ctivation

17.

Ikerva

arlicha

melijk

onge

mak

Iex

perie

nceph

ysical

discom

fort

“Not

atall”to

“Verymuc

h”0–

100

Som

atic

symptom

s

18.

Ikvo

elmege

waa

rdee

rdIfeel

valued

“ Not

atall”to

“Verymuc

h”0–

100

Self-es

teem

19.

Ikvo

elmeee

nzaa

mIfeel

lone

ly“N

otat

all”to

“Verymuc

h”0–

100

Lone

lines

s20

.Ik

hebhe

tge

voel

teko

rtte

schieten

Ifeel

Ifallsh

ort

“Not

atall”to

“Verymuc

h”0–

100

Worthlessne

ss

21.

Ikvo

elmeze

lfverze

kerd

Ifeel

confi

dent

“Not

atall”to

“Verymuc

h”0–

100

Self-es

teem

22.

Ikpiek

erve

elIworry

alot

“Not

atall”to

“Verymuc

h”0–

100

Worrying

23.

Ikbe

nsn

elafge

leid

Iam

easily

distracted

“Not

atall”to

“Verymuc

h”0–

100

Con

centratio

n/mindfulne

ss24

.Ik

vind

mijn

leve

nde

moe

itewaa

rdIfeel

my

life

isworth

living

“Not

atall”to

“Verymuc

h”0–

100

Worthlessne

ss/suicida

lidea

tion

25.

Ikbe

nva

nslag

Iam

unba

lanc

ed“N

otat

all”to

“Verymuc

h”0–

100

Stres

sreac

tivity

26.

Ikleef

inhe

thieren

nuIa

minthehe

rean

dno

w“N

otat

all”to

“Verymuc

h ”0–

100

Mindfulne

ss27

.Mijn

eetlu

stis

Myap

petiteis

“Muc

hsm

aller

than

usua

l”to

“Muc

hlarger

than

usua

l”0–

100

App

etite

(Continu

es)



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Tab

le2.

(Con

tinue

d)

Que

stion

Dutch

Trans

latio

nRes

pons

erang

eRan

geDes

criptio

n

28.

Hoe

druk

hebik

het?

How

busy

amI?

(1)Muc

htoobu

sy1–

5Tim

epres

sure

(2)Pleas

antly

busy

(3)Neu

tral

(4)Pleas

antly

quiet

(5)Muc

htooqu

iet

29.

Waa

rwas

ikhe

tafge

lope

nda

gdee

lde

mee

stetijd?

Whe

reha

veI

spen

tmos

tof

my

time

sinc

ethelast

mea

suremen

t?

(1)Ath

ome

1–9

Loca

tion

(che

ckbo

xes;

only

one

loca

tion

could

bech

ecke

d)(2)Atw

ork/scho

ol(3)With

family/friend

s(4)Ontheway

(5)Vac

ationho

me/ho

tel/

camping

(6)Hos

pital/h

ealth

facility

(7)Res

tauran

t/bea

nery

(8)In

nature

(9)Som

ewhe

reelse

30.

Wat

deed

ikhe

tafge

lope

nda

gdee

lde

mee

stetijd?

How

didIs

pend

mos

tof

my

time

sinc

ethe

last

mea

suremen

t?

(1)Res

ting/slee

ping

1–13

Activities

(che

ckbo

xes;

only

one

activity

could

bech

ecke

d)(2)Hou

seho

ld/groce

ries

(3)Working

/study

ing/

voluntee

ring

(4)Exe

rcising/walking

/cycling

(5)Yog

a/med

itatio

n/sa

una

visite

tc.

(6)Rea

ding

(7)Hob

by(e.g.g

arde

ning

,mak

ingmus

ic)

(8)Trip

(e.g.leisu

repa

rk,

conc

ert)

(9)Watch

ingtv

(10)

Web

surfing

/ga

ming/

social

med

ia(11)

Con

versing

(12)

Som

ething

intim

ate(e.g.

Cud

dling,

sex)

(13)

Som

ething

else

/allkind

sof

things

(Continu

es)



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Tab

le2.

(Con

tinue

d)

Que

stion

Dutch

Trans

latio

nRes

pons

erang

eRan

geDes

criptio

n

31.

Ikervo

erde

zeac

tivite

itov

erweg

endals

Iex

perie

nced

this

activity

mainlyas

“Veryun

plea

sant”,via“N

eutral”,

to“Veryplea

sant”

0–10

0App

raisal

ofac

tivity

32.

Iser

heta

fgelop

enda

gdee

liets

bijzon

ders

gebe

urd?

Did

something

spec

ial

happ

ensinc

ethelast

mea

suremen

t?

(1)No,

nothing

1–4

Spe

cial

even

t(che

ckbo

xes;

only

onebo

xco

uldbe

chec

ked).Ifno

thing,

jumpto

34,o

therwise33

.

(2)Yes

,som

ething

positive

(3)Yes

,som

ething

neutral

(4)Yes

,som

ething

nega

tive

33.

Waa

rha

dditm

eete

mak

en?

Thiswas

relatedto

(1)Myself

1–7

Con

text

ofsp

eciale

vent

(che

ckbo

xes;

only

one

box

couldbe

chec

ked).

(2)Hom

esituation/

clos

efamily/

sign

ifica

ntothe

rs(3)Frie

nds/

othe

rfamily/

acqu

aintan

ces

(4)Work/

scho

ol(5)Soc

iety/n

ews

(6)Pub

licsp

ace/

strang

ers

(7)Other

34.

Ikwas

heta

fgelop

enda

gdee

lgrotend

eels

Mos

tof

the

time

sinc

ethe

last

mea

suremen

tI

was

(1)Alone

1–2

Soc

ialc

ompa

ny(che

ckbo

xes;

only

onebo

xco

uld

bech

ecke

d).Ifa

lone

,go

to35

,followed

by38

.Ifin

compa

ny,g

oto

36–37

.

(2)In

compa

ny

35.

Ikwas

lieve

rin

geze

lsch

apge

wee

stIwou

ldrather

have

been

with

othe

rs“N

o,preferab

lyno

t”to

“Yes

,ce

rtainly”

0–10

0App

raisal

ofbe

ingalon

e

36.

Ikzo

ulieve

ralleen

zijn

gewee

stIwou

ldrather

have

been

alon

e“N

o,preferab

lyno

t”to

“Yes

,ce

rtainly”

0–10

0App

raisal

ofso

cial

compa

ny

37.

Ikvo

ndditge

zelsch

apov

erweg

end

Ifoun

dmyco

mpa

nypred

ominan

tly“Veryun

plea

sant”,via“N

eutral”,

to“Veryplea

sant”

0–10

0App

raisal

ofso

cial

compa

ny

38.

Ikhe

bin

hetafge

lope

nda

gdee

lgelac

hen

Since

thelast

mea

suremen

tIha

da

laug

h

“Not

atall”to

“Verymuc

h”0–

100

Laug

hing

39.

Ikhe

bin

hetafge

lope

nda

gdee

lietsvo

orieman

dku

nnen

beteke

nen

Since

thelast

mea

suremen

tIwas

able

tomak

ea

diffe

renc

e

“Not

atall”to

“Verymuc

h”0–

100

Mak

ingadiffe

renc

e (Continu

es)



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Tab

le2.

(Con

tinue

d)

Que

stion

Dutch

Trans

latio

nRes

pons

erang

eRan

geDes

criptio

n

40.

Ikbe

nhe

tafgelop

enda

gdee

lbuitenge

wee

stSince

thelast

mea

suremen

tIha

vebe

enou

tdoo

rs

“Not

atall”to

“Verymuc

h”0–

100

Being

outside

41.

Hoe

licha

melijk

actie

fwas

ikhe

tafgelop

enda

gdee

l?

Since

thelast

mea

suremen

tIwas

physically

active

“Not

atall”to

“Verymuc

h”0–

100

Phy

sica

lactivity

42.

Ikde

edding

en“opde

automatisch

epiloot”,

zond

erme

bewus

tte

zijn

vanwat

ikde

ed

Idid

jobs

ortasks

automatically

with

out

beingaw

areof

wha

tI

was

doing

“Not

atall”to

“Verymuc

h”0–

100

Mindfulne

ss

43.

Mijn

eige

nbe

lang

rijke

factor

Mype

rson

alim

portan

tfactor

“Not

atall”to

“Verymuc

h”0–

100

Perso

nalitem

Note:

Mom

entary

affect

andan

xietywereas

sessed

with

12ite

msfrom

thecircum

plex

mod

elof

affect,w

hich

describ

esaffect

interm

sof

twodimen

sion

s:va

lenc

ean

dac

tivation(item

s5–

16;B

arrettan

dRus

sell,

1998

;Yik

et?a

l.,19

99).Sleep

was

asse

ssed

with

oneite

mfrom

thePittsb

urgh

Sleep

Diary

(Mon

ket?a

l.,19

94)an

dtwo

self-co

nstruc

tedite

ms(item

s2–

4).Wead

dedfour

extraite

msreflec

tingtheDSM-V

crite

riaforde

pres

sion

notalread

yco

veredby

theaffect

andslee

pite

ms,

which

wede

rived

from

thePatient

Hea

lthQue

stionn

aire-9

(Wittka

mpf

et?a

l.,20

09)an

dtheInve

ntoryof

Dep

ressiveSym

ptom

atolog

y(R

ushet?a

l.,19

96),an

dad

aptedfor

daily

use(item

s20

,23

,24

,27

).Sub

jectivewell-b

eing

was

asse

ssed

with

asing

leite

mreflec

tingmom

ent-to-m

omen

tqu

ality

oflife(item

1:Barge

-Sch

aapv

eldet?

al.,19

99).Self-es

teem

,worrying,

lone

lines

s,ph

ysical

activity,p

hysica

ldisco

mfort,c

ontext

(loca

tion,

social

compa

ny,a

ctivities

),an

dtheap

praisa

lofthisco

ntex

twere

asse

ssed

with

itemsad

aptedfrom

prev

ious

ecolog

ical

mom

entary

asse

ssmen

tstudies

(item

s17

–19

,21,

22,2

9–31

,34–

37,4

2:Barge

-Sch

aapv

eldet?a

l.,19

99;C

ollip

et?a

l .,20

11;C

siksze

ntmihalyian

dLa

rson

,198

7;Doa

nean

dAda

m,2

010;

Myin-Germey

set?a

l.,20

09;S

avin-W

illiamsan

dDem

o,19

83;W

iche

rset?a

l.,20

11).Mind-

fulnes

swas

asse

ssed

with

twoite

msad

aptedfrom

theFiveFac

etMindfulne

ssQue

stionn

aire

(Bae

ret?a

l.,20

06,2

008)

andon

ese

lf-co

nstruc

tedite

m(item

s23

,26,

42).Wead

dedse

vense

lf-co

nstruc

tedite

msto

asse

ssstressfule

vents,

timepres

sure,m

akingadiffe

renc

e,laug

hing

,and

beingou

tside(item

s25

,28,

32,3

3,38

,39,

40).Participan

tsco

uldalso

define

ape

rson

alite

mthat

they

feltreleva

ntto

theirsituation(item

43;s

eeMetho

dsse

ction).



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

item, unless variability was too low (MSSD< 50) or skew-ness too high (z-skewness> 4). Only VARmodels with onetime lag were estimated, to prevent over-parameterization.Dummy variables for the part of the day (morning/after-noon/evening) were included in all models. Trend vari-ables denoting the assessment point (and the square of it)were included if necessary. Dummy variables for the daysof the week were included and variables were log trans-formed if this improved model fit. Autovar only consideredmodels that met the assumptions of stability, normality,homoscedasticity, and independence, and selected the best

model based on the Akaike Information Criterion. Detailson the procedure for automated network models are avail-able from the authors upon request.

Technological infrastructure of HowNutsAreTheDutch(HND)

An architectural overview of the HND web application,the external services involved, and the security of the con-nections is presented in Figure F22. The figure shows thatHND consists of three main components. The first

Figure 1. Example of a personal network model showing concurrent (left) and dynamic relationships (right) between diaryitems. Note: Red dots represent variables that tend to be perceived as negative (e.g. loneliness, sadness). Green dots rep-resent variables that tend to be perceived as positive (e.g. relaxation, mindfulness, feeling cheerful). The blue dot representsthe personal variable that participants could choose to add to the diary assessment. This variable could either be “negative”or “positive” and could be different for each participant. The size of the dot indicates its relative importance (i.e. the bigger adot, the more connections the variable has with other variables). The lines represent the connections between variables; thethickness indicates the strength of the relationship. A plus refers to a positive relationship; a minus refers to a negative rela-tionship. The arrowheads (only in the dynamic networks) indicate the direction of the relationships.

Colour

online,B&W

inprint

Figure 2. Architectural overview of the HowNutsAreTheDutch (HND) web application. Note: The cylindrical shapes repre-sent databases. The rectangular shapes depict (web) services. The hatched rectangle is the actual HND web application thatserves the website.

Colour

online,B&W

inprint



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

component is the HND application itself. This applicationis implemented in the Ruby on Rails framework (http://rubyonrails.com). The data is stored in two databases.The personal socio-demographic data (i.e. gender, birthmonth, birth year, and postal code area) are stored in aPostgreSQL database (http://postgresql.org). The otherquestionnaire data are stored in a MongoDB database(http://mongodb.com). The use of two separate databaseslowers the impact of a potential database security breach.Both databases would have to be compromised to tracequestionnaire data back to potentially identifying personalinformation. All web traffic to and from HND is encryptedusing a 128bit TLS/2048bit RSA Secure Socket Layer (SSL)certificate. This SSL certificate ensures that any data ex-changed between participants and the HND website appli-cation is unreadable for anyone else. The HND componentin Figure 2 serves the web application to the clients, con-tains the registration and login services, provides thecross-sectional questionnaires and provides the feedback.This component also interfaces to the other two compo-nents; the RoQua service and the Autovar service.

RoQua is a Software as a Service (SAAS) product that of-fers scheduling of momentary assessments and collection ofquestionnaire data (http://roqua.nl). HND communicateswith RoQua in order to schedule the measurement for eachparticipant. When a participant is scheduled to fill out aquestionnaire, RoQua notifies HND. HND then notifiesthe participants and redirects them to the RoQua service,which conducts the questionnaire and stores the result.

The Autovar service is a statistical service used to ana-lyse the diary questionnaire data (http://autovar.nl). In or-der to expose Autovar’s functions to the HND platform,Autovar uses a service known as OpenCPU (Ooms,2014). OpenCPU offers a RESTful (REpresentational StateTransfer) interface that allows R functions to be used byother systems and programming languages.

Ethics

The HND study protocol was assessed by the MedicalEthical Committee of the University Medical Centre Gro-ningen. The committee judged the protocol to be exemptedfrom review by the Medical Research Involving HumanSubjects Act (in Dutch: WMO) because it concerned anon-randomized open study targeted at anonymous volun-teers in the general public (registration numberM13.147422and M14.160855).

Sample comparisons

One has to realize that particular individuals are less likelyto participate in crowdsourcing studies. To explore

selection effects we compared the characteristics of theHND sample with (a) the general Dutch populationaccording to the Dutch Governmental Agency for Statis-tics (CBS) and (b) two representative samples from thenon-institutionalized Dutch population, that is, theNetherlands Mental Health Survey ( Q13NEMESIS-2, 2007–2009, n=6646, see de Graaf et al., 2010) and the Lifelinespopulation study from the north of the Netherlands (n=167,729, 2006–2013, see Scholtens et al., 2014).

Results

Cross-sectional study

Sample characteristics

Up to 13 December 2014 12,734 participants participatedin the cross-sectional study. We excluded 231 participantsfrom our analyses because they were younger than 18 (n=228) or provided unrealistic entries (e.g. birth year< 1900;n=3), resulting in a final sample of 12,503. The mean ageof the participants was 45 years [standard deviation(SD)= 15] and 65% were women. A detailed overview ofthe participants is given in Table T33. Participants were sam-pled from all regions of the Netherlands, as illustrated bythe heat map of the Netherlands in Figure F33.

The coverage concurs very well with population densityscores. However, compared to the Dutch population, theHND participants were more often women (65.2% versus50.5% in the population, NEMESIS= 55.2%, Lifelines =57.9%), on average slightly older (45 versus 39 years;NEMESIS= 44, Lifelines = 42), more often with a roman-tic partner (74% versus 58%), with whom they cohabitedmore often (61% versus 47%; NEMESIS= 68%). Most sa-liently, HND sampled few people from lower educatedstrata (2% versus 22%; NEMESIS= 5%), as well as me-dium education levels (16 years and more, 22% versus43%; NEMESIS= 60%); HND participants tend to behigher educated (>20 years, 76% versus 35%; NEME-SIS = 35%). Elderly were relatively well sampled in HND,as most participants were older than 45 (55%), and 9%of them were older than 65 (versus 19% of the population;Lifelines = 7.6%, NEMESIS did not include participantsolder than 64). To enable comparisons between thisHND sample and population samples, and to perform ro-bustness checks for our models, we calculated a selectionbias weight factor, based on population proportions de-rived from the CBS. Post-stratification weights were de-rived for 36 strata based on age (six categories), gender,and education level (three categories), see SupplementaryMaterial Table S1. Our weighted results are presented inTable 3.



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

Key results

Exactly 62,068 questionnaire modules were completed, onaverage five modules per participant (12,402 participantsfilled out≥ 1 of the 14 available questionnaires). The keyquestionnaire modules focusing on Affect/mood andWell-being were completed approximately 8000 and10,000 times, respectively (see Supplementary MaterialTable S2), while 5144 participants filled out all three keymodules (including life situation). All modules were com-pleted by 627 participants (except intelligence, which wasonly available in the last three weeks).

As presented in Table 3, on average participants reportedmore positive than negative affect (PANAS, mean=34 ver-sus 20, respectively, both SD=7 and range 10–50). How-ever, both the positive affect and the negative affect scalesshowed high variance. A visual representation of the rela-tionship between positive and negative affect in FigureF4 4 in-dicates that participants with a similar score on negative

affect varied substantially in their positive affect scores, andthe other way around, despite their strong correlation(r=�0.52, p< 0.001). Albeit women reported slightly morenegative (t=8.25, p< 0.001, d=0.19) and less positive affectthan men (t=4.54, p< 0.001, d=0.11) gender differenceswere minimal (see Supplementary Material Table S3).

Regarding mood, the Depression–Anxiety–Stress Scale(DASS) showed that symptoms of psychological stress weremost common, followed by symptoms of depression andanxiety. The large standard deviations and broad ranges ofthese scales indicate that substantial heterogeneity existsamong participants. Based onDASS cutoff values (Lovibondand Lovibond, 1995a) Q14, 14.8% of the participants in our sam-ple reported mild, 9.1% moderate, 3.9% severe, and 1.3%extremely severe depression symptom levels (in the pastweek). With respect to anxiety, 15.1% scored mild, 11.0%moderate, 4.6% severe, and 1.9% extremely severe. With re-spect to psychological stress, 17.9% scored mild, 9.5%mod-erate, 2.7% severe, and 0.4% extremely severe.

Table 3. Baseline characteristics of the cross-sectional HowNutsAreTheDutch (HND) sample

Module Topic N Range

Raw descriptives Weighted descriptivesa

SDMean SE SD Nb Mean SEc

Start Age 12,503 18 to 90d 45.3 0.13 14.6 12,189 40.7 0.17 13.7Education level 12,189 1 to 8e 6.9 0.01 1.2 12,189 6.4 0.02 1.5Duration of romanticrelationship in years

9,038 1 to 80 18.3 0.14 13.7 9,038 14.7 0.18 12.6

Living situation(socio-demography)

Number of children 12,190 0 to 12 1.2 0.01 1.2 12,189 1.1 0.01 1.2Height in centimetresf 11,035 100 to 213 174.7 0.09 9.1 11,034 175.1 0.12 9.1Weight in kilogramsf 11,034 20 to 190 74.6 0.14 14.7 11,034 74.7 0.20 15.2

Affect/ Mood PANAS Positive affect 8,031 10 to 50 34.2 0.08 6.9 8,030 33.5 0.11 7.1PANAS Negative affect 8,032 10 to 50 19.7 0.08 7.2 8,030 20.5 0.12 7.4DASS Depression 7,972 0 to 42 6.8 0.09 7.8 7,972 7.3 0.13 8.2DASS Anxiety 7,972 0 to 42 3.6 0.06 4.9 7,972 4.0 0.09 5.4DASS Distress 7,973 0 to 42 8.6 0.08 7.0 7,972 9.4 0.12 7.3

Well-being MANSA quality of life 10,181 12 to 84 62.1 0.09 8.6 10,180 61.4 0.13 9.0Happiness index 10,152 0 to 10 6.9 0.02 1.6 10,151 6.8 0.02 1.7SPF-IL 10,131 0 to 45 25.1 0.06 5.9 10,130 24.4 0.08 5.9Ryff total 10,033 46 to 234 166.6 0.27 26.6 10,133 163.7 0.38 27.4

Note: DASS=Depression-Anxiety-Stress Scale; MANSA=The Manchester Short Assessment of quality of life; PANAS=Positive and Negative Affect Schedule; SPF-IL = The Social Production Function Instrument for the Level of well-being.aThe calculation of the post-stratification weights can be found in Supplementary Material Table S2.bThe N for the weighted descriptives (N = 12,189) is smaller than for the raw descriptives because 314 participants (2.5%) didnot provide their education level correctly.cStandard errors based on Taylor series linearization in R-package “svy” (Lumley, 2014).dThe distribution of age and gender is presented visually in Figure S8 in the Supplementary Material.eEducation level ranged from one (elementary school not finished) to eight (academic degree).fThe lower thresholds of the height and weight range seem rather extreme, but only four individuals reported a height below150 cm (

There was considerable overlap between individualswith severe levels on the three subscales: all individualswho had severe symptom levels of stress and depressionalso had severe symptom levels of anxiety. Gender differ-ences were small (see Supplementary Material Table S3),with women reporting slightly more symptoms of stress(t=9.89, p< 0.001, d=0.23), anxiety (t=7.93, p< 0.001,d=0.18), and depression than men (t=2.64, p< 0.01,d=0.06). Finally, there were also many participants with-out a single symptom of anxiety (26.6%), depression(16.8%), or psychological stress (6.7%).

Most participants rated their quality of life fairly high (asmeasured with the Manchester Short Assessment of qualityof life (MANSA), mean=62, range 12–84). In terms of hap-piness the average rating was 6.9 on a scale from 0 to 10(SD=1.6, n=10,152, median=7.0. About 85% of the par-ticipants rated their happiness a six or higher, and 40% aneight or higher). Results of the Ryff total scale (range 46–234) indicate substantial individual differences in subjectivewell-being. On average, women reported slightly lower well-being than men (t=�3.52, p< 0.001, d=0.08), mainly dueto less self-acceptance (t=�4.59, p< 0.001, d=0.10) andlower autonomy (t=�21.77, p< 0.001, d=0.44). However,compared to men, women reported more positive social re-lationships (t=10.05, p< 0.001, d=0.21) and personalgrowth (t=7.77, p< 0.001, d=0.17). Finally, even the

Figure 3. Heat map of the cross-sectional study participants’ residence versus population density map. Note: The populationdensity map on the right is derived from CBS statline and presents Dutch population densities per municipality in 2010 interms of number of inhabitants per square kilometre: From low in green (21–250), via yellow (250–500) and orange (light:500–1000, dark: 1000–2500) to red (2500–6000). The pictures show that the study coverage concurs very well with popula-tion density scores.

Colou

ron

line,

B&W

inprint

Figure 4. Positive and negative affect in the cross-sectionalsample. A scatterplot with scores of the Positive and Nega-tive Affect Schedule (PANAS). The black lines in the figureindicate the mean values for positive affect (34.3) and neg-ative affect (19.7). The darker the orange the higher thenumber of people with that specific score. The black dotsrepresent actual observations.

Colou

ron

line,

B&W

inprint



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

subgroup of severely depressed, anxious, or stressed partici-pants (about the highest 5% scores on the DASS, n=578)rated their well-being fairly diverse (Ryff total score,range=46–206, mean=125, SD=27, median=123), justas their general happiness (range=0–10, mean=4.1, SD=2.2, median=4.0). Apparently, experiencing high symptomlevels does not necessarily preclude enjoying life.

Finally, we weighted our results for the selection bias fac-tor. The results in Table 3 suggest that mood symptoms areslightly underestimated in our sample, relative to the generalDutch population. Anxiety was more prevalent thandepression/stress in both the HND and NEMESIS sample(4.6%, past week, HND; and 12.4%, past 12 months, NEM-ESIS). In the HND sample 3.9% reported severe depression(past week), while 5.8% of the NEMESIS participants re-ported amajor depression in the past 12-months. In FigureF5 5we present the prevalence of the nine symptoms of depres-sion according to the DSM in the HND sample and in theLifelines sample, namely, depressed mood, diminishedinterest, weight loss/gain, insomnia/hypersomnia, psycho-motor agitation/retardation, fatigue or loss of energy,worthlessness/guilt, concentration problems, and suicidalideation. The HND participants reported more symptomsof depression (QIDS, self-report, past week) than the Life-lines participants (MINI clinical interview, past year), butthese differences dissipate at the higher total scores, in linewith a lower threshold for self-report and timing effects.

Diary study

Sample characteristics

Up to 13 December 2014 629 participants completed thediary study (5% of all HND participants). Of the diaryparticipants 532 (85%) also filled out additional cross-sectional modules. The 629 participants were 517 women(82%, mean age = 39, SD= 13) and 112 men (18%, meanage = 48, SD= 13), mainly Dutch (99%) and spreadthroughout the Netherlands; five participants were Belgianand one had another nationality. Diary study participants(n=629) were on average 5.4 years younger than the otherHND participants (t=9.78, p< 0.001, d=0.40), more of-ten women (χ2 = 84.51, p< 0.001, d=0.28), higher edu-cated (t=�5.67, p< 0.001, d=0.24), and they reportedlower well-being (t=3.23, p< 0.001, d=0.14; see Supple-mentary Material Tables S4 and S5). Details on the diarystudy are presented elsewhere (Van der Krieke et al., sub-mitted for publication b Q15 Q16).

Key results

Diary study participants completed 28,264 assessments intotal, with 45 assessments on average per participant(range 0–90, SD=32). Since analyses were run for eachparticipant separately, group-based results are not pre-sented here. About 48% (n=302) participants completed

Figure 5. The prevalence of the nine DSM depression symptoms. The prevalence for each number of depression symptomsin the HowNutsAreTheDuch (HND) sample and the Lifelines sample. The vertical axis shows the percentage of participantswith this particular number of symptoms.

Colour

online,B&W

inprint



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

enough assessments (t≥ 59) to receive basic feedback, and38% (n=238) completed enough assessments (t≥ 68) toreceive additional advanced feedback, including a concur-rent and dynamic personal network model.

Evaluation of the website

The evaluation questionnaire was completed by 3093 par-ticipants who were on average 48 years old (SD= 14), and65% were women. In the evaluation questionnaire partic-ipants scored six components of the cross-sectional studyof the HND website on a scale from 1 to 10. In short,the mean score for the lay-out of the website was rated7.6, the cross-sectional modules 7.7, the presented results7.3, and the overall judgement 7.7. The diary study wasevaluated separately, and these results are presented else-where (Van der Krieke et al., submitted for publication aQ17 Q18 ).

Missing data

In HND we use digital questionnaires of which most areprogrammed such that they need to be completed fromtop to bottom and no items can be left blank. As a result,the missing data within questionnaires is minimal. In thecross-sectional studymany questionnaire modules were op-tional, so not all participants completed all modules.Supplementary Material Table S2 shows how many partici-pants completed each module (range generally from 800 to12,000). In the diary study, many participants missed as-sessments, as we expected. To present reliable networkmodels, we only ran VAR analyses for participants whocompleted> 75% of the data (≥68 observations, n=238).Missing data of these participants were imputed usingAmelia-II imputation (http://cran.r-project.org/web/pack-ages/Amelia/Amelia.pdf). Time series data with many miss-ing observations are not suitable for time series analysis, butcan be included inmultilevel analyses and data-mining pro-cedures, which is what we intend to do in the future.

Comments

HND is primarily a scientific endeavour to explore ways inwhich the classification of psychiatric symptoms can beimproved, via data-driven studies of how mental vulnera-bilities and strengths are related and interact, includinghumour, empathy, and self-acceptance, and by zoominginto dynamic interactions between health-related vari-ables. Additionally, we aspire to contribute to the debateabout mental health in the Netherlands, aiming to reducethe stigma associated with psychiatric diagnosis (Hinshawand Stier, 2008). An emphasis on the dimensional natureof mental health and attention for individuals’ strengths

may contribute to opportunities to increase mental healthin people suffering from mental symptoms.

Nearly half of the population will meet current DSMcriteria for a mental disorder at some time in their life, butthis does not mean that they will all need treatment (Kessleret al., 2005). Health is best defined by the person (ratherthan by the doctor), according to his or her functionalneeds, which can be the meaning of “personalized medi-cine” (Lancet, 2009; Perkins, 2001). Doctors, then, are part-ners in delivering those needs. Diary approaches can play arole in this process of empowerment, as they enable for per-sonalized models that can shed light on etiology and per-sonal dynamics, as well as personalized solutions, andmerit the perspective of health as people’s ability to adaptto their environments and self-manage (Duckworth et al.,2005; Huber et al., 2011; Solomon, 2014). Goals and criteriafor “treatment success” often differ substantially betweenclinicians and their patients, which may explain part of thehigh drop-out rates (25–60%) for most psychiatric interven-tions (Perkins, 2001; Tehrani et al., 1996).

Mental symptoms may also be seen as more than “de-fects to be corrected”, as individuals’ differences may betheir very strengths. For example, individuals with autismmay be great scientists, mathematicians, or software testers(Mottron, 2011; Solomon, 2014), while anxious individ-uals may be rather creative, sensitive, and agreeable, thusperfect employees for social job tasks (George et al.,2002; Stossel, 2013). Antagonistic, mistrustful, uncoopera-tive and rude people, in contrast, may be excellent drillsergeants or bill collectors (George and Jones, 2002). Fur-thermore, genes underlying creativity also increase vulner-ability for schizophrenia and bipolar disorder (Poweret al., 2015). Everyone has both strengths and weaknesses,but which is which is a function of the context in which welive and grow (Darwin, 1859).

There can even be tension between identity and illness, assome “symptoms” can feel self-congruent and come with-out biographical breaks (e.g. in most personality disordersand autistic people), while other symptoms seem to invadean identity (e.g. in schizophrenia), and even “harm” oftenhas a normative component (Dehue, 2014; Gutiérrez et al.,2008; Solomon, 2014). A reconsideration of diversity and fo-cus on individual strengths and resources that may compen-sate for – or buffer against – the expression of mentalsymptoms, may help people to preserve acceptable levelsof mental well-being despite the presence of psychopathol-ogy (Harkness and Luther, 2001; Sheldon et al., 2011; Solo-mon, 2014). This would fit in with a concept of mentalhealth as a hybrid of absence of mental illness and the pres-ence of well-being and mental resources (Duckworth et al.,2005; Keyes, 2007).



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

In sum, our aim is to improve classification for re-search purposes, and one promising direction is a dimen-sional and dynamic approach that acknowledges the roleof the interactions between mental symptoms andstrengths (Duckworth et al., 2005). Although categoriza-tion may be helpful in clinical practice to reach treatmentdecisions, this procedure may be supplemented with apatient-tailored treatment via the introduction of diarystudies and personalized models (Van der Krieke et al.,submitted for publication aQ19 Q20 ).

Planned analyses

Data analyses of the HND project will be centred aroundthree sets of analyses. The first set of analyses will focuson identifying profiles of mental symptoms and strengthsbased on the cross-sectional data. For example, a studyof predictors for emotional and psychological well-beingacross the lifespan (Jeronimus et al., submitted forpublicationQ21 ), or of resources that enable people to preservesubjective well-being despite severe mental symptoms(Bos et al., in preparationQ22 ). The second set of analyses willzoom into the dynamic relationships between mood, cog-nitions, and behaviours at the individual level, for whichthe diary data will be used. To this end we will use time se-ries analysis and multilevel analysis. A first study showedthat prosocial behaviour and positive affect enhance oneanother in daily life, and may thereby maintain positivemental states (Snippe et al., submitted for publicationQ23 ). Athird set of analyses will apply machine learning tech-niques to find predictors for happiness (Wanders et al.,in preparationQ24 ), and to integrate dimensional and categor-ical approaches in the identification of affective subtypes(Wardenaar et al., in preparationQ25 ). Up to 1 July 2015, a to-tal of 32 research proposals have been accepted by theHND scientific board.

Dissemination of study findings

Apart from publication in scientific journals, study find-ings are presented to our participants, via the HNDwebsite. As described earlier, all participants receive indi-vidual results via the website after completing a question-naire module in the cross-sectional study, and aftercompleting the diary study. In addition, participants areinformed about study results via a newsletter (three to fourtimes a year). Study results are also communicated to thebroader Dutch population via news articles, radio inter-views, podium discussions, and presentations open to thegeneral public.

Strengths and limitations of the HND project

One of the strengths of the HND project is that we involvethe general public in mental health research and in the de-bate about how mental health should be conceptualized.The project website provides participants with the opportu-nity to gain insight into their mental health, whether or notby comparing their scores to scores of other participants.Moreover, the combination of (a) measuring mental symp-toms and strengths and (b) our longitudinal time-intensivedesign may allow for a more accurate and in-depth descrip-tion of the dynamics of mental health and ill-health thanmost studies are able to provide (Duckworth et al., 2005;Keyes, 2007; Lamiell, 1998; Molenaar and Campbell, 2009;Piantadosi et al., 1988). This broad range of assessed mentalstrengths set HND apart from previous studies such asNEMESIS and Lifelines.

The most salient limitation of our project is the prob-lem of representativeness (self-selection bias), especiallythe overrepresentation of highly educated strata andwomen. To estimate the extent to which selection effectscurved our results we weighted our sample against theproportions in the general Dutch population, and com-pared the HND sample with the NEMESIS-2 and Lifelinesstudies. Results suggest that scores of HND participantsare likely to deviate somewhat from population averageson several psychological characteristics (mainly those asso-ciated with differences in education), which might attenu-ate the generalizability of our results (just as in NEMESISand Lifelines). For this reason NEMESIS weighted their re-sults (de Graaf et al., 2010). Nevertheless, in the HND,NEMESIS, and the Netherlands Study of Anxiety andStress (NESDA) anxiety was more prevalent than depres-sion, and the small gender differences were even compara-ble in size (e.g. for anxiety in HND d=0.18 and NESDAd=0.11; for depression in HND d=0.06 and NESDAd=0.08, see Jeronimus et al., 2013).

A large random sample from the general Dutch popu-lation (without strong selection bias and non-response)would require immense resources. Note that only 58.6%of the random sample for NEMESIS-2 actually partici-pated in that study (de Graaf et al., 2010). In Lifelines only24.5% of the intended sample invited via their generalpractitioner participated, while two-thirds of the assessedsample resulted from self-selection via other means thanthe general practitioner (see Scholtens et al., 2014).

It also remains doubtful whether a random samplewould have yielded knowledge about individual dynamicsthat would be more applicable, informative, or transferable(“generalizable”) at the personal level, see Molenaar andCampbell (2009). Exactly therefore we implemented our



1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

60616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118

diary study. Moreover, we believe that the underlying facul-ties of the mind (Panksepp and Biven, 2012) as well as thestructure in our data (Kendler and Parnas, 2015) will notbe different in subsamples of the population. Selection ef-fects may thus, at worst, bias prevalence estimates (averagesymptom counts), but we deem it unlikely that selection ef-fects invalidate research into the associations and interac-tions between personal vulnerabilities and resources.

Self-selection is not necessarily problematic, as previ-ous crowdsourcing studies attracted more diverse partici-pants than any other means of recruitment did (Goslinget al., 2004; Revelle et al., 2010; Skitka and Sargis, 2006).For example, HND sampled more participants above age65 (9% versus 19% in the population) than Lifelines(7.6%) and NEMESIS, which excluded people older than64. This may reflect that the Netherlands are among thecountries with most and fastest internet connections percapita worldwide (90% of the households is connected).

Another limitation concerns the diary study. We allowedour diary participants to complete their questionnaire untilone hour after the prompt. Methodologically, the presenceof this time window may have biased the results. For in-stance, when participants received the prompt at a busy mo-ment, they had the opportunity to postpone their responseto a more quiet moment, in which different emotions wereexperienced and reported. However, our data indicated thatmost diary questionnaires were completed within 12 mi-nutes after the prompt (mean=18.0, SD=15.7), thus thismethodological bias is probably small.

Conclusion

The HND project has resulted in a rich dataset containingboth cross-sectional and intensive longitudinal data pro-viding information about mental symptoms and strengths,and their dynamic interactions. The data is used to providepersonalized feedback to participants about their mentalhealth, to study profiles of mental symptoms andstrengths, and to zoom into the fine-grained level of dy-namic relationships between variables over time.

Acknowledgements

The HowNutsAreTheDutch (HND) project is funded by a VICIgrant (no. 91812607) received by Peter de Jonge from the Neth-erlands Organization for Scientific Research (NWO-ZonMW)and by the University Medical Centre Groningen ResearchAward 2013, also received by Peter de Jonge. Part of the HNDproject was realized in collaboration with the Espria Academy.Espria is a health care group in the Netherlands consisting ofmultiple companies targeted mainly at the elderly population.The authors want to thank all participants of HND for their par-ticipation and valuable contribution to this research project.The authors also thank the RoQua team, Inge ten Vaarwerk, Es-ther Hollander, Jasper van de Gronde, Joris Slaets, Ester Kuiperand Chantal Bosman for their contributions.

Declaration of interest statement

The authors have no competing interests.

References

Abdel-Khalek A. (2006) Measuring happiness with

a single-item scale. Social Behavior & Personal-

ity: An International Journal, 34(2), 139–149.

Arrindell W.A., Ettema J.H.M. (1986) SCL-90.

Handleiding bij een multidimensionale

psyhopathologie indicator, Lisse: Swets &

Zeitlinger.

Baer R., Smith G., Hopkins J., Krietemeyer J., Toney

L. (2006) Using self-report assessment methods

to explore facets of mindfulness. Assessment,

13(1), 27–45. DOI:10.1177/1073191105283504.

Baer R.A., Smith G.T., Lykins E., Button D.,

Krietemeyer J., Sauer S., …Williams J.M.G.

(2008) Construct validity of the five facet

mindfulness questionnaire in meditating and

nonmeditating samples. Assessment, 15(3),

329–342, DOI: 10.1177/1073191107313003Q26Q27

Barge-Schaapveld D.Q.C.M., Nicolson N.A.,

Berkhof J., deVries M.W. (1999) Quality of life

in depression: daily life determinants and vari-

ability. Psychiatry Research, 88(3), 173–189.

Baron-Cohen S., Wheelwright S. (2004) The em-

pathy quotient: an investigation of adults with

asperger syndrome or high functioning autism,

and normal sex differences. Journal of Autism

and Developmental Disorders, 34(2), 163–175.

DOI:10.1023/B:JADD.0000022607.19833.00.

Barrett L., Russell J. (1998) Independence and bi-

polarity in the structure of current affect. Jour-

nal of Personality and Social Psychology, 74(4),

967–984.

Bevelander K.E., Kaipainen K., Swain R., Dohle S.,

Bongard J.C., Hines P.D.H., Wansink B. (2014)

Crowdsourcing novel childhood predictors of

adult obesity. PLoS ONE, 9(2), e87756.Q28

Blaauw F., van der Krieke L., Bos E., Emerencia A.

C., Jeronimus B.F., Schenk M., … de Jonge P.

(2014) HowNutsAreTheDutch: personalized

feedback on a national scale. Association for

the Advancement of Artificial Intelligence

(AAAI) Fall Symposium Series, DOI: 10.13140/

2.1.4203.1683Q29

Bolger N., Davis A., Rafaeli E. (2003) Diary methods:

Capturing life as it is lived. Annual Review of Psy-

chology, 54, 579–616. DOI:10.1146/annurev.

psych.54.101601.145030.

Bos E.H., Jeronimus B.F., de Jonge P., Snippe E. (In

preparation) Preserving mental wellbeing in

the face of psychopathology: compensatory

and buffering effects of personal strengths

and resources.Q30

Brabham D.C. (2008) Crowdsourcing as a model

for problem solving: an introduction and cases.

Convergence: The International Journal of Re-

search into New Media Technologies, 14(1),

75–90. DOI:10.1177/1354856507084420.

Brandt P.T., Williams J.T. (2007) Multiple Time

Series Models, Thousand Oaks: CA, Sage

Publications.

Clark L.A., Watson D. (1991) Tri

university of groningen hownutsarethedutch (hoegekisnl) van … · revised 10 july 2015; accepted...

Documents