development and validation of the diabetes bowel symptom questionnaire

Development and validation of the Diabetes BowelSymptom Questionnaire

C. QUAN*, N. J. TALLEY*, S. CROSS*, M. JONES*, J . HAMMER� , N. GILES� & M. HOROWITZ�*Department of Medicine, University of Sydney, Nepean Hospital, Sydney, Australia; �Universitatsklinik fur Innere Medizin

4 des AKH Wien, Abteilung fur Gastroenterologie und Hepatologie, Vienna, Austria; �Department of Medicine, University of

Adelaide, Royal Adelaide Hospital, Adelaide, Australia

Accepted for publication 26 February 2003

SUMMARY

Background: Although gastrointestinal symptoms occur

frequently, there is no validated measure of gastro-

intestinal symptoms in patients with diabetes mellitus.

Aim: To develop the Diabetes Bowel Symptom Question-

naire.

Methods: A questionnaire derived from previously vali-

dated symptom measures was compiled to assess all

relevant gastrointestinal and diabetes items. Face and

content validity were ascertained by expert review. One

hundred and sixty-eight patients with diabetes mellitus

completed the instrument, and reliability was evaluated

by a test–re-test procedure 1 week later. Concurrent

validity was evaluated by an independent physician

interview (n ¼ 33). Measures of glycaemic control

(glycated haemoglobin and plasma glucose) were

compared with self-reported glycaemic control on a

five-point Likert scale in diabetic out-patients (n ¼ 166).

Results: The questionnaire had adequate face and

content validity. There was good to excellent test–re-

test reliability for the gastrointestinal and diabetes items

(median kappa: 0.63 and 0.79, respectively); concur-

rent validity was good to excellent (median kappa: 0.47

and 0.65, respectively), except for the items assessing

the severity of gastrointestinal symptoms. Both glycated

haemoglobin (P < 0.0001) and plasma glucose

(P ¼ 0.005) correlated significantly with self-reported

glycaemic control.

Conclusion: The Diabetes Bowel Symptom Question-

naire appears to be a useful measure of gastrointestinal

symptoms and glycaemic control in diabetes mellitus,

and should have applicability in epidemiological and

clinical studies.

INTRODUCTION

It is now widely recognized, if only relatively recently,

that complications involving the gastrointestinal tract

represent an important cause of morbidity in patients

with diabetes mellitus.1–4 Gastrointestinal symptoms in

diabetes may reflect poor glycaemic control or auto-

nomic nervous system dysfunction;4, 5 complications

such as gastroparesis can, in turn, further aggravate

glycaemic control.6 Out-patient studies generally sup-

port the concept that gastrointestinal symptoms occur

frequently in patients with diabetes mellitus. For

example, in an unselected group of 136 out-patients

with diabetes, Feldman and Schiller reported that

constipation occurred in 60%, abdominal pain in

34%, nausea and vomiting in 29%, dyspepsia in 27%,

diarrhoea in 22% and faecal incontinence in 20%.7–10

These observations are concordant with clinical

impressions in practice, and it is likely that gastrointes-

tinal complaints have a negative impact on the quality

of life in diabetes, although this aspect has received

relatively little attention.11–13

Correspondence to: Professor N. J. Talley, Department of Medicine, Level 5

South Block, University of Sydney, Nepean Hospital, PO Box 63, Penrith,

NSW 2751, Australia.

E-mail: [email protected]

Aliment Pharmacol Ther 2003; 17: 1179–1187. doi: 10.1046/j.0269-2813.2003.01553.x

� 2003 Blackwell Publishing Ltd 1179

Although gastrointestinal symptoms are generally

believed to be frequent and troubling in diabetes, it is

notable that gastrointestinal symptoms are also common

in the general community;14, 15 indeed, somewhat

surprisingly, the results of community studies that have

specifically attempted to establish a link between diabetes

and gastrointestinal symptoms remain conflicting. Thus,

Schvarcz et al. evaluated the prevalence of gastrointes-

tinal symptoms in an unselected, population-based,

cohort of 110 patients with long-standing insulin-

dependent diabetes, compared with 210 age- and sex-

matched control subjects.16 In patients with diabetes,

there was an increased prevalence of upper, but not

lower, gastrointestinal symptoms.16 Moreover, patients

with higher glycated haemoglobin levels had more

symptoms than the remainder of the cohort.16 Similarly,

the population-based study by Bytzer et al. found an

increase in the prevalence of upper and lower gastroin-

testinal symptoms in patients with diabetes; an increased

prevalence rate of symptoms was also associated with

poorer levels of glycaemic control.10 Janatuinen et al., on

the other hand, found that the prevalence of upper

gastrointestinal symptoms, abdominal pain, constipa-

tion and diarrhoea was not significantly increased in

570 diabetic patients [481 of whom had type 2 (non-

insulin-dependent) diabetes] when compared with 635

controls.17 Similar, negative results were observed in a

population-based study from Olmsted County, Minne-

sota, of type 2 diabetes.18 There are a number of

potential explanations for these disparate findings, but

perhaps most critically the studies have failed to apply a

validated disease-specific symptom measure in subjects

with diabetes and appropriate controls.

In view of the increasing interest in the relationship

between diabetes, gastrointestinal dysfunction and

gastrointestinal symptoms, we aimed to develop a

reliable and valid questionnaire for the assessment of

gastrointestinal symptoms in diabetes. Potential deter-

minants of gastrointestinal symptoms in diabetes

include disordered motility, glycaemic control and

demographic variables.19, 20 Hence, we also wanted to

develop a measure that assessed diabetic disease status

and, in particular, glycaemic control with adequate

accuracy. The goal was to develop an instrument that

could be widely applied in both epidemiological and

clinical studies of patients with diabetes mellitus. We

report here our experience with a new questionnaire,

the Diabetes Bowel Symptom Questionnaire, developed

for this purpose.

METHODS

Questionnaire design

We revised a self-administered screening instrument

(the Bowel Symptom Questionnaire) that itself was

based on a previously validated US questionnaire.21 The

Bowel Symptom Questionnaire was administered to 361

out-patients at a gastroenterology clinic and was able to

identify chronic gastrointestinal symptoms.21 Twenty-

six of the 46 gastrointestinal items from the Bowel

Symptom Questionnaire were used in the Diabetes

Bowel Symptom Questionnaire; the Diabetes Bowel

Symptom Questionnaire also contained new items. In

addition, the Diabetes Bowel Symptom Questionnaire

contained items specific for diabetes and its complica-

tions. In extensive clinical studies of gastrointestinal

out-patients and population-based studies in the USA,

Germany and Australia, we have shown that the

gastrointestinal symptom items are well accepted, quick

to complete and highly comprehensible.21–24 The

individual items were also reliable on re-testing in the

same population, and produced results concordant with

a physician interview.21 However, individuals with

diabetes mellitus were not evaluated.

In this study, all relevant gastrointestinal questions

from our previous studies, questions relevant to the

assessment of the presence of diabetes, glycaemic

control and diabetic complications and demographic

items were compiled. Initially, an extensive Medline and

Current Contents search was conducted to identify

previous research, using the MeSH terms questionnaire,

diabetes and gastrointestinal. No disease-specific meas-

ure for gastrointestinal symptoms in diabetes was

identified by this search.

Gastrointestinal symptom items. The questionnaire con-

tained detailed information about upper and lower

gastrointestinal symptoms occurring within the last

3 months. The presence of individual gastrointestinal

symptoms was assessed on a five-point Likert scale (not

at all to almost always, or never to daily). Seven of the

symptoms/symptom complexes are listed below.

(a) Abdominal pain — abdominal pain occurring more

than one-quarter of the time and of at least

moderate intensity.

(b) Abdominal bloating or distension — abdominal

bloating/swelling or visible abdominal distension

occurring more than one-quarter of the time.

1180 C. QUAN et al.

� 2003 Blackwell Publishing Ltd, Aliment Pharmacol Ther 17, 1179–1187

(c) Nausea/vomiting — nausea/vomiting occurring

more than one-quarter of the time.

(d) Gastro-oesophageal reflux — heartburn or acid

regurgitation occurring more than one-quarter of

the time.

(e) Dysphagia — difficulty swallowing (food sticks on

the way down) occurring more than one-quarter of

the time.

(f) Diarrhoea — loose or watery bowel movements

occurring more than one-quarter of the time.

(g) Constipation — if any of the following two symp-

toms were present: (i) straining — straining a lot to

produce a bowel movement; (ii) hard or very lumpy

bowel movement; (iii) incomplete evacuation —

after finishing a bowel movement, there is still stool

that needs to be passed; (iv) anal blockage — a

sensation of blockage in the back passage; (v)

manual assistance in faecal evacuation — needing

to press a finger in or around the anus to help the

bowel movement; (vi) less than three bowel move-

ments a week.

The severity of each individual gastrointestinal symp-

tom was recorded on a five-point Likert scale ranging

from very mild to very severe.

Diabetes items. The questions related to diabetes inclu-

ded the type, onset, current treatment and microvascu-

lar and macrovascular complications. The diabetes

items were dichotomous (yes/no), for example presence

of proteinuria, eye damage and macrovascular compli-

cations, or based on a five-point Likert scale (not at all to

almost always, or never to daily), for example peripheral

neuropathy and hypoglycaemic episodes.

Subjects

Subjects were classified as having type 1 diabetes if their

age at diagnosis was < 30 years and they had used

insulin ever since they had been diagnosed with

diabetes.25 All other diabetic subjects were classified as

having type 2 diabetes.

The subjects were recruited from three distinct popu-

lations as follows: (i) a consecutive sample of 45 out-

patients attending the Diabetes Centre at the Nepean

Hospital, Sydney, Australia for their usual diabetes

clinic appointment; (ii) 111 subjects recruited from

the mailing list of Diabetes Australia (NSW Branch), a

non-profit organization with approximately 100 000

registered individuals; (iii) 66 subjects recruited from a

previous cohort of individuals who participated in

gastrointestinal research.

The questionnaire was administered to all subjects,

and those who returned the first questionnaire were

mailed a second questionnaire 1 week later for reliabi-

lity assessment; 168 of 222 (76%) completed and

returned the second questionnaire.

The overall mean age was 54 years (standard devi-

ation, 14.0 years) and 49% were males. There were 37

(22%) patients with type 1 diabetes, and the duration of

known diabetes was over 5 years in 64%.

Approximately 1 week after the second questionnaire

had been received, all subjects were interviewed by an

experienced gastroenterologist blind to the results of the

questionnaire (CQ). During the interview, the items in

the questionnaire were completed by the physician who

remained unaware of the answers provided by the

subject on the self-report assessment. The physician

probed for answers where there was uncertainty and

enquired about the clarity of questions. The physician

assessment was considered to be the ‘gold standard’.

Assessment of glycaemic control

A consecutive sample of 166 out-patients with docu-

mented diabetes mellitus was recruited from the Nepean

Hospital in Sydney and the Royal Adelaide Hospital in

Adelaide. There were 31 type 1 and 132 type 2 patients,

36% of whom were insulin requiring; no information

was available in three cases. The mean ages in type 1

and 2 patients were 33 years and 57 years, respect-

ively; 45% and 52% were male, respectively, and in

84% and 39%, respectively, the duration of known

diabetes was more than 5 years. They were initially

requested to complete the questionnaire. In addition

to checking on the ease of completion, these patients

had a fasting blood sample (20 mL) taken for the

measurement of glycated haemoglobin (Hb A1c) and

plasma glucose at 08.00–09.00 h.

Statistical analyses

Reliability and concurrent validity testing of individual

items were based on unweighted kappa statistics (or, for

multitype items, a weighted kappa).26 Perfect agree-

ment had a value of 1.0 and agreement no different

from chance had a value of zero. A kappa value between

DIABETES BOWEL SYMPTOM QUESTIONNAIRE 1181


zero and less than 0.4 indicated significant but poor

agreement, whereas values between 0.4 and 0.75

indicated good agreement; values above 0.75 indicated

excellent agreement.26–28

For each patient, the proportion of concordant

responses in the entire questionnaire for reliability

testing was calculated and modelled as a binomial

outcome in terms of age, gender, duration of diabetes

and insulin usage, using unconditional logistic regres-

sion.29 Odds ratios and 95% confidence limits were then

calculated.

Correlations between objective tests (Hb A1c) were

based on Spearman rank correlation coefficients due to

their non-normal distribution.

Sample size and power. The sample size for the record

review was chosen such that, should a clinically

minimum agreement be observed, the results could be

clearly differentiated from chance. Clinically minimum

conditions were defined as 80% observed agreement,

yielding a kappa value (i.e. after adjusting for random

agreement) of 0.6, with discrepancies evenly distri-

buted across disagreement categories. Under these

conditions, n ¼ 20 subjects would yield P < 0.01. A

conservative definition of significance was adopted

given the number of kappa values to be calculated. A

slightly larger total sample (n ¼ 33) was sought for

the concurrent validity assessment in order to allow

for drop-outs.

In assessing the sample size needs for the association

between self-reported glycaemic control and objective

test results, a moderate (v2/N ¼ 0.4) effect size was

defined as relevant. To achieve a power of 0.8 at the

0.05 level of statistical significance, n ¼ 50 patients

were required.

All calculated P values were two-tailed; the alpha level

of significance was set at 0.05.

RESULTS

Face and content validity

The questionnaire was reviewed by individual experts

in gastroenterology (n ¼ 2) and diabetology (n ¼ 2) in

Sydney and Adelaide. This was to ensure that all the

items included were meaningful and relevant. The

questionnaire was revised several times during this

process. To ensure content validity, a matrix of the

questionnaire domains and the questions in each

domain was composed. It was judged by the research

team that each item included was relevant to the goals

of the measure and all relevant domains were

adequately represented. The questionnaire was pre-

tested in a cohort used for the assessment of glycaemic

control (166 diabetic out-patients), and the instrument

was reported to be understandable and easy to

complete.

Reliability of individual questions

The reliability of the questionnaire items was good to

excellent. The median kappa values were 0.63 for the

gastrointestinal items (interquartile range, 0.56–0.71)

and 0.75 for the diabetes items (interquartile range,

0.59–0.87) (Tables 1 and 2). Only five items in the

questionnaire (pain or discomfort before meals; pain or

discomfort before meals and worse after eating; pain

improving with food or milk; vomiting; and less than

three bowel motions per week) had a kappa value

below 0.4. Overall, the individual gastrointestinal

symptom items measuring severity were reliable, with

a median kappa value of 0.66 (interquartile range,

0.52–0.71) (Figure 1). Sixty per cent of the gastroin-

testinal item kappa values were greater than 0.6, and

74% of the diabetes item kappa values were greater

than 0.6.

The effects of age, gender, duration of diabetes and

insulin treatment on the reliability of the gastrointesti-

nal and diabetes items were evaluated separately, and

did not predict any change in reliability responses.

Criterion (concurrent) validity

Individual questions. In the 33 patients who completed

the reliability assessment and were also independently

interviewed by a physician, the kappa values for the

individual gastrointestinal and diabetes items are

presented in Tables 1 and 2. Overall, the validity of

the individual items was good to excellent. The median

kappa values were 0.47 for the gastrointestinal items

(interquartile range, 0.24–0.64) and 0.65 for the

diabetes items (interquartile range, 0.51–0.77). Eight

gastrointestinal items had a kappa value of less than

0.4: pain improved with defecation, pain after meals,

pain improving with food or milk, anorexia, heartburn,

faecal urgency, incomplete faecal evacuation and

change in bowel habit. Five diabetes items also had

kappa values below 0.4: urinary urgency, abnormal

1182 C. QUAN et al.


sweating, presence of proteinuria, foot ulcers and

change in weight. In general, the severity of gastro-

intestinal symptoms had poor kappa values, with a

median of 0.14 (interquartile range, )0.87–0.40)

(Figure 2). However, the concordances for the severity

of gastrointestinal symptom items were good to

excellent. Overall, 42% of the gastrointestinal item

kappa values were greater than 0.6 (excluding the

severity items) and 63% of the diabetes item kappa

values were greater than 0.6.

Self-reported glycaemic control, glycated haemoglobin and

plasma glucose. The results obtained in the separate

sample of 166 out-patients with diabetes, who completed

the questionnaire and had their glycated haemoglobin

and plasma glucose values measured independently, are

shown in Table 3. There was a very strong correlation

between glycated haemoglobin and plasma glucose

(r ¼ 0.72, P < 0.0001). There was a statistically

significant change in both glycated haemoglobin

(P < 0.0001) and plasma glucose (P ¼ 0.02) across

Table 1. Reliability and concurrent validity of the individual gastrointestinal items

GI symptoms

Reliability Concurrent validity

Concordance Kappa s.e. n Concordance Kappa s.e. n

Abdominal pain 0.9 0.63 0.08 165 0.91 0.72 0.2 33

Abdominal pain and QOL 0.97 0.86 0.1 77 1 1 0 17

Pain better with bowel action 0.8 0.59 0.09 76 0.63 0.25 0.2 16

More bowel actions with pain 0.77 0.49 0.11 75 0.82 0.55 0.2 17

Fewer bowel actions with pain 0.92 0.58 0.16 74 0.88 0.43 0.3 17

Looser bowel actions with pain 0.84 0.63 0.1 75 0.76 0.43 0.2 17

Harder bowel actions with pain 0.93 0.67 0.14 74 0.88 0.67 0.2 16

Pain after meals 0.84 0.56 0.11 75 0.53 0.042 0.2 17

Pain better with food/milk 0.91 0.32 0.19 76 0.77 0.26 0.2 17

Pain occurred before meals 0.92 0.21 0.21 75 0.77 * 17

Pain before meals & worse after eating 0.89 0.37 0.18 75 0.88 0.68 0.2 17

Pain woke from sleep 0.92 0.68 0.12 77 0.88 0.67 0.2 17

Early satiety 0.94 0.63 0.11 166 0.94 0.63 0.2 33

Persistence of food in stomach 0.91 0.58 0.11 133 0.91 0.52 0.2 33

Abdominal bloating 0.94 0.8 0.06 167 0.88 0.64 0.2 33

Abdominal distension 0.93 0.71 0.08 166 0.91 0.52 0.2 33

Loss of appetite 0.95 0.7 0.1 167 0.91 0.37 0.3 33

Nausea 0.95 0.66 0.11 167 0.97 0.84 0.2 33

Retching 0.98 0.49 0.22 166 0.94 0.47 0.3 33

Vomiting 0.98 0.39 0.28 164 0.97 * 32

Heartburn 0.92 0.59 0.1 167 0.91 0.35 0.3 33

Acid regurgitation 0.95 0.59 0.12 167 0.97 0.78 0.2 33

Dysphagia 0.98 0.76 0.13 167 0.97 * 33

More than 3 bowel actions per day 0.9 0.67 0.08 166 0.84 0.6 0.2 32

Less than 3 bowel actions per week 0.93 0.39 0.15 165 0.97 0.65 0.3 31

Hard stools 0.92 0.62 0.09 167 0.91 0.62 0.2 33

Loose stools 0.93 0.73 0.08 166 0.91 0.71 0.2 32

Faecal urgency 0.91 0.62 0.09 166 0.81 0.29 0.2 32

Straining 0.95 0.81 0.07 167 0.91 0.52 0.2 33

Incomplete evacuation 0.89 0.61 0.08 167 0.7 0.19 0.2 33

Anal blockage 0.93 0.56 0.12 167 0.94 0.47 0.3 33

Manual assistance with defecation 0.99 0.92 0.08 166 1 * 32

Amount of faecal incontinence 0.77 * 39 1 * 3

Flatus incontinence 0.84 0.68 0.06 163 0.72 0.42 0.2 29

Change in bowel habit 0.8 0.62 0.06 165 0.59 ) 0.04 0.2 32

GI surgery 0.95 0.89 0.04 165 0.84 0.68 0.1 31

Abdominal or bowel disease 0.95 0.82 0.06 166 0.88 0.6 0.2 33

GI, gastrointestinal; QOL, quality of life.

* Unable to generate kappa due to asymmetric table.



the self-reported blood glucose control scale from very

poor to very good. This finding is reflected in the negative

correlations between an increasingly good self-report

and both lower glycated haemoglobin (r ¼ ) 0.38,

P < 0.0001) and plasma glucose (r ¼ ) 0.38,

P ¼ 0.005). The reported correlations suggest a moder-

ately strong relationship between patient self-reporting

and objective parameters of glycaemic control.

Reliability and validity of gastrointestinal

symptom complexes

For all the gastrointestinal symptom complexes, the

reliability and concurrent validity were good to excel-

lent, with the exception of constipation, which had a

kappa value of 0.31 (Table 4).

Table 2. Reliability and concurrent validity of the individual diabetes items

Diabetes symptoms



Duration of diabetes 0.96 0.91 0.035 165 0.85 0.7 0.12 33

Frequency of doctor visits re diabetes 0.87 0.65 0.071 166 0.82 0.58 0.15 33

Frequency of specialist visits re diabetes 0.93 0.39 0.15 166 0.94 * 33

Frequency of BSL monitoring 0.78 0.68 0.044 161 0.69 0.49 0.11 32

Average BSL readings 0.92 0.8 0.053 161 0.84 0.65 0.13 32

Frequency of hypoglycaemia 0.89 0.74 0.061 147 0.82 0.65 0.21 11

Self-reported glycaemic control 0.62 0.5 0.05 165 0.78 0.57 0.14 32

Urinary urgency 0.85 0.61 0.07 164 0.73 0.37 0.15 33

Frequency of urinary incontinence 0.92 0.76 0.064 165 0.97 0.93 0.072 33

Numbness of hands/feet 0.93 0.78 0.062 163 0.91 0.61 0.2 32

Tingling/prickling of hands/feet 0.9 0.59 0.089 164 0.97 0.84 0.16 32

Burning hands/feet 0.92 0.57 0.1 165 0.91 0.52 0.24 33

Postural dizziness 0.94 0.47 0.14 165 0.97 0.65 0.32 33

Abnormal sweating 0.93 0.75 0.071 161 0.87 0.27 0.26 31

Impotence 0.97 0.92 0.037 147 1 1 0 13

Insulin usage 0.97 0.94 0.042 69 0.83 0.67 0.2 12

Change in dose of insulin 0.85 0.7 0.08 81 0.92 0.75 0.23 12

Change in time of insulin 0.86 0.28 0.16 81 1 * 12

Usage of insulin pen 0.95 0.8 0.094 81 0.83 * 12

BSL monitoring 0.97 0.79 0.092 167 0.97 * 33

Nephropathy 0.96 0.86 0.056 164 0.94 0.72 0.19 33

Monitoring of protein/microalbuminuria 0.83 0.53 0.079 157 0.81 0.45 0.19 31

Presence of protein/microalbuminuria 0.93 0.85 0.055 107 0.65 0.29 0.19 23

Neuropathy 0.96 0.88 0.047 165 0.94 0.47 0.32 32

Pains in leg while walking 0.89 0.75 0.054 165 0.85 0.69 0.13 33

Claudication 0.84 0.53 0.15 50 0.83 0.67 0.2 12

Blurred vision 0.87 0.7 0.059 163 0.85 0.69 0.13 33

Flashes of light/black spots 0.88 0.63 0.075 165 0.85 0.57 0.17 33

Eye examined 0.95 0.48 0.16 165 1 1 0 33

Treatment for retinopathy 0.97 0.87 0.057 160 0.97 0.65 0.32 30

Diabetic foot ulcers 0.95 0.76 0.081 165 0.91 0.35 0.29 33

Angina 0.99 0.96 0.029 165 1 1 0 33

Myocardial infarction 0.99 0.95 0.037 164 1 1 0 33

CVA 0.98 0.9 0.055 165 0.97 0.84 0.16 33

Treatment for impotence 0.99 0.94 0.059 80 1 * 13

Gender 1 1 0 168 1 1 0 33

Smoking 0.98 0.96 0.022 166 0.91 0.83 0.09 33

Change in weight 0.79 0.63 0.056 164 0.56 0.39 0.12 32

Educational level 0.95 0.93 0.022 165 0.61 * 33

BSL, blood sugar level; CVA, cerebrovascular accident.


1184 C. QUAN et al.


DISCUSSION

Although a number of studies have evaluated gastro-

intestinal symptoms in patients or subjects with diabe-

tes,15–17 the lack of a disease-specific questionnaire has

been a potential limitation. It is conceivable that the

current controversy surrounding the link between

diabetes and gastrointestinal symptoms is related to

the utilization of suboptimal measures of variable

reliability and validity. We have developed a new

questionnaire, the Diabetes Bowel Symptom Question-

naire, to assess gastrointestinal symptoms in patients

with diabetes mellitus, and our observations support the

reliability and validity of the instrument. Although the

sample size may have been sufficient for the initial

validation, further validation may be required before the

questionnaire is used widely. In particular, it should be

recognized that the number of type 1 patients included

in our study was relatively small. The questionnaire

may have particular application in the setting of

pancreatic transplantation for type 1 diabetes, where

there is a high prevalence of gastrointestinal dysfunc-

tion.30,31

The items included in the questionnaire were derived

from clinical observation, an extensive literature search

of studies evaluating gastrointestinal complaints in

diabetes and a review of items used in available

validated instruments for evaluating functional gastro-

intestinal symptomatology.18, 22 Experts and the

research team reviewed multiple drafts of the question-

naire before final assembly to ensure that the instru-

ment had adequate face validity (that is, the question-

naire appeared to measure what it was supposed to) and

content validity (that is, the instrument incorporated all

the relevant domains).32,33

The reliability of each dichotomous item in the

questionnaire was assessed using kappa statistics.

Testing each item allows the user of the questionnaire

to discard unreliable questions. The kappa statistic

approach adjusts for the proportion of responses due to

chance, and is a conservative measure of reliability.26–28

The Pearson correlation coefficient, although frequently

used, can overestimate reliability and was not applicable

here.24 A kappa value above 0.4 is usually considered to

be adequate.28 Accordingly, the large majority of

symptom items were reliable. This applied to both upper

and lower gastrointestinal items, although a few of the

less important pain descriptors were of lower reliability.

The reliability of the symptom severity items was also

Reliability kappas for GI severity items

.95.85.75.65.55.45.35.25.15.05

Fre

quen

cy6

5

4

3

2

1

0

Figure 1. Distribution of the reliability kappa values for gastro-

intestinal (GI) severity items.

Concurrent validity kappas for GI severity items.93.80.67.54.41.28.15.02-.11-.24

Fre

quen

cy

3

2

1

0

Figure 2. Concurrent validity kappa values for gastrointestinal

(GI) severity items.

Table 3. Self-reported glycaemic control and mean (s.d.) glycated

haemoglobin and plasma glucose measurements

Self-reported

glycaemic control

Haemoglobin

A1c (%)

Plasma glucose

(mmol/L)

Very poor 9.5 (1.5), n ¼ 13 19.1 (0.8), n ¼ 2

Poor 8.4 (1.5), n ¼ 23 13.4 (3.3), n ¼ 5

Fair 8.1 (1.7), n ¼ 59 10.4 (4.2), n ¼ 20

Good 7.4 (1.5), n ¼ 45 9.5 (3.9), n ¼ 21

Very good 6.7 (1.0), n ¼ 9 6.7 (4.1), n ¼ 5

P < 0.0001 P < 0.02



high. Similarly, almost all of the diabetes items were

reliable.

Although an instrument may assess symptom status

reliably, it may be doing so in a systematically incorrect

way and thus a true picture of symptom status may not

be captured. This is why measures of validity beyond

face and content validity are important. Additional

assessment of validity is particularly relevant for

symptom items, as their presence or absence at a

particular time point cannot be established with

objective testing. In this study, we assessed concurrent

validity for the gastrointestinal and diabetes items

(comparing self-reporting with a blind physician inter-

view as the ‘gold standard’ using kappa statistics). The

physician interview and gastrointestinal symptom items

were generally highly concordant, supporting the

validity of the questionnaire, but, notably, measures

of gastrointestinal severity were not valid based on the

physician interview and therefore cannot be used. The

reason for the poor validity of the gastrointestinal

severity items is unclear, but may be due to the effect of

recall bias; more recent events may have had a greater

impact on the self-assessment of severity of each

symptom. The diabetes items were also, in general,

valid.

The relationship between gastrointestinal symp-

toms and glycaemic control remains controver-

sial.3, 6, 15, 34–37 A role for hyperglycaemia in the

aetiology of gastrointestinal symptoms, independent of

diabetes complications, is supported by both acute

physiological and cross-sectional epidemiological stud-

ies.10, 16, 19 For this reason, an evaluation of self-

reported glycaemic control and objective measures of

glycaemic control was undertaken, in an attempt to

ensure that questioning was an adequate proxy for

future epidemiological studies. The five-point self-report

scale of glycaemic control in the last 12 months

produced reasonably satisfactory results compared with

objective measures of chronic glycaemic control (gly-

cated haemoglobin) or acute control (fasting plasma

glucose). Although the relationship was, as expected,

somewhat imperfect, it should be adequate for epidemi-

ological studies.

In conclusion, the present study provides data on the

reliability and validity of a new questionnaire for the

assessment of gastrointestinal symptoms and glycaemic

control in diabetes mellitus. The questionnaire has been

modified based on the results, deleting the symptom

severity items. Further studies are needed to confirm the

validity of the self-report measures of diabetic compli-

cations with objective tests, and to assess the utility of

the gastrointestinal symptom measures in patients with

established gastrointestinal complications of diabetes,

such as gastroparesis, which were beyond the scope of

the present study. However, the results support the

utility of this new symptom measure for application in

epidemiological and clinical settings.

ACKNOWLEDGEMENTS

This work was supported by research grants from the

National Health and Medical Research Council of

Australia and Diabetes Australia.

REFERENCES

1 Horowitz M, Fraser R. Disordered gastric motility in diabetes

mellitus. Diabetologia 1994; 37: 543–51.

2 Koch KL. Diabetic gastropathy — gastric neuromuscular

dysfunction in diabetes: symptoms, pathophysiology, and

treatment. Dig Dis Sci 1999; 44: 1061–75.

3 Soykan I, Lin ZY, Sarosiek I, McCallum RW. Gastric myo-

electrical activity, gastric emptying, and correlations with

Table 4. Reliability and concurrent validity of the gastrointestinal (GI) symptom complexes

GI symptom complexes



Abdominal bloating/distension 0.91 0.74 0.063 166 0.85 0.6 0.16 33

Constipation 0.9 0.75 0.058 168 0.7 0.31 0.16 33

Nausea/vomiting 0.94 0.63 0.11 164 0.97 0.84 0.16 32

Reflux 0.89 0.5 0.098 167 0.91 0.52 0.24 33

Dysphagia 0.98 0.76 0.13 167 0.97 * 33

Abdominal pain 0.96 0.85 0.1 50 0.94 0.76 0.16 33

Diarrhoea 0.93 0.73 0.078 166 0.91 0.71 0.15 32


1186 C. QUAN et al.


fasting blood glucose levels in diabetic patients. Am J Med Sci

1999; 317: 226–31.

4 Annese V, Bassotti G, Caruso N, et al. Gastrointestinal motor

dysfunction, symptoms, and neuropathy in non-insulin-

dependent (type 2) diabetes mellitus. J Clin Gastroenterol

1999; 29: 171–7.

5 Bittinger M, Barnert J, Wienbeck M. Autonomic dysfunction

and the gastrointestinal tract. Clin Auton Res 1999; 9: 75–81.

6 Kong MF, Horowitz M. Gastric emptying in diabetes mellitus:

relationship to blood-glucose control. Clin Geriatr Med 1999;

15: 321–8.

7 Feldman M, Schiller LR. Disorders of gastrointestinal motility

associated with diabetes mellitus. Ann Intern Med 1983; 98:

378–84.

8 Ricci JA, Siddique R, Stewart WF, Sandler RS, Sloan S, Farup

CE. Upper gastrointestinal symptoms in a U.S. national sample

of adults with diabetes. Scand J Gastroenterol 2000; 35: 152–9.

9 Siddique R, Ricci J, Stewart WF, Sloan S, Farup C. Quality

of life in a US national sample of adults with diabetes and

motility-related upper gastrointestinal symptoms. Dig Dis Sci

2002; 47: 683–9.

10 Bytzer P, Talley NJ, Leemon M, Young LJ, Jones MP, Horowitz

M. Prevalence of gastrointestinal symptoms associated with

diabetes mellitus. A population-based survey of 15 000 adults.

Arch Intern Med 2001; 161: 1989–96.

11 Enck P, Dubois D, Marquis P. Quality of life in patients with

upper gastrointestinal symptoms: results from the Domestic/

International Gastroenterology Surveillance Study (DIGEST).

Scand J Gastroenterol Suppl 1999; 231: 48–54.

12 Farup CE, Williams GR, Leidy NK, Helbers L, Murray M,

Quigley EMM. Effect of domperidone on the health-related

quality of life of patients with symptoms of diabetic gastro-

paresis. Diabetes Care 1998; 21: 1699–706.

13 Bytzer PM, Talley NJ, Hammer J, Young LJ, Jones MP, Horo-

witz M. Gastrointestinal symptoms in diabetes mellitus are

associated with both poor glycemic control and diabetes

complications. Am J Gastroenterol 2002; 97: 604–11.

14 Talley NJ, Zinsmeister AR, Van Dyke C, Melton LJ III. Epi-

demiology of colonic symptoms and the irritable bowel syn-

drome. Gastroenterology 1991; 101: 927–34.

15 Talley NJ, Zinsmeister AR, Melton LJ III. Irritable bowel syn-

drome in a community: symptom subgroups, risk factors and

health care utilization. Am J Epidemiol 1995; 142: 76–83.

16 Schvarcz E, Palmer M, Ingberg CM, Aman J, Berne C. Increased

prevalence of upper gastrointestinal symptoms in long-term

type 1 diabetes mellitus. Diabetic Med 1996; 13: 478–81.

17 Janatuinen E, Pikkarainen P, Laakso M, Pyorala K. Gastro-

intestinal symptoms in middle-aged diabetic patients. Scand J

Gastroenterol 1993; 28: 427–32.

18 Maleki D, Locke GR III, Camilleri M, et al. Gastrointestinal

tract symptoms among persons with diabetes mellitus in the

community. Arch Intern Med 2000; 160: 2808–16.

19 Rayner CK, Samsom M, Jones KL, Horowitz M. Relation-

ships between upper gastrointestinal motor and sensory

function with glycemic control. Diabetes Care 2001; 24:

371–81.

20 Jones KL, Horowitz M, Berry M, Wishert JM, Guha S. The

blood glucose concentration influences postprandial fullness

in insulin-dependent diabetes mellitus. Diabetes Care 1997;

20: 1141–6.

21 Talley NJ, Phillips SF, Melton LJ III, Wiltgen C, Zinsmeister

AR. A patient questionnaire to identify bowel disease. Ann

Intern Med 1989; 111: 671–4.

22 Talley NJ, Phillips SF, Wiltgen CM, Zinsmeister AR, Melton LJ

III. Assessment of functional gastrointestinal disease: the

bowel disease questionnaire. Mayo Clin Proc 1990; 65:

1456–79.

23 Holtmann G, Goebell H, Talley NJ. Dyspepsia in consulters

and non-consulters: prevalence, health-care seeking behaviour

and risk factors. Eur J Gastroenterol Hepatol 1994; 6: 917–24.

24 Talley NJ, Boyce P, Owen BK, Newman P, Paterson K. Initial

validation of a bowel symptom questionnaire and measure-

ment of chronic gastrointestinal symptoms in Australians.

Aust NZ J Med 1995; 25: 302–8.

25 World Health Organization Study Group. Prevention of

diabetes. Geneva, Switzerland: WHO, 1994.

26 Armitage P, Berry G. Statistical Methods in Medical Research,

3rd edn. Oxford: Blackwell Scientific Publications, 1994:

443–7.

27 Cicchetti DV, Feinstein AR. High agreement but low kappa. II.

Resolving the paradoxes. J Clin Epidemiol 1990; 43: 551–8.

28 Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin

Epidemiol 1993; 46: 423–9.

29 Coughlin SS, Pickle LW, Goodman MT, Wilkens LR. The

logistic model of interobserver agreement. J Clin Epidemiol

1992; 45: 1237–41.

30 Hathway D, Abell T, Cardoso S, et al. Improvement in

autonomic function following pancreas-kidney versus kidney

alone transplantation. Transplantation Proc 1993; 25:

1306–8.

31 Gaber AO, Hathaway DK, Abell T, Cardoso S, Hartwig MS, El

Gebely S. Improved autonomic and gastric function in pan-

creas-kidney vs kidney alone transplantation contributes to

quality of life. Transplantation Proc 1994; 26: 515–6.

32 Last JM, ed. A Dictionary of Epidemiology, 3rd edn. New York:

Oxford University Press, 1995.

33 Streiner DL, Norman GR. Health Measurement Scales. A

Practical Guide to Their Development and Use. Oxford: Oxford

Medical Publications, 1989.

34 Clouse RE, Lustman PJ. Gastrointestinal symptoms in diabetic

patients: lack of association with neuropathy. Am J Gastro-

enterol 1989; 84: 868–72.

35 Ogbonnaya KL, Arem R. Diabetic diarrhea: pathophysiology,

diagnosis and management. Arch Intern Med 1990; 150:

262–7.

36 Chey WD, Kim M, Hasler W, Owyang C. Hyperglycaemia

alters perception of rectal distension and blunts the recto-anal

inhibitory reflex in healthy volunteers. Gastroenterology

1995; 108: 1700–8.

37 Kong MF, Horowitz M, Jones KL, Wishart JM, Harding PE.

Natural history of diabetic gastroparesis. Diabetes Care 1999;

22: 503–7.



development and validation of the diabetes bowel symptom questionnaire

Documents