development and initial evaluation of a new … · mark s. horswill1, andrew hill1,2, melany...

Mark S. Horswill1, Andrew Hill1,2,

Melany Christofidis1, Sophie Francis1,

and Marcus O. Watson1,2,3

1School of Psychology, The University of Queensland

2Clinical Skills Development Service, Queensland Health

3School of Medicine, The University of Queensland

Special thanks to Fiona McIver of Queensland Health.

We would also like to thank Helen Stark, Neville Board, Charles Mitchell, Merryn Thomae, Kerry Fitzsimons, Gerry Fegan, Glynis Ross, Glenda Gorrie, Peter Donovan, Trisha Dunning, and Kerry Newlin.

Contact: A/Prof Mark Horswill, School of Psychology, The University of Queensland, QLD 4072, Australia; [email protected]; 07 334 69520

22nd July 2015

DEVELOPMENT AND INITIAL

EVALUATION OF A NEW

SUBCUTANEOUS INSULIN FORM:

FINAL REPORT

2

DEVELOPMENT AND INITIAL EVALUATION

OF A NEW SUBCUTANEOUS INSULIN

FORM: FINAL REPORT

Contents

Project overview .........................................................................................................................8

The new subcutaneous insulin form ..........................................................................................8

Design rationales informing the new subcutaneous insulin form .......................................... 11

1. Supplemental insulin orders have been aligned with the BGL Monitoring area ............ 12

Modifications to BGL ranges ........................................................................................... 13

2. Routine insulin orders have been aligned with the daily BGL monitoring and insulin administration areas ............................................................................................................ 14

What to do when more than 6 BGL readings are anticipated for a single day ............... 15

What to do if the routine insulin orders do not coincide with a set meal time (e.g. the patient is nil by mouth) ....................................................................................................... 16

3. Stat/Phone orders have been moved to the top of the page ......................................... 17

4. Stat/Phone orders now include confirmation of whether they replace an existing order and whether the entry is a phone order ............................................................................. 17

5. Better perceptual separation of form elements ............................................................. 17

6. Increased use of tick boxes where appropriate .............................................................. 18

7. Checkbox to confirm NIMC has been checked ................................................................ 18

8. More information is included on how to complete the form ......................................... 19

9. Instructions on routines orders clarified ......................................................................... 19

10. Hypoglycaemia Management Flowchart re-formatted ................................................. 20

11. Hyperglycaemia Management – Table 1 ....................................................................... 21

12. Form title is now included on Page 4 ............................................................................ 21

Sources cited in design rationales ....................................................................................... 22

Development of a new heuristic evaluation protocol ............................................................. 23

Heuristic evaluations of the existing National Pilot Subcutaneous Insulin Form and the new draft form ................................................................................................................................ 24

GENERAL .................................................................................................................................. 26

Page Layout ......................................................................................................................... 26

Is non-safety-critical information displayed in an unnecessarily prominent way, such that it could potentially draw attention away from safety-critical information and/or increase visual clutter (e.g. hospital logo or name, administrative codes, etc.)? ........... 26

Is there a mixture of landscape & portrait-orientated information that may plausibly reduce readability (e.g. potentially requiring the form to be turned during use)? ........ 27

3

Are there any other page layout issues? ......................................................................... 27

Information Layout .............................................................................................................. 28

Is there a missing or ambiguous form name? ................................................................. 28

Does the form fail to organise information in order of importance (so that attention is naturally drawn to the most important elements first, and the least important elements last)? (e.g. blood glucose level before insulin administration.) ...................................... 29

Are any elements that ought to be perceptually separated not adequately separated (e.g. by a small space or double line)? ............................................................................ 30

Are headings/labels of the same level of importance formatted differently, potentially providing misleading cues? ............................................................................................. 31

Are any of the boxes for writing something (e.g. Diet, Comments) too small (size 14 font is considered to be a minimum handwriting size guideline)? ......................................... 32

Is there an unnecessarily large amount of space devoted to any particular element, potentially introducing unneeded clutter? ..................................................................... 33

Has the form failed to take advantage of the potential benefits of using a graph or graph-like presentation of quantitative data? ................................................................ 33

Is basic functionality not understandable intuitively (or with minimal explanation)? ... 34

Are there any other information layout issues? ............................................................. 34

Font ...................................................................................................................................... 35

Is there a problem with compressed font (e.g. Arial narrow) being used anywhere, given this may affect legibility? ................................................................................................. 35

Is the font too small (smaller than 11 point), such that it might impede legibility? ....... 35

Is font size potentially giving misleading cues (e.g. important information is in small font and unimportant information in large font)? .................................................................. 35

Is capitalization used too often (capitalization is good for highlighting elements but only if used sparingly; also capitalization slows reading)? ...................................................... 35

Are there any other issues with fonts? ........................................................................... 36

Colour Scheme ..................................................................................................................... 37

Has the form failed to use colour where it would have been advantageous to do so (e.g. to draw attention to important elements, to aid perceptual organisation, or as part of an alerting system)? ........................................................................................................ 37

If colour is used, are the colour shades used appropriate? E.g. if a colour-shaded area is to be written on, is the shade sufficiently light ("pastel") for the writing to be clearly visible? ............................................................................................................................. 38

If colour is used, are any of the colours used potentially deceptive in their meaning according to cultural norms (e.g. light blue used to indicate "bad"; red used to indicate "no problem")? ................................................................................................................ 38

If colour is used as part of a system, is there redundancy for when colours cannot be perceived easily (e.g., colour-blind users or low-light conditions)? ................................ 38

Are there any other colour scheme issues? .................................................................... 38

Does the form design minimise the user's cognitive load as much as possible? ................ 39

4

Is writing required, where a better option might be to provide options to circle or similar (where circling options can reduce cognitive load and minimize handwriting legibility problems and inappropriate entries)? .............................................................. 39

Does information have to be compared or transcribed over non-adjacent areas of the same page (potentially requiring users to hold information in memory)? ..................... 39

Are there any other issues related to the user’s cognitive load?.................................... 39

Language.............................................................................................................................. 40

Are there any grammatical errors (may impair readability)? .......................................... 40

Is there any mis-spelling or non-Australian spelling (may impair readability)? .............. 40

Are any expressions or instructions less clear and/or more jargonistic than they could be? ................................................................................................................................... 40

Are any abbreviations or acronyms used unnecessarily (assuming they may be less transparent than not using the abbreviation or acronym)? ........................................... 41

Are there any other language issues? ............................................................................. 42

General ................................................................................................................................ 43

Please use this section to provide general comments on form usability and also address issues that are not covered by any of the design heuristics above................................. 43

ACTIONS AND ALERTS.............................................................................................................. 46

Identification and demographics ......................................................................................... 46

Is the prescriber’s role less transparent than it could be? .............................................. 46

Is the administrant’s role less transparent than it could be? .......................................... 47

Are there any other issues with this action system?....................................................... 47

BGL frequency and notification instructions ....................................................................... 48

For this action system, has the form failed to make full use of colour to improve its usability (if appropriate)? ................................................................................................ 48

For this action system, does it involve mentally transcribing information between non-adjacent areas of the form (which may increase mental workload and hence the scope for errors)?....................................................................................................................... 48

For this action system, is its use less transparent than it could be? ............................... 49

For this action system, does the form fail to include instructions? (Users should ideally not have to refer elsewhere, to minimize errors resulting from inappropriate use.) .... 50


Additional instructions ........................................................................................................ 52

Is the prescriber’s role less transparent than it could be? .............................................. 52

Is the administrant’s role less transparent than it could be? .......................................... 52


Documenting diet ................................................................................................................ 53


5





Documenting BGL ................................................................................................................ 56



For this action system, does it fail to allow for modification of relevant values for individual patients where modifications may be required? ............................................ 56



Is information displayed as a graph or in a graph-like presentation assuming it was judged that this might reduce errors?............................................................................. 58

Does the graph look too small or cramped? ................................................................... 58

Is the graph label not clear or descriptive? ..................................................................... 58

Is the graph label written vertically with upright letters? ............................................... 59

Does the graph label fail to specify units of measurement? ........................................... 59

Does the graph label fail to provide an example of how data is to be recorded (e.g. numeric value, or ●)? ...................................................................................................... 59

Is the graph label font/formatting exactly the same as the Y-axis values font? ............. 59

Is the vertical axis NOT labelled on the left & right of the page? ................................... 59

Does the scale of Y-axis values change (e.g. going from .5 intervals to 1 intervals for the same length of the Y-axis)? ............................................................................................. 59

Are the Y-axis values not mutually exclusive (e.g. "5-10", "≤5")? ................................... 60

Are thick vertical lines NOT placed every 3-4 boxes? ..................................................... 60

Are the date boxes unnecessarily small (should accommodate size 14 text)? ............... 60

Are the time boxes unnecessarily small (should accommodate size 14 text)? ............... 60


Responding to abnormal BGLs ............................................................................................ 62

For this alerting system, has the form failed to make full use of colour to improve its usability (if appropriate)? ................................................................................................ 62

For this alerting system, does it involve mentally transcribing information between non-adjacent areas of the form (which may increase mental workload and hence the scope for errors)?....................................................................................................................... 63

6

For this alerting system, does it fail to allow for modification for individual patients where modifications may be required? .......................................................................... 64

For this alerting system, is its use less transparent than it could be? ............................. 65

For this alerting system, does the chart fail to include instructions? (Users should ideally not have to refer elsewhere, to minimize errors resulting from inappropriate use.) .... 66

Are there any other issues with this alerting system? .................................................... 66

Ordering routine insulin ...................................................................................................... 67



For this action system, does it fail to allow for modification of relevant values for individual patients where modifications may be required? ............................................ 68



Are any of the boxes for Name of Insulin too small? ...................................................... 70


Ordering supplemental insulin and determining the dose ................................................. 72



For this action system, does it fail to allow for modification of relevant values for individual patients, where modifications may be required? ........................................... 73





Ordering stat/phone insulin ................................................................................................ 76





7




Administer insulin and document ....................................................................................... 78








Ceasing orders ..................................................................................................................... 81







General ................................................................................................................................ 82

Does the form fail to include an action system where such a system would be of value? ......................................................................................................................................... 82

Does the form fail to include an alerting system where such a system would be of value? .............................................................................................................................. 82

Conclusion ............................................................................................................................... 83

8

Project overview

This report describes a research project to design and evaluate a new subcutaneous insulin form for potential national use, funded by and conducted in collaboration with the Australian Commission for Safety and Quality in Health Care. The development process for the new chart was informed by:

Simulation prototyping of potential high-level structures for the new form.

Engagement with multiple stakeholders, including the National Insulin Form Pilot Advisory Group.

Outcomes from heuristic evaluations of the pre-existing National Pilot Subcutaneous Insulin Form and a draft version of the new form (the current project also included the development of a new custom heuristic evaluation structure specifically tailored for insulin forms).

In this report, we first introduce the new form and describe the human factors design rationales that have informed its development.

Next, we present the results of the heuristic evaluations that we conducted on the previous National Pilot Subcutaneous Insulin Form and a draft of the new form. The evaluators judged overall that the new form was substantially safer than the previous form.

The new subcutaneous insulin form

The new subcutaneous insulin form is shown on the following two pages (the form is intended to fill both sides of an A3 printed page). The design process involved first reviewing the heuristic evaluation of the pre-existing National Pilot Subcutaneous Insulin Chart, as well as relevant previous research. We then generated mock-ups of potential new charts, designed to illustrate alternative options for the general layout of the form. These alternative prototype designs were subjected to informal micro-simulation testing with a senior diabetes educator. Next, an initial complete draft of the chart was created using Adobe InDesign CS6 (later modifications used Adobe InDesign CC), which was circulated to the development team for further discussion and comments. Changes were agreed and the draft modified and then resubmitted for further discussion and comments. This process was repeated over fifty times before reaching the final version of the new chart. Some of these iterations also incorporated changes arising from stakeholder feedback and the heuristic evaluation. At each iterative stage, if an initial consensus could not be reached on a particular design decision, then alternative versions of the form were created to facilitate decision-making. In addition, we sought out feedback from multiple expert stakeholders, which also informed the final product.

11

Design rationales informing the new subcutaneous insulin form

In this section, we present examples of some of the design rationales underlying the proposed new insulin form. The form is a modified version of the previous National Pilot Subcutaneous Insulin Form. We only discuss design rationales that reflect modifications made from the previous National Pilot Form and do not explain the rationale for features (such as the use of coloured range rows to display BGL) that were already present in the previous form.

Rationales were informed by:

(1) Empirical evidence from research carried out in our laboratory (e.g. Petersen 2012).

(2) Our previous Australian Commission for Safety and Quality in Health Care research on insulin charts (Christofidis et al., 2011), which included a partial task analysis of tasks involving insulin charts and a heuristic evaluation of a range of existing insulin charts.

(3) Prototype micro-simulation testing conducted as part of the current project.

(4) A new heuristic evaluation of the pre-existing National Pilot Subcutaneous Insulin Form conducted as part of the current project.

(5) Other Commission research on the National Pilot Subcutaneous Insulin Form (e.g. the National Subcutaneous Insulin Form Pilot Audit Report 2014 and feedback from individual pilot sites).

(6) Consultation with stakeholders.

12

1. Supplemental insulin orders have been aligned with the BGL Monitoring area

Experimental behavioural data indicates that aligning supplemental insulin orders with the BGL monitoring area results in fewer errors and faster determination of correct insulin doses (Petersen 2012). Note that we maintained the five-day recording period of the Pilot Form when implementing this change (one objection when we proposed this change previously was that it might require reducing the recording period to 4 days per form because of space limitations).

We anticipate that this change may have effects over and above those described above. For instance, it addresses the issue that supplementary insulin orders on the National Pilot Chart are presented in the opposite order to how BGL is charted in the monitoring area (i.e. lower BGL at the top, higher BGL at the bottom vs. higher BGL at the top, lower BGL at the bottom). That is, with the National Pilot Chart, users are required to mentally invert the information to determine the amount of supplemental insulin to administer. In the new form, both the insulin order and the BGL monitoring record are presented in a consistent sequence (higher BGL at the top, lower BGL at the bottom), resulting in reduced cognitive workload for nurses administering insulin.

Concern was expressed by one stakeholder site that the light blue border around the Supplemental Insulin Orders makes this section more prominent than the Routine Insulin Orders. In an attempt to address this concern, we created an alternate version of the chart in which we coloured the Routine Insulin Orders section header (as well as the associated instruction box and arrows) in green – and this did indeed have the effect of making the Routine Insulin Orders section more dominant on the page (see the illustration below).

However, this change comes at the cost of drawing attention away from other, arguably more safety-critical elements of the chart (such as the alerts section). In the final version of the chart, the use of colour on Pages 3 and 4 is reserved for situations associated with

13

abnormal BGL readings (including the administration of supplemental insulin). This makes it very noticeable when the patient is experiencing difficulties. We argue that also applying a colour to the Routine Insulin Orders would undermine this. It is a case where the over-use of highlighting reduces its overall effectiveness.

To address the possibility that inexperienced prescribers might ignore the Routine Insulin Orders section and instead prescribe only supplemental, we have changed the labelling and emphasis in the Supplemental Insulin Orders section (see illustration at right). The first instruction now reads “Supplemental insulin should NOT be prescribed for all patients.” The wording of the order itself also emphasizes that supplemental insulin is administered in addition to routine insulin. It now reads: “At the following intervals… administer additional insulin as specified below (dose depends on current BGL range row)”. Note that we included an acronym alongside one of the tick boxes associated with this instruction: “With meals only (unless NBM)”. While the general rule is to avoid acronyms where possible, in this particular case, we judged that the clutter introduced by writing out “nil by mouth” in full offset any potential ambiguity regarding interpretation of the acronym. In this instance, the context provides additional cues as to what NBM means. That is, it is referring to a situation when giving insulin with meals is not applicable and the non-abbreviated term “Nil by mouth” is written out in the corresponding diet record. Also, it is likely that a nil by mouth patient would be flagged in other ways (e.g. a sign above their bed).

Beyond this, we suggest that ensuring that inexperienced prescribers do not overlook routine insulin requirements is an issue better addressed through training.

Modifications to BGL ranges

Senior clinicians may wish to modify the BGL ranges associated with supplemental insulin. To do this in the new chart, they would cross out the current range and write the new range below it (within the same cell, see illustration below) in both the left and right hand label columns on the BGL monitoring record.

This solution means that the colours for ranges will remain consistent (unlike the modification strategy for the previous National Pilot Subcutaneous Insulin Form – where a modification results in inconsistency in the meaning of the colours between the Supplemental Insulin Orders and BGL record ranges, leading to an increased chance of error). We deliberately did not provide instructions for the modification procedure on the form itself because modifications should only be attempted by highly experienced senior clinicians (because modifications introduce new risks).

14

Instead, we recommend that the modification procedure should be described in the user guide for the new form.

2. Routine insulin orders have been aligned with the daily BGL monitoring and insulin administration areas

Experimental behavioural data indicates that aligning the routine and BGL monitoring sections results in significantly faster determination of correct insulin doses (Petersen 2012). The benefits observed were thought to be because this change reduces the cognitive workload involved in matching orders with both the BGL record and Administration Record.

In addition to reducing task time for determining the insulin dose, this change may reduce prescribing errors. For example, one typical omission involves a prescriber failing to order insulin for an upcoming time/day. The alignment of orders with the BGL record should make it more obvious when upcoming orders are missing.

We conducted prototype micro-simulation testing to determine whether the routine insulin order section should be placed underneath the Administration Record or above the BGL record. The former was preferred for the following reasons:

• For a left-handed prescriber (a substantial proportion of the health workforce is left-handed) the BGL record itself would be obscured by their writing hand when they complete a routine insulin order that is placed ABOVE the BGL record. For this option, it means that BGL trends would not be visible while writing the routine order, potentially reducing the likelihood that they would be taken into account.

• Placing routine orders BELOW the Administration Record should NOT introduce problems for a nurse recording the administration of a prescribed routine dose as their writing hand (whether they were left or right-handed) would not conceal the current day’s routine order (it would either cover the preceding or following day’s order, depending on handedness). That is, they should be able to transcribe the order directly without having to move their hand.

• In cases where the nurse must determine the appropriate doses for both routine and supplemental insulin, placing the Routine Insulin Orders BELOW the Administration

15

Record (rather than above the Monitoring Record) minimizes the travel distance of their gaze fixations around the chart for some sequences in which they may attend to the different chart elements, thus potentially reducing the risk of error.

• Note that the Routine Insulin Orders also could not be positioned immediately beneath the BGL record (i.e., above the Administration Record) because this would break the continuity of the time columns, and there is also value in the Administration Record (especially the past supplemental doses) being clearly visible to prescribers (along with the BGL values) when ordering insulin.

What to do when more than 6 BGL readings are anticipated for a single day

One issue that arises from aligning routine insulin orders with BGL readings is that it is

less clear how to deal with a situation in which a patient requires more than 6 BGL

readings in a single day. We would suggest that, in these cases, two or more daily

columns be combined, as illustrated in the following examples, in order to maintain the

alignment between routine insulin orders and BGL readings.

Up to 12 BGLs anticipated during a single day:

16

Up to 18 BGLs anticipated during a single day:

What to do if the routine insulin orders do not coincide with a set meal

time (e.g. the patient is nil by mouth)

We have ensured that the mealtime labels in the routine insulin section (breakfast, lunch, dinner) are formatted so that there is room to cross out the label and write a time next to it (as illustrated below). We have also changed the colour of these labels to dark grey (from black on the original form) to make them slightly less dominant, while still satisfying feedback on previous insulin forms that these labels should not become too unobtrusive.

17

3. Stat/Phone orders have been moved to the top of the page

Given the possibility of users overlooking entries in the Stat/Phone Order section (because they should occur at a much lower frequency than Routine Insulin Orders), this has been moved to a more conspicuous location at the top of the page in the new chart.

4. Stat/Phone orders now include confirmation of whether they replace an existing order and whether the entry is a phone order

Given the potential ambiguity over whether a stat/phone order is intended to replace or is in addition to an existing order, tick boxes have been added to capture this information. One potential issue is what users should do when there is no existing order. In such cases, the question may be regarded as ambiguous by some users. However, we suggest that, in this situation, there are no safety implications regardless of what is ticked (e.g. if there is no order then the new order could be regarded as replacing a non-existent order, in addition to it, or as neither – but the outcome would be the same from the patient’s perspective). The order type (stat or phone) is also presented as a tick box selection in the new chart.

5. Better perceptual separation of form elements

Overall, we have designed the new form to allow better perceptual separation of relevant elements, with strategies such as inserting more space between elements and using colour to group closely related items (e.g. blue borders and fills have been introduced to perceptually group all of the components involved with supplemental insulin). This should reduce the user’s cognitive workload by making it easier to perceive and understand the structure of the form, which ought to reduce errors and task times.

18

6. Increased use of tick boxes where appropriate

When options are unlikely to vary beyond a few standard possibilities, we have used tick boxes in place of text boxes. This should reduce errors by preventing problems associated with difficult-to-read handwriting and ambiguous information. As well as making communication less ambiguous and error-prone, the use of tick boxes should also reduce the time required to record information. As an example of where we have introduced tick boxes in the new form, we have included options for “Diet”; namely, “Full”, “Nil by mouth”, “TPN”, “Clear fluids”, and “Other”. Also, “BGL frequency” has new checkbox options (“pre-meals” and “21:00 hrs”) in addition to those already present. Note that, in the case of BGL frequency, the inclusion of these two new specific options negates the need for a defined “standard/default” option, which has been argued to be problematic (as documented in the heuristic evaluation outcomes below). Instead, the alternatives relating to the “standard/default” option (i.e. “pre-meals” and “21:00 hrs”) are now pre-ticked to reflect their default status.

7. Checkbox to confirm NIMC has been checked

Commission research has indicated that insulin omissions have occurred on discharge because the insulin order is not on the National Inpatient Medication Chart (NIMC). This might partly be because users sometimes fail to indicate on the NIMC that a separate insulin form is in use.

In an attempt to mitigate this issue, we have included a tick box below the patient ID label for prescribers to indicate that the NIMC has been marked. This is (1) to provide an active prompt for the user to cross-reference and mark the NIMC and (2) to provide a record that this has been done.

We have received some feedback suggesting that the “ID label has been checked” box is redundant if the patient name has been entered. Our argument for its inclusion would be that the prescriber should be checking more than just the patient’s name (given that two patients may have the same name); hence, simply transcribing the patient name does not necessarily indicate that the ID label has been checked adequately.

19

8. More information is included on how to complete the form

Where appropriate, we have included information on the form as to how it should be completed. Examples include the following:

For the tick boxes, we have included the instruction “tick to select; cross out to cancel”.

For time records, we have included the label “24 hr” to clarify what time format to use (where 24 hr and am/pm time were used inconsistently in the previous National Pilot Subcutaneous Insulin Form).

For date records, we have included water-marked DD/MM/YY labels within each cell to indicate that the day/month format should be used.

9. Instructions on routines orders clarified

For routine insulin orders, we included an instruction indicating that they should not be ordered more than 4 meals in advance.

This instruction was designed to reduce the chance that prescribers would either write orders too far in advance (where the patient’s situation might change) or not far enough in advance (e.g. the prescriber completes one day’s worth of orders in advance but omits the following morning’s order, which is likely to be required before they next visit the patient).

We also added an instruction, “nurses must consult doctor if expected dose is not ordered”, designed to create an incentive for doctors to act appropriately (to avoid being contacted by nurses following this instruction).

20

10. Hypoglycaemia Management Flowchart re-formatted

We have re-formatted elements of the hypoglycaemia management flowchart to improve usability. Changes include:

Colouring elements of the “unconscious/uncooperative” pathway in emergency response purple (in addition to thickening the arrows and using bigger arrowheads) in order to make it stand out from the other pathways, given that this procedure involves the most urgent situation.

Increasing separation between the flowchart elements and the carbohydrate table to avoid the table being perceptually grouped with the flowchart (and hence reducing the readability of the flowchart).

Adding colour and formatting cues to perceptually link the instructions related to giving fast or slow-acting carbohydrates with the lists of fast and slow carbohydrates.

Merging sequential flowchart boxes that did not contain sequential instructions (where the risk was that users might complete the instruction in the first box before reviewing the next box).

Using formatting to emphasize “and” and “or” decision points.

21

11. Hyperglycaemia Management – Table 1

We have reformatted Table 1 to improve usability. For example:

We colour-coded the BGL ranges and presented them in the same order as in other locations in the form.

We have clarified the two different decision paths (“previously on insulin” vs. “not previously on insulin”).

We have presented the two options for determining the doses in column format (this is to make it less likely that a user would interpret the first option as being what

to do first).

12. Form title is now included on Page 4

We included the form title in the vertical left margin of Page 4 to address concerns arising from previous research that the form might be misidentified if users only viewed Page 4 (which contains the hypoglycaemia treatment flowchart).

We also formatted the titles for the two treatment flowcharts to match other subheadings to make it clearer that these titles did not refer to the name of the form.

22

Sources cited in design rationales

1. Christofidis, M.J., Horswill, M.S., Hill, A., McKimmie, B.M., Visser, T., & Watson, M. O. (2012). Task Analysis and Heuristic Analysis of Insulin Charts. Final report prepared for the Australian Commission on Safety and Quality in Health Care.

2. Medication Services Queensland, Safe and Quality Use of Medicines Unit (June 2012). Register in response to Heuristic Analysis conducted by the School of Psychology, University of Queensland, on the Subcutaneous Order and Blood Glucose Record Form.

3. Australian Commission on Safety and Quality in Health Care (March 2014). National Subcutaneous Insulin Form Pilot Audit Report (Draft).

4. Feedback from individual pilot sites.

5. Petersen, L.D. (2012). The design of subcutaneous insulin charts affects calculation of patients’ insulin dose (Honours thesis).

23

Development of a new heuristic evaluation protocol

We developed a new heuristic evaluation protocol specifically tailored for insulin forms. This framework of design principles and issues was used by the heuristic evaluation panel to guide their assessments of the forms. We originally created such a protocol for our previous work for the Commission on insulin forms – but we took the opportunity to make substantial revisions for the current project. The new version is informed by research carried out since the original project. We also redeveloped the other materials for the heuristic evaluators (for example, insulin form protocols both from the prescriber’s and the administrant’s perspective, heuristic evaluator instructions, font guide, and a list of prescribed insulins).

24

Heuristic evaluations of the existing National Pilot Subcutaneous Insulin Form and the new draft form

We carried out heuristic evaluations of both the existing National Pilot Subcutaneous Insulin Form (“previous insulin form”) and a draft of the new form (“new insulin form”). The previous form was analysed by six evaluators all of whom had substantial human factors expertise (including two associate professors in human factors, a senior research fellow in human factors, a medical student currently completing a human factors PhD, a senior diabetes nurse educator, and a student currently undertaking a PhD on insulin form design). The aim of this evaluation was to inform the design process of the new insulin form. The new insulin form was evaluated by five of the six evaluators (one evaluator, the medical student completing a human factors PhD, was unavailable) when the design had reached a late draft in order to determine (1) whether it was, overall, considered to be superior to the previous form and (2) whether there were any further modifications that ought to be considered to further improve the form.

The previous insulin form is presented below:

25

The outcomes of both heuristic evaluations are summarized below, where comments about the forms are presented side-by-side for ease of comparison. Please note that the following should not be regarded as a list of recommendations, but merely a précis of the comments and suggestions raised by the evaluators.

26

GENERAL

Page Layout

Is non-safety-critical information displayed in an unnecessarily prominent way, such that it could potentially draw attention away from safety-critical information and/or increase visual clutter (e.g. hospital logo or name, administrative codes, etc.)?

Previous insulin form New insulin form

From the outside pages, the prominent Management Guidelines may cause unfamiliar users to miss the administrative purpose of the form.

On the inside pages, the most salient feature is the instruction to “...enter code (W) for withheld…” where the use of red font makes the instruction stand out above the alerts. It is questionable whether this instruction needs to be so prominent (e.g., could a colour other than red be used?).

Nevertheless, the form contains no superfluous non-safety information.

From the outside pages, the prominent Management Guidelines may cause unfamiliar users to miss the administrative purpose of the form.

Nevertheless, the form contains no superfluous non-safety information.

27

Is there a mixture of landscape & portrait-orientated information that may plausibly reduce readability (e.g. potentially requiring the form to be turned during use)?


On the outside front page, the form title is orientated differently in the margin, but this is conventional.

On the inside pages, the only information that is orientated differently is of very low importance (e.g., administrative codes, licensing information), and therefore not problematic.

On the outside pages, the form title is orientated differently in the margin. While this is conventional, there remains the possibility that users may only see Page 4 when they are searching for this form – and not notice the main form title (instead assuming the form to be guidelines for managing hypoglycaemia alerts). However, this was considered to be a necessary compromise (where it was unlikely to be a significant problem especially once the form was well established).

On the inside pages, the only information that is orientated differently is of very low importance (e.g., administrative codes, licensing information), and therefore not problematic.

Are there any other page layout issues?


The design is not optimized for any of the user groups.

Sections relevant to prescribers are not grouped together. The only practical way to achieve this by modifying the current design would be to move the Routine Insulin Orders up the page (between the Stat/Phone Orders) and the Monitoring Record. However, this would mean that left-handed prescribers would be covering the patient’s previous BGLs while completing the Routine Insulin Orders. It would also make the “Doctor to Notify” and “Special Instructions” boxes less prominent. Furthermore, alignment of the Routine Insulin Orders with days and Supplemental Insulin Orders with BGLs better supports the nurse’s task, and positioning of the Stat/Phone Orders at the top of the page may make these orders less likely to be missed in wards where they are a low-frequency occurrence.

28

Information Layout

Is there a missing or ambiguous form name?


The form name itself is somewhat ambiguous in that it may not clearly reflect the purpose of the form: the name only pertains to two of the form’s three major functions (i.e., ordering insulin and recording blood glucose levels).

This terminology was chosen to enable users to find and order the form on a catalogue system.

Although adding the final purpose to the form name (i.e., administering insulin) may address some concerns raised by previous research that the form could be missed by staff, this addition is likely to be cumbersome. One alternative suggested title (raised in previous research) “Insulin administration form” fails to differentiate between subcutaneous and IV administration (where IV requires a different form).

Also it is not clear from the name that the form also contains guidelines for managing hyper- and hypoglycaemia.

On the outside back page, the form name is missing. Consequently, the “Hypoglycemia Management…” heading may be wrongly viewed as the form name. This is consistent with concerns raised in previous research that the form may be missed (when it is being searched for) because the main functions of the form may not clear on a cursory viewing of these outside pages (though this issue might occur less frequently if the form goes into general usage as a national standard and people become familiar with it).

The form name itself is somewhat ambiguous in that it may not clearly reflect the purpose of the form: the name only pertains to two of the form’s three major functions (i.e., ordering insulin and recording blood glucose levels).

This terminology was chosen to enable users to find and order the form on a catalogue system.

Although adding the final purpose to the form name (i.e., administering insulin) may address some concerns raised by previous research that the form could be missed by staff, this addition is likely to be cumbersome. One alternative suggested title (raised in previous research) “Insulin administration form” fails to differentiate between subcutaneous and IV administration (where IV requires a different form).

Also it is not clear from the name that the form also contains guidelines for managing hyper- and hypoglycaemia. However, it was not considered not practical to include these elements in the form title as well.

29

Does the form fail to organise information in order of importance (so that attention is naturally drawn to the most important elements first, and the least important elements last)? (e.g. blood glucose level before insulin administration.)


The flow and perceptual grouping of information is not ideal. For example, the insulin orders are positioned towards the bottom of the page, unaligned with the Monitoring Record and Administration Record.

A potential solution is to align the Routine and Subcutaneous Insulin Orders with the Monitoring Record, where the Routine Insulin Orders are aligned with the day columns of the Monitoring Record (as they are contingent on time), and the Subcutaneous Insulin Orders are aligned with the BGL ranges (as they are contingent on BGL level). This solution is supported by empirical evidence, where novice participants were significantly faster at calculating the total dose of basal insulin required when Routine Insulin Orders were aligned with the Monitoring Record (vs. not aligned). This design strategy may also make the form more readily understandable and ergonomic.

The distance between the insulin orders and the Administration Record may also lead to column or row shift, given that the columns do not align.

However, from a doctor’s perspective, the priority position of the Monitoring/Notification Instructions box is logical. Similarly, for a nurse’s role, the position of the Monitoring Record relative to the insulin orders is appropriate for determining the required doses and seeking dose modifications based on BGLs.

The alerts are the most prominent feature (brightest red) followed by the BGL alert ranges. While supplemental insulin is next most prominent (being highlighted with the blue border), this is still associated with an BGLs that are abnormal (or at least approaching abnormality)

It could be argued that it is inappropriate for supplemental insulin to appear more prominent than routine insulin. However, it was argued that this was nonetheless the best solution (see Design Rationales for further discussion of this issue).

Aligning the routine insulin orders with the BGL record helps to draw attention down to them.

Arguably, the Stat/Phone Orders are not the “most important” information (in terms of placing them at the top of the form); however, the need to ensure that they are not overlooked (given that they should be used less frequently than routine orders) outweighs this consideration.

From a doctor’s perspective, the priority position of the Doctor to Notify and Special Instructions boxes is logical. Similarly, for a nurse’s role, the position of the Monitoring Record relative to the insulin orders is appropriate for determining the required doses and seeking dose modifications based on BGLs.

30

Are any elements that ought to be perceptually separated not adequately separated (e.g. by a small space or double line)?


The Routine, Supplemental and Stat/Phone Orders are not adequately separated from one another and appear perceptually joined (in an arbitrary way). A potential solution is to provide a small space between each order. This may also assist users’ understanding of the form by making it clear these are three separate “information units”.

Within the Monitoring Record: o the legend “BGL (mmol/L)…”

should be separated from the row of time boxes;

o the vertical line to the left of Day 1 Time 1 should be bold all along its length; and

o the vertical line after day 5 time 6 should be bold all along its length.

Within the Administration Record, the grey arrows (for nurse instructions) should not pass through the box containing legends (e.g., “Comments”) as this may cause confusion.

There are also inconsistencies in line thicknesses across the form that could be resolved (e.g., the thick line underneath the Supplemental Insulin Order instructions).

If the Routine Insulin Order remains in its current form, thicker lines could be used around each “Meal/time” row to better perceptually separate the Meal/time based orders.

There is also disconnected flow between the Supplemental Insulin Orders and the instructions (disconnected right-to-left and bottom-to-top flow).

The Pharmacy review and Diabetes treatment prior to admission boxes appear to be joined (despite the fact that they ought to be perceptually separated).

There is no clear perceptual distinction between supplemental and routine insulin in the Insulin administration box,

The “date” label within the routine box is placed in the same cell as “name of insulin”. [This issue has been addressed in the final draft of the chart by converting the date label box into an arrow shape to provide an additional cue to information grouping.]

31

which may lead to row shift errors when documenting the different types of insulin administered to patients (this, in turn, may be problematic for prescribers, who may use this information to alter a patient’s insulin orders).

Are headings/labels of the same level of importance formatted differently, potentially providing misleading cues?


The headings for the Hypo- and Hyper-glycaemia Management Guidelines on the outside pages are more prominent than all of the other major headings on the form, including those that are more important for the core task information (e.g., the form name on the front page).

In the Supplemental Insulin Orders, the “Start date” and “Start time” are as important as the other information (i.e., BGL and insulin dose) because these three pieces of information have to be interpreted jointly. Given that there has been some confusion about the fact that these are start dates (as opposed to administration dates), then these headings (or at least the word “start” within them) arguably also warrant bold text.

The dark band used to separate the Administration Record from the Monitoring Record could be made thicker so that it is consistent with all the other equivalent headings, although this may compromise space.

A potential suggestion is to use smaller font for the headings “Diabetes treatment prior to admission” and “Pharmacy Review” so they are formatted differently from the other major form headings (however, their position towards the bottom right side of the form may already satisfactorily signal their lower priority).

A potential suggestion is to use smaller font for the headings “Diabetes treatment prior to admission” and “Pharmacy Review” so they are formatted differently from the other major form headings.

However, the new position of these sections towards the bottom right of the form was considered to satisfactorily signal their lower priority. In addition, diabetes treatment prior to admission and pharmacy review are now separated by a substantial white space. This was also regarded as having the effect of appropriately reducing their prominence. [In the light of these prominence-reducing factors, it was not considered necessary to action the suggestion of reducing the title font.]

“Diabetes treatment prior to admission” – the first letters of this title are not capitalized (i.e. in title case); whereas other section titles are in title case. [This point has been actioned in the final draft.]

32

Are any of the boxes for writing something (e.g. Diet, Comments) too small (size 14 font is considered to be a minimum handwriting size guideline)?


Prescriber name boxes in the Routine, Supplemental and Stat/Phone Orders may not be suitable for long names.

Other areas include the: o “Other……” spaces within the

Monitoring Record; o “Comments” boxes in the

Administration Record; o “Time” cells (may require writing

on the diagonal or over 2 lines to achieve size 14 font);

o “Nurse 1/2 initials” diagonally split cells;

o “Date” cells in the Supplemental Insulin Orders and Pharmacy Review

o “Date/time of dose” cells in the stat/phone insulin orders box.

Prescriber name boxes in the Routine, and Stat/Phone Orders may not be suitable for long names. Note that there is some space available to increase the length of this box in the Stat/Phone orders section while still maintaining adequate separation from the Supplemental Insulin Orders section. [This point has been actioned for the final draft.]

Other areas include the: o “Other……” spaces within the

Monitoring Record (but noting that the length of the line has been increased significantly from the pilot form);

o “Comments” boxes in the Administration Record;

o “Time” cells o “Nurse 1/2 initials” diagonally

split cells; o “Date” cells in the Supplemental

Insulin Orders. o The “name of insulin” box in the

Routine Insulin Orders is slightly narrow than before, though this was not considered problematic for commonly-prescribed insulins).

33

Is there an unnecessarily large amount of space devoted to any particular element, potentially introducing unneeded clutter?


The number of rows within the Stat/Phone Insulin Orders is questionable.

The boxed instructions that relate the Routine and Supplemental insulin Orders to the Administration Record may be somewhat oversized to fill the available space.

A potential solution to reduce overall clutter is to remove the need for doctors’ printed names and signatures (even for the initial order) if this information is not required. It is worth noting that Queensland Health have indicated that pharmacy departments should keep a log of all prescribers on record and that, because it is a written order (not a “prescription” within the meaning of the Health, Drugs, and Poisons Act (Qld)), there is no legislative requirement to provide a the Medical Officer’s name in print. If it is not feasible to remove the name and signature altogether, then one suggestion is that, because this information is not safety-critical, we REDUCE the size of the name and signature box and move it away from the more safety critical elements (e.g. insulin name, dose). That is, we contain the non-safety-critical clutter to minimize its impact on the safety-critical form elements.

As mentioned previously, the spaces provided for the Hypo- and Hyper-glycaemia Management Guidelines headings on the outside pages are unnecessarily large.

No issues identified.

Has the form failed to take advantage of the potential benefits of using a graph or graph-like presentation of quantitative data?


No issues identified. No issues identified.

34

Is basic functionality not understandable intuitively (or with minimal explanation)?


The form functionality is complex and requires training. If form re-design cannot reduce the subjective experience of feeling overwhelmed (as reported by nurses in previous research), then the instructions themselves may benefit from a careful re-design.

The areas of the form that prescribers need to attend to have not been flagged specifically. In a new design, it might be possible to direct their attention to the relevant areas.

There is also no consistency in flow in the Hypo- and Hyper-glycaemia Management Guidelines.

Some functions may be better without instructions, if they should only be performed by sufficiently skilled staff (such as adjusting BGL ranges, where novice prescribers should not be encouraged to make adjustments).

The form functionality is complex and requires training. Since the subjective experience of initially feeling overwhelmed may remain for some users, the instructions themselves may benefit from a careful re-design.

The areas of the form that prescribers need to attend to have not been flagged specifically. However, it was considered that the design compromises necessary to achieve this outweighed the potential benefits.

Some functions may be better without instructions, if they should only be performed by sufficiently skilled staff (such as adjusting BGL ranges, where novice prescribers should not be encouraged to make adjustments).

Are there any other information layout issues?


The labels for insulin in the Administration Record (e.g., “Name of routine insulin”) seem odd because they also act as the row label. A potential solution could be to present these labels as (with an instruction for the nurse to print the name):

o “Routine Insulin 1 (print name)” o “Routine Insulin 2 (print name)”

etc. Or alternatively:

o “Routine Insulin 1 …………………..” o “Routine Insulin 2 …………………..”

etc.

However, these options also have costs in terms of space and clutter.

The labels for insulin in the Administration Record (e.g., “Name of routine insulin”) seem odd because they also act as the row label. A potential solution could be to present these labels as (with an instruction for the nurse to print the name):

o “Routine Insulin 1 (print name)” o “Routine Insulin 2 (print name)”

etc. Or alternatively:

o “Routine Insulin 1 …………………..” o “Routine Insulin 2 …………………..”

etc.

However, these options also have costs in terms of space and clutter. [This suggestion was therefore not actioned in the final draft.]

35

Font

Is there a problem with compressed font (e.g. Arial narrow) being used anywhere, given this may affect legibility?


No issues identified. “Breakfast” label has slightly compressed font – but this could be considered a necessary compromise to allow room to cross it out and write a time if needed. However, this was not believed to affect readability (only the evaluator who actually introduced this font noted this issue).

Is the font too small (smaller than 11 point), such that it might impede legibility?


Very small fonts are used in various places on both sides of the form; however, this may be unavoidable and may not be an important issue in some instances (e.g., where the label is predictable, units/initials).

Very small fonts are used in various places on both sides of the form; however, this may be unavoidable and may not be an important issue in some instances (e.g., where the label is predictable, units/initials).

Is font size potentially giving misleading cues (e.g. important information is in small font and unimportant information in large font)?


As mentioned previously, the font size for the Hypo- and Hyper-glycaemia Management Guidelines headings on the outside pages are unnecessarily large.


Is capitalization used too often (capitalization is good for highlighting elements but only if used sparingly; also capitalization slows reading)?


Capitalization is used sparingly when appropriate (e.g. “NOT”, “ALERTS”).

Although “DO NOT WRITE IN THIS BINDING MARGIN” is also capitalized, this is unlikely to have safety consequences.

Capitalization is used sparingly when appropriate (e.g. “NOT”, “ALERTS”).

Although “DO NOT WRITE IN THIS BINDING MARGIN” is also capitalized, this is unlikely to have safety consequences.

36

Are there any other issues with fonts?


Overall, there are instances of odd uses of title case, inconsistent uses of bold, and (arguably) inappropriate uses of red font.


37

Colour Scheme

Has the form failed to use colour where it would have been advantageous to do so (e.g. to draw attention to important elements, to aid perceptual organisation, or as part of an alerting system)?


One possibility is that colour could be used to indicate which sections of the form are for doctors and nurses. For example, two different colours (for doctors and nurses, respectively) could be used to draw attention to the relevant elements of the form (e.g., with box-and-arrow instructions, where the colour of the boxes and arrows indicates the target audience). However, this is not straightforward given that some areas are shared.

This approach could also be used in the instructions on the outside pages, where colour is poorly applied.

Alternatively, coloured font could be used to increase the salience of the Routine Orders instruction “must be ordered for each day”, and the Supplemental Orders instruction, “valid until changed or ceased”. An endocrinologist has reported regularly shading the Supplemental Orders instruction with a yellow highlighter as a signal to other form users.

In Table 1 (on page 1), colour could be used to reinforce the correspondence between the BGL ranges here and in the Supplemental Insulin Orders.

One possibility is that colour could be used to indicate which sections of the form are for doctors and nurses. For example, two different colours (for doctors and nurses, respectively) could be used to draw attention to the relevant elements of the form (e.g., with box-and-arrow instructions, where the colour of the boxes and arrows indicates the target audience). However, this is not straightforward given that some areas are shared.

Alternatively, coloured font could be used to increase the salience of the Routine Orders instruction “must be ordered for each day”, and the Supplemental Orders instruction, “valid until changed or ceased”.

However, both these strategies would undermine the use of colour to ONLY flag abnormal BGLs and related procedures. In addition, the redesign of the supplemental orders box has used alternative strategies to achieve the same end. [These suggestions were therefore not actioned in the final design.]

38

If colour is used, are the colour shades used appropriate? E.g. if a colour-shaded area is to be written on, is the shade sufficiently light ("pastel") for the writing to be clearly visible?


The colour shades used in the Hypo- and Hyper-glycaemia Management Guidelines are inappropriate.


If colour is used, are any of the colours used potentially deceptive in their meaning according to cultural norms (e.g. light blue used to indicate "bad"; red used to indicate "no problem")?


The red font used for the instruction to “...enter code (W) for withheld…” is potentially deceptive in meaning and confusing.


If colour is used as part of a system, is there redundancy for when colours cannot be perceived easily (e.g., colour-blind users or low-light conditions)?



Are there any other colour scheme issues?


No other issues identified. As the supplemental insulin box has a coloured border to perceptually link it to the Administration Record, this raises the issue as to whether the routine insulin box should also have a coloured border. Concern was expressed by one stakeholder site that the light blue border around the Supplemental Insulin Orders makes this section more prominent than the Routine Insulin Orders. [Note that this issue is discussed in the design rationale section, where this option was attempted but then rejected].

39

Does the form design minimise the user's cognitive load as much as possible?

Is writing required, where a better option might be to provide options to circle or similar (where circling options can reduce cognitive load and minimize handwriting legibility problems and inappropriate entries)?


The area for documenting a patient’s diet may benefit from tick-box or circling options (e.g., full, nil by mouth, total parenteral nutrition, clear fluids, other).


Does information have to be compared or transcribed over non-adjacent areas of the same page (potentially requiring users to hold information in memory)?


Most of the form requires the comparison or transcription of information over non-adjacent areas (e.g., insulin orders, BGL monitoring, insulin administration, management guidelines, or determining doses using Table 1).

For instance, monitoring instructions are recorded in two places, with the potential for transcription errors that cannot be distinguished from an intended change. A potential solution is to remove the general instructions and require boxes to be ticked for each day when routine insulin orders are reviewed.

Some form functions require comparison over non-adjacent areas (e.g., consulting management guidelines or using Table 1 to determine dosage).

Are there any other issues related to the user’s cognitive load?


Although the “units” labels help cue users to the appropriate cells to record insulin doses, they also clutter that area of the form.

Although the “units” labels help cue users to the appropriate cells to record insulin doses, they also clutter that area of the form.

40

Language

Are there any grammatical errors (may impair readability)?


Commas are used where semicolons or full stops would be more appropriate.

Full stops are used inconsistently in the Management Guidelines.

The sentence “Check with doctor if order replaces, or is in addition to, other insulin orders” is ambiguous, where a potential solution is to replace “if” to “whether”.

There is also some telegraphic writing (e.g., “If on insulin infusion, stop insulin infusion, continue glucose infusion”); however, the meaning is not ambiguous (and the reduced word count reduces visual clutter).

There is some telegraphic writing (e.g., “If on insulin infusion, stop insulin infusion and continue glucose infusion”); however, the meaning is not ambiguous (and the reduced word count reduces visual clutter).

Is there any mis-spelling or non-Australian spelling (may impair readability)?



Are any expressions or instructions less clear and/or more jargonistic than they could be?


The Management Guidelines and insulin ordering instructions are very unclear. These guidelines, and the form generally, may benefit from a review for plain English (e.g., “markedly labile” may not be understood by all users).

More specifically, the instruction “Patient is receiving food orally or by tube” may be better phrased as “Is the patient receiving food orally or by tube?” to be consistent with other decision points (and better phrased as question to prime it as decision point). However, it is arguably not a “decision point” because it is the last remaining alternative after the three prior decision points have already been negotiated.

The Management Guidelines may require further refinement.

For example, the instruction “Patient is receiving food orally or by tube” may be better phrased as “Is the patient receiving food orally or by tube?” to be consistent with other decision points (and better phrased as question to prime it as decision point). However, it is arguably not a “decision point” because it is the last remaining alternative after the three prior decision points have already been negotiated. [Hence, this suggestion was not actioned in the final draft.]

41

Are any abbreviations or acronyms used unnecessarily (assuming they may be less transparent than not using the abbreviation or acronym)?


Several acronyms are used in the Management Guidelines (e.g., DKA, RN, IV, IM, BGL); however, some of these might be considered high frequency enough uses (e.g. IV, BGL, RN) or contextually unambiguous (e.g., glucagen IM) that they are understandable to all relevant users. However, this assumption should be verified.

In contrast, although DKA is defined in full (diabetic ketoacidosis), the acronym is not used again. This is only useful if DKA is more likely to be understood by some form users than the full name.

In the patient identification label area, “I” is used as the abbreviation for “indeterminate”. However, the Australian Government Guidelines on the Recognition of Sex and Gender (2013) stipulate that “X” should be used as the abbreviation for “indeterminate/intersex/unspecified” when gender information is collected and recorded in a personal record. This is arguably less intuitive than “I” (although “I” is not particularly intuitive either). However, since there is not sufficient room for “indeterminate” (or “indeterminate/intersex/unspecified”) to be printed in full in the patient ID label area (the size of which is constrained), the use of an abbreviation is arguably necessary and it is probably preferable to comply with the guidelines.

The label “Hypo intervention” abbreviates “Hypoglycaemia” unnecessarily.

Several acronyms are used in the Management Guidelines (e.g., DKA, RN, IV, IM, BGL); however, some of these might be considered high frequency enough uses (e.g. IV, BGL, RN) or contextually unambiguous (e.g., glucagen IM) that they are understandable to all relevant users. However, this assumption should be verified.

In contrast, although DKA is defined in full (diabetic ketoacidosis), the acronym is not used again. This is only useful if DKA is more likely to be understood by some form users than the full name. [We left this acronym in the final draft in case this might be true for certain users.]

42

Could the position of clinicians’ initials required for authorisation cause confusion with close abbreviations or acronyms?


The bottom row of the Supplemental Insulin Orders table could potentially be confused for the “8.1-12 mmol/L” range (which also has a white background).

The prescriber initials in this row could also be confused with the insulin units that correspond to the “Greater than 20 mmol/L” row. For example, initials of B, D, I, L, O, S, Z, (e.g. IO, SO, OI, OS, IZ, ZI, SS, ZZ, BO) could be seen as numbers when handwritten (in a previous incident in Queensland, the initials SS were read as 55).

A potential solution to both these issues is to shade the row as per the other similar rows (i.e., those for prescriber signatures, names and initials).

While it is still possible to confuse initials with doses, this is less likely with the new design.

Are there any other language issues?


No other issues identified. No other issues identified.

43

General

Please use this section to provide general comments on form usability and also address issues that are not covered by any of the design heuristics above


It seems likely that no design will remove the need for training and educational resources. However, there is scope for improving the usability of the existing instructional materials – something we would recommend for a future project.

Although it is probably the case that having the treatment advice at hand is more likely to minimise risk, it might be worth considering the option to include form instructions in the outside form pages and move the Management Guidelines elsewhere (e.g., a separate sheet in the same folder). This option may be worth considering if a large of number of people do not understand how to use the form though it should also be noted that if the form is adopted as a national standard and becomes ubiquitous, this may become less of an issue. One advantage of having form instructions on the outside pages is that it may reduce the chances of users missing the form as the document is less likely to be mistaken for an information sheet on how to treat hypo- and hyperglycaemic events (as implied by previous research). Again it is probably the case that the advantages of having treatment advice close at hand outweighs this in terms of patient safety.

The Management Guidelines themselves are complicated, difficult to follow and likely to induce significant cognitive load. Specific issues are listed below.

o Hyperglycemia protocol : Not clear which user

group(s) this is aimed at, and how it fits with the alerts.

Items to assess may not be in the most logical order.

It seems likely that no design will remove the need for training and educational resources. However, there is scope for improving the usability of the existing instructional materials – something we would recommend for a future project.

Although it is probably the case that having the treatment advice at hand is more likely to minimise risk, it might be worth considering the option to include form instructions in the outside form pages and move the Management Guidelines elsewhere (e.g., a separate sheet in the same folder). This option may be worth considering if a large of number of people do not understand how to use the form though it should also be noted that if the form is adopted as a national standard and becomes ubiquitous, this may become less of an issue. One advantage of having form instructions on the outside pages is that it may reduce the chances of users missing the form as the document is less likely to be mistaken for an information sheet on how to treat hypo- and hyperglycaemic events (as implied by previous research). On balance, it was considered likely that the advantages of having treatment advice close at hand outweighs the availability of instructions in terms of patient safety.

The Management Guidelines themselves are complicated, difficult to follow and likely to induce significant cognitive load. Specific issues are listed below.

o Hyperglycemia protocol : Not clear which user

group(s) this is aimed at, and how it fits with the alerts.

Items to assess may not

44

o Hyperglycemia flow-diagram : Not clear where the

start-point is. However, it does conform with Western reading conventions (i.e. top left is read first).

Table 1 is confusing (at least initially).

o Hypoglycaemia flow-diagram : Again, not clear where

start-point is. Inconsistent use of

colour and capitals. Inconsistent directions

for “yes” and “no”. In the second box in the

left-hand column, it is confusing if the user assumes that “if on insulin infusion” applies to all of the instructions in that box, not just the first.

Up arrows on the left do not conform to expected flow. The design of the red boxes (with a solid line all the way down the middle) implies that the arrow only references the left-hand side information.

There are many other formatting issues with the flow diagram.

Two boxes are presented sequentially in the chain but refer to two alternative choices: “if unconscious and BGL greater than 4, manage decreased level of consciousness” and “if patient conscious follow up with appropriate oral or IV treatment”.

o Diabetes treatment review instructions :

Only apply to doctors but are not signposted

be in the most logical order. [Revisions to this section have been made according to the stakeholder advice for the final version of the form that may address this issue.]

o Hyperglycemia flow-diagram : Not clear where the

start-point is. However, it does conform with Western reading conventions (i.e. top left is read first).

o Hypoglycaemia flow-diagram : Inconsistent directions

for “yes” and “no”; however, this is unavoidable due to space constraints.

Up arrows on the left do not conform to expected flow. However, alternatives were more problematic and other design changes have made this issue less problematic.

One box (at the bottom left) presents two alternatives but is formatted differently from the other boxes that do likewise. [This last point actioned in final draft.]

o Diabetes treatment review instructions :

Only apply to doctors but are not signposted as such. However, this was considered sufficiently self-evident in context.

Arrangement of information in point 4 is not consistent with points 2 and 3 (where letters and roman numerals represent

45

as such. Inconsistent

arrangement of information.

It is questionable whether it would be obvious to both nurses and doctors who the hyper/hypoglycaemia flowcharts were directed at.

Despite these issues, on-site trials revealed that although users needed support upon implementation, this experience of being “overwhelmed” by the form was reduced when users had enough time to work through it.

In general, the instructions are unclear. For example, it is not entirely clear which instructions are directed at prescribers and which are directed at administrants.

either/or alternatives). A better alternative might be to present the information in point 4 as bullet points. [This suggestion has been actioned in the final draft.]

It is questionable whether it would be obvious to both nurses and doctors who the hyper/hypoglycaemia flowcharts were directed at. However, the additional clutter added by including extra information regarding this point was considered not worth the potential benefit. [This point therefore not actioned.]

46

ACTIONS AND ALERTS

Identification and demographics

Is the prescriber’s role less transparent than it could be?


The instructions beneath the patient identification label area say “First Prescriber to Print Patient Name and Check Label Correct,” above a line for printing the patient’s name. However, it is unclear whether the prescriber is just confirming the patient’s name, or whether they are supposed to confirm all of the other information. If the latter, there is no mechanism for recording that the confirmation has occurred. Perhaps compliance would be higher if the first prescriber had to initial to confirm that the check has been performed.

Some prescribers have been reported as misunderstanding this instruction and instead write their own name.

A potential solution may be to change the label to “First prescriber to print patient name below and check ID label correct”. However, the problem may be that they don’t read beyond the first two words. Therefore, an alternative might be “Print patient name below and check ID label correct (first prescriber)” or some variant.

In addition, the title case used for the direction (capitalizing the first letter in every word) seems inappropriate and may impede readability.

It is also possible that more than one label may be used (hence the term “ID label” is included in the possible replacement phrase above as an alternative).

Generally speaking, the completion of the patient identification details might be a compliance issue, where some sort of checking procedure may be advantageous (e.g., where the original person responsible for completing the


47

patient identification section is contacted if some information is missing).

Is the administrant’s role less transparent than it could be?


The form could more clearly signal to nurses which parts they can complete. For example, doctor-only sections (e.g., Routine Insulin Orders, Monitoring/Notification Instructions, and Supplemental Insulin Orders) could be in a different colour and given a “doctor only” label, icon or colour. However, this could be potentially confusing given that these sections are all used by nurses and that the Stat/Phone Insulin Orders section can be written in by both doctors and nurses. The use of too many colours is also likely to be problematic.

To improve cross referencing to the NIMC insulin order, one potential solution is to prompt doctors by including a check box on the insulin form to allow people to confirm that the NIMC chart has been marked.

A check box could also be utilised to signal that a new insulin form has been started.

The form could more clearly signal to nurses which parts they can complete. For example, doctor-only sections (e.g., Routine Insulin Orders, Monitoring/Notification Instructions, and Supplemental Insulin Orders) could be in a different colour and given a “doctor only” label, icon or colour. However, this could be potentially confusing given that these sections are all used by nurses and that the Stat/Phone Insulin Orders section can be written in by both doctors and nurses. The use of too many colours is also likely to be problematic.

A check box could potentially be utilised to signal that a new insulin form has been started. However, this was considered impractical due to space constraints.

Are there any other issues with this action system?



48

BGL frequency and notification instructions

For this action system, has the form failed to make full use of colour to improve its usability (if appropriate)?


As per previous suggestions, the Monitoring/Notification Instructions could utilise a particular colour to indicate that this section is only for doctors to complete.

As per previous suggestions, the Monitoring/Notification Instructions could utilise a particular colour to indicate that this section is only for doctors to complete.

For this action system, does it involve mentally transcribing information between non-adjacent areas of the form (which may increase mental workload and hence the scope for errors)?


Having two disconnected BGL frequency sections (in the general monitoring instructions and at the top of each date column) is potentially problematic. For example, the general instructions are redundant if the tick boxes at the top of each column are used when routine insulin is reviewed each day. Potential ambiguities may also arise when there are conflicts between the sources of information. A possible solution is to remove the general monitoring instructions (which will also free up space). From a clinical perspective, changes to BGL frequency are uncommon given that the default option of “standard” involves 4 tests per day (at the very least).


For this action system, does it fail to allow for modification of relevant values for individual patients where modifications may be required?


Frequency can be modified. However, consideration should be given to whether there are some other common options that could be added as tick-boxes to minimize the need to write.


49

For this action system, is its use less transparent than it could be?


Although the instruction “Medical officer to notify” presumably refers to who to call if a BGL triggers an alert, it is not entirely congruent with the BGL alerting instruction to “notify doctor immediately” (which could mean notify any doctor). Either the header could be altered to “Doctor to notify” or the alert instruction could be altered to “notify medical officer”.

It is also unclear whether a change recorded at the top of a date column carries over to the next day or whether the frequency defaults back to the general instructions.


50

For this action system, does the form fail to include instructions? (Users should ideally not have to refer elsewhere, to minimize errors resulting from inappropriate use.)


Users in previous focus group research have suggested changing the wording of “standard” monitoring frequency to “default”. However, we considered this to be contentious.

On one hand, it seems reasonable to change “standard” (where standards may vary) to default (implying one given state). This change could also lead to the removal (or at least minimization) of the line “if not instructed, default is standard” (perhaps adding a prompt such as “assume default if not marked”).

On the other hand, “default” could be considered just as unclear and ambiguous as “standard”. To improve clarity, a potential solution is to split “standard” into two separate boxes, “pre-meals” and “at 21:00hrs”.

These boxes could be pre-ticked, and the instruction “tick all that apply” could be changed to “tick or strike out as required”. This would prevent the nurse from having to know (a) the meaning of “standard” or “default” in this context, and (b) the instruction that states that the standard frequency should be used if the prescriber has not indicated the frequency.


51



In the Monitoring Record, the “Change BGL to” system is ambiguous. For example, on one form, this value was changed from “standard” and “2:00am” to just “standard”. However, the BGL continued to be monitored at 2:00am. A potential solution is to instruct users to cross out those that no longer apply and tick those that do.

Alternatively, doctors could be instructed to tick the appropriate boxes for each day, instead of having the separate general instructions, so that there can be no conflict. This would also ensure that the monitoring frequency information was consistently kept in the same place, close to the date and time information to which it is functionally related.

In addition, 24 hour time is not used consistently (i.e., 21:00hrs vs. 02:00am).


52

Additional instructions

Is the prescriber’s role less transparent than it could be?


The box for recording “Diabetes treatment prior to admission” is not perceptually linked with the other areas of the form that the first prescriber must attend to (which are all down the left-hand side of the page). It is possible that further improvements in documentation could be had if the workflow could be improved (e.g., depending on the re-design, this box could be grouped with “Special Instructions”).

The box for recording “Diabetes treatment prior to admission” is not perceptually linked with the other areas of the form that the first prescriber must attend to (which are all down the left-hand side of the page). It is possible that further improvements in documentation could be had if the workflow could be improved (e.g., depending on the re-design, this box could be grouped with “Special Instructions”).

Is the administrant’s role less transparent than it could be?






No other issues identified. No issues identified.

53

Documenting diet






Diet needs to be reviewed and written each day. This current design aims to create a cognitive link between diet and insulin doses.

However, diet interventions are not recorded, where possible interactions of diet and insulin changes can impact on BGL.

Diet needs to be reviewed and written each day. This current design aims to create a cognitive link between diet and insulin doses.

Diet interventions are not recorded, where possible interactions of diet and insulin changes can impact on BGL. However, the benefits of adding this information into the form were considered not worth the costs in terms of added clutter. [This point was therefore not actioned in the final draft.]

54



From the perspective of the nurse, dietician and diabetes educator, there is no ability to document a carbohydrate plan for patients that dose in response to carbohydrate loads.

Indeed, modifications to insulin only occur after BGL has changed which means that unrecorded carbohydrates can induce unexpected changes.

From a doctor’s perspective, the diet areas are positioned in an area where a doctor may not tend to look (i.e., in the Monitoring Record).

Finally, the use of blank “Diet” boxes allows anything to be written.

From the perspective of the nurse, dietician and diabetes educator, there is no ability to document a carbohydrate plan for patients that dose in response to carbohydrate loads.

Indeed, modifications to insulin only occur after BGL has changed which means that unrecorded carbohydrates can induce unexpected changes.

However, the benefits of adding this information into the form were considered not worth the costs in terms of added clutter. [These points were therefore not actioned in the final draft.]

From a doctor’s perspective, the diet areas are positioned in an area where a doctor may not tend to look (i.e., in the Monitoring Record). However, it was considered that addressing this point would have adverse effects on other elements of the form. [This point was therefore not actioned in the final form.]



As mentioned previously, the area for documenting a patient’s diet may benefit from tick-box or circling options (e.g., full, nil by mouth, total parenteral nutrition, clear fluids, other).

Also, no predictive time frames or quantum are given for carbohydrates on BGL.

No predictive time frames or quantum are given for carbohydrates on BGL.

However, the benefits of adding this information into the form were considered not worth the costs in terms of added clutter. [These points were therefore not actioned in the final draft.]

55



No instructions for completing “Diet” boxes are given. It is unclear who is to document this information (e.g., the doctor or nurse) and how it is to be used (in practice, both can document this).

It is also questionable whether all users (e.g., inexperienced nurses or interns) would be familiar with the appropriate terminology, and whether errors typically occur.

The above suggestion of tick-box or circling options may alleviate the need for instructions.

Although instructions for completing “Diet” boxes are given, it remains unclear who is to document this information (e.g., the doctor or nurse) and how it is to be used (in practice, both can document this). However, the benefits of adding this additional information into the form were considered not worth the costs in terms of added clutter. [These points were therefore not actioned in the final draft.]

Although some users (e.g., inexperienced nurses or interns) may not be familiar with some of the terminology (e.g. TPN), the tick-box system only requires recognition.




56

Documenting BGL









Because of the paper-based nature of the form, the alerting ranges associated with each colour are fixed (unlike the Supplemental Insulin Order ranges) – potential mismatches caused by modified supplemental insulin order ranges may lead to errors.

BGL ranges for alerting and Supplemental Insulin Orders can be adjusted jointly (except “Greater than 20” and “Less than 4”), making changes more evident, eliminating a source of potential mismatches and reducing the risk of error. Nevertheless, there is a risk that the right-hand range will be adjusted and the left-hand range will not. This needs to be addressed in training for prescribers (how to make changes correctly) and nurses (what to do if there is a mismatch). However, it is questionable whether: (a) the benefits of being able to modify the ranges outweigh the costs and risks associated with this, and (b) insulin dosage is a precise enough science for this to be justified. If not, it may be preferable to remove the ability to modify.

57



The instruction, “BGL (mmol/L) Write in corresponding range box”, is buried. This instruction is positioned too close to the y-axis “Time” label such that it appears as if the BGL should be recorded in the time row. This also makes the “Time” row unnecessarily tall and cumbersome.

One option might be to move the alerts to the right-hand side (which might also be a more intuitive place for them with a left-to-right workflow), allowing more space for the legend to be made clear on the left-hand side.

A suggestion from previous research was to leave the insulin prescription on the NIMC to mitigate nurses’ confusion with the form’s requirement to document the time of BGL, separate from the documentation of “Time” and “Nurse 1/2 initials” for insulin administration. If implemented, this may make it more difficult to determine when supplemental insulin should be given. Also, doctors prescribing even routine insulin should take into account BGL readings. Moving the insulin prescription to the NIMC will make this harder.

This concern can be better addressed through the alignment of insulin orders and administration, as well as training.




Although instructions are provided, they are not strong and might easily be missed.


58

Is information displayed as a graph or in a graph-like presentation assuming it was judged that this might reduce errors?



Does the graph look too small or cramped?


The graph cells are of a reasonable size, where legibility should not be affected.

To institute the rounding of BGLs to the nearest whole mmol/L, in an effort to reduce visual clutter within the graph, would require a huge culture change. BGL meters also have around a 20% variance which could impact whether users act on a BGL or not.

The suggestion from previous research to extend the duration of the form by using an alternative form just for monitoring/alerts and hypo- and hyper-glycaemia treatment may disconnect continuity. For example, if a prescriber does not look at both the “standard” and “hyper/ hypo” forms, periods of abnormal BGLs could be missed and users might assume that all BGLs are within range as they would not be documented sequentially.

The graph cells are of a reasonable size, where legibility should not be affected.

To institute the rounding of BGLs to the nearest whole mmol/L, in an effort to reduce visual clutter within the graph, would require a huge culture change. BGL meters also have around a 20% variance which could impact whether users act on a BGL or not.

The suggestion from previous research to extend the duration of the form by using an alternative form just for monitoring/alerts and hypo- and hyper-glycaemia treatment may disconnect continuity. For example, if a prescriber does not look at both the “standard” and “hyper/ hypo” forms, periods of abnormal BGLs could be missed and users might assume that all BGLs are within range as they would not be documented sequentially.

Is the graph label not clear or descriptive?


The graph title “Monitoring Record” could be more specific (e.g., “BGL Monitoring Record”), except that it also includes ketones.

Further, the “BGL” heading does not conform to the other heading structures.

The graph title “Monitoring Record” could be more specific (e.g., “BGL Monitoring Record”), except that it also includes ketones.

Although the “BGL” heading still does not conform to the other heading structures, it is now much clearer which rows it relates to, and its prominence is justified given its central importance.

59

Is the graph label written vertically with upright letters?



Does the graph label fail to specify units of measurement?



Does the graph label fail to provide an example of how data is to be recorded (e.g. numeric value, or ●)?


Although an example of how data is to be recorded is not provided, this may be confusing in the case of a numeric value.

Although an example of how data is to be recorded is not provided, this may be confusing in the case of a numeric value.

Is the graph label font/formatting exactly the same as the Y-axis values font?



Is the vertical axis NOT labelled on the left & right of the page?


No issues identified. Values are presented on both sides, but the label “BGL (mmol/L)…” is only presented on the left, due to the Supplemental Insulin Orders table on the right. However, since BGL is the only parameter with values listed at left and right, and the parameter label on the left is very prominent, this is not viewed as problematic.

Does the scale of Y-axis values change (e.g. going from .5 intervals to 1 intervals for the same length of the Y-axis)?


The scale of the y-axis values only changes in the case “Greater than 20”, and this is unlikely to be problematic.

The scale of the y-axis values only changes in the case “Greater than 20”, and this is unlikely to be problematic.

60

Are the Y-axis values not mutually exclusive (e.g. "5-10", "≤5")?



Are thick vertical lines NOT placed every 3-4 boxes?



Are the date boxes unnecessarily small (should accommodate size 14 text)?



Are the time boxes unnecessarily small (should accommodate size 14 text)?


Time boxes are small, but this is unavoidable if the form is to cover 30 time-points. It may be necessary for some users to write times diagonally or over two lines.

Time boxes are small, but this is unavoidable if the form is to cover 30 time-points. There may be the need for some users to write times diagonally or over two lines.

61



Dates may be illegible because of free-form date boxes. Legibility may be improved by adding forward slashes to separate day from month, and month from year. (This also applies to the Stat/Phone Insulin Orders section.)

There is also some risk that clinicians from countries where MM/DD/YY is the conventional order may not comply with the Australian DD/MM/YY convention. Therefore, light grey text for “MM”, “DD” and “YY” (or “YYYY”, as required) may prevent ambiguous dates that may otherwise be problematic for auditing and/or cause minor confusion at the bedside. (Likewise in other parts of the form where dates are recorded).

The conceptual grouping of the “BGL (mmol/L)…” label with the range rows below is poor. It should not touch the time boxes. (This also applies on the right-hand side of the page).

The partial bold vertical line between the BGL ranges and the first date column seems out-of-place and means that the other labels (e.g., “date”, “diet”, etc.) seem more closely linked with the first date column than the others. Suggest making the entire line bold; likewise the last vertical line on the right.

The time boxes are an inappropriate shape. This is a by-product of the odd labelling of the BGL area, itself a result of the odd placement of the alerts box. This could be moved to the right (which will be more practicable if the monitoring record etc., is moved down the page to allow for alignment of the supplemental orders) without losing a day of recording space.


62

Responding to abnormal BGLs

For this alerting system, has the form failed to make full use of colour to improve its usability (if appropriate)?


A potential suggestion is to use mauve in the Hypoglycaemia Management Guidelines to perceptually link them to the corresponding BGL range row.

However, it is possible that the use of mauve in the Monitoring Record may confuse users, as a mauve alert on the ADDS chart signals users to place or at least consider an emergency call (whereas that would only be appropriate for a subset of hypoglycaemia cases).

The hypoglycaemia protocol fails to use colour to link fast and slow acting carbohydrates.

No problems identified (mauve now used in the Hypoglycaemia Management Guidelines, but only for the emergency pathway).

63

For this alerting system, does it involve mentally transcribing information between non-adjacent areas of the form (which may increase mental workload and hence the scope for errors)?


Having the alerts on the left is not as intuitive as having them on the right, given a left-to-right workflow. A potential solution is to move the alerts to the right (rather than duplicating them), in conjunction with moving the monitoring record down and to the left, so that a day does not have to be lost.

Users need to turn the page to access the guidelines for dealing with seriously abnormal BGLs.

Once actions for hypo/hyperglyceamia alerts have been taken, users need to return to the inside pages of the chart to document various information related to responding to these alerts (e.g., ketones, doctor notified – which are recorded at the bottom of the monitoring record). Nevertheless, this information is probably better left where it is, as opposed to being documented on the same page as the alert guidelines, as all patient-specific information is located on the inside pages of the chart.

Having the alerts on the left is not as intuitive as having them on the right, given a left-to-right workflow. However, the value of moving them to the right (or repeating them there) is outweighed by the value of aligning the Supplemental Insulin Orders.

Users need to turn the page to access the guidelines for dealing with seriously abnormal BGLs. However, actions are flagged at the alert point.

Once actions for hypo/hyperglyceamia alerts have been taken, users need to return to the inside pages of the chart to document various information related to responding to these alerts (e.g., ketones, doctor notified – which are recorded at the bottom of the monitoring record. Nevertheless, this information is probably better left where it is, as opposed to being documented on the same page as the alert guidelines, as all patient-specific information is located on the inside pages of the chart.

64

For this alerting system, does it fail to allow for modification for individual patients where modifications may be required?


Although modifications cannot be made to the altering ranges in the Monitoring Record, they are arguably general enough to apply to most, if not all, patients.

However, because these ranges cannot be changed while the corresponding ranges for supplemental insulin can be changed, mismatches may result.

Note that it would in principle be possible to cross out ranges on the form or specify modifications in the special instructions area. However, both of these options are likely to increase errors.


If the option to modify ranges is kept, then instructions on how to do this must be included in training, as this is not intuitive and there are associated high risk caveats.

Modifications are constrained to some degree. For example, there is no space for additional ranges. [It was not considered practical to action this point in the new form.]

65

For this alerting system, is its use less transparent than it could be?


Although the instructions appear clear, it is important to note that responding to some alerts requires mental processing where users need to count consecutive readings (to a different total in each row and cumulatively across different coloured rows in some cases). Unlike with the alerting system on the ADDS chart, this means that a problematic case may not always “pop out” perceptually. This issue may be amplified by the fact that consecutive BGL data may not be in adjacent cells: for example, when columns are skipped at the end of a day because fewer than 6 readings per day are taken. This system may in fact require training interventions.

The form does not make it clear what needs to be recorded in the case of hypo intervention. A potential solution is to add a tick icon to the legend, for example, “Hypo intervention ()”. The same applies for “Doctor notified”.

Also, it is not clear whether the “Doctor notified” box should also be ticked for a “hypo intervention”, where certain steps involve the doctor.

The order of the Ketones, Hypo intervention and Doctor notified rows is also questionable. For example, it may be better to perceptually group the Hypo Intervention row with the mauve hypoglycemia row in the Monitoring Record.

Finally, the instruction to “Notify if x consecutive BGLs greater than...” could include ‘Doctor’ to be more specific (and consistent with the ‘Doctor Notified’ row).

There is no instruction in the “guidelines for managing hyperglycaemia alerts” box that the results of the ketone test need to be documented in the monitoring record on the first page of the chart.

Although the instructions appear clear, it is important to note that responding to some alerts requires mental processing where users need to count consecutive readings. Unlike with the alerting system on the ADDS chart, this means that a problematic case may not always “pop out” perceptually. This issue may be amplified by the fact that consecutive BGL data may not be in adjacent cells: for example, when columns are skipped at the end of a day because fewer than 6 readings per day are taken. This system may in fact require training interventions.

It is not clear whether the “Doctor notified” box should also be ticked for a “hypo intervention”, where certain steps involve the doctor. [Clarifying this issue was not considered worth the costs of introducing the extra information into the form; this point is therefore not actioned.]

Finally, the instruction to “Notify if x consecutive BGLs greater than...” could include ‘Doctor’ to be more specific (and consistent with the ‘Doctor Notified’ row). [This suggestion not actioned due to space constraints.]

There is no instruction in the “guidelines for managing hyperglycaemia alerts” box that the results of the ketone test need to be documented in the monitoring record on the first page of the chart. [This comment not actioned due space constraints.]

66

For this alerting system, does the chart fail to include instructions? (Users should ideally not have to refer elsewhere, to minimize errors resulting from inappropriate use.)


The BGL notifications do not specify who should be notified for observations within the 12.1-16 mmol/L and 16.1-20 mmol/L range rows.

Although the hypoglycaemia row refers users to the Hypoglycaemia Management Guidelines on the outside back page, the hyperglycaemia range row(s) do not refer users to the corresponding guidelines on the outside front page. However, this may be because they are meant exclusively for doctors, and the other instructions embedded in the form are aimed at nurses.

The fact that these instructions are over the page may also be a problem.

The BGL notifications do not specify who should be notified for observations within the 12.1-16 mmol/L range row. [This suggestion was not acted upon due to space constraints.]

Although the hypoglycaemia row refers users to the Hypoglycaemia Management Guidelines on the outside back page, the hyperglycaemia range row(s) do not refer users to the corresponding guidelines on the outside front page. However, this may be because they are meant exclusively for doctors, and the other instructions embedded in the form are aimed at nurses.

The fact that these instructions are over the page may also be a problem. However, this was considered to be unavoidable.

Are there any other issues with this alerting system?



67

Ordering routine insulin



Although there is good use of greyscale to help prevent row shifts, colour could be further utilised.

As mentioned previously, colour could be used to clarify that this area is for doctors only (especially, given that this was an issue raised by previous research).

Alternatively, one colour could be applied to the Routine Insulin Orders and the routine insulin administration rows to make it clearer that the two are linked.

Another option is to use two alternating colour shaded cells to: (a) perceptually separate each day’s routine order, and (b) encourage the daily review of routine orders. This shading could also encourage the ordering of the following morning’s insulin by using the same colour shade (i.e., one shade for day 1 + day 2’s breakfast, another shade for day 2 + day 3’s breakfast, etc.).

Although there is good use of greyscale to help prevent column shifts, colour could be further utilised.

As mentioned previously, colour could be used to clarify that this area is for doctors only (especially, given that this was an issue raised by previous research).

Alternatively, one colour could be applied to the Routine Insulin Orders and the routine insulin administration rows to make it clearer that the two are linked.

Another option is to use two alternating colour shaded cells to: (a) perceptually separate each day’s routine order, and (b) encourage the daily review of routine orders. This shading could also encourage the ordering of the following morning’s insulin by using the same colour shade (i.e., one shade for day 1 + day 2’s breakfast, another shade for day 2 + day 3’s breakfast, etc.).

However, all these suggestions undermine the use of colour to denote abnormal BGL (and associated consequences). [These suggestions were therefore not actioned.]

68



Routine Insulin Orders should take into account the patient’s condition up to that point; however, this section is not perceptually linked to the monitoring and Administration Record. A potential solution is to align them with the day columns, which may also encourage daily orders.

Prescribers may need to turn the pages of the chart back and forth when prescribing insulin (i.e., they will need to look at the back page for prescribing guidelines and at the front page in order to document insulin orders). When prescribing, it may be more beneficial for clinicians to have the routine and supplemental insulin boxes on the same page as the prescribing guidelines.


Prescribers may need to turn the pages of the chart back and forth when prescribing insulin (i.e., they will need to look at the back page for prescribing guidelines and at the front page in order to document insulin orders). When prescribing, it may be more beneficial for clinicians to have the routine and supplemental insulin boxes on the same page as the prescribing guidelines. [This issue not addressed due to space constraints.]



Suggestions from previous research to (a) allow space for routine orders to indicate 6-8 units for patients who self-administer insulin, and (b) include the capacity to prescribe a range of doses with respect to intake, requires a solution that does not lead to excessive clutter. This is not likely to be trivial and it is unclear whether these issues need addressing.

Suggestions from previous research to (a) allow space for routine orders to indicate 6-8 units for patients who self-administer insulin, and (b) include the capacity to prescribe a range of doses with respect to intake, requires a solution that does not lead to excessive clutter. [Due to space constraints, this cannot be addressed without compromising the safety of other elements.]

69



The Routine Insulin Orders probably constitute the most important part of the form. Ideally, a form design would ‘force’ a dose review if none are ordered, while not mandating a dose (e.g., patients may be NBM).

Also, new insulin (U500) incidents add complexity.

Users having to cross-reference between the date columns in the monitoring and Administration Record and the Routine Insulin Orders may be problematic. Once again, a potential solution is to align with respect to the date.

Further, the Routine Insulin Order could be positioned above the Monitoring Record to group this near the other tasks of the prescriber (e.g., the Monitoring/Notification Instructions box). This would also avoid a potential disconnect between BGL, ketone, and hypoglycemia documentation, as per Qld Health’s response to our previous suggestion of positioning Routine Insulin Orders immediately underneath the Monitoring Record. However, another way to address this concern would be to leave the BGL, ketone and hypoglycemia rows adjacent to the Monitoring Record and position the Routine Insulin Orders below them.

Finally, there could be a problem with the ambiguity of a blank cell. For instance, a blank cell could mean that insulin should be given but the prescriber has failed to order it, or that the order is ceased (where the patient no longer needs insulin) and the prescriber has not followed the steps in the separate guide to indicate that the order is ceased (this is not described on the form itself).


70



The instructions are not clear or particularly prominent.

A key error of prescribers is not completing the Routine Insulin Orders (i.e., where nurses do not call the prescriber to get a replacement phone order). On the simplest level, a clearer directive to prescribers might help. For example, the instruction such as “if blank, ring the prescribing doctor immediately” could be added to “must be ordered each day”, making the procedural consequences of blank orders clear to both nurses and doctors.

The instruction “must be ordered for each day” is also ambiguous. First, it is not made clear that the prescribing should actually be done on a daily basis. Second, even if interpreted that way, it does not make it clear that it is appropriate to order the following morning’s dose in advance. A potential suggestion is to change the wording to “must not be ordered more than 36 hours in advance” (or similar).


Are any of the boxes for Name of Insulin too small?


Boxes for routine insulin names may be too small, especially for long names. This may reduce legibility. However, it is worth noting that the insulin medicine with the longest name is rarely used.

Boxes for routine insulin names may be too small, especially for long names. This may reduce legibility. However, it is worth noting that the insulin medicine with the longest name is rarely used.

71



Given that the form can be used just for BGL monitoring (i.e., some patients will not be receiving insulin), there should be a clear way of prescribing nothing, which can easily be distinguished from an omission.

Given that the form can be used just for BGL monitoring (i.e., some patients will not be receiving insulin), there should be a clear way of prescribing nothing, which can easily be distinguished from an omission. However, to prevent clutter, it was suggested that this issue might be better dealt with via training rather than introducing further instructions onto the form itself.

72

Ordering supplemental insulin and determining the dose



If the ranges for supplemental insulin are modified, the colour system will no longer work. It is questionable whether: (a) the benefits of being able to modify the ranges outweigh the costs and risks associated with this, and (b) insulin dosage is a precise enough science for this to be justified. If not, it may be preferable to remove the ability to modify.

Another suggestion is to reverse the supplemental insulin ranges to conform to the Monitoring Record, and align them to improve user-performance. This suggestion is also supported by empirical evidence where novice participants were significantly faster and more accurate at calculating the total dose of basal insulin required when Supplemental Insulin Orders were aligned with the Monitoring Record (vs. not aligned).

Once again, colour could be used to clarify that this area is for doctors only.

Alternatively, the Supplemental Insulin Order box could be made a certain colour that matched the supplemental insulin administration row to make the link between the two clearer (as the arrows are not particularly salient).


73

For this action system, does it involve mentally transcribing information between non-adjacent areas of the form (which may increase mental workload and hence

the scope for errors)?


To reduce the need for users to mentally transcribe information between non-adjacent areas, once again, the BGL record could be aligned with the Supplemental Insulin Orders (again, this should be done in conjunction with moving the monitoring record down the page and to the left to preserve the five-day duration of the form).


For this action system, does it fail to allow for modification of relevant values for individual patients, where modifications may be required?


Users involved in previous focus group research have suggested the addition of 1-2 rows to allow for tighter control. If clinically justified, additional range rows may be a viable option if the trade-off is removal of the ability to modify the ranges. However, there is no clear evidence for smaller levels of adjustment due to range, and this change would also require reducing the row heights, which would be detrimental for usability.

The impact of modifications around individual tolerances for abnormal BGL might lead to a greater acceptance of abnormal BGL in unmodified patients.

As discussed above, modifications to the ranges of the Supplemental Insulin Orders are allowed but may cause the system to fail. Also, it may not be immediately clear to novices and nurses how to use the modifications (i.e. training might be needed).

Modifications to supplemental ranges may not be necessary if the current supplemental ranges are accepted as clinically sound.


The impact of modifications around individual tolerances for abnormal BGL might lead to a greater acceptance of abnormal BGL in unmodified patients.

Modifications to supplemental ranges may not be necessary if the current supplemental ranges are accepted as clinically sound.

74



The instructions are not easy to identify (also, “if unsure, seek advice” may have little effect on those who do not understand).

As suggested above, the “start date” and “start time” legends could be made more salient with bold text. Perhaps the legend suggested by previous research could be added if space permits.

As mentioned previously, coloured font could be used to increase the salience of the Supplemental Orders instruction, “valid until changed or ceased”. However, making these instructions more prominent may serve to de-emphasize other text.

Alternatively, the “valid until changed or ceased” instruction could be moved closer to where the actual doses are recorded (e.g., in the “it is not necessary” box).

It may also be helpful for prescribers to cross through superseded supplemental orders so that there is only one readable set of supplemental insulin.

It is not really clear which instructions are meant for nurses (if any) and which are meant for prescribers

“If unsure, seek advice” may have little effect on those who do not understand.

As mentioned previously, coloured font could be used to increase the salience of the Supplemental Orders instruction, “valid until changed or ceased”. However, making these instructions more prominent may serve to de-emphasize other text.

It may also be helpful for prescribers to cross through superseded supplemental orders so that there is only one readable set of supplemental insulin orders. This could be addressed in training.

It is not really clear which instructions are meant for nurses (if any) and which are meant for prescribers

75



Although instructions are provided, they are unclear. For example, the position of the label “If the BGL is:” (i.e., where the administrant has to first track down the ranges) relative to the label “then administer additional” (i.e., where the administrant then needs to track right), creates a “sharp turn” pathway that may not be completely obvious. A potential improvement could be to move the “If the BGL is:” label to the left of the ranges and include arrows to show the intended flow.

In terms of the frequency box, one option is to specify a “default” frequency (e.g. “If not instructed, default is “with meals only”), given there is a similar default for the BGL frequency). However, given that supplemental insulin is not appropriate for all patients, this may cause confusion.




The box for the supplemental insulin name may be too large, as it does not correspond to other box heights.




There are inconsistencies in the prescriber signature and “print your name” area between routine and supplemental insulin orders.


76

Ordering stat/phone insulin



As per previous suggestions, colour could be used to flag that that the Stat/Phone Insulin Orders (at least the first component) can be completed by nurses who are taking instruction from a doctor over the phone. However, this is complicated by the “dual use” nature of this section of the form.

As per previous suggestions, colour could be used to flag that that the Stat/Phone Insulin Orders (at least the first component) can be completed by nurses who are taking instruction from a doctor over the phone. However, this is complicated by the “dual use” nature of this section of the form. [This suggestion was therefore not actioned.]



A potential solution to help avoid doses being missed when a phone order is made for a future time is to place the Stat/Phone orders in a more prominent position at the top of the page (assuming that the Monitoring Record, etc., are to be moved down).

A tick-box could also be added as a secondary check on whether it has been administered (plus some additional instructions on how to record and deal with stat/phone orders, space permitting).

Other options might include separating stat and phone orders. However, this solution may not be practicable if it takes up more space on the form.

A tick-box could be added as a secondary check on whether insulin has been administered (plus some additional instructions on how to record and deal with stat/phone orders, space permitting). On balance, it was judged that including this extra information was not worth the additional clutter it would introduce. [These suggestions were therefore not actioned in the final draft.]

Other options might include separating stat and phone orders. However, this solution was not considered to be practical as it would take up additional space on the form.




77



The instruction to “check with doctor if order replaces, or is in addition to, other insulin orders” is problematic as nurses are told to seek this information without the ability to record it in the form. A potential solution is to include two checkboxes for each phone order (e.g., “this order replaces all other orders at this time” vs. “this order is IN ADDITION to all other orders at this time”).




The instructions are inadequate.

It is not entirely clear that the stat/phone doses go in the “Routine insulin” rows of the Administration Record. A potential solution is to change the label to “Name of insulin (routine/stat/phone)” (or similar).

Instructions to nurses could be also improved, space permitting (this might also avoid the need to radically alter the “name of routine insulin” legends).

This section of the form is mostly self-explanatory.

However, it is not entirely clear that the stat/phone doses go in the “Routine insulin” rows of the Administration Record. A potential solution is to change the label to “Name of insulin (routine/stat/phone)” (or similar).

While it was acknowledged that the instructions to nurses were not completely clear, this issue could not be addressed on the form itself without increasing clutter.



Boxes for insulin names may be too small for long insulin names, although the insulin with the longest name is rarely used in practice.


78



The “Date/time of dose” box is too small.

Where a phone or stat order is initially recorded by nursing staff, would it make sense for them to record the prescriber’s name? If so, should “Print your name” be replaced with “Print prescriber name” (or similar)?

Where a phone or stat order is initially recorded by nursing staff, would it make sense for them to record the prescriber’s name? If so, should “Print your name” be replaced with “Print prescriber name” (or similar) to allow this flexibility? [This suggestion was actioned in the final draft.]

Administer insulin and document



Although the rows are not alternately shaded, they each have different borders at the top or bottom.

As noted previously, this section could use colour to match either the Routine or Supplemental Insulin Orders.

Although the rows are not alternately shaded, they each have different borders at the top or bottom, which serves the same function.



Routine and Supplemental Insulin Orders are not aligned with dates and BGL levels, respectively. This could easily lead to errors as the sections are not laid out congruently (where substantial mental effort is required).

Although the arrows linking the Routine and Supplemental Insulin Orders to the Administration Record are somewhat effective, they are not salient enough (i.e., a user cannot spot, at a glance, how orders are linked to administration).


79






There is some ambiguity. For instance, a user cannot necessarily determine which insulin administration a comment refers to.

There is some ambiguity. For instance, a user cannot necessarily determine which insulin administration a comment refers to. [There was not sufficient available space to address this issue.]



In accord with the point made by previous research, there may be confusion about nurses’ and pharmacists’ authorizations to transcribe the insulin name to the Administration Record.

A solution to this issue is critical given reports that nurses can sometimes be reluctant to transcribe insulin names.

Hence, a specific instruction could be provided so that nurses and pharmacists have the authorization to do so.

Nurses are explicitly authorized to transcribe the insulin name to the Administration Record, but will pharmacists infer that they can too (given that it is now clear that this task is not restricted solely to doctors)? If not, does this even matter, given that nurses are now instructed to do so?





80




81

Ceasing orders



No issues identified. No issues identified



If an order is ceased, a new form is required. This is likely to be unavoidable.

There is also the possibility that a routine order maybe labelled as “ceased” in the routine section, but a phone order might be present that is not crossed though and is in a different area of the form.

If an order is ceased, a new form is required. This is likely to be unavoidable.

There is also the possibility that a routine order maybe labelled as “ceased” in the routine section, but a phone order might be present that is not crossed though and is in a different area of the form. [There was not the space needed to address this concern via design.]






The procedure for ceasing orders is entirely given within the instructional document, with no clues on the form itself.

The procedure for ceasing orders is entirely given within the instructional document, with no clues on the form itself. [This was considered to be unavoidable given the lack of additional space on the form.]

82



Although instructions are not provided on the form itself, a national standard system in place for ceasing orders (as per the NIMC).

Although instructions are not provided on the form itself, a national standard system in place for ceasing orders (as per the NIMC).




General

Does the form fail to include an action system where such a system would be of value?



Does the form fail to include an alerting system where such a system would be of value?



83

Conclusion

Overall, the heuristic evaluation panel judged that the draft of the new subcutaneous insulin form that they reviewed was likely to be considerably safer than the previous form. Further modifications suggested by the panel were then incorporated, where practicable, into the final draft of the insulin form presented at the beginning of this report. That is, we would consider the final draft of the form to be potentially even safer than the draft subjected to the heuristic evaluation.

We therefore recommend the final draft of the new subcutaneous insulin form be progressed to clinical trial. This conclusion is subject to the following caveats:

(1) The heuristic evaluation technique reported above is a qualitative assessment method based on systematically canvassing human factor expert opinion and does not constitute an empirical assessment.

(2) There was overlap between membership of the design team for the new form and the heuristic evaluation panel, which may have affected the evaluation outcomes. (However, the same was true for the ADDS Chart, and there is now considerable empirical evidence to suggest that it yields superior performance in chart users relative to prior designs; that is, the overall conclusion of the panel was subsequently validated through objective experimental research.)

In the light of these caveats, we recommend that a program of empirical behavioral research is also required to verify the conclusions from the heuristic evaluation study (which was outside the scope of the current project). This could provide a much stronger evidence base to inform best practice in insulin form design, both at the level of the overall form design and at the level of individual design features.

development and initial evaluation of a new … · mark s. horswill1, andrew hill1,2, melany...

Documents