evaluation of ilead 2015 - uis · evaluation of ilead 2015 center for state policy and leadership...

Evaluation of ILEAD 2015 Center for State Policy and Leadership

University of Illinois Springfield March 2016

ILEAD is a program of the Illinois State Library (ISL) to train librarians in the use of participatory technologies, commonly referred to as Web 2.0 (and now beyond). The program began with a pilot training of Illinois librarians in 2010. That was followed by another Illinois cohort in 2011, an extension of ILEAD to four other states, plus Illinois, in 2013, and a fourth cohort of Illinois librarians in 2014. The most recent iteration of the program – the subject of this evaluation report – occurred in 2015 with a replication in ten states including Illinois. To date, 414 librarians from 12 states have participated in ILEAD training. In the ILEAD model, librarians learn about participatory technologies by serving on small teams of up to five librarians, typically, although not always, from two to three different libraries. Each team develops a project using one or more of these technologies, and the teams learn about relevant technologies principally through participating in three in-person training conferences held at different times during the nine-month duration of the program. Each team is assigned a mentor, usually a seasoned librarian, often with prior participation in ILEAD, who is available to provide guidance and troubleshoot issues. Each team also interacts with one or more non-librarian representatives from their community to help provide a patron perspective. In 2015, 175 librarians served on 45 different teams in the 10 participating states. For evaluation purposes, participants were asked to complete four surveys: 1) a baseline survey in January-February of basic demographic characteristics, job experiences, and preliminary project team activities; 2) a survey in April-May of perceptions of project team functioning and learning and use of content from the first in-person training conference in March; 3) a survey in August-September of project team functioning and learning and use of content from the second in-person training conference in June; and 4) a final survey in November-December evaluating the ILEAD experience. (Copies of all four survey instruments can be found in Appendix A.) Mentors also completed a baseline survey as well as surveys in April-May and August-September asking for their perceptions of team learning and use of training content. The same schedule of surveys has been used in the evaluation of all ILEAD cohorts, although the specific questions have changed some over time. Throughout the report, results for 2015 are compared with results for the four preceding ILEAD cohorts. The comparisons are done so that a reader, going from right to left, can track changes, if any have occurred, over time.

Participant Baseline Characteristics

Participants in the 2015 ILEAD USA program, like those in the 2014 Illinois cohort, were somewhat older and more experienced than the previous three groups (Table 1). The 2015 group was also slightly more likely than all but the 2011 Illinois cohort to include librarians whose jobs entailed supervising others. Interestingly, 2015 participants, representing ten different states, on average had known one another before ILEAD for almost twice as long as was the case for all four of the earlier groups. In other words, team members in 2015 were more likely to be familiar with one another. Table 1: Demographic Characteristics

2

Demographic Characteristics ILEAD USA 2015

Illinois 2014

ILEAD USA 2013

Illinois 2011

Illinois 2010

Years of experience as librarian (median) 8.0 10.0 6.0 6.0 6.0

Years with current employer (median) 5.0 7.0 5.0 6.0 3.5

Years in current position (median) 3.0 3.0 2.8 3.0 2.0

Median age 40 to 49 40 to 49 30 to 39 30 to 39 30 to 39

Percent who supervise others 60.0% 50.0 53.3% 62.2% 51.3%

Mean number supervised 7.0 4.0 6.6 7.5 8.2

Percent female 83.0% 85.0 81.0% 75.7% 92.3%

Mean length of time have known other team members (in years)

2.2 1.2 0.8 1.0 0.5

Percent with Master's degree or above 81.0% 94.0 77.7% 81.1%

While the majority of 2015 participants reported working in a public library, they were similar to the 2014 Illinois cohort and 2013 ILEAD USA cohort in having fewer from this type of library than the first two Illinois cohorts (Table 2). The 2015 group was least likely of any previous cohort to report working in a school, but more likely than all but the first Illinois cohort to indicate working in another type of library. Table 2: Types of Library Where Participants Work

Library Type

ILEAD

USA

2015

Illinois

2014

ILEAD

USA 2013

Illinois

2011

Illinois

2010

Public 56.9% 56.5% 54.4% 67.6% 69.2%

School 12.3% 21.7% 14.6% 18.9% 12.8%

Academic 27.1% 34.8% 27.2% 35.1% 53.8%

Specialty 14.5% 13.0% 10.8% 21.6% 23.1%

Other 13.8% 8.7% 7.0% 8.1% 17.1%

The baseline survey asked participants to indicate how much independence and interdependence they have in their job and to rate themselves on group skills (Table 3), attributes that might be important to ILEAD project team functioning. 2015 participants reported a lower level of job independence than any previous cohort and the highest level of group skills. Their job interdependence tied for highest with the 2011 Illinois group. Most likely, relatively high interdependence and low independence stemmed in part from the high percentage of supervisors in both cohorts. Supervision, by its nature, is interdependent. Table 3: Job Characteristics

3

Job Characteristics (median scores) 1= High, 7 = Low

ILEAD

USA

2015

Illinois

2014

ILEAD

USA 2013

Illinois

2011

Illinois

2010

Extent to which current job calls for independence 3.5 2.3 2.5 2.5 2.8

Extent to which current job calls for interdependence 2.0 3.0 2.3 2.0 2.3

Level of group skills 2.6 3.3 3.5 3.5 3.5

The baseline survey asked participants questions designed to measure their motivation to perform and learn, since motivation is important to how members might function on their teams and derive value from the training (Table 4). Performance motivation has to do with being motivated by external recognition, while learning motivation concerns being motivated by an intrinsic desire for mastery of a subject or method. The four prior evaluations of ILEAD measured performance motivation as one index variable1 and learning motivation as another. The 2015 evaluation took advantage of recent research on this topic to measure motivations more precisely. Instead of just measuring performance and learning motivation, it asked questions that get at whether one is motivated: 1) to perform to acquire success (approach) or to avoid failing to succeed and 2) to learn to master something (approach) or to avoid failing to master it. These four different types of motivation are not mutually exclusive; a person can have different amounts of each. Prior evaluations, on average, found both performance and learning (mastery) motivations to be relatively strong among participants, with learning motivation slightly stronger than performance motivation. In the 2015 evaluation using the more refined measures, a somewhat different pattern emerged. Participants showed a high mastery approach motivation, a mid-range mastery avoid motivation, and low performance motivations of both types. Since ILEAD is a training program designed to promote learning, strong mastery orientations are most likely an advantage. Table 4: Participants’ Motivations

Motivations 0=None, 3=High

ILEAD

USA

2015

Performance approach motivation 1.0

Performance avoid motivation 0.0

Mastery approach motivation 3.0

Mastery avoid motivation 2.0

Participants were asked questions about the extent of preliminary activities by their project team prior to the first in-person training conference in March 2015 (Table 5). Table 5: Preliminary Project Team Activity

1 An index variable is a variable made up of more than one item or question on a similar topic.

4

Preliminary Project Team Activities ILEAD USA 2015

Illinois 2014

ILEAD USA 2013

Illinois 2011

Percent saying team has met in person 56.0% 44.0% 36.6% 35.1%

Percent saying team has met online or by conference call 54.0% 65.0% 66.7% 64.9%

Frequency of communication among members (median response) Once a week

Once a week

Every other week

Every other week

Percent saying project team has selected a goal 81.0% 88.0% 76.7% 56.8%

Team goal specified so clearly that all members should know exactly what trying to accomplish (median scores). 1 = Strongly agree, 7 = Strongly disagree

3.0 3.0 3.0 3.0

The most notable difference between 2015 and previous cohorts (preliminary activities were not measured during the pilot test of ILEAD in 2010) is the fairly high percentage of 2015 participants saying their team had met in person prior to the first in-person training. One of the findings from the initial evaluations of ILEAD was the apparent operational advantage enjoyed by teams that had organized themselves well enough before training. Meeting in person is a key ingredient in this early organizing activity, as are selecting a team goal (i.e., choosing a team project to work on) and communicating frequently. The baseline survey has been used throughout the evaluations of ILEAD to obtain a pre-ILEAD measure of participants’ perceptions of their confidence in using common Web 2.0 technologies (Table 6). The hope at the beginning of ILEAD in 2010 was that this measure could be taken again at the end of training to see whether or not self-reported Web 2.0 confidence had improved. However, since training content has been designed based on identified needs at the time and available expertise for each cohort, the coverage of these technologies in the training has not been consistent. Consequently, a post-ILEAD measure of confidence has not been feasible. Nevertheless, continuing to take the pre-ILEAD measure has been a useful way to gauge Web 2.0 familiarity. Table 6: Confidence with Web 2.0 Technologies

Web 2.0 Technologies (median scores) 1 = High self-efficacy, 7 = Low self-efficacy

ILEAD

USA

2015

Illinois

2014

ILEAD

USA 2013

Illinois

2011

Illinois

2010

Blogging tools 3.0 2.5 3.8 2.0 3.5

Digital/audio podcasting 5.0 5.0 4.0 4.0 6.0

Digital photography 3.0 3.0 2.0 3.0 3.0

Gaming 4.0 5.0 4.0 4.0 4.0

Instant messaging 1.0 2.0 1.0 1.0 2.0

Photo-sharing websites 2.0 2.0 2.0 2.0 2.0

RSS feeds 4.0 3.5 2.5 2.0 3.0

5

Web 2.0 Technologies (median scores) 1 = High self-efficacy, 7 = Low self-efficacy

ILEAD

USA

2015

Illinois

2014

ILEAD

USA 2013

Illinois

2011

Illinois

2010

Social bookmarking 3.0 3.0 2.0 3.0 4.0

Social networking 1.0 1.5 1.0 2.0 2.0

Tagging 3.0 3.5 3.0 4.0 5.0

Videoconferencing 3.0 3.0 3.0 4.0 4.0

Virtual referencing 3.0 3.0 3.0 2.0 3.0

Web conferencing 3.0 3.0 3.0 4.0 4.0

Wikis 3.0 2.0 2.0 2.0 3.0

Overall Web 2.0 self-efficacy at baseline 3.3 3.0 2.8 2.5 3.3

There has been remarkable consistency over time in ILEAD participant confidence with the different Web 2.0 technologies. With most of these technologies, only a point on the 7-point measurement scale has separated the high from the low scores for the different cohorts. 2015 was no exception. As noted at the beginning, mentors also were asked to complete a baseline survey. All of the questions were the same as those included in the participant survey, except for the section on preliminary team activity, which is excluded from the mentor survey because mentors were not familiar with their team at that point. While across all cohorts mentors have been older and more experienced than participants, the 2015 group mentors was especially notable in this regard. They were, on average, about 15 years older than participants and with more than twice as much experience in library work. They were more likely to be supervisors and to supervise more people. The 2015 mentors reported a higher level of group skills than participants did, but they were somewhat less confident in their Web 2.0 abilities. Mentors were much more apt to say they worked in an academic library or other type of library. On job interdependence, job independence, and performance and mastery motivations the mentors were very similar to participants.

Learning and Use of Training Content The spring and fall surveys asked participants to evaluate the previous in-person training conference – the March conference in each state for the spring survey and the June conference in each state for the fall survey. The surveys posed three questions about the content of each of the conference sessions participants attended: how much did you learn, how much has your team used that content, and how much have you personally used that content. Across all ten states in the 2015 cohort, each state-specific training conference included some of the same sessions along with sessions specifically tailored to the identified needs in each state. Because of wide variations across states in the content taught, it is not possible to analyze the perceived effects of the training by content area. Instead, the evaluation computed for each of the three questions the median response for all states across all sessions for each of the two conferences (Table 7). Table 7: Learning and Use of Training Content

6

Learning and Use of Training Content (median scores) 1 = A lot, 7 = Not at all

ILEAD USA 2015

Illinois 2014

ILEAD USA 2013

Illinois 2011

Illinois 2010

How much did you learn?

March in-person program 2.8 2.5 4.2 2.0 NA

June in-person program 2.0 2.3 3.0 3.0 2.7

How much has team used?



How much have you used?



Overall, the 2015 cohort produced results similar to those for the 2014 Illinois cohort and notably better than the previous multistate replication of ILEAD in 2013. Throughout the history of ILEAD, the second in-person training program in June has tended to receive somewhat higher ratings from participants than the March training, and that was the case again in 2015. The 2015 training showed particular improvement in team use of training content between the March and the June conferences. There was also marked March-to-June improvement in team use during the 2013 program, although the latter ratings were less favorable overall than those for 2015. Evaluation data do not afford an explanation for why ratings improved between March and June for the 2015 and 2013 groups. Most likely, June training content has been more influenced by team needs that are more easily articulated once the program is underway and teams gain a clearer idea of what they are trying to accomplish with their projects. While the learning and use ratings for 2015 were not quite as positive as those for the 2014 Illinois cohort, they were close, which is noteworthy given the vastly greater size and complexity of the effort in 2015. This suggests that the Illinois State Library has continued to learn from its experience in developing the capability to implement the model on a larger scale in ways that meet the needs of participating librarians. There is still room for improvement as open-ended comments in the spring, fall, and final surveys indicate (see Appendix B for responses to open-ended questions from the 2015 surveys). But, there can be no doubt that progress has been made. For the 2015 cohort, mentors were asked in the spring and fall surveys to rate how much their team had learned from each training session and how much they were using the content from that session. The median mentor rating for team learning from the March conference was 2.9 on the 7-point scale, almost the same as the median participant rating of their own learning. The median mentor rating for team use of content from March was also 2.9, more than a point better than participant ratings of team use. The mentor rating of team learning associated with the June conference was 2.5, not quite as favorable as the median participant rating of learning but close to the same. The mentor rating for team use of June conference content was 3.7, more than a point less positive than the median participant rating of team use. For participants, team use of what they learned from training improved between March and June, but for mentors it declined. The reason for this disparity in perceptions between mentors and participants is not clear.

Team Functioning

7

Because teams are central to the ILEAD experience, how they function is an important part of the evaluation. Much of the learning that participants do occurs through or in direct relation to their team. Although completing the team project during the nine-months of the program is not a requirement and teams are told at the outset to put more emphasis on the journey than the destination, bringing the project to fruition is an unavoidable expectation for all teams. So, how team members work together on their projects matters. Evaluation of team functioning focuses on six areas that research has shown affect team performance and the ways and degree to which team members gain from the experience. Team direction refers to the influence of goals and the goal or direction setting process. Team structure has to do with how the work of the team is organized. Team capability concerns the team’s wherewithal to achieve goals and get work done. Team cohesion denotes the quality of the relationships among team members and member identification with the team. Team communication has to do with how often members communicate with one another and the helpfulness of the different ways in which this communication can happen. Team commitment refers to the ways in which member behavior reflect devotion to the team and what the team is trying to accomplish. Except for commitment, which includes measures of actual behavior, the evaluation measures all areas of team functioning by asking ILEAD participants about their perceptions. This section of the report presents findings on team functioning as ILEAD USA unfolded in 2015. These findings are drawn from the spring and early fall surveys. Team functioning was also addressed in the final survey which occurred after the conclusion of ILEAD in October. Findings from that survey are reported in the next section. In what follows an * next to a survey item means the item in the original survey that participants completed was framed negatively and has been rephrased in a positive way for the report. Giving all items the same valence makes for easier interpretation. The median response on the 7-point scale has been reversed accordingly. For example, an item in the survey might read “what the team is supposed to accomplish remains unclear,” and the median response to this item might be a 6, indicating disagreement. For the report, this item would be reworded as “what the team is supposed to accomplish remains clear,” and the median response would be flipped to a 2 to indicate agreement. Best practice in survey design calls for using both positively and negatively valenced items in order to reduce the response bias that can occur when all items have the same valence.

Team Direction The spring and fall surveys asked participants questions about the direction of the team (Table 8). The setting of direction plays a critical role in providing participants with the reason for the team’s existence and their reason for being a member of it. When direction is poorly defined or takes too long to establish, member motivation to participate is apt to lessen, and conflict is more likely to arise as a function of uncertainty. Table 8: Team Direction

8

Team Direction (median scores) 1 = Strongly agree, 7 = Strongly disagree

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Illinois 2010 Fall

Team goal is specified so clearly that all members should know exactly what teem is trying to accomplish.

3.6 3.6 2.0 2.0 2.0 2.5

What team supposed to accomplish remains clear.*

2.0 1.0 2.0 2.0 5.5

Goals and priorities of team are clear enough.*

2.0 1.0 2.0 2.5 2.0 2.0 4.0

Conflicting priorities do not exist on the team.*

3.0 2.0 2.0 2.0 3.0 3.0 2.0 4.0 5.0

There is not much uncertainty and ambiguity about what the team is supposed to accomplish.*

3.0

The goal of the team makes a difference to people not on the team.*

2.0

Team maintains a high standard of work at all times.

2.0

Team does not have much difficulty actually carrying out the plans it makes to proceed toward its goal.*

3.0 2.0

2.0

4.0 3.0

Team goal of great consequence for those we serve.

2.0 3.0 2.0 2.0 3.0 2.0 3.0 3.0 3.0

Median 2.0 3.0 2.0 2.0 2.3 2.0 2.0 4.0 3.0

The team direction results for 2015 are largely consistent with earlier cohorts starting with Illinois in 2011, although there are differences worth noting. Overall they show participants agreeing that team direction was clear and teams were pursuing their goal appropriately. However, there are indications that 2015 teams may have been somewhat more challenged in establishing a clear direction and being able to act on that direction. The median score of 3.6 for members being very clear about the team goal at both points in time is close to the neutral position on the 7-point scale, a full one and a half points less favorable than in the three preceding cohorts. The lack of improvement on this measure between the spring and fall suggests that at least some teams weren’t entirely able to overcome the ambiguity that always defines organized work at the outset. Continued ambiguity might then have contributed to more struggle on teams, as the decline in agreement between the spring and fall with the item on not having conflicting priorities seems to imply. Interestingly, while goal conflict seemed to have increased some, belief in the consequentiality of the team goal improved. The shifts here are not dramatic, and resonate with the normal experience of scaling up model programs. Maintaining consistency becomes more difficult with increasing scale.

9

Team Structure Team structure is the necessary counterpart of team direction (Table 9). A team can have a clear direction, but if that direction is not embodied in the organization of roles and responsibilities, then the team may founder. The work needed to pursue goals may fall through the cracks, and members may wind up duplicating one another’s efforts. Good structure helps a team operate efficiently. Table 9: Team Structure

Team Structure (median scores) 1 = Strongly agree, 7 = Strongly disagree

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Individual roles on team are very clear and we don't stray from them.

4.0 4.0 4.0 2.0 2.5 4.0 4.0

There is a clear team leader who guides what we do.

3.0 4.0 2.0 3.0 4.0 3.0 4.0 4.0

Team follows a very structured work schedule.

4.0 4.0 4.0 5.0 5.0 5.0 5.0

Team is just the right size to accomplish its goal.

4.0 2.0

Members of the team agree on how members are expected to behave.

2.0

Standards for behavior on the team are clear.*

2.0

Median 4.0 2.0 4.0 4.0 4.0 3.0 4.0 4.0

Strong team structures have generally not been characteristic of ILEAD, and this is evident in the results for 2015. While the spring survey showed participants agreeing that standards of conduct were clear at that point, they were ambiguous about the presence of a specific team leader. Identifying a leader is not a requirement of the model, and teams throughout ILEAD’s history have often seemed to prefer a flatter, more egalitarian to a more hierarchical structure. As can be seen, in addition to being ambiguous about leadership, teams by the fall survey also seemed ambiguous about structuring roles and work. How important team structure is to fulfilling the educational purpose of ILEAD is not clear. It may be that only minimal structure is necessary for teams to accomplish or make sufficient headway with their projects. But, even if more structure was desirable, it may be difficult to achieve within the ILEAD approach. ILEAD teams typically include people from different libraries, many if not most of whom have full-time jobs that limit how much time and effort they can devote to extramural projects. Strong structures require an intensity of interaction that may simply not be possible, on average, in ILEAD. Structure may matter more at the individual level. Individuals who have a role that is meaningful to them may be more willing and able to do their part to help the team. Since the 2011 Illinois cohort, the evaluation has measured individual role perceptions using a standard set of items that differentiate between a specialized and more general role. Specialized roles, when they match a person’s abilities and

10

interests, are likely to be more motivating and to result in a more solid contribution to the work of a team attempting to complete a specific project. Table 10: Individual Member Roles on Team

Role on Team (median scores) 1 = Strongly agree, 7 = Strongly disagree

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

I am responsible for a particular task or set of tasks.

2.0 2.0 2.0 2.0 2.0 2.0 2.0 3.0

Every member of the team has the same role at this point.

3.0 3.0 3.0 4.0 4.0 4.0 4.0 3.0

I am on the team to represent my library.

3.0 3.0 2.0 3.0 3.0 3.0 2.0 2.0

I am on the team because of my particular expertise and/or skills.

2.0 3.0 2.0 2.0 2.0 3.0 3.0 3.0

I am on the team to learn more about Web 2.0 technologies.

3.0 3.0 2.0 2.0 2.0 3.0 2.0 2.0

The role perceptions results for 2015, like those for the 2014 Illinois cohort, present a mixed picture (Table 10). On the one hand, and particularly by the fall of last year, the items that measure a more specialized role (responsible for particular tasks, on team because of particular expertise) attracted a healthy level of agreement. On the other hand, the items that represent a more general role (same role, represent my library, learn more about Web 2.0), while the agreement with them was somewhat weak in both the spring and the fall, it was still agreement (i.e., median score of 3). By contrast, in cohorts before 2014, participants were inclined toward the neutral position on the item that most captures generality – everyone having the same role (median score of 4.0) It is difficult to envision a team being effective when all of its members perceive themselves to be doing the same work. Consistent with the predictions of research, there is some evidence from further statistical analysis that role specialization may have benefited teams in 2015. When the two items measuring specialization in the spring survey are converted into a single index variable, this variable is positively correlated with a factor measuring team direction at the time of the spring survey (.267, p=.005) and a factor measuring team structure at the time of the fall survey (.293, p=.004).

Team Capability Team capability speaks to member beliefs about the suitability of the skills and knowledge of team members. A team can have a clear, compelling goal and a sound structure, but if doesn’t have the necessary talents, achieving its goal will be difficult. Table 11: Team Capability

11

Team Capability (median scores) 1 = Strongly agree, 7 = Strongly disagree

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Members have more than enough talent and experience for our goal.

2.0 2.0 2.0 2.0 2.0

No member lacks the knowledge and skills he/she needs to do his/her part of the team's work.*

2.0 2.0 2.0 2.0 2.0

Information and knowledge available to team has been more than adequate.

2.0 2.0 3.0 3.0

Everyone on team has the special skills needed.

2.0 2.0 2.0 2.0 3.0

Team has a broad enough range of experiences and perspectives.*

2.0 1.0 2.0 2.0

Team has nearly an ideal mix of members.

2.0 2.0 2.0 2.0 2.0

It has been easy for the team to get the information it needs to do its job.*

3.0

All members of the team carry their share of the overall workload.*

Team members know what skills and knowledge they each possess relevant to the team goal.

2.0 3.0 2.0 3.0 2.0 2.0

Team members are assigned to tasks suited to their knowledge and skill.

2.0 3.0 2.0 2.5 2.0 4.0

Team members have a good map of each other's talents and skills.

3.0 3.0 3.0 3.0 2.0 3.0

The team often comes up with innovative ways of proceeding that turn out to be just what's needed.

2.0 2.0 3.0 3.0

Members of the team actively share their special knowledge and expertise.

2.0

Median 2.0 2.0 2.0 2.0 3.0 2.0 3.0

Participants’ ratings of their team capability have been uniform throughout the history of ILEAD. As can be seen in Table 11 above, surveys have used a wide variety of items to measure capability.

12

Consistently, participants have agreed that their team has the needed skills, knowledge, and experiences. Agreement here has not been as strong as it could have been, but it has been strong enough to give participants’ confidence in their team’s wherewithal. Sometimes reported agreement has been less in the spring survey and higher in the fall survey – 2015 being a case in point. This suggests that participants may perceive their team as becoming more capable with the passage of time, which makes sense. Next to team cohesion, which is discussed in the following subsection, team capability has garnered the most reliably favorable views from participants. A team may struggle some to set direction and clarify its structure, but it seems unlikely to doubt its capability. This may be because people perceive abilities as mainly being possessed by and under the control of individuals, while direction and structure are group attributes.

Team Cohesion Team cohesion focuses on the emotional dimension of teamwork. It gets at how well members get along with one another and their sense of belonging to the team. When people enjoy being on a team and see the experience as meaningful and emotionally satisfying, they may be more motivated to do what they can to help the team succeed. Table 12: Team Cohesion

Team Cohesion (median scores) 1 = Strongly agree, 7 = Strongly disagree

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Illinois Fall

2010

Members of the team are not dissimilar to work together well.*

1.0 1.0 1.0 2.0 4.0 4.0

How seriously members ideas are taken does not depend that much on who the person is rather than how much he or she actually knows.*

2.0 2.0 2.0 2.0 2.0 4.0

Working together energizes and uplifts members of the team.

2.0

No members of the team do not get along with others.*

1.0

I enjoy the kind of work we are doing on the team.

2.0

Working on the team is not an exercise in frustration.*

2.0

I do not feel that the other members of the team will judge me on the things that I say.

2.0 1.0

I would feel guilty if I left the team now.

1.0 1.0 1.0 1.0 1.0 1.0 1.0

I feel a strong sense of belonging to the team.*

1.0 1.0 2.0 2.0 2.0 2.0 2.0

13

Team Cohesion (median scores) 1 = Strongly agree, 7 = Strongly disagree

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Illinois Fall

2010

I would not leave my team because I have a sense of obligation to it.

1.0 2.0 1.0 1.0 1.0 1.0

Members of the team have to depend heavily on one another to get work done.

2.0 3.0 3.0 2.0 2.0

The team has a great deal of personal meaning for me.

2.0 2.0 2.0 3.0 3.0 2.0 3.0

Team handles differences of opinion, not privately or offline, but rather, directly.*

3.0 3.0 3.0 3.0 3.0 3.0

Median 2.0 2.0 2.0 2.0 2.0 1.5 2.0 2.0 3.5

The 2015 cohort gave among the highest marks yet to team cohesion, with several items receiving a median response of 1, meaning strong agreement (Table 12). From the evaluation data available, it is not obvious why cohesion has been rated as highly as it has been by participants. One possibility is that it’s compensatory. Strongly identifying with a team may be a way in which members compensate for the challenges they otherwise experience setting direction and structure under the rules of more limited interaction inherent in the ILEAD model. That identification may be further reinforced by members feeling that ILEAD is putting them on the cutting edge of innovation in the library world. As one participant put it in comment in the final survey in 2015, “Great overall experience! Library systems across the nation have always been about changing with the needs of the community, and ILEAD is at the forefront of providing insight on what the appropriate innovative adjustments should entail.”

Team Communication The team orientation of ILEAD places a premium on good communication among team members. The surveys provide participants with a list of communication modes and ask them to judge each mode in terms of its frequency and helpfulness. For the 2015 cohort, these questions were only included in the fall survey, since previous cohorts indicated that frequency and helpfulness for each mode tended not to change much between the spring and the fall. Table 13: Frequency of Communication

Frequency of Communications (median scores) 1 = Very often, 7 = Not at all

ILEAD USA Fall

2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Email 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Electronic meetings 7.0 6.0 6.0 7.0 6.0 4.0 3.0

Teleconferencing 6.5 6.0 6.0 7.0 7.0 6.0 7.0

Videoconferencing 6.0 6.0 6.0 7.0 7.0 7.0 7.0

14

Frequency of Communications (median scores) 1 = Very often, 7 = Not at all

ILEAD USA Fall

2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

One-on-one phone calls 5.0 5.0 7.0 5.0 6.0 4.0 4.0

File sharing 2.0 1.0 2.0 2.0 2.0 2.0 4.0

Voicemail 7.0 6.0 7.0 7.0 7.0 7.0 7.0

Face-to-face meetings 3.0 2.0 2.5 3.0 3.0 2.0 3.0

Median 5.5 5.5 6.0 6.0 6.0 4.0 4.0

The reported frequencies of different modes communication in 2015 fit well into the pattern demonstrated by previous cohorts, with the exception of the 2011 Illinois cohort, which made more use of electronic meetings and phone calls (Table 13). Email was the dominant mode in 2015, as it has been throughout ILEAD since the start. Newer, more costly means of communicating, including electronic meetings along with videoconferencing and teleconferencing, have generally been used far less often than more established methods. The general ILEAD pattern has been for participants to rely mainly on email and file sharing, asynchronous modes of communicating, with face-to-face meetings used when real-time interactions are needed. Table 14: Helpfulness of Communication

Helpfulness of Communications (median scores) 1 = Very helpful, 7 = Not at all

helpful

ILEAD USA Fall

2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Email 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Electronic meetings 6.0 4.0 4.0 6.0 7.0 2.0 2.0

Teleconferencing 4.0 6.0 5.0 4.5 4.0 4.0 3.5

Videoconferencing 3.0 5.0 4.0 6.0 4.0 3.0 3.5

One-on-one phone calls 3.0 6.0 4.0 3.0 3.0 1.0 1.0

File sharing 1.0 1.0 1.0 1.0 1.0 1.0 2.0

Voicemail 7.0 6.5 5.0 4.5 4.0 3.0 3.5

Face-to-face meetings 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Median 3.0 4.5 4.0 3.8 3.5 1.5 2.0

For 2015, the modes used most often – email, file sharing, and face-to-face meeting – were also regarded as the most helpful (Table 14). This has been the case for nearly all of the history of ILEAD, except, again, for the 2011 Illinois group when phone calls also received the highest score for helpfulness. What may be most interesting is how even some modes of communication used infrequently have been regarded as more rather than less helpful. This is evident in comparing the median scores for frequency and helpfulness of all modes combined (last row of each table), where the helpfulness scores for all years, including 2015, are clearly more positive than the frequency scores. In ILEAD, nearly all forms of communication are perceived to have value in helping teams accomplish their work.

15

Team Commitment Team cohesion measures emotional or affective commitment to the team. Commitment can also be measured more behaviorally. Both forms of commitment are typically helpful to a team, although demonstrated behavioral commitment is likely to be more directly related to the team’s ability to get work done. ILEAD has measured behavior commitment in the spring and fall surveys by asking participants how much time they have spent on ILEAD, how much of a priority it is for them, and their sense of the extent to which it conflicts with their other job responsibilities (Table 15). Also, in the spring survey for 2015, an additional item was included soliciting participant perceptions of the time commitment of their team members. Table 15: Member Commitment

Member Commitment

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014

Spring

Illinois 2014 Fall

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Illinois 2010 Fall

Time given to ILEAD since last training (median scores)

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

1-2 hours/

wk

Priority of ILEAD for you 1 = High priority, 7 = Low priority

3.0 3.0 3.0 3.0 2.0 2.0 2.0 3.0 2.0

Extent to which your other responsibilities have conflicted with ILEAD 1 = A lot, 7 = Not at all

3.0 3.0 4.0 3.0 2.0 3.0 3.0 3.0 2.0

Members of the team demonstrate their commitment by putting in extra hours to help the team succeed. 1= Strongly agree, 7= Strongly disagree

2.0

The median time that participants report giving to ILEAD has been the same since the beginning, one to two hours per week. The priority that participants give to ILEAD has also been consistently positive, although this measure was one-point less positive for the 2015 multistate cohort and the 2014 Illinois cohort than for previous cohorts. It’s possible that ILEAD is experiencing a common pattern in the diffusion of innovation, where those initially drawn to an innovation are more eager to be on the cutting edge than those who follow after them. Alternatively, it could be that librarians have simply become more pressed for time in the past couple of years as resources tighten. As the findings for conflict between one’s job and ILEAD show, the program has, for the most part, consistently been perceived as at least somewhat competitive with a participant’s other responsibilities. That comports with the positive response to the spring 2015 item on putting in extra hours (last row of table), indicating that the median reported time commitment to ILEAD of 1-2 hours a week was perceived as additional effort.

16

Additional Measures of Team Functioning Beginning with the 2011 cohort, the spring and fall surveys have been used to obtain two other ratings of team functioning. One rating has asked participants to evaluate the importance to themselves and to their team of three key team attributes – goal accomplishment (direction), clarity of roles and responsibilities (structure), and sense of belonging (cohesion). Research has found that importance judgments offer a more precise test of the strength of people’s attitudes or perceptions. The other rating has asked participants to assess how clear they are about what’s expected of their team. While other survey items call for evaluations of clarity, this rating breaks the work of the team down into more specific categories. Table 16: Importance of Direction, Structure, and Belonging

Importance of Key Aspects of Team (median scores) 1=Very important, 7=Not at all important

ILEAD USA Fall

2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Importance to you personally that team accomplishes its goal.

1.0 1.0 1.0 2.0 1.0 1.0 1.0

Importance to your team that it accomplishes its goal.

1.0 1.0 1.0 2.0 1.0 2.0 2.0

Importance to you personally that roles and responsibilities on team are clear.

2.0 2.0 2.0 2.0 2.0 2.0 2.0

Important to your team that roles and responsibilities are clear.

2.0 2.0 2.0 3.0 3.0 3.0 3.0

Importance to you personally that you have sense of belonging to team.

1.0 1.0 1.0 2.0 2.0 2.0 2.0

Importance to team that every member have sense of belonging.

1.0 1.0 1.0 2.0 2.0 2.0 2.0

As can be seen, ratings of importance have been consistently high across cohorts, across dimensions of team functioning, and between one’s personal rating and one’s view of the team’s rating (Table 16). Judgments have tended to be higher for the importance of goal accomplishment and belonging than for the importance of structural clarity. This lends further support to the observation made above that in the ILEAD context, structure may not be quite as salient to participants as team direction and cohesion. As with so many other aspects of team functioning reviewed in this report, participants’ reports of their clarity about the work of their team have been very similar across time (Table 17). With only a couple of exceptions in the fall 2014 survey for Illinois, the highest level of clarity has eluded ILEAD. But, clarity has still generally fallen into what might be considered the acceptable range for all aspects of the work teams have to do. Also, for every cohort since 2011, the overall median score for clarity has improved between the spring and the fall surveys. This indicates that clarity improves as the work gets done. It may be that because ILEAD involves activity external to or on the periphery of most participants’ regular

17

jobs, relatively modest time commitments, and limited interactions among team members, high levels of clarity will always be difficult to attain under the current model. Table 17: Clarity of Teamwork

Clarity about Work of Team (median scores) 1=Very clear, 7=Not clear at all

ILEAD USA Fall

2015

ILEAD USA

Spring 2015

Illinois 2014 Fall

Illinois 2014

Spring

ILEAD USA 2013 Fall

ILEAD USA 2013

Spring

Illinois 2011 Fall

Illinois 2011

Spring

Each of the tasks that need to be accomplished.

2.0 2.0 1.0 2.5 2.0 2.5 2.0 4.0

Dividing up time among tasks. 3.0 3.0 2.0 3.0 2.0 3.0 2.0 4.0

What each member personally responsible for doing.

2.0 2.5 1.0 2.0 2.0 2.0 2.0 3.0

When each task will be completed.

3.0 3.0 3.0 3.0 2.0 2.0 3.0 4.0

What constitutes success. 2.0 3.0 2.0 3.0 3.0 3.0 3.0 4.0

Criteria used to evaluating final team product.

3.0 4.0 3.0 4.0 3.0 4.0 3.0 4.0

What final output of team will look like.

2.0 4.0 3.0 3.0 2.0 3.0 2.0 4.0

Median 2.0 3.0 2.8 3.0 2.0 3.0 2.0 4.0

Judging the ILEAD Experience

The final survey for ILEAD occurs after the end of each year’s program in November and December. It asks participants to assess both their team and their overall ILEAD experience, the support they received for their participation from the library where they work, and the effects of their team project.

Team Functioning Participants are asked to look back and evaluate the direction, capability and cohesion their team displayed while ILEAD was underway. These are the dimensions of team functioning most closely aligned with how ILEAD operates. Prior to 2014, team direction was largely neglected in the final survey for ILEAD cohorts. In 2015, five items on direction were included in the final survey as a way of assessing whether the process of setting direction had worked or not for teams (Table 18). The results showed that participants had a fairly healthy level of satisfaction with their team’s goal and the process they followed in setting and acting on that goal. Two of these items were also on earlier surveys in the year – lack of conflicting priorities and not having difficulty carrying out plans. Agreement with not having difficulty carrying out plans strengthened by the time of the final survey, while there was no change in perception of the lack of conflicting priorities. Agreement on the latter had deteriorated by a point between the spring and fall surveys and stayed that way through the final survey.

18

Table 18: Looking Back at Team Direction

Team Direction (median scores) 1=Strongly agree, 7=Strongly disagree

ILEAD USA 2015 Final

Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Looking back, the team chose the right goal.*

2.0

The team's goal was challenging; achieving it was not well within our reach.*

2.0

2.0

Conflicting priorities did not exist on the team.*

3.0

2.0

I would not have chosen a different team goal.*

2.0

1.0 2.0 2.0 2.0

Team did not have much difficulty carrying out the plans it made to process toward our goal.*

2.0

Median 2.0 2.0

The final surveys in both 2015 and 2014, as can be seen in Table 19, included several items measuring post hoc perceptions of team capability. This was done to reflect the likelihood, based on previous evaluations, that capability would be strongly associated in participants’ minds with how successful their team had been in completing or working toward completion of its project. Table 19: Looking Back at Team Capability

Team Capability (median scores) 1=Strongly agree, 7=Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Team members had more than enough talent and experience for the goal we tried to accomplish.

2.0 2.0

Team had nearly an ideal mix of members.

2.0 2.0

No member lack the knowledge and skills they need to do their parts of the team's work.*

2.0 2.0

The team had a great deal of personal meaning for me.

2.0

19

Information and knowledge available to the team in pursuing its goal was more than adequate.

2.0 3.0

Everyone on the team had the special skills that were needed for what we are trying to accomplish.

2.0 2.0

The team often comes up with innovative ways of proceeding that turn out to be just what's needed.

2.0 2.0

This was one of the most effective teams which I have ever served.

2.0 2.0 3.0 2.5 3.0

Team members worked very hard to keep one another up to date on their activities.

2.0 2.0 2.0 2.0 2.0

Median 2.0 2.0 2.5 2.3 2.5

Just as with direction, there was good agreement in 2015 with all of the statements intended to measure views of team capability. The results for 2014 were only slightly less favorable. Only a couple of items were used to represent capability in the final survey for cohorts before 2014, but they, too, showed agreement. Readers will recall that above, in discussing team functioning during ILEAD, cohesion was identified as the dimension with the most consistently positive assessments from participants. This was also the case in the final post-ILEAD surveys in 2015 and 2014, although with an interesting exception (Table 20). In both years, agreement with the statement about handling differences of opinion directly rather than privately was notably weaker than with most of the other statements measuring cohesion. This item is also one that has been measured at one time or another in all earlier cohorts, where it earned less favorable marks. Arguably, because ILEAD involves relatively infrequent face-to-face communication where differences can be more easily ironed out, members may be more likely to resort to “offline” communication in resolving problems. Table 20: Looking Back at Team Cohesion

Team Cohesion (median scores) 1=Strongly agree, 7=Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Members of the team were not too dissimilar to work together well.

1.0 1.0

Team handled differences of opinion, not privately or offline, but rather, directly.*

3.0 3.0

20

How seriously members' ideas were taken on the team did not depend on who the person is but on what they knows.*

1.0 1.0

The team had a great deal of personal meaning to me.

2.0 1.0

The team gave me enough opportunities to contribute.

1.0 1.0 1.0 2.0 2.0

I did not feel other members of the team would judge me on the things I said.

1.0 1.0 1.5 2.0 2.0

Median 1.0 1.0 1.3 2.0 2.0

Value of ILEAD The final survey has always included questions that invite participants to make a global assessment of the value of the ILEAD experience along with more specific assessments of key components of the program. 2015 participants’ ratings of the overall value of ILEAD, as shown in Table 21, were not quite as positive as those from previous cohorts. As will be discussed later in the report section on predictors of ILEAD team functioning and outcomes, there are several variables from the spring and fall surveys that may help to explain why the value of ILEAD in 2015, while still positive, didn’t quite match the reviews of earlier cohorts. What those data won’t show is the effect of much greater scale. Expanding the program to ten states in 2015 simultaneously most likely led to more variation in participants’ experiences, and some portion of that variation would have likely leaned in a less favorable direction. Table 21: Global Assessment of ILEAD

Value of ILEAD (median scores) 1=Strongly agree, 7=Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

I got as much out of ILEAD as I thought I would.*

2.0 2.0 3.0 2.0 2.5

ILEAD was one of the most enjoyable experiences of my career.

3.0 2.0 2.0 2.0

My participation in ILEAD is more beneficial to me professionally than personally.

4.0

ILEAD was one of the most useful experiences of my career.

3.0 1.5 2.0 2.0 3.0

Compared to other learning experiences, ILEAD ranks among the best.

3.0 1.0 2.0 2.0 2.0

Median 3.0 1.50 2.0 2.0 2.0

21

In addition to the global assessment, the final survey asked participants to evaluate the different groups of people who were tasked with supporting them during the program, including community representatives, instructors, and mentors. The assessment of community representatives was, as it has been in all previous evaluations, only somewhat positive (Table 22). The challenge since the start of ILEAD has been to identify a clear role for community representatives in helping teams carry out their projects. There has always been some ambiguity around this role, and that shows in how 2015 participants rated their team’s use of community representatives. Table 22: Ratings of Community Representatives

Ratings of Community Representative Involvement (median scores) 1=Strongly agree, 7=Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Team's community representative(s) involved from the start.

4.0 3.0 3.0 3.0 3.0

Did not always make good use of community representative(s).

3.0 3.0 3.0 4.0 2.0

Community representative(s) were best possible people given team goal.

3.0 3.0 3.0 3.0 4.0

Community representative(s) got along very well with team.

3.0 2.0 2.0 2.0 3.0

Median 3.0 3.0 3.0 3.0 3.0

The instructors received favorable marks from 2015 participants, consistent with the finding from previous evaluations (Table 23). Also, the 2015 scores are identical to the previous multistate replication in 2013, and both are not quite as positive as the versions of ILEAD carried out in Illinois. Again, as with other aspects of the program, greater scale may have made achieving consistent quality more challenging. Table 23: Ratings of Instructors

Ratings of Instructors (median scores) 1=Strongly agree, 7=Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Instructors, as a group, were outstanding.

2.0 2.0 2.0 2.0 1.0

Some instructors clearly better than others.

2.0 1.0 2.0 1.0 2.0

22

Without what we learned from instructors, our team could not have made as much progress.

3.0 2.0 3.0 1.0 1.0

Median 2.0 2.0 2.0 1.0 1.0

In something of a reversal from the experience with instructors, the 2015 ratings of mentors, like those from 2013, were more positive than mentor ratings for the Illinois-only cohorts in the other years (Table 24). Table 24: Ratings of Mentors

Ratings of Mentors (median scores) 1=Strongly agree, 7=Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Team could not have gotten as far without help from mentor.

2.0 4.0 2.0 3.0 3.0

Mentor was always available when needed.

1.0 2.0 2.0 2.0 1.5

Quality of advice from mentor was uneven.

6.0 5.0 6.0 5.5 6.0

Mentor always knew what to do when team had problems.

2.0 3.0 2.0 2.5 2.0

Median 2.0 3.5 2.0 2.8 2.5

Support from Library People cannot participate in ILEAD unless the library where they work agrees to it. Also, the stronger the support from their library, the more likely they are to have a hospitable environment for using what they learn. The final survey includes several items designed to elicit participants’ sense of this support. The findings from the 2015 survey indicate that, as with previous cohorts, participants were supported fairly well by their home libraries (Table 25). Having the time for ILEAD and library awareness and backing of their participation earned high marks in the final survey. Somewhat lower marks were received by the tension between a participant’s job and the demands of ILEAD participation, which was also evident in evaluation of the similar item in the spring and fall surveys as discussed earlier in this report. Also, how much someone’s participation in ILEAD would matter to their library has been consistently judged somewhat ambiguously – positive, but not that positive. Table 25: Support from Library

Support from Library (median scores) 1 = Strongly agree, 7 = Strongly disagree


Illinois 2014 Final


Illinois 2011 Final

Illinois 2010 Final

Never felt that my participation got in the way of my other work.

4.0 3.0 4.0 4.0 3.5

My library gave me all that time I needed to participate.*

1.0 1.0 2.0 2.0 1.5

23

My participation was well-known to my co-workers.

2.0 3.0 2.0 1.5 2.0

Everyone in the library where I work was supportive of my participation.*

2.0 3.0 2.0 2.0 3.0

My participation mattered as much to my library as it did to me.*

3.0 3.5 3.0 3.5 3.0

Median 2.0 3.0 2.0 2.0 3.0

Impact of Team Projects Before 2014, the final survey did not delve into the status of team projects. The 2014 survey in Illinois added a section on project status, and this was repeated in the 2015 survey for the ten-state replication (Table 26). Table 26: Impact of Team Project

Impact of Team Projects


Illinois 2014 Final

Percent saying team completed its project.

62.4% 63.0%

If project completed, how much are your library patrons benefiting from it? (median response)

Some A lot

Percent saying impact of project will increase over next 12 months.

87.9% 54.0%

Percent saying project not complete yet, efforts are being made to finish it.

22.9% 31.0%

If efforts being made to finish project, when will it likely be completed? (median response)

4-6 months

4-6 months

There are both similarities and differences between 2015 and 2014. The percent of responding participants who said their project had been completed was almost identical in both years, at close to two-thirds. In 2015, this average, however, masked differences among the ten participating states. Participants from one state reported 100 percent project completion, while those from another reported only 17 percent. The median response on the expected benefit of the project for library patrons was not as strong in 2015. Again, there were notable differences among the 2015 states, with a few states providing a median response of “a lot” of benefit, and one state with no participant indicating a lot of benefit. For 2015, almost 9 in 10 responding participants said they expected the impact of the project to increase in the next year, considerably more than the 5 in 10 who said this in 2014. Among the 2015 states, the range in response on this question was 100 percent to 58 percent anticipating

24

increased impact. A slightly smaller share of 2015 participants, compared to 2014 participants, indicated projects were not finished and efforts were underway to complete them. Variations on this question among states in 2015 were not pronounced, nor were they for the follow-up item asking for an estimate of when completion might occur, which was the same as that for 2014.

Predictors of ILEAD Team Functioning and Outcomes An important feature of the evaluations of ILEAD is to try to understand what aspects of the program and its participants appear to shape how teams function and how this functioning, in turn, may affect program outcomes as measured in the final survey. Understanding these influences may then point to ways in which ILEAD could be modified or strengthened to improve outcomes. To get at what affects what in ILEAD, two types of analyses are done. First, attempts are made to reduce the many variables (items) that are measured in the various ILEAD surveys to a smaller number of factors. A smaller number of factors make analysis easier and interpretation of the results more straightforward. The surveys consist mostly of specific scales that each assesses a given area of evaluative importance. For example, the baseline survey uses a number of items to measure performance and learning motivation, and the spring, fall, and final surveys use a wide range of items to measure the different dimensions of team functioning. Factor analysis is used to find out how respondents categorize these survey items. Similar responses to a given set of items would indicate that participants view those items as measuring the same underlying construct or factor. The hope is that participants’ constructs match those the evaluation is designed to measure – for instance, that the items measuring team cohesion would also be understood by participants as measuring team cohesion. The second type of analysis, linear regression, uses the resulting factors and other variables from the surveys – referred to as independent variables – to examine which of them, if any, are significantly associated with desired outcomes – referred to as dependent variables. A regression is an equation that uses data from the evaluation to produce predictions of the extent to which a dependent variable appears to be explained by one or more independent variables.

Predictors of Team Functioning In examining team functioning, factor analysis yielded factors that are roughly consistent with the dimensions of team functioning the evaluation was designed to measure. From the spring 2015 survey, the analysis identified, in order of strength: 1) a factor for team capability that included most of the items intended to measure capability2, 2) a factor for team cohesion, 3) a factor for team direction, 4) a factor for team structure, 5) a factor for having a generalized role on team and a factor for having a specialized role, and 6) a factor consisting of the clarity scale from the survey (Table 17 above). From the fall 2015 survey, the analysis identified, in order of strength: 1) a factor for team cohesion, 2) a team capability factor, 3) a team structure factor, and 4) a team direction factor. Regression analysis was not able to identify any significant relationships between variables measured by the baseline survey and the factors that emerged from the spring survey. All of the statistically significant relationships were found in treating the factors from the fall survey as dependent variables that could be predicted by the spring survey variables and baseline measures.

2 A few of the items intended to measure team capability loaded on another, weaker factor. This factor was dropped from the analysis as duplicative.

25

The team cohesion factor from the fall survey was predicted by four factors derived from the spring survey. The two strongest factors were early team cohesion and early team capability, both having a positive association with team cohesion in the fall. The two other predictors were negatively associated with fall team cohesion: the spring measures of clarity and having a generalized role on the team. Combined, these four independent variables/factors explained 67 percent of the variance in the team cohesion factor from the fall survey, which is a strong result. This finding suggests that being cohesive and capable early on, combined with some differentiation in member team roles, helped to sustain cohesion, that is, helped to keep members working together well. Why the clarity factor has a negative association here is not obvious. It may be that too much clarity in the early stages was experienced by ILEAD participants as rigidity that got in the way of relationships.

The team capability factor from the fall survey was predicted by two variables: 1) the team capability factor from the spring survey and 2) the baseline item asking participants if their team had selected a goal. These two variables were included in a regression equation with two other independent variables that turned out to be statistically insignificant3, which together explained 45 percent of the variance in the team capability factor from the fall survey. That team capability in the spring would predict team capability in the fall is not surprising. More interesting is the influence of having chosen a team goal at the outset. Previous evaluations of ILEAD have found that deciding on a goal early tended to benefit team functioning down the road, most likely because it gives a team direction in identifying and mobilizing its capabilities.

The team structure factor from the fall survey was predicted by two factors from the spring survey: 1) team capability and 2) team direction. They were in a regression equation along with the spring clarity factor (which turned out not to be significant), and that equation accounted for 44 percent of the variance in the fall team structure factor. The logic here is fairly straightforward. Structure, which has to do with clarity about the work and how it will be organized, benefits from a team being clear early on about the purpose that work is supposed to serve and the team’s perceived wherewithal in carrying it out.

Predictors of Outcomes Factor analysis of the final survey responses yielded four factors: 1) a factor consisting of survey items that participants appeared to associate with team success (e.g., members worked hard to keep one another up-to-date, one of the most effective teams I ever served on), 2) a factor that included most of the survey items measuring the value of the ILEAD experience, 3) a factor that included items measuring participants’ lack of felt conflict between their job and ILEAD participation, and 4) a factor that included items measuring participants’ sense of having felt “psychologically safe” on the team (i.e., team was safe for interpersonal risk taking, like voicing one’s opinion). Regression analysis showed that all four factors were predicted by measures from the earlier surveys. Earlier measures also predicted one of the final survey items evaluating project impact – impact of project will increase, decrease, or stay the same over time. There were no significant predictors of any of the other impact measures.

3The insignificant variables were included in the regression because they had strong, statistically significant bivariate correlations with fall team capability. Once those variables were considered with the other two together in the regression, their significant relationship with fall capability disappeared.

26

The team success factor from the final survey was predicted by the team cohesion and team structure factors from the fall survey and a factor consisting of the most helpful modes of communication from the fall survey (i.e., email, file sharing, face-to-face meetings). Combined these three factors explained 68 percent of the variance in the team success factor, a strong result. This suggests that in looking back over their experience, participants who perceived their team to have been successful were ones three months earlier who had seen their team as being cohesive, well-organized, and reliant on particular forms of communication. Note that spring or fall team capability did not predict perceptions of team success.

The factor measuring the value of the ILEAD experience from the final survey was predicted by several different factors and variables from the fall and spring surveys: 1) the team capability factor from the fall survey, 2) a factor that included only the sense of belonging items from the fall team cohesion factor, 3) both the fall and the spring survey items asking participants to rank ILEAD’s priority for them, 4) an item from the fall survey that asked participants if they expected their project to be completed by the end of ILEAD, and 5) the conflict with job item from the spring survey, which was negatively associated. Together, these variables accounted for 62% of the variance in the value of experience factor, again, a strong result. The story that appears to be told here is, at least partly, one of self-fulfilling prophecy, in a good sense. Participants who made ILEAD a priority, who didn’t experience significant conflict between it and their job, and who perceived themselves to be on a capable team were more likely to see the experience as having been especially valuable.

This next regression covers some of the same ground as the previous one, and probably tells a similar story. The final survey factor that measured the extent to which participants experienced ILEAD as being less, rather than more, in conflict with their job was predicted by five items from the fall survey and a demographic variable from the baseline survey: 1) the conflict with job item from the fall survey, 2) the amount of time given to ILEAD as measured in the fall survey, 3) the fall survey item asking if the team project would be completed by ILEAD’s end, 4) the team cohesion factor from the fall survey, and 5) whether or not a participant was a supervisor from the baseline survey. This regression equation explained 42 percent of the variance in the lack of job conflict factor.

The psychological safety factor from the final survey was predicted by the team structure factor from the fall survey. It was included in a regression with three other insignificant variables/factors, although one of these, the factor measuring the most helpful modes of communication in the fall, came close to significance. The equation only accounted for 7 percent of the variance in psychological safety. Possibly, perceiving one’s team as having a clear structure in the final stretch also meant that the team had established the healthy norms for member contributions on which psychological safety depends.

The final survey item asking about expected change in the impact of the team project was predicted by one item from the fall survey, the sense of belonging factor. It was included in a regression with three other variables that turned out to be statistically insignificant. The equation accounted for 24 percent of the variance in the expected change in project impact item. The suggestion here is that participants who by the fall felt a stronger sense of belonging to their team were more likely to perceive their team project in positive way by the end.

27

Appendix A

Survey Instruments

28

ILEAD Baseline Survey 2015

What is your role in ILEAD USA?

Participant

Mentor

Other

Has your ILEAD U project team met in person?

Yes

No

How long have you known each of the members of your team?(To help answer this question, please put

the first name of each of the other members of your team on a line. Then next to it, indicate how many

years you have known the person. We will not record the names; this is just to assist you in answering

the question.

Team Member 1

Number of Years Knowing Team Member 1

Team Member 2


Team Member 3


Team Member 4


Has your ILEAD U project team "met" online or had a conference call?

Yes

No

Excluding any member of your team from the library as you, how often have you communicated with

other members of your project team since you became a member?

Daily

Every other day

Twice a week

Once a week

Every other week

Once a month

Less than once a month

29

Has your ILEAD U project team selected a goal?

Yes

No

If yes... Please indicate your level of agreement with the following statement: Our project team's goal is

specified so clearly that all members should know exactly what we are trying to accomplish.

Strongly agree

2

3

Neither Agree nor Disagree

5

6

Strongly DISagree

How much do you agree or disagree with EACH of the following statements as it applies to your current

job. Please indicate your answer using the scale below ranging from Strongly Agree (1) to Strongly

Disagree (7).

Strongly

Agree 2 3

Neither Agree nor Disagree

5 6 Strongly DISagree

I work closely with others in doing my job.

I often have to coordinate

my efforts with others.

I work fairly independently

of others in my job.

I can plan my work with

little need for coordination with others.

30

On a scale from 1-7, in which 1 represents "Very often" and 7 represents "Not at all," how often, if at all,

do you use the following means of communication in your job?

Very often 2 3 4 5 6 Not at all

Email

Electronic meetings

Teleconferencing

Videoconferencing

Phone calls (one-on-one)

File Sharing

Voice mail

Face-to-Face

On a scale from 1-7, in which 1 represents "A lot of experience" and 7 represents "No experience at all,"

please rate how much experience you have in the following areas.

A Lot of

Experience 2 3 4 5 6

No Experience

at All

Developing new project

teams

Negotiating formal

agreements with other

organizations or groups

Resolving conflicts among

stakeholders in the

organizations where you

have worked

Working as a member of a

team

31

Teaching Others Selected Technologies

On a scale from 1-7, in which 1 represents "Very certain," and 7 represents "Not at all certain," please

rate how certain you are that you can teach other people how to use each of the following participatory

technologies.

Very

certain 2 3 4 5 6

Not at all certain

Blogging tools

Digital/audio podcasting

Digital photography

Gaming

Instant messaging

Photo-sharing websites

RSS feeds

Social bookmarking

Social networking

Tagging (folksonomies)

Videoconferencing

Virtual referencing

Web conferencing

Wikis

Librarian Experience, Employment, and Demographics

Number of years of experience as a librarian? (Number of years)

Number of years of employment with current employer? (Number of years)

Number of years in current position? (Number of years)

32

Do you supervise other employees?

Yes

No

If so, how many employees (both full-time and part-time) do you supervise? (Do not count volunteers)

Please select the following kinds of libraries you have worked during at least one year of your career.

(Choose all that apply)

Public Library

Elementary/Secondary School Library

Academic Library

Specialty Library

Other (Please specify below) ____________________

The following questions ask you about your most important goal in your work. Please choose the option

that best describes you.

In my work, my most important goal is...

to do better than others

not to do worse than others


to do better than I did before

not to do worse than I did before








33






The following questions are to help when analyzing results. They will not be used to identify you as a

respondent.

Gender

Female

Male

Other, please specify: ____________________

Prefer not to say

What year were you born? (YYYY)

What is the highest grade of school or college that you completed?

High School Diploma

Some College

Bachelor's Degree

Master's Degree

Degree beyond Master's (e.g., PhD, law degree)

34

ILEAD Spring Survey 2015

Do you agree to continue as a voluntary participant in this evaluation?

Yes

No

What state's ILEAD USA program are you participating in?

Delaware

Illinois

Maine

New York

North Dakota

Ohio

Pennsylvania

South Carolina

Utah

Wisconsin

Please select the option that best describes your participation in ILEAD USA.

Mentor or Apprentice

Participant

Section I: Role and Involvement

How much do you agree or disagree with EACH of the following statements regarding your role on your

ILEAD USA project team? Please indicate your answer using the scale below ranging from Strongly Agree

to Strongly Disagree.

35

Strongly

Agree 2 3 4 5 6

Strongly Disagree

I am responsible

for a particular

task or set of tasks.

Every member of

the team has the same

role at this point.

I am on the team to

represent my library

I am on the team

because of my particular

expertise and/or skills.

I am on the team to

learn more about Web

2.0 technologies.

36

How much time have you been giving to ILEAD USA since the in-person training program in March?

less than 1 hour/week

1 to 2 hours/week

3 to 4 hours/week

more than 4 hours/week

On a scale of 1-7, in which 1 represents "High Priority" and 7 represent "Low Priority," how much of a

priority is ILEAD USA for you?

High Priority

2

3

4

5

6

Low Priority

On a scale of 1-7, in which 1 represents "A lot" and 7 represent "Not at all," to what extent have your

other responsibilities conflicted with your involvement in ILEAD USA?

High Priority

2

3

4

5

6

Low Priority

Section II: Measures of Team Functioning. How much do you agree or disagree with EACH of the

following statements regarding your role on your ILEAD USA project team? Please indicate your answer

using the scale below ranging from Strongly Agree to Strongly Disagree.

37

Strongly

Agree 2 3 4 5 6

Strongly Disagree

There is great

uncertainty and

ambiguity about what the team is

supposed to accomplish.

The team's goal is

specified so clearly that

all members should know exactly what the team is

trying to accomplish.

The goal of the team doesn't

make much of a

difference to anybody

else.

The team's goal is of

great consequence

for the libraries

represented on the team.

Conflicting priorities

exist on the team.

The team is just the right

size to accomplish

its goal.

38

There is a clear team leader who guides what

we do.

Members of the team are

too dissimilar to

work together

well.

The team has a nearly ideal "mix"

of members - a diverse

set of people who bring different

perspectives and

experience to the work.

Members of the team

have more than enough

talent and experience for the goal

we are trying to

accomplish.

Everyone on the team has

the special skills needed

for team work.

Some members of

the team lack the

knowledge and skills

they need to do their

parts of the team's work.

39

Team members do

not know what skills

and knowledge they each possess

relevant to the goal of the team.

Team members

are assigned to tasks well

suited to their

knowledge and skill.

Team members

have a good "map" of

each other's talents and

skills.

Standards for behavior on the team

are vague and unclear.

Members of the team agree on

how members

are expected to behave.

The team maintains a

high standard of work at all

times.

40

It has been easy for the team to get

the information it needs to do its job.

Members demonstrate

their commitment to the team by putting in

extra time and effort to

help it succeed.

Some members of the team do

not carry their share

of the overall

workload.

The team often comes

up with innovative

ways of proceeding

that turn out to be just

what is needed.

The team has a great

deal of difficulty actually

carrying out the plans we

make for how we will

proceed toward our

goal.

41

How seriously a member's ideas are taken by others on the team

often depends on

who the person is

than on how much he or she actually

knows.

Members of our team actively

share their special

knowledge and

expertise with one another.

Working together energizes

and uplifts members of

our team.

Some members of our team do

not get along.

I enjoy the kind of work we are doing on the team.

Working on the team is an exercise

in frustration.

42

Generally speaking, I

am very satisfied with the

team.

I feel that other

members of the team will judge me on

the things that I say.

Section III: Communication and Discussions

How effective has communication been among the members of your ILEAD USA project team?

Very Effective

2

3

4

5

6

Not at all Effective

On a scale of 1-7, in which 1 represents "This is very clear" and 7 represent "This is not at all clear,"

when reflecting on your discussions with your team, please rate the clarity of the following issues:

43

This is Very

Clear 2 3 4 5 6

This is Not At All Clear

What each of the tasks are that need to

be completed to accomplish the team goal

How the tem should divide

its time among the

various tasks

What you personally

are responsible for doing on

the team

When each of the tasks will be completed

What constitutes "successful

performance" for the team

What criteria will be used

to evaluating the final

product of the team project

What the final output

of your team's work will look like

If you have anything else to tell us about your experience in ILEAD USA so far, please use the following

space for this purpose.

ILEAD USA Early Fall August 2015

44


Yes

No

What state's ILEAD USA program are you participating in?

Delaware

Illinois

Maine

New York

North Dakota

Ohio

Pennsylvania

South Carolina

Utah

Wisconsin

Please select the option that best describes your participation in ILEAD USA.

Mentor or Apprentice

Participant

Section I. Role and Involvement


ILEAD USA project team (mark a bubble on the Strongly Agree to Strongly DISagree scale for EACH

statement)?

45

Strongly

Agree 2 3 4 5 6

Strongly DISagree

I am responsible

for a particular

task or set of tasks.

Every member of

the team has the same

role at this point.

I am on the team to

represent my library.

I am on the team

because of my particular

expertise and/or skills.

I am on the team to

learn more about Web

2.0 technologies.

46

How much time have you been giving to ILEAD USA since the in-person training program in June? Mark

the bubble that best describes this.

less than 1 hour / week

1 to 2 hours / week

3 to 4 hours / week

more than 4 hours / week

On a scale of 1-7, in which 1 represents "High Priority" and 7 represents "Low Priority," how much of a

priority is ILEAD USA for you (mark the bubble on the scale below)?

High

Priority 2 3 4 5 6

Low Priority

How much of a

priority is ILEAD USA for you?

On a scale of 1-7, in which 1 represents "A lot" and 7 represents "Not at all," to what extent have your

other responsibilities conflicted with your involvement in ILEAD USA (mark the bubble on the scale

below)?

A Lot 2 3 4 5 6 Not at All

To what extent have your other

responsibilities conflicted with

your involvement in

ILEAD USA?

47

How much do you agree or disagree with EACH of the following statements (mark a bubble on the

Strongly Agree to Strongly DISagree scale for EACH statement)?

48

Strongly

Agree 2 3 4 5 6

Strongly DISagree

What our project team is

supposed to accomplish

remains uncertain and

ambiguous.

I really feel as if this team's problems are

my own.

The team goal is so

challenging that we have to stretch to

accomplish it.

Our individual roles on the project team are very clear and we don't

stray from them.

The team goal is of great

consequence for those we

serve.

Team members do

not know what skills and

knowledge they each possess

relevant to the goal of the

team.

I would feel guilty if I left

my team now.

There is a clear team leader who guides what we do.

49

Team members are assigned to

tasks commensurate

with their knowledge and skill.

The goal and priorities of

the team are not clear enough.

The team has a "good map"

of each other’s talents and

skills.

I do not feel a strong sense

of belonging to this team.

The team follows a very

structured work schedule.

Our project team does not have a broad enough range of experiences

and perspectives

to accomplish its goal.

Our project team's goal is specified so

clearly that all members

should know exactly what the team is


50

I would not leave my team because I have

a sense of obligation to

it.

Members of our project

team have to depend

heavily on one another to get

the team's work done.

Section II. Communication and Discussions

On a scale of 1 to 7 where 1 indicates "Very Effective" and 7 indicates "Not at all Effective," how

effective has communication been among the members of your ILEAD USA project team.

Very Effective

2

3

4

5

6

Not at all Effective

On a scale of 1-7, in which 1 represents "Very Often" and 7 represents "Not at All," how often, if at

all, has your team used each of the following communication methods since it was first formed?

Very Often 2 3 4 5 6 Not at All

Email

Electric meetings

Teleconferencing

Videoconferencing


File sharing

Voice mail

Face-to-Face

51

On a scale of 1-7, in which 1 represents "Very Helpful" and 7 represents "Not at All Helpful," how

helpful has each of the following methods of communication been to the work of your project team so

far (if it does not apply, please leave it blank)?

Very

Helpful 2 3 4 5 6

Not at All Helpful

Email

Electric meetings

Teleconferencing

Videoconferencing


File sharing

Voice mail

Face-to-Face

In reflecting on your discussions with your team, to what extent has the team discussed and developed

clarity about each of the following issues. On a scale of 1-7, in which 1 represents "This is Very Clear"

and 7 represents "This is Not at All Clear," when reflecting on your discussions with your team, please

rate the clarity of the following issues:

52

This is Very

Clear 2 3 4 5 6

This is Not at All Clear

What each of the tasks are that need to

be completed to accomplish the team goal

How the team should

divide its time among the various

tasks

What you personally

are responsible for doing on

the team

When each of the tasks will be completed

What constitutes "successful

performance" for the team

What criteria will be used in evaluating

the final product of the team project

What the final output

of your team's work will look like

53

Section III. Importance of Selected Team-Related Characteristics

On a scale of 1-7, in which 1 represents "Very Important" and 7 represents "Not at All Important," please

rate the importance of the following items.

54

Very

Important 2 3 4 5 6

Not at All Important

How important is it

to YOU PERSONALLY

that your project team

accomplish its goal?

How important to

YOUR TEAM is accomplishing

its goal?

How important is it

to YOU PERSONALLY that roles and

responsibilities on your team

are clearly defined?

How important to

YOUR TEAM is it that roles

and responsibilities of its members

be clearly defined?

How important is it

to YOU PERSONALLY that you have

a sense of belonging to your team?

How important to

YOUR TEAM is it that every

member feel a sense of

belonging to the team?

55

How much do you agree or disagree with each of the following statements?

56

Strongly

Agree 2 3 4 5 6

Strongly Disagree

Members of our project team

are too dissimilar to

work together well.

Our project team's goal is not especially challenging;

achieving it is well within

reach.


have more than enough talent

and experiences for the goal we

are trying to accomplish.

Conflicting priorities exist on our project

team.

How seriously members' ideas

are taken by others on our

team often depends more

on who the person is rather

than on how much she or he actually knows.

Our project team has a

nearly ideal mix of members.

Some members of our project team lack the

knowledge and skills they need to do their parts

of the team's work.

57

This team has a great deal of

personal meaning for

me.

Our project team tends to

handle differences of

opinion privately or off-line, rather than

addressing them directly as

a group.

The information and knowledge available to our project team in

pursuing its goal has been more than adequate.

Everyone on our project

team, has the special skills

that are needed for what we are


Our project team often

comes up with innovative ways of proceeding

with our project that turn out to

be just what was needed.

Our team has a great deal of

difficulty actually carrying

out the plans we make for how we will

proceed toward our goal.

58

I am certain that our project team's goal will

be accomplished

by the completion of the cohort in

October.

Our project team will

continue to work together

after the completion of the cohort in

October.

Strongly

Agree 2 3 4 5 6

Strongly Disagree


are too dissimilar to

work together well.

Our project team's goal is not especially challenging;

achieving it is well within

reach.


have more than enough talent

and experiences for the goal we

are trying to accomplish.

Conflicting priorities exist on our project

team.

59

How seriously members' ideas

are taken by others on our

team often depends more

on who the person is rather

than on how much she or he actually knows.

Our project team has a

nearly ideal mix of members.

Some members of our project team lack the

knowledge and skills they need to do their parts

of the team's work.

This team has a great deal of

personal meaning for

me.

Our project team tends to

handle differences of

opinion privately or off-line, rather than

addressing them directly as

a group.

The information and knowledge available to our project team in

pursuing its goal has been more than adequate.

60


team, has the special skills

that are needed for what we are



comes up with innovative ways of proceeding

with our project that turn out to

be just what was needed.

Our team has a great deal of

difficulty actually carrying

out the plans we make for how we will

proceed toward our goal.

I am certain that our project team's goal will

be accomplished

by the completion of the cohort in

October.


continue to work together

after the completion of the cohort in

October.



61

ILEAD USA Final Survey 2015


Yes

No

Please type in your unique project identification number which is found in the letter that was sent to

you.

Section I: Measures of Involvement. How much do you agree or disagree with EACH of the following

statements regarding your role on your ILEAD USA project team? Please indicate your answer using the

scale below ranging from Strongly Agree to Strongly Disagree.

62

Strongly

Agree 2 3 4 5 6

Strongly Disagree

My supervisor or other superior

asked me to participate in

ILEAD.

I never felt that my

participation in ILEAD got in the way of my

other work.

I did not get as much out of ILEAD as I

thought I would at the beginning.

My participation

in ILEAD is more

beneficial to me personally

than professionally.

ILEAD has been one of

the most enjoyable

experiences of my career.

My library did not give me

all of the time I needed for

ILEAD.

Looking back, my team

should have been chosen a

different project goal.

63

My participation in ILEAD was

well-known to my co-

workers at the library where I

work.

ILEAD has been one of

the most useful

experiences of my career.

My participation

in ILEAD mattered

more to me than the

library where I work.

ILEAD ranks among the

best learning experiences I

have had.

Not everyone in the library where I work

was supportive of

my participation

in ILEAD.

64

Section II: Statements about Your Project Team

How much do you agree or disagree with EACH of the following statements? Please indicate your

answer using the scale below ranging from Strongly Agree to Strongly Disagree.

65

Strongly

Agree 2 3 4 5 6

Strongly Disagree

Members of our project team were

too dissimilar to work together

well.

Our project team's goal

was not especially

challenging; achieving it

was well within reach.

Members of our project team had more than

enough talent and experience for the goal we tired to accomplish.

Conflicting priorities

existed on our project

team.

How serious members' ideas are taken by

others on our team often depended

more on who the person was rather

than on how much she or he actually

knew.

66

Our project team had nearly an

ideal mix of members.

Some members of our project team lacked

the knowledge and skills

they needed to do their parts of the

team's work.

This team had a great

deal of personal

meaning to me.

Our project team tended

to handle differences of opinion

privately or off-line,

rather than addressing

them directly as a group.

The information

and knowledge available to our project

team in pursuing its

goal has been more

than adequate.

67


team had the special skills

that were needed for

what we are trying to

accomplish.


came up with innovative

ways of proceeding

with our project that

turned out to be just what was needed.

Our project team had a

great deal of difficulty actually

carrying out the plans we

made for how we

proceeded towards our

goal.

Our project team's goal

was accomplished

by the completion

of the cohort in October


continue to work

together after the

completion of the cohort in October.

69


ILEAD USA project team? Please indicate your answer using the scale below ranging from Strongly Agree

to Strongly Disagree.

Strongly

Agree 2 3 4 5 6

Strongly Disagree

This was one of the most

effective teams which I have ever

served.

Our individual

roles on the team were very clear.

Our team did not give me

enough opportunities

to contribute.

I felt like the other

members of the team

would judge me on the

things I said.

I would have chosen a

different goal for our team.

Members of the team

worked very hard to keep one another

up to date on their

activities.

The team did not maintain

a high standard of work at all

times.

70

Section III: Opinions about Other ILEAD Participants

How much do you agree or disagree with EACH of the following statements on community

representatives? Please indicate your answer using the scale below ranging from Strongly Agree to

Strongly Disagree.

Strongly

Agree 2 3 4 5 6

Strongly Disagree

Our team's community

representative/s were involved from the start.

We did not always make

good use of our team's

community representative/s.

Our community representative/s

were the best possible people given our team's

goal.

Our team's community

representative/s got along very well with us.

If you have one suggestion to make for improving the community representative role in ILEAD USA it

would be:

How much do you agree or disagree with EACH of the following statements on instructors? Please

indicate your answer using the scale below ranging from Strongly Agree to Strongly Disagree.

71

Strongly

Agree 2 3 4 5 6

Strongly Disagree

The instructors, as a group,

were outstanding.

Some instructors

were clearly better than

others.

Without the things we learned

from one or more

instructors, our team could not

have made as much

progress as it did.

72

How much do you agree or disagree with EACH of the following statements on mentors? Please indicate

your answer using the scale below ranging from Strongly Agree to Strongly Disagree.

Strongly

Agree 2 3 4 5 6

Strongly Disagree

Our team could not

have gotten as

far as if did without the

help of your

mentor.

Our mentor was always

available when

needed.

The quality of advice

we received from our mentor

was uneven.

Our mentor always

knew what to do when

out team had

problems.

If you have one suggestion to make for improving the mentor role in ILEAD USA it would be:

73

Section IV: Measures of Project Impact

Did your team complete its project?

Yes

No

If YES, how much are the patrons of your library benefiting form the results of this project?

A Lot

Some

Little

None

Do you expect the impact of the project on your library to increase, stay the same or decrease over the

next 12 months?

Increase

Stay the same

Decrease

If the team did not complete its project, are efforts being made to finish the project?

Yes

No

If so, when will the project likely be completed?

In the next month

One to three months

Four to six months

More than six months



74

Appendix B

Participants Comments

75

COMMENTS FROM SPRING 2015 SURVEY

All the sessions in March were exceptional. I am so thankful to be in this program. I have been working on keeping all that I heard alive for myself in my profession. As a team I do not think everyone feels the way I do. For the first few weeks after the March session I was frustrated as my team members were not responding. I tried to keep the purpose of ILEAD alive for myself and not be influenced by my team members and would work on posting things on our team blog. I contemplated talking to my mentor about this but did not. The learning I took for myself in all this is patience and not to be affected by the others in the team. The thing I have not understood yet is that the team project is affected by all this and I am not too happy, I try to nudge my team members and the response I have got is to chill. Of late due to a few challenges at work I was on the verge of wanting to give up ILEAD. On the other hand I did not want to let down my team.

At this point I am frustrated because my team is struggling with communication and team dynamics. It is much harder to help out since they are a three hour drive away. Meeting online is not always productive.

For the sessions, both on video and in person, it would be great if they could be more practical. The Color Theory thing, for instance, was not at all helpful. Personality theory is iffy at best. The project management one could have been useful, but there was not much I could take away from it. Also, fewer videos and more in person sessions would be better.

Glad I'm participating!

Going in to the experience, I wasn't sure what to expect or what was really going to be expected of me. Perhaps a little more information prior to beginning the program might have been helpful.

Great concept and experience! I hope it leads to some great library programming.

Had I known that the learning outcomes would be so pathetic at the March session I would have opted not to sign up for the ILEAD USA program. The keynote talks were interesting, but did not have much impact on our project. The rest of the training seemed to be aimed toward people who have never turned on a computer, have little to no team experience, and have never completed a project. Spending a day talking about what a podcast is, how to use Wordpress, and Outcome Based Evaluation was a significant drain on my team's time. We are smarter than that and would have benefited from new and interesting information, not old technologies and techniques to evaluate a program that doesn't even have a proper scope. The team I am working with is fantastic (and very smart) and I believe we will come up with something interesting by the end due to our own project management skills and abilities rather than an outcome of ILEAD influence. I apologize, I do not mean for this comment to come off as rude. Perhaps my team is not the right audience for this program. When I read "a nationwide leadership immersion program..." in the description of this program I thought it would be something significant that would cultivate my skills as a leader. Unfortunately, this is not the case. I apologize for the haste in writing this message. If you need clarification or have further questions I am happy to help.

I am enjoying my experience with Ilead. It is making me think about bigger ideas and giving me the freedom to explore without feeling I am wasting time. The experience is giving me permission to spend time dreaming and thinking outside of the box.

I am excited to see if we can pull off the terrific idea that we have.

76

I am feeling very frustrated with the lack of expectations or the cases where there are expectations and they are not clearly stated. The program says that there are no mistakes but our group made some priority related calls that clearly did not meet the expectations of the program organizers. I am still not sure what the expectations are going into the second session? What do we need to have completed upon arrival? How much time are we expected to devote to preparing for out poster session? Is there going to be enough time to accomplish that. Also, I don't think that the environment is right for all libraries. My team is all academic and we are just wired differently. We appreciate specific requirements and deadlines. Instructions like "be sue to post to Wordpress", "participate in intersessions" and "make a 5 minute video" left my team with about a zillion questions. We spend more time on the logistics of creating the video than the content and we all would have rather spend the time working on discussing the actual project. The tension is rising as we approach the second session and a poster session without any parameters or expectations. I have really enjoyed the speakers and when it is over will most likely be counting this experience in the positive column.

I am really enjoying this learning experience. I like that it is structured around a project so that we have ways of applying what we are learning and can see further application through the projects of other teams. I love all of the ideas!

I feel like we have received insufficient instruction on the intersession activities requirements. Otherwise, it has been a wonderful experience and I am so thankful for the opportunity.

I feel so lucky and blessed to be part of this experience. The feel that the sessions have broadened my knowledge much further than for just my team project. Amazing!

I feel that sometimes we are running blindly. There doesn't seem to be communication from ILEAD USA down to the different groups

I found it very frustrating when presenters kept mentioning other presentations that we (in other states) were not privy to. In particular, the accessibility presentation.

I hope our next session in June will not last for 12-16 hours a day. I love libraries and want to continue being a great contributor to the library services; however, in order to serve our best, my team members and I need some rest. ;) Thank you.

I think the ILEAD Program is predicated upon a strong Mentor or team leader. The possibilities that arise from discussion need to be reined in by someone who can see the forest and the trees in their proper context.

I was a team member last year and am a mentor this year. They really are two different experiences. In some ways, I feel that the mentor role is more challenging, because I'm removed somewhat from the immersive experience that being a team member is, with different tasks to complete and goals and deadlines to meet. It's much more nerve-wracking to be in the role of a cheerleader and hoping for your team's success and offering whatever guidance possible.

I've quite enjoyed the community building that is happening through ILEAD and have been inspired by the presentations. Because of ILEAD I now know so many more librarians in the State and that has been wonderful.

ILead has been an excellent experience so far. It's a reminder and great lesson on how to work well with others in accomplishing a few goals and learn from the experience.

77

It is a great experience working with public librarians. I am a school librarian. Many things we talk about and do aren't actually geared towards me; however, it can be altered to fit my needs. I kind of wished we had a solid schedule worked out so I knew what I was supposed to be doing at each week’s checkpoint.

It seems surprisingly disorganized and vague.

It would be great to have access to expertise that is relevant to the work our team is doing. It seems as if public librarianship is the focus of the ILEAD leaders and that is not the entire makeup of the room. In addition, team members are traveling from all over NYS to work together as a team. It would be ideal to have more time built in that allows for in-person work.

It would be nice to have more time to work on projects when we meet in person. Our team is all from different libraries or different types and a long way apart. While online collaborations works, in person sessions tend to be more productive. The days were so packed we had very little time to work together.

It's been a great opportunity to make new connections, deepen existing relationships and have time to work on a special project with a talented team. I appreciate all that Mary Linda Todd has done to bring this to New York!

It's been a very good program.

It's been a wonderful learning experience so far, and very inspirational.

It's been fantastic! I've noticed a huge change in myself both professionally and personally, and even coworkers have commented how much more involved, confident, and innovative I've become since starting ILEAD.

It's fantastic!

More information about the grant earlier would have been way more helpful

More time with our team while we are in Springfield would be very helpful

Our team is in a bit of a slump, but I know it is not indicative of what we have done and what we will do. We have had great moments and I have full confidence we will find and focus the direction of our project so that we can complete a doable deliverable. Our mentor has been wonderful, the state support and the first session were wonderful, and the participants on our team are talented, dynamic women. We are adjusting to the research we have done and that is a bit frustrating, but we are committed to working this through. Thank you for the opportunity!

Several of our team members were very busy with preparing for our state library conference, so now we feel like we're running to catch up. I also volunteered to take more of a leadership role (my title is Whip-Cracker) to make sure we're getting our individual tasks completed. I hope I can help us do better!

Simultaneously the most challenging, rewarding, and inspirational event of my career in libraries.

So far, I have truly enjoyed ILEAD USA. I think it is a wonderful experience to be collaborate with librarians, gain knowledge and be creative and innovative.

Some amount of energy is lost during the live stream presentations from Illinois.

Super! Looking forward to the next session and further development of our respective projects!

78

The internet feed programs are not easy to watch, it makes it hard to concentrate on them.

The needs of the team are very different - making effective use of group time during scheduled ILead training difficult. Some teams could benefit from more team time, other more training, others more hands on demonstrations, other general team building activities....Someone is always going to be disappointed with the schedule. Most important would be to have an engaging and likable trainer. When that fails, a variety of instructors would be better so that more learning styles/preferences are addressed.

The positive and encouraging atmosphere in Springfield made it one of the best things I have been involved in. I am really looking forward to the June session.

The presenters were well prepared and had good information but you are preaching to the choir- we already know these things and share the same views. We need more time to work on our projects. My biggest frustration is the lack of communication about what is expected, when it needs to be completed, and the lack of time given to work with our groups. It would be very helpful to see examples of past projects or to have a rubric of expectations. This whole experience has been so ambiguous. We have had no clear definition of what we are doing, and why we are doing it. I love the collaboration opportunity, and my team is great- we just have no clear definition of our task

The video speakers were hard to watch and hear.

This experience has been really disorganized and frenetic. Quite possibly one of the most disorganized projects I have ever been involved with not only on the macro level but within my team as well. Information has come to us in fits and starts and sometimes has been vague and confusing. I do not look forward to this project at all. The only good part is that it has been an exercise in restraint for me as I usually take over when other people are failing or floundering. I hope it gets better, I really do because the ideas and concepts are fantastic. That’s why I signed up.

This has been a great experience - I have really enjoyed getting to know and working with my team, the ILEAD mentors, instructors and the state library staff. I think programs of this sort will go a long way towards libraries being able to work better together to make a positive impact on their communities.

COMMENTS FROM FALL 2015 SURVEY

All of the ILs staff, the mentors, and the coordinators have been amazingly helpful. They are the key to our success and have done a wonderful job.

Being selected for this program is one of the best things that could have happened to me. It's exactly what I needed at this moment in my career.

Don't make it mandatory for all members of a team to be a librarian. To encourage collaboration and partnerships within the community, it may be easier and more beneficial to bring in people to teams who aren't necessarily librarians.

79

Even though my team's project wasn't helped by the June session they DID find it very useful for other aspects of work in their libraries.

Everyone has been great and glad to have had the experience but what I discovered was that as a one man show for my job this ended up being way more than I thought it would be for the amount of time needing to put into it plus keep up with my regular work I would really recommend giving better clarity to new folks looking to be part of the next ILead program.

Great Experience! I wish that there were more connections to the outside world when projects come to an end. Having more support for what libraries across the nation are doing could help build a campaign bigger than one state's library's.

I am disappointed that there is not enough time built into the ILEAD sessions for us to really get work done. While we video conference regularly, the face to face time is critical to accomplishing anything significant. I also believe it would have been more helpful to have access to the tech development we need rather than what is readily available. The tech development has not served our needs.

I appreciate the breakout sessions though I think they somewhat missed their mark and were not helpful, specific enough or presented/planned effectively. Our team has really bonded and we are doing well. The grant money process could be more structured with guidelines. My library went one formal direction that took much time to get while other libraries were very loose. No guidance was given and I think that needs to be done. It would also be helpful if there was one central place we could see what other teams are doing so if we needed to contact someone we could to collaborate or get assistance, etc.

I had trouble evaluating the past ILEAD sessions for this survey in terms of what our team as "used". I loved the keynote sessions for inspiration and leadership training; they have been meaningful and important for me to experience and have started to change how I think about librarianship. But they don't really have practical implications for our ILEAD project - they offer different ways of thinking, which is useful but not practical. I'm not sure if that makes sense, but I think there is a difference between training sessions that give practical help on the project, and inspirational sessions that give motivation, and I think both are important for ILEAD.

I have been a mentor and had an amazing team. This entire experience has been one of the best of my 20 year career. I am spreading the word far & wide about ILEAD.

I have only taught how to use Wordpress so the teams could create their ILEAD Wordpress sites. Thank you.

I love ILEAD. A lot. I get so much out of the sessions. I'm just swamped with work and feel like I was spread thin. I got much more out of the in person sessions about leadership than I did out of the group project. I am wondering if this imbalance might indicate of a universal experience others are having. Would it make sense to make ILEAD focus more on the leadership bootcamp than a project?

I loved the national speakers, however, they were hard to follow and watch at times. One presenter had handouts that weren't received by our location until well after the presentation started. And the video quality could be improved. At least they weren't as bumpy as the first sessions, but there is still a long way to go so your outlying states don't feel alienated!

80

I really appreciate the chance to learn new technology along with my team members. I try to stay in the shadows and act as sounding board during major project work. The team is working very well together and shows great excitement for their project.

I really enjoyed sharing our project with the other teams at the poster sessions. It was a nice way to see what other teams are doing and to get to know them better.

I think I could have given better feedback two weeks out from the last session. Also - for children's librarians, June and July are the WORST times to be out of the building. I am enjoying the process, in spite of the large commitment of time. It is worth it to me personally, I hope my administration finds it to be as well.

I very much wish I had known how much time I would need to devote to this program outside of the in-person sessions. With the intersession requirements and scope of our project, this program has become a much larger time investment than I was expecting and it would have been nice to plan accordingly ahead of time.

I was greatly saddened by the change of venue. The new hotel was an utter disappointment and really killed the vibe for me. After a day of sessions and group work it just feels better to be staying at a nice hotel with decent service. Oh well, I suppose it could be a lot worse....

I was hoping to learn more hard technology skills, but it's more of the same soft people skills but with technology tools that I already know. I'm glad that my participation and our project will help our library and community, even if that's all we get out of it.

I was unable to answer any of the questions, because I fell ill and could not attend the last institute.

I wish I was on a team that focused more on promoting technology driven programs, and usage in the library instead of solely educational with elementary aged children. There is much we can offer our patrons in regard to interactive gaming clubs, and material other than books to facilitate creativity and community through animation, music, and video. That is where my passion lies. Therefore, our team seems to be a team of two because we are focused on the collective passion of only two members instead o incorporating each member’s strength, knowledge, and passion. For example, one of our team members already has an established and successful connection with teens in her library through innovative programing, and her strength, knowledge and passion have not been incorporated into our project.

I wish the training had been MUCH more advanced. We are attending leadership training and learning things that anyone with an MLS should know. That said, I do appreciate this experience as a whole but the training itself has been lackluster.

I would reconsider the requirement that instructors have to be present for the entire session, every session. Or is that just a local rule we made for ourselves? It takes too much of their time.

81

ILEAD has been an interesting experience. It has been frustrating because of our individual group's experience and the way that we came together, our group project, and whether or not it will continue to be supported in our school district. It has also been a bit frustrating to feel that most of the information given at the federal level is for only public libraries, which leaves us figuring out how to apply it to our library work. That said, it has been an excellent experience in many ways. I have enjoyed the opportunity to talk about libraries with librarians and raise my sights to the horizon to see what other librarians are doing and experiencing. Nothing bad can ever come from that.

ILEAD has been great. This survey is way, way too long though.

ILead has really given a sense of group cohesion and collaborative fire to our group that might not have developed as well otherwise and our project has really benefited from it. We will definitely continue our work after ILEAD is finished.

ILEAD USA has been a wonderful experience so far. Team building and working toward a goal have been the most helpful. Thank you for offering this opportunity.

In regards to the previous question, none of the sessions presented in June directly had any impact on our project...they provided interesting and fun new things to learn, but because our project is fairly specialized and focused in a particular area, the sessions were informative, but not altogether relevant.

It has been a great experience and a chance to work on a project that I have been developing for a few years but did not have the time or resources to pursue.

It has been an awesome experience and I'm going to be a little sad when it's over. I'm very thankful for the chance to participate in this. It's really helped me grow professionally and get back some of my librarian mojo.

It has been difficult for everyone on the team to find the same amount of time to devote to the project, outside of our meetings.

It has been excellent, there is a lot to do, my biggest challenge is finding enough time to do it all.

It has definitely taken awhile for our team and the project to gel and for us to refine the focus of our project, but things seem to be coming together as we approach this last formal session of ILEAD. Overall, it has been a positive and valuable experience and I hope to use the skills I learned at ILEAD going forward in my professional life.

It is hard to find time to take such in=depth surveys and I don't remember since it was quite a while ago

It would be nice to have the order of ILEAD team presentations shuffled- we were always last

It's been an amazing experience, and a fantastic opportunity to transcend institutional borders and work with such a great team!

82

It's been quite the journey! Our team has grown quite a bit from our first session. As individuals we've learn to let go of some of the control that we felt we needed to have over the project and let those that are the experts do what they do best. I've enjoyed getting to know the members of my team and those on the other teams. We have so much knowledge in Wisconsin Libraryland!

Love the experience of ILEAD USA and have found it very valuable. I also was able to meet and interact with people from all over the state which I had not had contact with before, so that was a wonderful experience. This group is very enthusiastic and working on some great projects! I will be sad to see the experience end in October.

My primary reason for participating in ILEAD was to gain experience and techniques for being a better leader, but the session content has not done much to develop leadership skills.

My team gained a lot of exposure to the newest technology, and even though much was not applicable to their project, they are implementing ideas at their home libraries already.

My time in ILEAD has been great. I have learned about blogging and using Twitter not only for my project, but to use in my job. I have enjoyed working with my team, as they are the best. We encourage each other, brainstorm and flesh out ideas that we ca actually use for our project. We trust each other to make decisions on spot if needed. Our mentor is terrific; she makes suggestions that are on point. In addition, she has kept us on track, and rallied us when needed. I enjoyed getting to know the Mane State Library staff and working with them and the instructors. I really do not want ILEAD to end.

Our mentor was well meaning and engaged in the topic. However, because she doesn't work in this field, I think she was not as effective a mentor as she could have been.

Our project has been very challenging and educational for me. I'm at a point that I want to learn more about technology skills that I am lacking; this project has helped me, at least introducing me to technologies I was not even aware of prior to the project. I'm the oldest, least tech savvy member of our team but the team has been very courteous and respectful of me. I have the least to contribute but what I do contribute is appreciated. I'm receiving more than giving at this point. I'm learning new skills Time constraints have been a real issue for me. I thought I would have more time going into the project than I do. Most if not all the time I spend on our project is my personal time rather than work/library time. I'm glad that I am participating! Thank you!

Out team has learned so much. While I am not yet confident that I could teach Makey Makey or Arduino right now, I know how to play around with it to teach a program later. Keynotes were interesting but not really applicable to our project. Our team needed new tech skills and we got them. Thank you!

Over all I'm enjoying the training and developing our project. I think there should be a bit more accountability for actually connecting with our communities.

Raspberry Pi guy was awesome and very knowledgeable.

83

So far it's been a great experience and we've learned a lot as a group. While some of the topics for the last sessions hands-on workshops were great, there was little or no content, which made me frustrated and feeling like I was wasting time that could have been spent doing work back home. I was particularly frustrated with the 3D Printer as not only did we not get to play with the scanner, but there was no instruction. I could sit at my office and play with Tinkercad. The Raspberry Pi class was top notch and how all the workshops should have been. Having everyone work on the same design/project is a great way to structure it. Wish the 3D Printing class had been like that, not just here's Tinkercad, design something. People new to a technology or software need more guidance than that in my opinion. The keynotes have been great and really inspiring. It's also been a good experience in team building for the members of the team from my library.

So far it's been an amazing experience. Thank you for making it possible!

The hardest part was that the grant distribution was done through the other library. It took us very long time to get all papers in order and get approval from the library who agreed to work with us. We lost a lot of valuable time by waiting for all documents to be processed and funding released to us. We could have spent more of this time on promoting the project to SC librarians. Please consider giving the grant to the group or to the state library that was hosting ILEAD.

The lack of material by one of the keynote's speakers made that presentation fail miserable, but I enjoyed the rest of the keynotes speakers emmensely . I have learned so much in the past 6 months and I looking forward to the October session.

The last couple questions on which sessions were most useful to the project were difficult to answer. Some information I expect to be useful, but we haven't actually used it yet (e.g., sessions grantwriting and on shooting and editing the video). Others were very interesting, but not specifically related to my team's particular project (e.g., the sessions on maker spaces and community engagement, as well as the keynotes). And since we didn't get this survey until three months after the last session, I frankly don't remember a lot of what was covered then, or who said what. So my responses were mostly a combination of guesses and random choices.

The link up to the live keynote presentations often failed to work well.

The opportunity has been interesting and our team has had a lot of challenges. So the experiential part has been good. The speakers are across the board. Some are great and others not so much. The technology is still incredibly problematic and I finds it hard to believe after this many years and months we can't get it right. It seems that the biggest challenge comes from Illinois. It would be nice to be able to count on the technology.

The sessions have been informative and mind-expanding. However, they did not relate directly to my team's project, so I am unaware of whether or not the members have made use of what they learned in their own libraries.

The sessions in the second ILEAD session were not particularly relevant to our project.

84

The training sessions have not helped at all and have been very frustrating We have made several attempts to ask for better sessions and no one seems to listen. I like my team but the 4 days of ILEAD "trainings" have been mostly a waste of time. I could have taught most of them and done a better job.

There was not enough time provided teams to get the actual work accomplished. And, the workshops were not geared to all the needs in the room. Most, if not all, of the workshops were tailored to public librarianship. As such, those in other aspects of librarianship were left out and did not have their PD needs met.

This experience has been one of the most memorable professional development opportunities that I have had. Without this program, I would not have been able to experience technology and leadership training at this magnitude.

This experience has been transformative. Thank you!

This has been a fabulous experience and it will extend beyond October when we wrap it up. The state has paid for a number of us to take an expensive course offered by ALA on app development that will be invaluable as we move forward.

This has been a great chance to connect with some terrific library staff, whom I would not have met otherwise, and to learn from them.

This has been a positive experience personally and as a team. I have felt prepared and empowered by the ILEAD process because the sessions have prepared us for our project work.

This has been an incredible experience, and I've grown a great deal, professionally and personally. I'm very grateful I've been able to participate. Thank you.

This has been awesome. It has really changed the way I think of my job.

This has been the most intensive, and I believe will come to be the most useful learning experience of my career. Thank you

This has been the most unsuccessful learning opportunity that I have participated in. The in-person sessions are poorly structured. I do not feel that I have been provided with instruction that will help me build leadership capacity. The project feels like busy work to say that the program has led to something. This was a missed opportunity considering the 9 days we have each devoted to ILEAD.

This immersive process has been very beneficial to my professional growth. To keep re-visiting the learning and having a focus on the team goals is very effective. I appreciate the efforts of Denise and Ryan. It has been an extremely positive experience.

This is a wonderful experience. Although I ranked accordingly as to what you asked, I would have ranked differently as to what I learned more from. We use things differently for our projects than what we take back and use otherwise.

85

This survey has some issues. First, what are "Electric meetings"? The communications section had a lot of overlap--what's the difference with teleconference/videoconference? teleconference/conference call? Also the part about sessions -- our group missed the streaming lectures due to bad internet so the responses are worthless. Also, we shouldn't be forced to choose 5 sessions most valuable to individual/team if there aren't 5 sessions you say that about.

Through working on developing a Homeless APP, I have gained more insight and compassion for the homeless.

Very Inspiring Initiative

We have met some nice people. The quality of the trainings has been low, the organization of the meetings has been disappointing, and I still haven't received any leadership training.

Wonderful keynotes once again!

Would have liked to hear more from Dr. Matteson. Evan's Trust Circle is very fun, but also a useful, safe place, to discuss some difficult topics and concerns. And I hope we have another S'mores Bonfire at the October session. It was a lot of fun and areal bonding experience, collectively and as teams.

Would like to emphasize the importance of presenters from OUTSIDE the Library field.

COMMENTS ON USER REPRESENTATIVES FROM FINAL SURVEY

Allow those without email addresses to participate as community reps

As Delaware's ILEAD Project Director, I felt I had a good understanding of the CR role and that I communicated it effectively to our teams. I think communication about this on the national level could be enhanced.

Be more involved. we heard little and if we responded to his "how are things" we didn't get much response back

Being more clear about what the role is and having only 1 representative per team.

Choose the community representative after the June session. At that time we had a better idea of what our final project would like.

Choosing a community rep later in the process. Our project switched focus and the community reps were no longer relevant.

Clearer definition of role and/or datelines for check-ins and accountability.

Clearer guidelines for participation

Clearer guidelines of how to choose the community representative.

Clearly define whether mentor should be involved in team activities, or be more a facilitator for making connections

Don't be a go-between between team and state librarians. It leads to miscommunication.

Eliminate the community representative piece of the project -- not terribly useful nor committed to the project - no skin in the game so to speak

Even more clarity on the function of the representative to the team member.

86

Fewer representatives--It was hard to get everyone together.

Find reps that are as invested in your project as you are.

Give a little more time to allow an idea to form. Had I known where the project was going, I think I would have chosen a different representative. Choosing a representative from the get-go doesn't necessarily work will with the evolution of a project idea especially one that is radically different than the idea with which the group may have originally applied for the program.

Give concrete ideas for what community representatives can/should do.

Give them the opportunity to share in some of the learning opportunities, if possible

I don't think I would make it mandatory

I felt that mine was used a lot, but others not so much. I'm not sure how to improve that.

I had a great community representative, but others didn't; mine was involved from the start, we made good use of her expertise, she was the best person for the role, and we got along well, but others had the opposite experience. I think it might be better to focus on one relationship with one community representative for the whole group, that way someone who is reliable and relevant to the project can be chosen and involved more closely.

I had a great relationship with my community representative before ILEAD USA so it was very easy for me to keep her informed and get her opinions. I don't think this was true with everyone else on the team. I think there was only one other community representative who took an active role in our project. I feel like a community representative might be better if it were set up to be someone who is a professional mentor, or someone you look up who you can bounce ideas off of, instead of someone who might be directly in your community or directly benefit from the project.

I loved our mentor, but I think it was odd to have them at everything. It seems that a mentor should be someone you check in with occasionally or get advice from, but if they are at everything, I feel that it changes the team dynamic. It almost makes the mentor a team member instead of just a mentor.

I was under time constraints to find one. So i didn't get my first or second choice.

I would have a community partner for the team instead of each person having a community representative

I would recommend selecting your CUR after the first in person session. It wasn't until after that session that our team had a defined goal and then some of us had to change CURs.

I would recommend that once a community representative is selected, the team members should submit their email to ILEAD so ILEAD can send supplemental information about the program. This might be one area of oversight that could help keep communication open between teams and their community reps. Booyah!

I'm a mentor, so I am not sure how this section applies to me.

Identify this person as part of the team from the beginning when possible.

Invite them for multiple days during the second and third session.

Invite them to participate in the institutes

It didn't seem as though they were a necessary component. In fact, I think we only utilized one of our reps, and I didn't hear any other teams mention theirs

Longer time, perhaps a year on the goal rather than 9 months

Make sure they don't move while you're in grant process!

87

Mandatory tasks for the CURs, perhaps a set of suggested questions and times in the project to engage them. A direct training session on "how to use your community representative." No one on my team reached out to or relied on our community representative. They felt kind of perfunctory and not really meaningful.

Match library types of team members and mentors more closely.

Maybe have the reps meet up at the onsite if their schedule's allow

More clarity on what the role is.

more clear as to their responsibilities

More concrete tasks for community user representative.

More specific guidance about what they should be doing and how much they should be involved.

More suggestions on how to involve our CUR

Nothing, it worked very well.

Perhaps wait until projects are more well-defined to choose CURs

Picking the community representative later in the process, after the community being served has been completely identified.

Please note that I was a project director and did not have a community rep., so this section is moot for me.

Propose or suggest ways of looping them in sooner and more often.

Spell out the required interactions more clearly. I didn't really consult my rep for much input, I would have made a different choice in hindsight.

That not everyone on the team choose a CUR---one or two members should choose CURS that would work best with our group.

The rep should be someone outside of the library where you work--i.e. someone in the community or a patron.

They are key to the projects!

to have some sort of product with a community representative required, even just a write-up of interaction, by the end, and maybe making sure every person has a community representative by the time they get to the middle session

to have the mentor role more clearly defined from the beginning so that they know exactly what they should be doing

COMMENTS ON MENTORS FROM FINAL SURVEY

A clearer explanation of that their role is, although I can't think of anything that hasn't already been tried!

Again, she was perfect for us.

be more involved and actively engage with us. We are all so busy it was hard to remember to contact him. We didn't need him on calls but it would have been nice so he could see what we did.

Be sure it is someone who is invested in and knowledgeable about the projects being done--and someone who is willing to devote the time to it.

Being more clear about the role and expectations of the mentor

88

Better match on the mentor and team project

can't think of anything

clearer instruction/support from the state as to the mentor's responsibilities

Give them a more defined role, or make their role clearer to teams.

He was great and became another team member

Hey, you spelled make right this time!

I am the mentor for the Team

I did not feel that each of the team's mentors had expertise or engagement in the type of project each team was undertaking. I think seeking mentors that specifically match the team's project - that can provide expertise as well as guidance - would be great.

I think the role is well-designed, but there's no accounting for personality differences between teams and mentors; I don't have a suggestion for that.

It is important for mentor's to strike the perfect balance between letting the team move independently and offering help and advice. Our mentor was fantastic at achieving this balance.

Maybe select mentors after the teams are formed and see what qualities are needed to work with the group? Ohio has strong mentors and clear outlines of the role-- that has never been an issue. But it may have been useful to find mentors for the group one they have formed and we can see what the needs are to balance them out? That may be more easily said than done, however.

Maybe use past ILead participants as mentor's so they could share past experiences in teams.

More clear cut roles and duties

More sincerity and believing in us.

Our experience with our mentor was fantastic. No suggestions.

Our mentor was not a great match for our team, but she was very nice. We were very self-directed.

Our mentor worked hard to keep us communicating. I'm very thankful for her guidance and contribution.

Our team was very amicable as we all knew each other well before this project. If we had not known each other, I'm sure our mentor would have handled disagreements smoothly. There just were not that many conflicts to smooth over.

Provide a mentor that had expertise in the area we were pursuing.

Tell the Ilead organizers to use the mentors better by telling them everything they need to know

The mentors should be chosen based on skill set in relation to the project. Our mentor was very nice and helpful, but did not have any subject knowledge.

This was the same question two screens back. Is that an error? I would have the mentor around less so that the team has time to try things out on its own and then go to the mentor instead of having them around all the time.

to make the mentor's role clear from the start

89

FINAL COMMENTS FROM FINAL SURVEY

Project already completed but continuing.

As one of the off-site locations we often had technical difficulties with the keynote presentations out of Illinois. Improving that would be very helpful. All in all it was a very beneficial experience. I hope the project can continue.

Because our project is to circulate a kit of small robotic (FinchBot, Moss Zombo Nitron etc, ) circuitry (Squishy Circuits, Snap Circuits, etc.) and computing resources (Raspberry PI, Minecraft server) for public libraries in the State of SC to trial for a month the impact is not only for the library system I'm in it is for the county library systems in the state that will receive the kit -- provided the State Library of SC circulates it -- over the next year or so hopefully longer. The team I'm with has Been a pleasure to work with even with our different personalities. We formed a great group. To say an exact completion would be ... If the State Library of SC decides not to circulate it then our project ends! If not the project continues on until the Equipment falls apart due to use or becomes outdated.

Due to delays with items we needed to purchase, our timeline got pushed back and we were not able to have a completely finished product by the October presentation. I wish we would have had more time between when we got our money and the final presentation. I think that if we would have had another month prior to the presentation, we would have been ready with a finished production

Great overall experience! Library systems across the nation have always been about changing with the needs of the community and ILEAD is at the forefront of providing insight on what the appropriate innovative adjustments should entail.

Higher quality training at the state level. Choose community reps as we go, it was difficult to determine at the beginning who we would need.

I appreciated, as I know the other members of my team did too, the openness of the ILEAD setting; the ability to experiment, learn, and question with constructive criticism but not fault-finding. Our projects and the many projects within them were a chance to try new skills, new ideas, even new professional identities. The team-building exercises, for the whole group and the teams, were essential to this environment, and, with the generally amazing speakers and faculty members, helped make it an exceptional PD experience. This was my first venture back into PD experiences after several years for various personal and professional reasons, and it was a wonderful and inspiring kickstart. Thank you.

Higher quality training at the state level. Choose community reps as we go, it was difficult to determine at the beginning who we would need.

I don't feel like it is really a project that ends or is complete. We completed the groundwork and it is now replicable by others, but ongoing for users.

I don't think that teams should be made up of people from the same library when one of them is the direct supervisor of another. I can't speak for everyone, but there was initially a lot of pressure to participate and it was difficult to establish myself as an equal in the group with this sort of dynamic.

90

I feel as if our group would have been better served by having nine days of work time to complete the project. The presentations were mostly unrelated to our project and those meant to be inspiring or to discuss leadership were not successful. The actual work time provided during the ILEAD sessions were so scattered that they did not allow for deep, meaningful work to happen. The largest amount of work time was provided during the 3rd session so that we could complete our seed and grow videos and ta-da presentations. I would have preferred that time earlier in ILEAD so that we could work on the project.

I found it useful to write up for myself what I learned about myself as a leader. I did it because I thought it was expected of us, but am glad I made the mistake because it gave me a chance to think about it in a focused manner and reflect more on the talks and activities throughout ILEAD. It probably would be useful to do this exercise with the technology component of ILEAD too.

I like the concept of ILEAD for a variety of reasons. I like the project management piece and the fact that we bring them training and expertise based on their needs as they progress. The struggle for us is getting people to participate. The construct f the program is such that there is a limited amount of time to decide on a project in order to meet the deadline to apply, finding a community person to 'support' the project yet they are rarely involved ever again, in my experience. These 'rules' for the program make it difficult for potential participants to understand the concept and thus meet the criteria -- it seems overly artificial. And it seems to me also that a year is rarely enough time for any of the projects to be completed-- they need 18 moths. That also, to me, seems like setting them up to fail-- the program is over, the money is essentially gone, and no support left for them to see the project through. Although having a deadline is part of work-life. I struggle with this aspect of ILED and don't know if there are good solutions available.

I thought this was a rather incredible experience with its various presentation, leadership training, time to work. I do think there were times we really overthought our project too much. I am not sure we always needed all the presentations. We could have used more team time in the beginning sessions.

i was the state program manager so these questions did not relate to me

ILEAD has been a revolutionary experience in my career. There is nothing else like it or the professional challenges posed by it. It has spurred tremendous growth for me professionally and I have seen the impact it has made on everyone has been involved--team members, mentors, and instructors alike. We have all learned from each other, all helped one another, and all grown as librarians, leaders, and innovators. We've also developed a strong and diverse network amongst us. Thank you for this opportunity!

ILEAD USA was a fantastic experience. I learned so much professionally and personally. The keynote speakers from Illinois were wonderful and inspiring. The local presenters were almost all terrific and shared important information specific to our state's needs. I would recommend the ILEAD USA experience to anyone!

ILEAD USA was a great experience for our team. It gave us time away from our libraries to deepen our relationships, to work on and launch a complex project and to make something magical happen. I would definitely do it again -- I think it helped to have clear idea of what we wanted to do (although the process did help us refine our project into something more manageable).

ILEAD USA was an amazing experience. I feel like I've learned and grown so much throughout the process. It has made me much more confident in my abilities to be a leader and an

91

innovator in my field. Once our project is completed and up and running I think it's going to provide an amazing resource to libraries and library staff.

ILEAD was instrumental in moving the needle in Delaware.

It was a great experience and I appreciate all of the effort that the WIS DPI team put into making this a great program, at a great place, and with great learning opportunities

It was a great learning experience professionally and personally. I enjoyed working with my team and with my mentor.

It was amazing and I am so thankful to have been able to been a part of this cohort. I am very thankful for each of the staff for putting this event together and making it such an enriching place.

It was an awesome experience, and I'm lucky to have been involved in such a unique program.

It was an incredible experience that I hope to draw upon for a long time.

It was difficult to answer the question about whether we completed our project. We did not complete the initial project, but it morphed into something bigger.

It was great to have dedicated time to work with the team. However how our state rolled out assignments often got in the way of the project work. The state-specific technology presentation seemed dated. The chief organizers from our state were not inspiring nor were they tech-savvy.

It would have been good to have a clearer timeline--our project was just about launched for the October session, so we had hardly any results yet.

Make sure the streaming of the speakers works. The first in-person session, our streaming was bad, the image on the screen flickered and made us nauseous, etc. This is very important!

Our Ohio ILEAD experience was very good. The sessions were well thought out, especially the October one. I don't think we got as much from dialing in to the keynotes. With the distance and unreliable internet connection it wasn't as engaging and I don't think some of the presenters were engaging or inspiring. I would also like to see a way ILEAD USA could open up channels between the teams, see what they are doing, have a space/opportunity for contact. Will there be anything at PLA or ALA next year to showcase projects or for alum to meet?

Our project completion relies on the State Library taking over the distribution of the resources. They have had some staffing changes in the last few months, which have made this difficult. We are hopeful that, once these staffing transitions are complete, they will be able to begin circulating the resources.

Our project is on track to be completed. We are presenting it to library administration on December 7 and we hope to implement by January 15. Our team is very proud of our project and I could not be happier to have had the opportunity to work with such wonderful people throughout the process. This has been an experience I will always cherish and I think the outcome will be both fun and impactful for our patrons and staff.

Our project is ready for checkout, it has gone out once so far. It's only been ready a week. I'm sure it will have more impact as more patrons use it.

Our project is such that we hope to continue it indefinitely. The ILEAD team may not be the people that work on the project in the future but other librarians may step up and fill in.

92

Our project presented suggestions that will need to go through a statewide Steering Committee. Our group will continue to meet and continue to work towards our goal of pushing statewide initiatives. We feel that developed a model and are working to make more videos to show other librarians. We want all librarians throughout Delaware to build a relationship with their school librarians. We will also continue to go to school events, and network through different school channels.

Our project was to put together a proposal. I think that the proposal will take life and our continued participation in the new program will go on. We are looking to establish a new program that will take place every summer.

Our project will be ongoing and is becoming attached to a larger 3-yr state initiative.

Our team felt more like a high school group than a professional team. Some team members didn't quite understand that they signed up to dedicate time to ILEAD. A lot of work was done by some members of the team while very little work was done by others. Th level of knowledge and professionalism of our team members was uneven and in some cases our mentor didn't do a good job addressing this or swaying team members in the right direction.

Project will be completed depending on funding. I had a great experience with ILEAD and our team worked well together. We completed each other--some were young and very techy savvy; others of us a bit older and had the wisdom and experience about other things.

Team is using Google Analytics to track usage, including repeat visits.

Thank you for the opportunity to work with people in my field, but from various locations. It introduced me to great resources I would not have known otherwise.

Thank you for the opportunity.

Thank you so much for the opportunity to participate in ILEAD. I have grown so much personally and professionally during these past 9 months. I can't thank you enough for the opportunity. I hope ILEAD is offered again in the future. It was such a valuable experience!

The above question is out of place, does not reference previous question. My team's impact will be measured over the long-term. They are hoping to see a shift in the mindset of girls and technology.

The project is ongoing - It is functional as a tool for programming librarians

There should be different surveys for participants and mentors. I am not able to answer most of these question because they are not geared towards mentors.

This experience was life-changing both personally and professionally. Everything I learned has been, and will continue to be useful at work and also to train others. I would never have been able to create this educational video without the inspiration from the mentors, others at ILEAD, the grant money, and most importantly my team members. Thank you for this opportunity!

This program was poorly organized, poorly communicated and poorly designed. I really liked my team but the Ilead group was pretty miserable. This program needs better organization, better instructors, different process around community reps (we never used ours and why does each team member need one? It makes no sense and is more trouble). I wouldn’t recommend this to anyone.

This was a HUGE time commitment. Outside of the in-person sessions, I spent at least 4, sometimes 8-24 hours on ILEAD work every week for most weeks from March through October. I wish I had known that ahead of time so that I could have properly prepared. also believe that there was more "busy work" than necessary, such as over-evaluation of

93

member's experiences (I've filled out 4 surveys in the last week, along with feedback at the end of every conference day and at the end of the previous two sessions).Also, the requirement for 5 intersession hours was a lot to handle along with everything else we were doing to bring our project to fruition.

We've all decided to go beyond our original project goal, and of course much marketing and implementing of our programs are still to come.

evaluation of ilead 2015 - uis · evaluation of ilead 2015 center for state policy and leadership...

Documents