exploring experienced professionals’ reflections on computing education

Exploring Experienced Professionals’ Reflectionson Computing Education

MARISA EXTER, Indiana UniversityNICHOLE TURNAGE, Oklahoma Baptist University

This exploratory qualitative study examines computing professional’s memories of their own formal andnon-formal educational experiences, their reflections on how these have prepared them for their professionalroles, and their recommendations for an “ideal” undergraduate degree program. Data was collected throughsemi-structured interviews of experienced computing professionals. Ongoing on-the-job learning is a nat-ural part of professionals’ work lives. Participants indicate that important elements in an undergraduatedegree program include foundational computing topics, development of critical thinking and communica-tions skills, and a strong emphasis on experiences similar to those encountered in a professional position,such as complex, realistic group projects. Specific programming languages and technologies should be usedonly as practice in solving problems and learning on one’s own.

Categories and Subject Descriptors: K.3.2 [Computers and Education]: Computer and InformationScience Education—Computer science education, curriculum; K.6.1 [Management of Computing andInformation Systems]: Project and People Management—Training

General Terms: Design, Documentation, Experimentation, Theory, Management

Additional Key Words and Phrases: Computing education, Computer Science education, non-formallearning, self-learning, university, higher education, undergraduate, adult education, continuingprofessional development

ACM Reference Format:Exter, M. and Turnage, N. 2012. Exploring experienced professionals’ reflections on computing education.ACM Trans. Comput. Educ. 12, 3, Article 12 (July 2012), 23 pages.DOI = 10.1145/2275597.2275601 http://doi.acm.org/10.1145/2275597.2275601

1. INTRODUCTION

Education gives basic foundations. [Working in] business gives therefinements.

Preparation in school was a nice thing but wasn’t necessarily what I reallyhad to know to do [the job]; there were a lot of things I still had to learn.

These statements made by participants of this study highlight the importance oflooking at the impact of both formal and non-formal education on the development ofcomputing professionals.

Understanding how computing professionals continue to learn throughout their pro-fessional careers will help those who design university programs prepare students forthis important aspect of professional life. This study examines how the formal and

Author’s address: M. Exter, Department of Instructional Systems Technology, W.W. Wright Education Build-ing, Room 2276, 201 North Rose Avenue, Bloomington, IN 47405-1006; email: [email protected] to make digital or hard copies of part or all of this work for personal or classroom use is grantedwithout fee provided that copies are not made or distributed for profit or commercial advantage and thatcopies show this notice on the first page or initial screen of a display along with the full citation. Copyrightsfor components of this work owned by others than ACM must be honored. Abstracting with credit is per-mitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any componentof this work in other works requires prior specific permission and/or a fee. Permission may be requestedfrom Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701, USA, fax +1 (212)869-0481, or [email protected]© 2012 ACM 1946-6226/2012/07-ART12 $10.00

DOI 10.1145/2275597.2275601 http://doi.acm.org/10.1145/2275597.2275601

ACM Transactions on Computing Education, Vol. 12, No. 3, Article 12, Publication date: July 2012.

12:2 M. Exter and N. Turnage

non-formal education they have received prepares computing professionals for theroles they play throughout their professional lives. Gaps or areas for improvementin educational preparation and continuing education as identified by computing pro-fessionals are highlighted. We then discuss possibilities for improving educational sup-port for computing professionals.

2. REVIEW OF LITERATURE

2.1. Lifelong Learning

Research on professionals across many fields of design indicates that expertise contin-ues to develop beyond the period of formal education. In fact, it typically takes about10 years for designers to gain expertise in their area of specialization [Cross 2004].Clearly, designers continue to learn beyond their initial university experiences.

Traditionally, education has been viewed as a “period of preparation and training”that takes place within primary, secondary, and university settings, and is “followedby a period of action” [Lengrand 1989]. In order to fully prepare students to succeed inlife, schooling must therefore “cram the pupils’ heads with all kinds of facts” that willallow them to build a satisfactory amount of “accumulated capital” [Lengrand 1989].

Taking lifelong learning into account can result in stronger and more useful prepa-ration for professionals, and eliminate the need to “cover [all] the ground” in formaleducational programs [Houle 1980], by treating education as an important componentof each phase in a person’s life. Early education can focus on the skills necessary toacquire knowledge and communicate with others, rather than the acquisition of largenumbers of specific facts, while universal education of adults can be seen as a normalprocess that is just as important as the education of children [Cropley 1989; Lengrand1989].

Based on this view, ongoing adult education can and should take multiple forms,depending on the topic and need. Different terms have been used over time, but thetypes of adult education generally fall into three categories [Radcliffe and Colletta1989].

— Formal Education is used to describe hierarchically structured, graded educationsystems. This includes primary and secondary school, universities, and technicaland professional training programs.

— Informal Education includes the daily experiences with family and the communitythat help individuals acquire attitudes, values, skills, and knowledge.

— Non-formal Education describes any organized education that takes place outsideof the established system of formal education. Adults generally seek out non-formaleducational experiences when they need to acquire skills to address a specific prob-lem or client need.

This article will focus primarily on the formal and non-formal educational expe-riences of software designers and ways that formal education did or did not supportthem in their role as lifelong participants in non-formal education, as well as their roleas computing professionals.

Tough [1989] describes adult learning as an iceberg, with self-instruction as theunseen base and more formal, instructor-led instruction as the visible top. Self-directed learning projects are typically aimed at acquiring skills and knowledgedirectly applicable to an anticipated task. According to a number of studies acrossmultiple countries in the 1970s and 1980s, approximately 80% of the adult popula-tion were “continuously engaged in a series of learning projects, of which only 20% areoccurring in formal classes” and typical adults pursued approximately five distinctlearning projects per year [Tough 1989]. More up-to-date statistics are hard to find;


Exploring Experienced Professionals’ Reflections on Computing Education 12:3

Livingstone [2001] indicates that most North Americans spend on average 10 hoursor more on informal learning per week but that most studies do not distinguish be-tween instruction and self-teaching. A 2004-2005 survey of adults in the United Statesfound that 83% of adults in professional/managerial professions pursued some formof informal instruction, while 93% were enrolled in employer-supported, work-relatedcourses, or training. Work-related, self-directed learning was not addressed [O’Donnell2006]. Although self-instruction by definition does not involve pre-planned orinstructor-led educational experiences, it clearly forms a crucial component of lifelongeducation.

Self-directed learning projects are most commonly motivated by an anticipated ap-plication of the knowledge or skill being explored. Self-directed learning may be cho-sen for a number of reasons, including desire to learn at one’s own pace, desire to usea flexible learning style that matches one’s own preferences and to structure one’s ownproject, and the absence of restrictions such as the time and location a course is offeredand transportation issues. Although self-instruction by definition does not involve pre-planned or instructor-led educational experiences, it clearly forms a crucial componentof lifelong education [Driscoll 2000].

Continuing Professional Education (CPE) allows professionals to “advance froma previously established level of accomplishment to extend and amplify knowledge,sensitiveness, or skill”, and can take many different forms [Houle 1980]. A usefulframework for understanding CPE is seeing it as an integrated learning system includ-ing professional practice, context, and knowledge [Daley 2000]. For pre-professional(university) education, this may be achieved through internships and projects duringwhich CPE is a resource [Knox 2006]. Practitioners frequently use information ac-quired through CPE in different ways than the program designers intended, but thisis not necessarily detrimental [Daley 2000]. For professionals already immersed ina real-world setting, the need for education typically arises from work tasks. CPE ismost valuable when it is seen as having a reciprocal relationship with work [Knox2006; Mott 2000].

2.2. Computing Education

Standards created by the professional organizations, such as the Association forComputing Machinery (ACM), the Institute for Electrical and Electronics Engineers(IEEE), and the Liberal Arts Computer Science Consortium, tend to focus on generalskills that these organizations indicate all computing professionals need [Atlee et al.2006; LACS 2007]. However, when asked, managers and practitioners indicate thatthey believe that skills beyond those identified by the standards are needed, includ-ing knowledge and skills related to business strategy and applications, technology in-frastructure and support, and organization and management [Andriole and Roberts2008]. There is clearly a need to balance theory and practice in order to addressrapidly changing industry needs [Callahan and Pedigo 2002; Dawson 2000; Tucker2002]. Educators are further challenged to teach “soft” (social) skills within comput-ing courses [Arrdis et al. 2008; Van 2006], including listening, receiving and providingfeedback, communicating orally and in writing, planning, reflecting on what has beenlearned, and interacting with peers, including dealing with conflict and making ethicaljudgments.

Continuing debates among computing educators focus on a large number of top-ics relating to both the content and the structure of curricula in computing-relatedfields, including the specific programming languages used, and the use of team projectsand other less traditional teaching techniques. Ongoing discussions also address thespecific topics covered during undergraduate (as well as graduate) computing-related



educational programs. As pointed out by Kumar [2010] some languages and program-ming development environments may be especially attractive to students in introduc-tory courses, because these environments encourage playfulness through the abilityto quickly design and implement interesting and fun applications. Others are con-cerned that it is all too easy for students to overlook the fundamental structures andalgorithms they should be mastering early on in their degree program because of theavailability of extensive libraries provided by some modern programming languages[Dewar and Astrachan 2009; Dewar and Schonberg 2008]. They also highlight theimportance of prioritizing other fundamental skills important in creating complex,reliable systems. These include topics such as mathematics, formal specification, re-quirements engineering, systematic testing, formal proofs of correctness, and struc-tural modeling [Dewar and Astrachan 2009]. Other educators discuss the value ofspending time and energy on projects that may encourage student enthusiasm [Dewarand Astrachan 2009], help students to learn higher-order thinking skills [Hauer andDaniels 2008], and give students experience dealing with clients, struggling with com-munication and people skills, developing timelines, learning about new programminglanguages or technologies on their own, and dealing with complex and stressful prob-lems similar to those they would encounter in the real world [Ghassan Alkadi andSchroeder 2010]. A related discussion examines the benefits and drawbacks of teach-ing within a context versus more generalizable, decontextualized instruction, and inwhat situations each approach should be used [Cooper and Cunningham 2010; Gudzial2010]. Nearly all of these topics are part of a larger discussion of whether educationin the field has been dumbed down over time by removing or reducing topics once feltimportant, or whether recent changes have not gone far enough to meet the changingneeds of the current workplace [Dewar and Astrachan 2009; Dewar and Schonberg2008]. Yet, as Walker [2010] points out, not every topic can be adequately coveredin a four-year curriculum. Therefore, curriculum designers need to set priorities andprovide a foundation for long-term study.

Because technology changes so rapidly, the need for ongoing education beyond theperiod of formal education has been recognized since the early days of computer sci-ence. In 1978, Fischer, Alvarez, and Taylor surveyed practicing programmers to deter-mine how they kept up to date and made suggestions on the implications of this studyto computer science education [Fisher et al. 1978]. They found that the majority ofprogrammers kept up to date by conversations with colleagues, shop standards, andbooks and manuals. They found that programmers tended to be more up to date thannon-programmers, and suggested that managers ensure that they themselves were upto date and that managers leverage programmer’s inclinations to learn from one an-other by hiring new programmers with up to date skills, and keeps shop standards andan in-house library of published books up to date.

A much more recent study addressed the types of knowledge most important tosoftware professionals through a survey instrument that asked participants to re-spond to each of 75 topics identified from university curricula and SWEBOK stan-dards. For each topic, respondents indicated how much they had learned at school,how much they currently knew, and how useful each topic had been to them. Theresults of this survey allowed the researchers to determine areas in which profes-sionals had to augment their learning on the job, and topics that were apparentlyextensively covered at the university level but then largely forgotten as they werenot needed on the job [Lethbridge 2000]. Similarly, Andriole [Andriole and Roberts2008] found that a number of areas important to practitioners were not addressed byACM Task Force standards, including knowledge and skills related to business strat-egy and applications, technology infrastructure and support, and organization andmanagement.



University programs need to provide a foundation for students to succeed in design-ing and developing increasingly complex systems, and faculty members and the fieldat large continue to pursue ways to improve educational programs to fill this need.However, as was pointed out in Fisher et al.’s [1978] study, in a field that focuses onrapidly changing technologies, keeping up to date will always be important. Under-standing how and what computing professionals learn after they have left the univer-sity may help instructors and designers of university programs prepare students forthe important role self-instruction will play in their future careers. The aim of thisstudy is to explore how formal and non-formal education prepares computing profes-sionals for their future roles, how they seek these types of information, and how theybelieve educational programs could be improved to meet the needs of future computingprofessionals.

3. METHODS

Research questions for this study include the following.

(1) How have computing professionals’ formal educational experiences prepared themfor the roles they currently play in their professional positions?

(2) How have computing professionals’ non-formal educational experiences preparedthem for the roles they currently play in their professional positions?

(3) How do computing professionals seek educational experiences outside of formaleducation?

(4) How do computing professionals recommend improving formal educational pro-grams to best prepare students for the roles they will play during their professionallives?

3.1. Participants and Setting

For the purposes of this study, the term computing professionals includes those whoparticipate in the high-level design of software systems, design and development ofprogramming code, and/or interface design. Participants included professionals witha range of experience levels and who are currently working in the field or recentlyretired. Participants from a range of industries were recruited via a snowball tech-nique, beginning with the researchers’ own personal contacts. At the end of each in-terview, the interviewee was asked to pass the invitation to others who might be in-terested and/or to provide e-mail addresses for potentially interested individuals whowere then contacted directly by the researchers. Colleagues of the researchers alsopassed invitations on to potentially eligible computing professionals. The invitatione-mail itself described criteria for acceptance to the study, and requested that he recip-ient forward it to potentially interested individuals. Those who forwarded invitationswere not informed whether or not those they passed the e-mail to participated in thestudy.

Twenty-three interviews were conducted. Interviews were conducted at the partici-pants’ work place, in person outside of the workplace, or remotely using phone, skype,or instant messaging. In order to focus on the experiences of experienced computingprofessionals, transcripts were omitted from this study if one or more of the followingwas true.

— The participant had been working in the field for less than five years (five partici-pants).

— The participant did not meet the researchers’ definition of “computing professional.”(three participants).



— The participant’s answers did not pertain to the questions being asked and didnot provide useful information for the purpose of the research questions (twoparticipants).

— There were significant missing or unclear portions in the recording or transcriptbecause of technical difficulties (one participant).

Fourteen interviews were included in this study. Participants had a range of ex-perience levels (between six and 35+ years) and came from the following industries:telecommunications, middleware development, academic support, education and re-search (data management, research, and educational applications), and corporate (re-tail, insurance, sports analysis, and parking lot management). All industries were notequally represented, with an overrepresentation in telecommunications (seven partic-ipants). Only one female was interviewed, and her transcript was omitted from thestudy as it did not meet the criteria described above. The rest of the participantswere male. Participants were spread across the United States, with an overrepre-sentation of participants from the Midwest. Participants’ work responsibilities weretypically spread between multiple roles, including architecture, software development,software/hardware integration or firmware development, and specifications writing.

3.2. Data Sources

Each semi-structured interview lasted approximately 1 1/2 hours and similar questionswere used regardless of the medium. The two researchers were each present for all butone interview (one researcher was not available for that interview, but the recordingand transcript were later reviewed by the second researcher).

Guiding questions addressed the participant’s current role(s), professional history,formal educational background, current approach to continued professional education,and ideas for improving undergraduate computing education programs. Ongoing mod-ifications were made to the semi-structured protocol to better elicit the desired re-sponses, based on experiences in earlier interviews. As appropriate, questions werere-stated and follow-up questions were given based on participant responses. The fi-nal set of baseline questions are included in Table I.

Interview recordings were stored digitally. Each recording was transcribed verbatimby one of the two researchers. If a portion of the recording was unclear or included ter-minology unfamiliar to the researcher doing the transcription of a particular interviewrecording, the second researcher was asked to review the recording and transcriptionfor that segment. If proprietary information (such as the name of a company or vendor)was mentioned, the specific information was replaced with a tag such as “<employer>”or “<vendor: large telecommunications company>”. If a small segment was not deci-pherable after multiple reviews of the recording, this was also marked in a bracketedcomment. In one case, the recording device failed and a portion of the recording wasmissing. In that case, the interviewers’ notes were used, with a note that this portionof the transcript was not verbatim. The researchers both reviewed these notes imme-diately after the interview to verify their correctness and completeness. No quotationsfrom the non-verbatim portion of the transcript were included in this presentation ofthe data.

3.3. Data Analysis Process

We used a constant comparative method [Lincoln and Guba 1984] to compare andcontrast data within and across interview transcripts into a meaningful construction.The unit of analysis is a transcript segment that stands alone as a single coherentidea. We informally reviewed each transcript shortly after the interview took place



Table I. Final Semi-Structured Protocol

1. Role1.1. Where do you currently work? Is your current job related to programming or software design? [If

not, ask where they previously worked]1.2. Could you describe what your current role at <company name> is/was?1.3. What is your official title at <company name>?1.4. How do you describe your role when you talk to family, friends, or acquaintances? OR If an

acquaintance asked you what you do, what would you say?1.5. What is your work history prior to starting this position?

[If role has recently changed, follow up with more detailed prompting about their previous position/role:]Could you describe what your previous role at <company name> was?

2. Education2.1. What is your formal educational background? [Referring to post-secondary unless they feel they

had relevant earlier experiences]2.1.1 What are the most useful things you learned from Computer Science [or related] courses that

apply to your current role?2.1.2 What are the most useful things you learned from non-computer science courses [or majors] that

apply to your current role?2.2. What other educational experiences, if any, have you had that have helped you in your current

position? These could be formal or informal. [If they do not mention each of the following, follow up withprompts].

2.2.1 Learning from others at your workplace.2.2.2 Learning from vendors or others in the field.2.2.3 Workshops sponsored by employer or external.2.2.4 Self-taught (How? Websites, books, etc.)2.2.5 Anything else that comes to your mind?2.3 If you encounter a situation where you need to learn new things, how do you go about learning

what you need? [Follow up as necessary, for example, see the following questions.]2.3.1 If they grab a book or other resource, ask how used.2.3.2 If they say “online”, ask to show a typical website or web search and discuss how they use this.2.3.3 If they refer to samples provided by other people through discussion, white-boarding, code sam-

ples, etc., ask Who provided them? How did you reach them? In what ways did you use them?3. If you could design an ideal undergraduate software design program, what would it be like?

and made ongoing modifications to the semi-structured protocol to better elicit thedesired responses, based on experiences in earlier interviews.

Both researchers were involved in the entire data analysis process. As is recom-mended by Lincoln and Guba [1984], we used our own previous experiences and tacitknowledge to develop ourselves as “human instruments”. This informed the develop-ment and refinement of interview questions as well as our understanding of the dataduring the data-coding process. At the time data analysis was conducted, the first au-thor held Bachelor’s and Master’s degrees in Computer Science and had 14 years ofexperience developing software and managing software development projects in sev-eral domains. The second author has a degree in Computer Science, has had sevenyears of software development experience, and was, at the time of this study, an ad-junct faculty member in a Computer Science department.

We initially reviewed three transcripts in order to create preliminary codes. Thenwe went through each transcript coding each relevant unit. Tentative categories werenoted to ensure that coded units remained within the scope of the study and thatcategory notes were consistently applied. Then we placed transcript units and notesabout the code, category, and location in the transcript for each unit on note-cards.Then, a card sort technique was used. The tentative categories were used only for



clarification purposes, and new categories were allowed to emerge. As necessary, wereferred back to the transcripts to understand the context of a unit of text. If we wereunsure of where a card should go or were in disagreement about the card, we put itaside. At the end we looked through all remaining cards to determine which categorythey belonged in. Eleven cards were rejected as the researchers determined that theydid not fall into any of the emerging themes, and did not address one of the researchquestions. The remaining cards were placed in relevant categories. The researchersdiscussed and reorganized until they were in complete agreement about the categoriesand criteria for acceptance into each. Finally, the card stacks were used to determinerules for inclusion for each category and sub-category that emerged. Coding and in-formation categories were worked on collaboratively, and negotiation was done on anongoing basis.

The high-level themes that emerged during the analysis are presented in Table II.They will be discussed in detail in the Findings section.

4. FINDINGS

The themes that emerged during the data analysis (summarized in Table II) are dis-cussed in detail in the following sections. We will describe participants’ formal ed-ucational background and the resources used by participants in pursuing their ownlearning goals. Finally, participants’ own recommendations for an ideal educationalprogram for software designers will be discussed.

4.1. Formal Education

Formal educational experiences as described in this study include anything studentslearned in their college or university coursework. Participants received Associates,Bachelors, Masters, and Doctoral degrees in a range of fields. Seven have one or moredegrees in Computer Science. Eight have degrees in a related field such as Telecommu-nications or Electrical Engineering. Still others had unrelated degrees in fields such asFine Arts, Physics, Mathematics, and Business. Those who have been working in thefield for many years would not have had the opportunity to pursue a degree in Com-puter Science or similar topic, and tended to have degrees in Electrical Engineering.

When asked “What are the most useful things you learned that apply to your currentrole?” participants highlighted general skills they had gained, as well as describingspecific courses that were particularly useful.

4.1.1. Learning about Thinking and Learning. Participants indicated that one of the mostuseful aspects of their formal education was the opportunity to learn techniques thathelped them to teach themselves what they needed to know. As one participant putit, “. . . you learn how to learn in your undergrad degree. You learn to, I guess, acquireknowledge on your own, versus at the high school and junior high level, everything ispretty much done by instruction, right?”

Critical thinking was also considered a highly important skill. Participants in-dicated that critical thinking skills were often developed in non-computing-relatedcourses in courses, such as math and physics. One participant mentioned that hespent a lot of time “doing critical thinking and writing about said thinking” while pur-suing a liberal arts degree. Lessons learned included the importance of understandingthings from a very low level, and the necessity of putting aside pre-existing paradigmswhen learning something radically new and different.

A general approach to problem solving was learned by practicing with a varietyof problems during the degree program. As one participant noted, “if [I] have been



Table II. Themes Identified

1. Formal Education1.1. Learning about thinking and learning

1.1.1. Critical/analytical thinking (general)1.1.2. Problem solving1.1.3. Learning how to learn (how to acquire knowledge on one’s own)1.1.4. Specific lessons learned

1.2. Computer Science courses1.2.1. General computing skills and knowledge1.2.2. Programming language concepts1.2.3. Specific Computer Science courses that were most useful

1.3. Real-world projects1.3.1. Team-based1.3.2. Sufficiently large scope1.3.3. Use of realistic techniques and tools

1.4. Non-Computer Science courses1.4.1. Hardware-related1.4.2. Math and Physics1.4.3. Communication-related1.4.4. Other non-technical courses

2. Non-Formal Education2.1. Self-learning practices

2.1.1. Value of self-learning2.1.2. Learning is need-driven

2.2. Learning from experience2.2.1. Techniques and style improve over time2.2.2. Experience enables you to understand problems2.2.3. New technologies are similar to old ones2.2.4. Underlying concepts same across programming languages

2.3. Learning from Previous Designs or Code2.3.1. Use of previous designs/coding experiences2.3.2. When to reuse code2.3.3. Samples of other’s work2.3.4. Experimentation

2.4. Learning resources2.4.1. The Internet2.4.2. Books2.4.3. Peers2.4.4. Mentors2.4.5. Training2.4.6. Vendor-provided resources2.4.7. Other

2.5. Selection of resources3. Suggestions for an Ideal Program

3.1. The basics3.1.1. Theory3.1.2. Programming languages

3.2. Design concepts and problem solving3.3. Realistic team projects3.4. Other real-world experiences

3.4.1. Realistic tools and practices3.4.2. Maintaining code over time3.4.3. Working with buggy code and “pilot error”

3.5. Non-Computer Science Courses3.5.1. Communications3.5.2. Relevant math courses

3.6. Other suggestions



through a variety of problem solving areas, I kind of feel like I can tackle [learningabout new technologies] on my own.” Participants indicated that problem solving skillswere learned in a variety of courses, with courses such as math and physics beingparticularly good at providing experience with challenging types of problems.

4.1.2. Learning from Computer Science Courses. Participants gained exposure to comput-ing skills and knowledge from their computer science courses. University-level courseswere considered especially important in getting a “big picture” of higher-level theoriesand concepts, something that sometimes may not occur when learning by one’s self.This included the understanding of systems: “. . . all systems can be described at somelevel by looking at the interactions between them. . . all other technical knowledgechanges fast[er] than you can learn it.”

Interviewees frequently mentioned that knowing good programming practices al-lowed them to learn new programming languages easily. Participants emphasizedthat they had to initially learn one or more specific language(s), but that it was notimportant which language was learned. Once they had learned specific concepts ortechniques within one language, they knew what to look for when taking up a newlanguage either within their degree program or once in the field. An exploration ofunderlying algorithms regardless of the language used was an important part of earlycomputer science courses.

Specific useful courses mentioned covered underlying technologies such as hard-ware/assembly language, operating systems, and networking. Conceptual courses werealso seen as important. These included “database design” and “data structures and al-gorithms”. Early Computer Science course work inspired non-computer science majorsto consider pursuing an additional degree in Computer Science or a related field.

When asked about useful aspects of courses they had taken, participants describedlearning specific programming concepts and developing, debugging, and testing skills.Practice in designing reusable code was an especially important lesson that some indi-cated they learned while in school, although this theme was described more frequentlyas something learned on one’s own either during their years of schooling or after grad-uating. Some participants developed their own software libraries that were used againand again in course projects and later further refined and used on the job.

4.1.3. Learning from “Real-World” Projects. Course projects with real-world aspects wereseen as an especially important component of Computer Science and other courses.Ideally, these projects required team-work by a realistically sized team (typically in-cluding two to three members). Projects that transferred to real life were sufficientlylarge in scope and required interaction with complex systems, such as time-shared sys-tems, systems with multiple inter-related hardware components. Large, team-basedprojects required students to use version control software and testing techniques,which are necessary in real projects but not often covered within formal course ma-terial. Such projects required students to learn to communicate effectively, segmentwork among team members, and check each other’s work.

4.1.4. Learning from Non-Computer Science Courses. Participants valued courses that re-quired them to use thought processes similar to those they currently use on the job.For example, one participant indicated that highlights of courses taken as part of a de-gree in Electrical Engineering included “logic development, formal ways of doing logicdesign, and . . . methodology [that] can be applied to [Computer Science].” Hardware-related courses included hands-on exposure to hardware that is similar to platformsthat would be encountered in the work world years later, helping students understandhow things work at a low level. Participants who had deep experiences with hardware



in Electrical Engineering or Telecommunications degrees indicated that they under-stood the underlying systems better than co-workers who did not have such a back-ground. One said, “often you can get insights into a problem because you have theelectrical engineering background to do it.” He later explained, “I know what our sys-tem’s trying to do when it interfaces with cable modems. . . I can see the CS majorsoften have a hell of a time with this. . . It’s not that they couldn’t [understand], theyjust don’t. They abstract it more.”

Math and Physics were seen as useful if they applied to the job or the type of think-ing done on the job. Particular courses and topics mentioned included Discrete Mathe-matics, Statistics, math problems used in Physics courses, and Boolean logic. However,others mentioned that much of what was taught in the large number of required mathcourses, especially Calculus, was never or rarely used on the job. One participant saidabout those courses, “you take it and then it never comes up again”. One intervieweeexpressed his perception that these courses were required merely to “weed people out”.

Participants indicated that some of the most important things learned as part oftheir formal education were verbal, written, and visual communication skills. Com-municating with team members and customers has been an important componentof their jobs. Technical writing was necessary for creating specifications documentsand other design materials. Several mentioned specifically that these skills were notlearned in Computer Science or Software Engineering courses, and that they were gladto have learned them in various non-technical courses. Speech, English, or writingcourses helped them gain communication skills even if they were not focused on tech-nical writing. Fine arts courses helped a few gain skills in developing useful “visuals”.Courses such as philosophy and history required students to learn to state their ideasclearly.

Individuals mentioned other non-technical courses that had an influence on theircurrent work. For example, one said that business classes taught him “how to lookat things from a higher perspective,” as well as helping him understand what occursonce a product goes to market. Other courses helped participants understand othercultures, which became relevant when working with people from around the world,as well as when developing software to be used in other countries. One mentionedrequired readings for English and other courses (particularly Science Fiction) were aninspiration for his current work as a system architect.

4.2. Non-Formal Education

Non-formal education for computing professionals is primarily driven by self-learning.Self-learning can take the form of learning from experience, or can be supported by theuse of instructional materials found online or in books, support from co-workers andmentors, on-site or vendor-sponsored training, and a variety of resources provided byvendors and other sources.

4.2.1. Self-Learning. Participants recognized self-learning as an important part oftheir role. One stated, “I think I describe myself as an auto-dictat”. He indicatedthat he learns from “a combination of lots of reading, trying, examples, people”, ex-plaining that it is easy to find examples and tutorials on the Internet or within the“proliferation of books that are made for you to learn from.” Several interviewees indi-cated that “if you manage to teach yourself something, you are going to remember it alot better than if just you go and use somebody like a reference”. Similarly, somethinglearned in order to pass a quiz or certificate exam would not be as useful as somethinglearned through “hands-on use of the technology”. Another participant recognized thathe could not remember everything, and that strategies to (re)learn material have beennecessary for him on the job.



The computing professionals interviewed indicated that their on-the-job learning isgenerally self-directed. One indicated that his previous experiences allowed him togeneralize from one system to another by abstracting what he had previously learned.This in turn helped him to guide his own learning. “I can direct myself; I don’t need tohave someone tell me what to do. That is a big advantage.”

In the participants’ experience, work-related learning occurred on an as-needed ba-sis. As one mentioned, “. . . I run across things I feel I should know more about, [but]I don’t have the time to dive into all of it.” Another indicated that “when one gets anassignment to work [on] a new project, then it is time to look in deep detail about [whatis needed for the project].” However, he indicated that he may read the introductoryportions of specifications documents not directly related to his current work, just sohe “will know what the document is about”, and, presumably, be able to go back laterif this material turned out to be relevant. Similarly, when describing learning newprogramming languages, he indicated that although he might try something out onhis own, “when there was a good reason to use it on a project; that is when the deeperlearning kicks in.”

4.2.2. Learning from Experience. Learning from experience is an important componentof self-learning. The software designers indicated they were aware of their own devel-opment over time. For example, one mentioned that his coding style had improved,noting that “there are a lot of times I. . . look back at code and say ‘what were youthinking?”’ Another mentioned that he learned iterative development skills on the job.He came to realize that the first version does not have to be perfect, and that in factyou need to be able to quickly generate preliminary documents in order to get reviewsbefore you can iterate and design something even better.

At a more general level, participants indicated that their experiences helped themto understand problems better. As experienced designers, they felt they could under-stand problems and come to solutions more quickly than novices. One indicated thatas a novice, he was “more naı̈ve”, causing it to “take longer to do things.” Ironically,participants indicated that novices often under-estimate the time it will take to solve aproblem. Previous experience allowed them to determine the difficulty level of a prob-lem and how long it would take to resolve it. One described his ability to step back andsee a problem in its entirety; “. . . I will step back and look at what we’ve mapped out,and I’ll think to myself, ‘It’s going to take us 3 years to do that’, and a lot of this has[more] to do with the time and effort than [asking if] . . . the technology exist[s] to dothat [sic].” Understanding a problem at this level “is something that young engineersdon’t have an appreciation for. Collectively it becomes part of your instinct.”

Being exposed to many designs over time also allowed interviewees to look at a pro-posed design and understand it more quickly than they had as novices. “My ability tolook at a design, whether it’s code or schematic or whatever, and understand what isbeing done. . . my experience level allows me to do that a lot quicker than when I was ajunior engineer.” The specific type of experiences professionals encountered was deter-mined by the roles they played over time. One indicated that when he was a developerhe had to have in-depth knowledge about the specific part of the design assigned tohim. However, now as an architect, he needs to have a wider understanding of allaspects of the design. The participants who were currently architects had previouslybeen developers (programmers or chip designers). They indicated that that experiencehelped them understand the underlying technologies they later incorporated in theirarchitecture-level designs.

Previous experiences also made it possible for the computing professionals to copewith an ever-changing technological landscape. As one participant put it, “I don’tknow what people would do if they had to relearn all the time, it would be really



hard, if everything’s new.” One of the most important lessons computing profession-als learned from experience was the concept that “new technologies are similar toold ones”, and that technologies with a similar purpose must also have similar func-tionality. As one participant noted, “usually you look at a device or system and you. . . put some pattern around it. I haven’t seen a hell of a lot of systems that wereso startlingly new that I’d never seen anything like it [sic]. . . and that’s probablywhy I’m able to do this at my age and everything, because I don’t have to build anew paradigm.” However, he cautioned making too many assumptions based on pre-vious experience could be dangerous. “I’m very conscious not to be so tied to [think-ing something is similar to something else he has done before] that I make stupidmistakes.”

Similarly, participants noted that learning new programming languages was notdifficult, because they realized that while they needed to learn a new syntax, the un-derlying concepts remained the same across languages. One indicated that althoughhe used to need five to six weeks of organized training per year, he now could eas-ily teach himself new programming languages, by recognizing that the same conceptswere simply expressed differently across different languages.

4.2.3. Learning from Previous Designs or Code. Previous design and coding experienceshelped interviewees learn what to do and what not to do in a variety of specific types ofsituations. Memories of past solutions could be used to resolve similar problems in thefuture. In some cases, repositories of solutions and self-created code libraries providedresources that could be directly reused. Reusing previously developed code was seenas having a number of advantages. Using existing code may save the time it wouldtake to write new code. Because it had been tested in other contexts, there was lessrisk of introducing new and potentially buggy code. However, in their experience, codereuse has not always been worth it. If the previous solution did not fit a new problemvery well, most of the existing code could not be reused. Often something written toaddress a specific need would be more computationally efficient than a generalizedsolution designed to address a different context.

In addition to reusing their own previous solutions, the software designers inter-viewed frequently learned from the work of others. Often this came in the form of codesamples, pieces of programming code someone else created for the purpose of sharinga concept or a best practice. Sometimes code samples were the only way the softwaredesigners could learn about a very new piece of equipment or software their code hadto interact with.

Although software designers often talked about “stealing” code from others, theytypically did not copy these code samples verbatim. “If I am really interested in [atechnology or technique I have heard about], I will download the software and try toplay with it, see if I can understand it better.” “Playing with it,” or interacting withthe code in order to understand it better, was a common theme. “I think the best wayto [learn about a program] is to take the code and rewrite it. Then you really know it.”

After using the code sample to understand how others have solved the problem inthe past, designers often found that the solution was not a direct fit with what theywere hoping to accomplish. However, they could write their own code to fit their ownspecific needs based on what they had learned from interactions with the code sample.Participants also challenged themselves to improve on the code sample.

Finally, software designers learned from experimenting with examples they find orcreate. They might start by writing their own code samples based on examples foundin books or online in order to learn or relearn a technique or programming language.This would allow them to generate a small demo program to determine whether their



attempted solution would work. Once this had been accomplished, they could do some-thing similar within a larger design.

4.2.4. Resources Used in Non-Formal Education. Books, the Internet, people, and train-ing events all can be used as resources for self-learners. They can be used as sequencedinstruction, reference materials, or sources for code-samples and other types of exam-ple solutions that software designers can learn from. This section describes how andwhy participants choose from a variety of commonly used resources.

4.2.4.1. The Internet. The Internet was considered particularly useful in addressinga specific, easily defined topic or problem.

If I have a very easily defined problem, I will definitely go and look up thatproblem online. And 99% [of the time] I will succeed with that in like 5minutes. If that doesn’t work. . . there could be two things: I cannot clearlyexplain what I am looking for and then. . . I might just need to find a dif-ferent way to explain what I’m looking for. Or the problem could be justbigger.

In addition to searching, participants indicated that they frequently used the listsof links provided in Wikipedia entries as a starting point. Topic-specific discussionforums were also frequently mentioned.

The Internet was also considered a good source for examples and code samples,which could be found on official websites (such as the Microsoft Developer Network),discussion forums specific to a particular language or technology, or in articles or blogpostings created by other developers. Participants indicated that they might look formultiple sources to understand all aspects of a problem, especially a problem that was“hot off the presses” (or in many cases, had not yet reached the presses). Error mes-sages observed while developing new software or interacting with a piece of technologycould be entered into a search engine. Frequently forum postings from people with thesame problem would be found, along with responses from experts describing how toresolve or work around the issue. Code samples would also be used for inspiration andexperimentation, as described in the previous section.

The Internet was the only source for learning about very new technologies andprogramming languages, or for technologies that for other reasons do not have muchpublished documentation. One participant described how he would go about learningabout a particular topic online.

“. . . let’s say for example when I had to learn RIP. . . I go to the Inter-net. . . and I write down ‘RIP tutorial’, ‘RIP examples’, ‘RIP Wikipedia’,‘RIP. . . ’. . . you know. . . The open source community will very likely havean explanation.”

Without the Internet, he explained, he could not have learned what he needed forthe job at hand.

“If . . . I just have to understand [these types of] concepts I will go to theweb. . . . I wouldn’t know what to do if I couldn’t learn from these things. . .you couldn’t take courses [for these topics]. There’s nothing you can do.”

Finally, a few participants used the internet to look for information on industrybest practices, which could be found in articles or blog entries written by respectedmembers of the community. This could allow the participants to see how others haveattacked a general type of problem: “I’ll go out and do searches on the web. . . type thegeneral subject I’m trying to work at, just get sources, ideas, look at open-source code



to see what others have done. Even if it is in another language or operating system[I can still learn from it].” Others followed favorite authors to learn about higher-leveltopics such as software engineering practices.

4.2.4.2. Books. Books were considered well-structured and good at giving a big pic-ture. They were found especially useful when starting to learn a new language andlearning about a new technology, because books lay out a topic in its entirety. Oneparticipant explained that although in the past he bought books every month or two,he now buys a book about every three months, and that “usually the books I do buyare not about specific programming things, [but] more high-level things, like how tobe a better programmer, programming processes, and things like that. More abstracttopics.” Books were seen as useful reference materials only for things that could notbe found on the internet. However, particular books were seen as useful for particularsubjects. Old textbooks were used to look up specific algorithms or design patternsby two participants respectively. Another participant indicated that when learninga new language, he always picked up a book in a particular series, which provides aconsistent format and structure across titles.

4.2.4.3. Peers. Software designers used other people as a resource in several situ-ations. Some people were considered particularly good at explaining things at a con-ceptual level. Participants also recognized that one software designer could not knoweverything, and explained that particular individuals had specific areas of expertisewithin their organization. They would ask these subject matter experts questions thatwere completely outside of their own area of expertise. Subject matter experts wereespecially likely to be asked questions if they were easy to access (e.g., sat close by)and/or could answer questions quickly.

Despite the valuable experience of their peers, the software designers interviewedconsidered asking co-workers for assistance only as a last resort. This appeared tobe for a number of reasons. Several indicated they did not want to cause take others“out of their groove” by distracting them or “waste their time”. Participants also in-dicated that they did not want to look bad or stupid by asking questions. If they feltthey needed to ask a question, they would prepare as well as they could with otherresources first. Based on the discussion of mentors (below) and participants’ expressedwillingness to share their own expertise with others, it would seem that novices are ex-pected to ask questions and require assistance. Participants also described frequentlyworking collaboratively with peers, indicating that other types of interactions wouldnot be considered “wasting someone’s time”.

4.2.4.4. Mentors. Participants recalled mentors being valuable early in their careers.Some organizations had formal mentorship programs, while other mentors appear tohave been acquired informally. Interviewees recalled that their mentors taught themgood programming practices. One recalled that his mentors helped him in “. . . justtrying to start at a high level, doing more drawings of what you are going to do. Tryingto be very thorough to bring in all the cases, the sunny-day kind of stuff as well asthe rainy-day kind of stuff. . . they teach the best way or a good way of being through,how to come up with a good architecture, design.” Mentors guided novices’ learning bydemonstrating techniques and critiquing mentees’ work. Mentors were also a sourceof information on getting along in the corporate culture of their organization.

4.2.4.5. Training. Interviewees described their experiences with on-the-job trainingoffered on-site by their employers, off-site training developed by 3rd parties on com-mon topics, and specialized training offered by vendors. Especially good courses wereoften offered by a third party or vendor and were typically used as an introduction or



“catalyst” to learn more about a topic. These courses included valuable lab exercises,an expert teacher to answer questions and provide assistance during labs, and weresufficiently long to cover a topic in some depth (typically three to five days).

A number of participants currently or previously worked in organizations whichhad mandatory education plans which required a certain number of training hoursper year. These mandatory plans and the employer sponsored training which filled themajority of the delegated training hours were generally considered a waste of time,as they encouraged software designers to take courses that were not directly relevantto their work or were focused on soft skills which were considered unnecessary orobvious to the participants or were repeated year after year. Even when the trainingcovered relevant topics, they were often of little value because they were too short,not in-depth, and not hands-on. However, one interviewee found the mandatory fiveto six weeks’ worth of mandatory education time advantageous, as it gave him formal“company time” to learn things. Once that his organization no longer enforced thisplan, he found he had to learn new things on his own time, “shoehorned in as needed,”because of quick pace of the work environment.

4.2.4.6. Vendor-Provided Resources. These resources are offered by companies whichprovide proprietary software or hardware used by the organization software designersare a part of. Vendor-provided training courses were taught by experts on the specifictechnology and generally provided a good framework. Materials and demonstrationsoftware were the only way to learn about the peculiarities of the propriety technol-ogy, which was too new or too obscure to have other forms of documentation. Samplesoftware provided by the vendor were used to explore what the proprietary technologycould actually do (which was especially useful if the technology was “bleeding edge”or a beta or early release without the full promised functionality). Sample code couldalso be used as a starting point in learning to interact with the new hardware or soft-ware. Once a sample program was run to see what it could do, the interviewees wouldexpand from it and adapt it to their own purpose.

Vendor employees were seen as a particularly useful source of information. Partic-ipants found vendor employees to be valuable resources who knew their own propri-etary technology well. While some vendor employees were experts in only one aspect ofthe proprietary technology (for example, the installation of specialized hardware), thesoftware designers could learn a lot from observing and interacting with them. Someparticipants had experience with vendors who provided access to people mandatedto provide training and support to customers. Because these vendor employees wereprovided as a resource to the organization, interviewees did not mind “wasting” theirtime. Such contacts could answer obscure questions, provide sample code, and contactsoftware or hardware designers at the vendor location to help resolve obscure issues.

4.2.4.7. Other Resources. The interviewees mentioned a number of additional re-sources which were used for specific purposes. Magazines and trade journals wereconsidered helpful for keeping up with the latest trends, but not very useful for findinginformation on a specific problem or topic needed at a specific time. Standards docu-ments, which provide an extremely detailed set of technical specifications for a partic-ular technology that the interviewees needed to use or create, were universally seenas difficult to read and search through. Unfortunately, these documents were oftenthe only source for this level of information. Standards documents must also be lookedat when writing new specifications, to find out whether an idea already exists. Aca-demic conferences were seen as a jumping off point, inspiring the interviewees to lookinto new technologies, programming languages, or techniques, and were also an op-portunity to see what competitors were doing. However, academic conference sessions



were generally too short to learn much in detail. Valuable organizational resourcesincluded company repositories (in which all employees share useful information), indi-vidual repositories, and work-place “bootcamps” or “chalk talks” offered by individualpeers, which could be used to bring a small group of employees up to speed on a par-ticular technology or technique. Finally, individual participants mentioned keepingan eye on news media for current trends that might impact their work, learning fromteaching, and improving certain types of skills through playing computer games.

4.2.5. Choosing a Resource. Internet resources were free and easily searchable, mak-ing information easy to find. Internet articles are often more up-to-date than otherresources, and may be the only source for bleeding edge topics. Community groupsor discussion forums on the Internet were useful for finding information on partic-ular technologies or languages, looking up specific error messages, and finding codesamples. Because of the ease of accessing and searching Internet resources, most par-ticipants used the Internet as their first source for most information. One explained,“It seems like I always start on the Internet first, that kind of guides me. Then [if itseems useful] I’ll recommend to my boss I take a class or we all take a class in it. Orif [the Internet sources] recommend a book, I’ll go buy the book. But it seems I alwaysstart [on] the web.” However, the Internet was hard to use for large topics, as the web-based format tended to be hard to page through or piece together to get an idea of theentire subject.

Books were generally seen as useful for high-level or abstract information. A par-ticular advantage of books was that they could be physically held and read anywhere.One interviewee noted “there is just something . . . comfortable about sitting down andreading something.” The tactile nature of books was also mentioned. “Books are nicebecause the information stays put. I can remember where I read it [within the book]and get back to it if I want to go over it again.” In the absence of books, one participantwould print out online materials, which allowed him to lay them across his desk to geta better overview of the information.

People (including colleagues and vendor employees) were seen as especially goodat teaching organizational processes and best practices (such as testing techniques).Subject matter experts may be the only resource for particular types of information,especially proprietary information (within their own company, or within a vendor or-ganization). Peers and mentors could also recommend other good resources.

Participants frequently used resources in combination. For example, a one-daytraining event might give a high-level picture. However, an entire topic could notbe mastered in one day, but a book could allow one to go into more depth by providing“everything you need to know” about the topic. Similarly, a book might be skimmed toobtain a big-picture understanding, after which the participant would start “playingwith” the technology, only returning to the book as reference if he got stuck.

In summary, all of these resources, including training, were used as necessaryto aid in the computing professionals’ self-directed learning. Mandatory trainingwas often seen as having little value, while resources chosen to meet a currentneed were most useful. Use of resources was nearly always combined with hands-onexperimentation.

4.3. Suggestions for an Ideal Program

Interviewees offered a range of suggestions when asked what an ideal program wouldbe like for someone planning to enter a job similar to their own. While some mentioneda few specific courses that they felt were useful and others focused on recommenda-tions for “real world projects”, others outlined elaborate four- or five-year programs. Wedescribe common aspects recommended by interviewees in the following sub-sections.



4.3.1. The Basics. Participants indicated that students need to learn some basicsbefore they can go on to more interesting real-world problems. The basics includetraditional underlying theoretical courses, such as “Algorithms & Data Structures” and“Database Design”. Experience with specific programming languages was consideredvery important. However, interviewees stressed that it did not matter which particularlanguage was learned. Rather, learning a language would allow students to learn otherconcepts. “Just knowing concepts doesn’t stay in your brain for long. You need both,you need concepts, and the way they can be used in practice. . . different people areusing different languages. . . the syntax of the languages is not important,” explainedone participant. He went on to describe his own experience: “I haven’t used [Fortran]since I graduated . . . but the concepts are the same. So lab [exercises in the specificlanguage] would complement the theory.”

4.3.2. Design Concepts and Problem Solving. Intense experiences with design conceptsand related problem solving within courses would help students understand why theyneed to learn to design. “All too often, people [come] out of technical degrees knowingwhat to do, but not why to do it. . . . The technical specifics [of learning something likethe Java programming language] are important to be productive, but . . . to do some-thing useful, you need to be able to think.” Such courses would encourage studentsto learn general problem solving: “Something that [teaches] how to solve problems,not just how to write code. Not just pure algorithms, pure math, but really, how doyou identify the problem”. Courses in Object Oriented (modular) design would also bestressed.

4.3.3. Realistic Team Projects. Large, team-based, real-world projects would be centralto most of the participants’ recommendations. As one explained, “I remember in a lot ofmy classes it was just little teeny tiny classes with all code samples. It seems like thereshould be bigger projects with a big team involved, something [that involves] workingtogether to create a big project.” The problem would be carefully designed to ensurethat students learned to work effectively in teams, as they would at a real work place.“I would probably make it project centered with students in teams. In a project that’slong enough that students would get in trouble if they didn’t apply all these practices.”Projects themselves would also simulate what happens on the job. Students would berequired to gather requirements from customers, learn new programming languagesand technologies, and engage with “really difficult concepts like performance and multiprocessor/networked system.”

4.3.4. Other Real-World Experiences. Either within or outside of the team projects, stu-dents in participants’ ideal programs would be required to engage in other experiencesthat occur in the real world but are often not covered in depth in university programs.These include Software Engineering practices such as version control, commentingcode, modularity, and code reuse. One way to foster such skills is to involve studentsin an ongoing project, which requires them to debug, test, and add to a growing baseof code. As they would in a real-world setting, students would need to make changesto code that already exists.

One recommendation to achieve this would be to have each student work on a cohe-sive set of code all the way through.

The stuff you wrote as a freshman, although it wouldn’t be that great. . .you’d still be using when you’re a senior. You know the libraries and stuffyou wrote as a freshman you would still be using later on. . . One of theproblems coming out of undergrad, at least for myself, was. . . you neverhad to maintain anything. You write it for a project, the project takes a



week or a month to get done, and then you throw away that code and younever use it again. And in real life, you know, you’re using that code for yourentire career at a place, or even across multiple places if you bring your codebase with you.

An alternative suggestion would be to have different students work on the sameproject year after year. New students have to see “the kludges [previous students] did,with no documentation, and add on. This is what really happens, but you don’t haveanything like that in an education setting.”

Similarly, students could be required to work with buggy code and “pilot error” de-signed by the instructor. One interviewee suggested that students work over time onwhat he called “bad example” projects. “Tell them ‘this is your project today’ and justgive them a one-line description of a problem [like you would get from a real client].”A teaching assistant would simulate a typical customer interaction by doling out in-formation over time, some of it incorrect.

4.3.5. Other Classes. Participants frequently mentioned the lack of technical writ-ing classes in their own formal educational experiences, and recommended that oneor more English or technical writing courses be included in the “ideal program”. Alsomirroring comments made about the relatively large proportion of math courses withinthe course load in computer science and other computing-related programs, one rec-ommended against too much math, indicating that today “math requirements are toostrict. . . going all the way up to calculus.” However, another recommended four years’worth of math and what he called “math related courses”, “including basic electronicsand mechanics.”

After describing the value of learning about hardware in their own formal educa-tional experiences, several interviewees recommended one or more courses that re-quired hands-on experiences that would provide students an opportunity to interactwith real hardware.

4.3.6. Other Ideas. A number of other interesting ideas were proposed by individualparticipants. Two participants specifically mentioned that a good program should in-clude some type of co-op or real world work experience. The researchers were surprisedthat internship or co-op experiences were not mentioned more frequently, but note thatthis was not specifically asked about in the interview.

Other ideas proposed by participants include the following.

— Use visual aids.— Provide good physical resources (i.e., lab equipment).— Assign small real-world problems, which would help students learn concepts in a

realistic context without being too large to do as a homework assignment.— Have three groups of students and have them switch roles each week of the

semester. Roles would include developers, architects, and testers. This allows stu-dents to see how different each role is.

— Don’t just show examples of good code. Give both good and bad examples, and showhow to code and debug them.

— Don’t use typical case studies, which are generally too high level and therefore notvery useful. Instead, have students write their own case studies, including all theconsiderations that go into designing an entire system or making decisions aboutone component in a larger system.

— Make exams useful. They should be short answer and essay rather than multiple-choice. Students should be allowed to bring a sheet of notes.



— Allow students to gain experience with specific topics such as wireless technologies,networking.

— Provide independent specialized work in later years.

5. DISCUSSION

The findings of this study provide an insight into what in-practice professionals feelare the necessary skills and knowledge to work in this field. Our participants workedin a range of industries and have been involved in developing a range of types of soft-ware. Some have focused primarily on end-user experience, while others have spentmuch of their careers developing computationally heavy analysis software or designingfirmware, which requires detailed knowledge of specific types of hardware.

What was particularly interesting to us as researchers was the consistency of the re-sponses from these participants across industries, experience levels, and roles played.Many of the participants exhibited intense interest in the topics covered in the inter-views and it was clear that these in-practice professionals had given this topic a lot ofthought. We are pleased to be able to give these participants a means to share theirexperiences and their suggestions.

Non-formal learning was considered an essential and natural part of working inthe computing field by all participants. This was reflected in their recommendationsfor formal education programs, along with their critique of their personal formal ed-ucational experiences and what they believe is the current state of Computer Scienceeducation. Their recommendation to move quickly to large, team-based projects couldbe seen as an argument that they believe that self-learning and just-in-time instruc-tion while engaging in realistic, complex problems will provide the best preparationfor work in this field. Indeed, when discussing their own non-formal educational expe-riences several participants specifically noted that they remembered and understoodmaterial much better if it was learned in the context of a real project or problem theyhave been engaging in.

This is not to say that participants did not feel that theory is important. Rather,they appeared to see theory as a foundation to build on during hands-on experi-ences. Participants focused on the importance of building strong critical-thinking,problem-solving, and communication skills along with what several participants called“the basics”. They stressed the need for “traditional” theoretical courses such as“Algorithms & Data Structures” and “Object Oriented Design” and experiences withlow-level access to hardware. They also felt that students should learn to use oneor more programming languages early on in their program, but did not focus on thespecific language(s) to be learned. Rather, they saw experience with one or more lan-guages as a basis for working on realistic projects. Early language experience wouldalso provide students with an example which would help them teach themselves addi-tional programming languages in the future.

Only with this foundation could students meaningfully engage with realisticprojects. The real-world problems they would encounter while tackling these projectswould in turn require students to learn to use techniques and technologies that inter-viewees found were lacking in their own schooling but so important in the real world,such as technical writing, test techniques and strategies, and version control. Again,these recommendations highlight the importance of encountering the complexities ofthe types of situations that real professionals encounter on the job, complexities theybelieve cannot be well covered in traditional theoretical courses.

In the literature review, we discussed the ongoing debate about the appropriate fo-cus of computing education programs, and the impact of the nature of the program on



computing professionals. Lethbridge’s [2000] review of important topics for computingprofessionals listed 75 areas identified in the standards documents and computer sci-ence curricula reviewed at that time. That study clearly showed that not all topicswere mastered by the end of a university program, and it would seem difficult to imag-ine a program that could adequately cover all 75 topics. Similarly, Andriole [Andrioleand Roberts 2008] identified additional topics which may not be sufficiently addressedin the current standards. Clearly, as Walker [2010] stressed, university curricula mustfocus on some areas while giving less time and attention to others.

Some experts suggested that the reduction of the number of programming lan-guages (especially those that interact at a relatively low level with computer hardware)as well as trends such as the decrease in the required number of math courses wouldresult in the dumbing down of the U.S.’s computer science programs and would causestudents to be unprepared for many types of positions [Andriole and Roberts 2008;Dewar and Schonberg 2008]. They recommended the re-introduction of topics thatare missing or insufficiently covered in most curricula today. Alternative suggestionsincluded a focus on computer science principles and business strategies [Andriole andRoberts 2008] or a focus on interesting problems that would motivate the best studentsto return to this field [Dewar and Astrachan 2009].

The experienced computing professionals interviewed as part of this study were notall products of what some of the authors cited above described as recently dumbed-down Computer Science or Software Engineering programs. In fact, many of themexperienced math-intensive degrees in fields such as engineering and physics beforeComputer Science was a common area of study, while others experienced the type oftraditional Computer Science educational experiences that those authors claim arelacking in the current system. We are aware that the participants’ discussion of theirown formal educational experiences cannot be seen as either an endorsement or a cri-tique of the computing education programs available in 2012. Rather, their reflectionon their own experiences and their responses to them help illustrate what was trulyimportant to their own development as computing professionals.

The implications of participants’ recommendations cannot be met by making smallchanges to individual courses. They seem to imply changes at the curriculum level,and potentially even in the way that university degree programs are structured.Suggestions such as having students maintain code for several years would neces-sitate organization across semesters and between individual instructors. Allowingsufficient time for such significant hands-on experience would require trade-offs inthe curriculum, including reducing the amount of theoretical material covered—orat least, the amount covered within a traditional lecture or small-lab-project format.Participants seemed to be aware of this need, and recommended topics and coursesthat could be eliminated or subsumed into just-in-time instruction and self-learningexperiences.

These recommendations provide a warning to curricular designers considering go-ing too far in either reintroducing large bodies of theoretical knowledge, or introducingspecific languages, topics, or development environments solely based on how appealingthese may appear to potential new students.

In summary, findings from this study suggest that formal educational programstake into account the important role that non-formal educational experiences play inthe lives of even the most experienced computing professionals. If students can be pre-pared to make the most out of non-formal learning experiences, the burden of teachingstudents “everything” or even a complete set of “the right things” is lowered, as longas formal educational experiences foster critical thinking and problem solving skills,and provide the foundations students need to continue developing as they progressthrough their careers.



6. LIMITATIONS AND AREAS FOR FUTURE RESEARCH

Participants were obtained through a snowball technique. The number of participantswas sufficient for us to obtain clearly recurring themes. Therefore, we determined wehad reached saturation for the purpose of this initial study. However, the participantgroup is not a representative sample of all computing professionals. Many industriesare not covered, and those that are included are disproportionately covered. Similarly,geographic regions are not evenly covered. The sampling technique also does not guar-antee that the proportions of years of experience and roles played by participants arerepresentative of all computing professionals. A planned follow-up study will survey alarger group with a more representative sample frame.

Also, the sample is not necessarily representative of other demographics. Becauseit was not the focus of this study, demographics such as race, ethnicity, and genderwere not asked for. Only one female was interviewed, and her transcript was omittedfrom the study as it did not meet the criteria for inclusion. Other females initiallycontacted were not able to participate because of scheduling conflicts. The researcherscould not determine a gender bias in the invitation process, and are themselves bothfemales with backgrounds in computer science. It is possible that the use of the snow-ball technique had some impact, but the researchers did not note any evidence of thisfrom interview data or responses by females during scheduling-related discussions.

Other studies look at trends in education and retention of minorities and women,but a future study might include these factors when investigating preferences andstrategies of non-formal education or suggestions for ideal computing educationprograms.

With the possible exception of the telecommunications industry, there were notenough participants within any industry to determine whether there were any pat-terns in attitudes towards formal and non-formal educational experiences within in-dustries. A subsequent study [Exter 2011] took a deeper look at software design-ers working on educational software. The follow-up study being planned will sur-vey participants from a range of industries, allowing us to investigate whether thereare statistical trends in attitudes toward education or the topics that are consideredmost important by professionals working in different domains or on different types ofsoftware.

ACKNOWLEDGMENTS

Our thanks to Elizabeth Boling for all of her guidance and input.

REFERENCESANDRIOLE, S. J. AND ROBERTS, E. 2008. Technology curriculum for the early 21st century. Comm. ACM

51, 27–30.ARRDIS, M. A., CHENOWETH, S. V., AND YOUNG, F. H. 2008. The “soft” topics in Software Engineering

education. In Proceedings of the 38th ASEE/IEEE Frontiers in Education Conference (FIE’08).ATLEE, J. M. J., LEBLANC, R. J., LETHBRIDGE, T. C., SOBEL, A., AND THOMPSON, J. B. 2006. Reflections

on software engineering 2004, the ACM/IEEE-CS guidelines for undergraduate programs in softwareengineering. In Software Engineering Education in the Modern Age. Springer, Berlin, 11–27.

CALLAHAN, D. AND PEDIGO, B. 2002. Educating experienced IT professionals by addressing industry’sneeds. IEEE Softw. 19, 57–62.

COOPER, S. AND CUNNINGHAM, S. 2010. Teaching computer science in context. Inroads 1, 5–8.CROPLEY, A. J. 1989. Lifelong education: Interaction with adult education. In Lifelong Education for Adults:

An International Handbook, Pergamon, Oxford, 9–12.CROSS, N. 2004. Expertise in design: An overview. Design Stud. 25, 427–441.DALEY, B. J. 2000. Learning in professional practice. New Direct. Adult Contin. Learn. 86, 33–42.



DAWSON, R. 2000. Twenty dirty tricks to train software engineers. In Proceedings of the International Con-ference on Software Engineering (ICSE’00). 209–218.

DEWAR, R. AND ASTRACHAN, O. 2009. Point/counterpoint: CS education in the U.S.: Heading in the wrongdirection? Comm. ACM 52, 41–45.

DEWAR, R. AND SCHONBERG, E. 2008. Computer Science education: Where are the software engineers oftomorrow? CrossTalk 21, 28–30.

DRISCOLL, M. P. 2000. Motivation and Self-Regulation in Learning. In Psychology of Learnign for Instruc-tion, Allyn and Bacon, Needham Heights, MA.

EXTER, M. 2011. The educational experiences of software designers working in education/instructional tech-nology related fields. In Instructional Systems Technology, Indiana University, Bloomington, IN.

FISHER, J., ALVAREZ, J., AND TAYLOR, R. 1978. A survey of how practicing programmers keep up-to-date first results including their implications for computer science education. In Proceedings of the 9thSIGCSE Technical Symposium on Computer Science Education (SIGSCE’78). 67–72.

GHASSAN ALKADI, T. B. AND SCHROEDER, R. 2010. The sometimes harsh reality of real world comptuerscience projects. Inroads 1, 59–62.

GUDZIAL, M. 2010. Does contextualized computing education help? Inroads 1, 4–6.HAUER, A. AND DANIELS, M. 2008. A learning theory perspective on running open ended group projects

(OEGPs). In Proceedings of the 10th Australasian Computing Education Conference (ACE’08).HOULE, C. O. 1980. Continued Learning in the Professions. Jossey-Bass, San Fancisco.KNOX, A. B. 2006. The continuum of professional education and practice. New Direct. Adult Contin. Educ.

86, 13–22.KUMAR, D. 2010. Reflections: Languages, wars and false dichotomies. Inroads 1, 10–11.LACS. 2007. A 2007 model curriculum for a liberal arts degree in computer science. J. Educ. Res. Comput.

7, 1–34.LENGRAND, P. 1989. Lifelong education: Growth of the concept. In Lifelong Education for Adults: An Inter-

national Handbook, Pergamon, Oxford, 5–9.LETHBRIDGE, T. C. 2000. What knowledge is important to a software professional. Comp. 33, 44–50.LINCOLN, Y. S. AND GUBA, E. G. 1984. Processing the naturalistically obtained data. In Naturalistic In-

quiry, Sage, Beverly Hills, CA, 332–256.LIVINGSTONE, D. 2001. Adults’ informal learning: Definitions, findings, gaps and future research. In

T-Space at The University of Toronto Libraries: Centre for the Study of Education and Work.MOTT, V. W. 2000. The development of the professional expertise in the workplace. New Direct. Adult Contin.

Educ. 86, 23–31.O’DONNELL, K. 2006. National household education surveys program of 2005: Adult education participation

in 2004–2005. In National Center for Education Statistics.RADCLIFFE, D. J. AND COLLETTA, N. J. 1989. Nonformal education. In Lifelong Education for Adults: An

International Handbook, Pergamon, Oxford, 60–63.TOUGH, A. M. 1989. Self-directed learning: Concepts and practice. In Lifelong Education for Adults: An

International Handbook, Pergamon, Oxford, 256–261.TUCKER, A. 2002. On the balance between theory and practice. IEEE Softw. 19, 94–97.VAN VLIET, H. 2006. Reflections on Software Engineering Education. IEEE Softw. 23, 55–61.WALKER, H. 2010. Eight principles of an undergraduate program. Inroads 1, 18–20.

Received September 2011; revised April 2012; accepted April 2012


exploring experienced professionals’ reflections on computing education

Documents