StackSkim: Breadth-First Visual Search for ProgrammingProblem-Solving

Andrew HeadUniversity of California, Berkeley

Berkeley, CA 94704andrewhead@berkeley.edu

Codanda AppachuUniversity of California, Berkeley

Berkeley, CA 94704appachu@berkeley.edu

ABSTRACTWhen finding technical solutions, programmers select an-swers from search results with a variety of formats and ap-proaches. Modern search engines excel at providing rele-vant examples based on query terms and community rank-ings. However, programmers benefit from considering mul-tiple answers, discovering common solutions, and exploringexamples in formats that fit their learning needs. We presentStackSkim, a visual breadth-first search interface for rapidcomparative exploration of examples that enables consider-ation of common classes and programming concepts acrossexamples. Through a 5-subject preliminary study, we findthat users consider code-to-text ratio, dependency familiarity,and search history while finding solutions. StackSkim ad-dresses these challenges by providing visual representationsof code and text amounts, commonly used dependencies, anda snippets bank for comparing answers from StackOverflow.Following a 4-subject user study, we show that StackSkimenables users to detect common dependencies across manyexamples, albeit requiring an atypical search workflow.

Author KeywordsVisual search; Programming; Search user interfaces;

ACM Classification KeywordsH.5.m. Information Interfaces and Presentation (e.g. HCI):User Interfaces

INTRODUCTIONThe proliferation of programming resources on the web hasenabled the growth of an enormous programmer community.Relevant content for solving programming problems residesacross an abundance of tutorials, MOOCS, programming ex-amples, Q&A sites and technical blogs. These resources arepopular for programmers of varying skill levels and fill in theinadequacies of traditional documentation and formal refer-ences. Today’s coding workflow contrasts starkly from thatof a decade ago. Programmers retrieve web resources andengage in self-led learning during each iteration of code im-plementation and foraging [5] [4].

A search for a solution to a programming problem begins witha Google search, leading to blog posts, tutorials, referencedocumentation, forums, and online Q&As. Searchers assessthe relevance of online examples prior to extensive manualinspection of documents in multiple ways. First, they rely

on their search engine to rank results such that those at thetop of the page will provide a relevant solution. Second, theyexamine the textual excerpts returned with each result for ad-ditional context. Third, for online Q&A like StackOverflow,they assess which examples will be most helpful based onvotes and user reputation.

Whether a programmer can successfully understand and inte-grate code examples into his code depends on many factors.Code examples should build upon familiar libraries and ex-planatory text should use known terms or provide a definitionof them. If they contain errors, the programmer should beable to find related examples with the same classes and func-tions to check his work. The programmer may already knowwhether he is looking for long textual responses or short codesnippets. Modern search engines should offer a way for auser to quickly and effectively assess which examples con-tain a promising amount of explanation, usable libraries andterminology.

By conducting a preliminary user study with 5 programmers,we examine users’ obstacles as they search for solutions tounfamiliar programming problems. We identify 3 challengesusers face using general search interfaces. First, users searchfor results of particular format, like tutorials or code snip-pets. Second, they engage in backtracking after discoveringthat previously abandoned examples contain well-supportedlibraries that they were trying to avoid. Third, they maintaina bloated history through a trail of tabs in the browser of ex-amples they may want to revisit later.

We develop StackSkim, a search user interface that providessolutions to these problems through visual features. Our toolrepresents text and code of code examples in colored columnsto allow easy assessment of content distribution for eachsearch result. It generates charts of commonly used classesand coding concepts to help users with early detection of fre-quently used dependencies used for solving a problem. A pre-view panel provides a way to rapidly skim through the contentof multiple examples without requiring navigation throughtabs or links. A snippet bank enables users to maintain anin-tab history of promising code segments and explanationsand to compare two examples side-by-side. StackSkim lever-ages the StackExchange API to gain access to answers to over700,000 questions on StackOverflow, providing an overviewof dozens of relevant answers with each query.

This work concludes with a 4-subject user study to evalu-ate the usefulness of our tool. We determine that StackSkim

Figure 1: StackSkim user interface. Upper left. Search results for StackOverflow query. Each column represents a relatedanswer. Colored blocks represent plaintext sentences or lines of code in examples. Upper right. Preview window that highlightslines of answers as the user hovers the mouse over results. Bottom left. Control widgets sort answers by content type. Chartsdescribe common Java classes and concepts (functions, objects, etc.) in the results. Bottom right. Users store code snippets forlater examination and comparison by clicking and dragging over regions of the answer bars.

enables tasks like finding the most commonly used classesacross examples where a baseline search interface does not.Participants find it useful to be able to pinpoint structural codeelements like functions, viewing them across many examplesat once. They also see value using the system in the earlystages of learning when required to use libraries in unfamil-iar programming tasks. We finish with a discussion of futuredirections, focusing on the challenges of encouraging usersto use a breadth-first approach in when finding programmingsolutions.

RELATED WORK

Visual SearchThere have been several attempts to represent features in largetext corpora to aid search tasks. TileBars [9] uses com-pact iconic displays to represent relative document length,query term frequency, and query term distribution, enhanc-ing a user’s ability to quickly judge relevance of a text docu-ment. Relation Browser [21] facilitates faceted visualizationof large information collections. Insyder [17] combines sev-eral previously-studied visualizations in one interface to sup-port information seeking of general web content. StackSkimsimilarly represents features of a large text corpus, focusingon the domain of programming examples.

In the domain of code visualization, SeeSoft [8] uses a similarcolored column encoding of multiple source code texts ren-dering a compact display to reveal statistics about text. Tools

for supporting web code search like Mica [18] combine doc-umentation and source code but continue to represent resultsprimarily with text. Similarly, Assieme [10] presents API in-formation from several web resources by using a novel tech-nique for finding implicit references in code. Blueprint [4]helps users find and integrate code directly in the program-mers coding environment.

Search for Programming LearnersPast work has observed programmer search tasks and inter-faces to aid these tasks. Visualizations and interfaces havebeen produced for analyzing learning algorithms [11], pro-gram execution [7] [1], program comprehension [12] and pro-gram maintenance [3] [6]. Search behavior and interfaceshave also been studied in the domains of craft knowledge [19]and selecting tutorials [16] [14].

Ko et al. categorize six types of programming learning bar-riers - design, selection, coordination, use, understandingand information [13]. Our StackSkim interface primarily ad-dresses selection and use barriers in the terminology of Ko etal. Our answer bars and concept and class charts allow pro-grammers to quickly identify most commonly used classesand concepts and, as a result, what to use to solve a program-ming problem (selection barrier). Furthermore, the ability toannotate specific answer bars with classes and concepts helpssearchers view a variety of ways in which a programmingconstruct is used (use barrier).

Brandt et al. extend the work of Ko et al. by describing howprogrammers overcome these barriers [5] [13]. They iden-tify three types of intentions when foraging for informationon the Web: just-in-time learning of new skills, clarifyingand extending existing knowledge, and assisting with recallof specific information. They observe that programmers typ-ically open several Web browser tabs before evaluating qual-ity, rapidly skimming through their content, sometimes skip-ping prose to begin experimenting with the code sections.StackSkim’s color-coded answer bars provide a quick wayfor users to view and zone in on the code sections of manyanswers.

Research on the StackOverflow community has revealed thatusers benefit significantly from multiple answers from a di-verse group of experts [2]. Furthermore, the vertical positionof answers positively biases early answers toward best answerstatus [2]. One of the reasons cited for the success of Stack-Overflow is its ability to capture the competitive spirit of theprogramming community through gamification [15]. How-ever these built in incentives can make distinguishing expertsfrom very active users difficult [20]. By making the content ofmany StackOverflow answers accessible, StackSkim allevi-ates some of these issues. In addition, we believe StackSkimbetter facilitates the exploration of answers to some types ofquestions that require significant contextual knowledge or ob-scure technologies which have previously been identified asbeing poorly supported by StackOverflow [15].

PRELIMINARY STUDYTo discover programmers’ search habits when looking forcode examples in unfamiliar domains, we conducted a pre-liminary 5-subject study. We refer to the subjects as P1 - P5.Of the subjects (2 female), 3 studied engineering or informa-tion science (P2, P3, P5), 1 was a full-time industrial engi-neer (P4), and 1 had no technical background outside of asemester of introductory programming (P1). All had writtencode in the last month.

We conducted semi-structured interviews with all participants(P1 - P5) to understand how they solved programming prob-lems using online resources. For four participants (P2 - P5),we provided a task in a language they had used previouslybut claimed they were not comfortable with, and observedhow they looked for solutions online.

During interviews, we asked questions to understand howusers found answers in unfamiliar domains of programming:

1. What sources do you use for locating code examples?2. What are the features of a useful code example?3. How many examples do you typically have to look at be-

fore discovering one that is useful?4. What sources help you to learn the most while you are pro-

gramming?

For the coding task, we asked participants to use a languagethey had used in the last 6 months which they felt uncom-fortable using. We assigned them simple tasks in these lan-guages, but ones that they would be likely to forget how todo had they not performed the task recently. For example, weprovided P4 with a task to write a program in C that printed

time to the screen with a period of 1 second between eachprint. P2 and P3 were asked to perform string concatenationin C++ and write the result to the screen. P5 was asked toprint a random number between 1 and 1,000,000 in bash.

Findings about Search BehaviorWe observed several trends that informed the design of Stack-Skim as a search tool:

Programmers prefer different sources at different learningstages. The novice programmer (P1) claimed that whensearching for answers to programming problems in herclasses, she most frequently referenced class notes. Afterthis, she relied on general web search to search for exact an-swers to her problems. For her current online course in pro-gramming, she would look for online blogs of past studentswho had encountered and solved the same problem. Pro-grammers more used to solving open-ended problems withcode (P2, P3) stated that they frequently used answers theyfound on StackOverflow. One user (P5) claimed he regu-larly used textbooks when looking up tiny examples related tohis problems. However, he clarified that textbook exampleswere often insufficient to provide comprehensive answers tothe larger system-level problems he encountered at his IT in-ternship.

Users revisit abandoned search results when later searchesare not successful. Two programmers (P3, P5) revisitedsearch results they had previously abandoned while search-ing for examples during coding tasks. They abandoned earlyresults when they seemed complex or unfamiliar. However,if later results also lacked promise or used the same unknownlibraries, participants sometimes backtracked. A trail of tabskept history of the pages they had visited. Participants ac-cessed old examples by looking back through old tabs.

Finding a usable solution can require iterative search. Twousers (U4, U5) refined their queries during the coding taskto improve their search results. One user (U5) performed 10separate queries through Google to discover how to print arandom number using Bash. Many queries were variationson a theme, for example bash masking, bash command mask-ing, and bash command escape. Queries also targetted dis-tinct steps of the full problem, including generating a ran-dom number, saving the value, and retrieving a larger randomvalue than the default range.

We noticed several trends that inspire future additions toStackSkim.

Code examples often cannot be run as posted online. P3wrote C++ code using XCode, a development environmentfor Macintosh platforms. This environment did not requireexplicit std:: scoping for strings that were present in the ex-ample he found. He manually inspected compiler errors todiscover why the code would not run, and adapted the codefrom the online examples for his coding context using thisfeedback. P4 told us of a similar experience in recent historywhere he found the appropriate function for his task online,but could not determine the right configuration parameters ofthe function without a coworker’s help.

Users leverage existing code to perform tasks. One program-mer P3 found it easier to complete his coding task by openingexisting, running code and copying in source code from theanswers he found online. Another P5 told us that when atwork, he was most likely to check the company repositoryfor related code before searching online for solutions.

Users begin their search from Google. Each of the program-mers (P1 - P5) started in the search bar or in Google to findresources. Search tools that look to provide an overview overa general collection of code examples from various resourcesmight benefit from comparing themselves to the baseline ofGoogle.

These preliminary studies inspired the design of StackSkim.We set the following goals:

• Help users distinguish between code-heavy and text-heavydocuments.

• Demonstrate which libraries are core to solving a problem,and provide access to external documentation for thesemodules.

• Enable users to quickly locate sections of the code thatleverage key libraries.

• Store code snippets in compact location allowing for laterinspection and comparison.

• Integrate visual features into a UI that supports iterativesearch.

PARSING STACKOVERFLOW ANSWERSStackSkim allows users to perform queries on answers to over700,000 Java questions indexed on StackOveflow, an onlineprogramming Q&A site. On StackOverflow, users pose ques-tions to the community. Other members provide answers,which are voted to be helpful or unhelpful. One questioncan have multiple answers. For StackSkim, answers to ques-tions related to a user’s query make up the code examples theysearch through for solutions.

Using StackOverflow as a central corpus for code examplesoffered three advantages. First, developers can access thethe site’s full answer index the StackExchange API, whichallows 10,000 queries per day with authentication. Second,in our own experience we have found StackOverflow to pro-vide high quality answers over a range of topics. Third, eachexample is preformatted into distinct code and text sectionswhen authored in a StackOverflow markdown editor, makingseparation of text and code in examples simple. We restrictourselves to only Java-related answers on StackOverflow, theprogramming language with the most questions.

We parse three classes of features from examples:

Content blocks and types. StackOverflow answers have threetypes of content (Figure 2): explanatory text, source code,and in-code comments. Explanatory text is wrapped inHTML <p> and <li> elements. It can be divided into sen-tences by splitting this content on periods. Code is wrapped in<pre> elements, which can be divided into lines by the new-line character. In-code comments can be detected throughregular expression matching of double-slash and slash-starpatterns.

Figure 2: An answer to a Stack Overflow question. Thisexample contains lines of code, explanatory text, and inlinecode comments.

Class usage. After extracting content blocks from an answer,we detect classes. First, we search in the explanatory textfor single-word phrases contained within <code> tags. Ifthe term begins with a capital letter, we assume it to be aJava class. Second, we perform a regular expression max onthe code to detect all class names used in the declaration ofa variable, or coupled with a new keyword to instantiate anobject.

Concept usage. We detect whether examples use conceptslearned in the CS1 classroom like loops, functions and objectsby using regular expressions for Java keywords, symbols, andsequences of tokens.

We expect that more accurate detection of classes and con-cepts could be achieved by parsing code examples to their ab-stract syntax trees and inspecting the nodes of this tree. How-ever, we note that many code snippets on StackOverflow areincomplete and cannot be compiled. For this reason, we usedregular expression matching to capture class and concept usefor this version of StackSkim.

StackSkim: VISUAL SEARCH FOR PROGRAMMING SO-LUTIONSStackSkim helps users select suitable code examples by en-abling visual inspection of code-text ratio, inspection andlocalization of common classes, and comparison of similarexamples. We describe the design of visualizations to helpthese tasks. This includes content-colored columns, class fre-quency charts with brushing and linking to the source exam-ples, and a snippet bank. We build the UI for StackSkim toencourage rapid systematic skim over many examples and de-crease the time of link-based context switches.

Figure 3: A StackOverflow answer and its representation as acolumn in StackSkim search results. Plaintext sentences be-come blue blocks, code lines orange blocks, and inline com-ments green blocks. By inspecting the resulting column, onecan see that the example begins with short plaintext, demon-strates the idea with code, provides additional text, and endswith another segment of code.

Figure 4: A vertical bar represents an answer from Stack-Overflow. In this image, you can see 13 explanatory answersmade completely of plaintext and several examples with over20 lines of code.

Code Example ColumnsWe expect some users to look for text-heavy explanatory doc-uments like tutorials, and others to look for short segmentsof code to insert into their programs. We encode differenttypes of content like source code, plaintext, and code com-ments, as different colored blocks. One block corresponds toone sentence of plaintext or one line of code. These blocksare stacked from top to bottom to maintain the metaphor of avertical programming example text from StackOverflow. Ananswer converted to the visual space of StackSkim is shownin Figure 3.

Bars are packed horizontally for users to rapidly assess whichprogramming examples contain the most content overall (seeFigure 4). The vertical window size fits around 80% of theanswers we viewed during our own use of the tool. This en-ables users to view the full column of most displayed results.

To help users quickly locate search results that match theircontent preference, we provide a simple sorting interface. Us-ing a dropdown menu (Figure 5c), users can choose whetherto sort search results and their representative columns by total

(a) Sorting by total content length. (b) Sorting by plaintext sentences.

(c) Sorting widget.

Figure 5: Sorting code examples by content type (in this case,total content and plaintext sentence count) in StackSkim.

example length (Figure 5a), lines of code, sentences of plain-text (Figure 5b), votes or author reputation.

Source Text Preview PaneWe enable users to inspect the source text of examples fortwo reasons. First, they must determine if the solution the barpresents is relevant to their query. Second, if the example isrelevant, users will want to copy text from the example intotheir own programs.

StackSkim enables users to skim the source of dozens of ex-amples across multiple questions with small movements ofthe mouse. By positioning the cursor over a block of an ex-ample, the preview pane highlights that block of code amidst10 - 20 blocks of surrounding context (Figure 6). We assumedusers would be predominantly scrolling down through exam-ples. Therefore, we saved 30% of the preview pane’s verticalsspace to show blocks above, and 70% for blocks below thefocused block. The window animates vertical scrolling be-tween different blocks in the same example. Users can shifttheir focus to the source text of another example by movingthe mouse horizontally to another example column.

We used a preview window instead of tooltip-based inspec-tion due to space constraints. A horizontal space of around60 characters shows source code content without frequent line

Figure 6: The preview pane shows the source text of an exam-ple a user is hovering over. The pane highlights the currentlyhovered block in pale blue. The focused block is surroundedby text from the blocks immediately preceding and followingit to provide the user with context.

wrap or horizontal scrolling. A vertical space of 10 - 15 linesshows source code before and after the highlighted region orseveral sentences of explanatory text that come directly be-fore or after. A tooltip showing 10 lines of text 60 characterswould be massive or have a small font. To avoid occludingother bars of text, we opted to decouple the preview pane fromthe mouse position, and place it in a separate region at the topof the screen.

Class and Concept InspectionWe show users commonly used libraries in code examplesto direct them to corresponding documentation prior to usingthe examples. We provided details about classes and conceptswhich were most used across all examples with easy accessto the regions of code that used those classes and concepts.

First, we present a markup pane with bar charts that providea quick visual description of what material would be help-ful for a user to examine before examining the search results(Figure 7). Bar charts show the frequency of Java classes andconcepts (objects, functions, loops, etc.) ordered from great-est number of occurrences to least.

Second, we enable users to gain additional backgroundknowledge on these libraries quickly. Labels for classes linkto the Java API for access to more comprehensive documen-tation.

Third, users can mark regions of examples that use of a class,to find alternate approaches to leveraging the same classesand typical modes of use. By hovering over a library depen-dency in the summary pane, all code examples that use thatdependency are highlighted in the search results. By click-ing on one of those library dependencies in the markup pane,users flag all blocks of all examples that instantiate that classor use that concept. Users can then visually filter for or ex-clude examples that contain that dependency.

Colors of class and concept markers have greater saturationthan the colors used to mark the block content to make themstand out against the blocks they mark. Classes and conceptmarkers are colored using the Tableau 10 colors and assignedbased on frequency of class or concept occurence.

Figure 7: The Markup pane presents which dependencies andcode techniques are used most often. By clicking on bars,users highlight all blocks that use that class or concept. La-bels for the class bar chart link to Java API documentation.

(a) User clicks and drags over blocks of text.

(b) Source text from the selected regions are added to the snippetspane for saving multiple relevant results for later inspection.

Figure 8: By clicking and dragging over a search result col-umn, users create example snippets and store them in thesnippet bank.

Saving and Comparing Code ExamplesIn our preliminary study, programmers revisited old code andkept many tabs open. We introduce a snippets bank for pre-serving past potentially relevant code. This concentrates allrelevant code in one location for skimming through past re-sults.

Users click and drag over blocks of code in a single examplethat they want to keep (Figure 8a). After releasing the mouse,a new snippet is pushed to the top of the Snippets stack (Fig-ure 8b). This ensures that the most recent results are instantlyviewable. Users can edit titles of snippets to later recall whythey saved the snippet. Individual snippets and be collapsedand expanded to enable a user to open several examples fordirect comparison.

In the future, we will provide a way to sort, edit, combine andmarkup snippets. We will also link snippets visually to thesearch columns they were extracted from.

Iterative Search UIThe utility of StackSkim requires that users have multiple ex-amples to skim through. During search tasks, this is not al-ways the case. Some StackOverflow questions have only oneanswer. We also observed users end their search after consid-ering only one relevant question and answer.

Figure 9: Users select questions related to their query in thequestion selection popup after each query. By selecting re-lated questions, users populate the search results with relevantanswers.

To make StackSkim pertinent for solving programming prob-lems with multiple examples, we developed search UI ele-ments to help users discover many relevant answers prior toexamining search results. After a user performs a query inthe search box at the top of the page, we fetch the 20 relevantquestions from StackOverflow. We list the title of each ques-tion in a list shown to the user (Figure 9). The user selectsquestions relevant to their query. For example, for a query“play audio”, a user might select questions titled Java audioplaying solution?, Play audio file in java, Playing audio inJava, and Reading and playing audio file. StackSkim popu-lates the search results with every answer to these questions.

During later queries, StackSkim maintains all snippets thatusers have selected from past queries. In this way, users cancompare examples between initial and later, refined queries.They can also store snippets from multiple steps in a sequencethat will eventually form a longer program.

At this point, users cannot view the questions that answers ad-dress. Future work could explore how to represent questionsin our interface as multiple answers share the same question.In addition, question selection could be augmented by show-ing excerpts from the question body and counts of answersto help users both assess question relevance and breadth ofanswers covered by their selection.

EVALUATIONWe conducted an informal 4-subject user study to assess theusefulness of StackSkim for finding programming solutions.

We recruited 4 students (3 male, 1 female) from a local uni-versity. All users had experience programming in Java. Werefer to users with names U1 - U4.

Users completed the tasks on a 15” MacBook Pro with RetinaDisplay, 2 GHz Intel Core i7 processor and 16 GB 1600 MHzDDR3 RAM, laptop trackpad and builtin keyboard. Usersperformed coding tasks in the Koding IDE 1, a web-hostedvirtual machine for developing code in Java and other lan-guages.

TasksBecause of varying availability, some users only performed asubset of tasks. All users (U1 - U4) followed a tutorial thatintroduced the visual representation of bars, preview panel,sorting examples by text length, marking examples based onclasses used and Java concepts, saving snippets, and query-ing for more examples. Most users (U2, U3, U4) performedwarm-up exercises to demonstrate their knowledge of inter-face features before performing coding tasks. Warm-ups in-cluded locating all examples that contained a common class,sorting examples by the amount of text and choosing exam-ples with a fixed number of text sentences, and saving snip-pets with descriptive names.

Example Discovery TasksUsers U1, U2, and U4 performed example discovery tasks.We preloaded StackSkim with a set of 26 answers to 6 ques-tions related to the query “record audio”. We provided userswith 3 tasks:

1. Locate 5 answers with 2 lines of code or less, and paste thefull text of their body into a document.

2. Write the name of the class that is instantiated the mostoften in the answers.

3. Locate 5 code snippets that explicitly declare a new func-tion, and paste their content into a document.

Users completed tasks with both StackSkim and a baselineinterface. For the baseline, users were given a static Stack-Overflow search page with search results for each of the 6relevant questions. Answers could be accessed by clicking onquestion links. While users in our preliminary study most fre-quently used Google web search to find examples, we wantedto provide an alternate list-based search interface that userswere likely to use that represented the same data as Stack-Skim.

Users were asked to ‘talk aloud’ to share their reactions tothe interface and search process. We timed each task forboth interfaces (Table 1). Due to time constraints, users werestopped after 3 minutes of each task with one interface. Or-der of interfaces for each task was counterbalanced betweenparticipants. Users were allowed to refresh the page betweentasks.

Open-Ended Programming TasksUsers were asked to perform two programming tasks in Java,one using StackSkim (U1, U2, U4) and one with StackOver-flow (U1, U2):1https://koding.com/IDE/

1. Read text from a file and print it to screen.2. Replace all instances of the word ‘function’ in this text with

a single random character.

These tasks were non-trivial enough to require users to lookup assistance and simple enough that they could be performedin around 10 minutes. We counterbalanced the order of searchinterface used for both conditions across users U1 and U2.

ResultsTable 1 lists the times users took to complete code discoveryand programming tasks. Significant qualitative findings fromthe study include the following:

StackSkim helps users find frequently-used classes. In thetimings in Table 1, only Task 2 shows a clear time advan-tage for using StackSkim in example discovery tasks. Userswere not sure how to approach Task 2 with StackOverflow’suser interface. One user (U2) attempted to use tally marksto count instantiations for each class. Another (U4) lookedat the answers by eye and tried to keep a mental idea of howoften each class occurred. Class charts in the markup panemade this information quickly accessible. Users performedthis task in less than 30 seconds with StackSkim.

Users saw our visual features as useful for future tasks. U3appreciated that the coloring of the code bars allowed him toswiftly shift attention to code sections across multiple exam-ples. U1 told us that he could see himself using the class barsto point out relevant classes when approaching an unfamiliarprogramming domain.

For some users, long fixations were more likely with Stack-Overflow. We noticed that U2 fixated on answers he pulledup on StackOverflow for tens of seconds at a time. We didnot notice the same behavior when he was skimming exam-ples with StackSkim.

Users preferred copying source from the preview pane insteadof the snippet bank. We intended for programmers to selectimportant example text into the snippets bank and later con-struct code from these snippets. We found that all users (U1,U2, U3, U4) attempted to copy text from the preview win-dow. Most users (U1, U3, U4) used the preview window asthe main source of copying text. U3 explained that copyingfrom the preview pane is more like copying text from a pageyou scroll through. The snippets bank separated selectablecode from the pane where it was discovered.

StackSkim is not as strong as the baseline for answering ‘re-minder’ queries. Some users (U1, U2) told us that they knewa specific class would perform a task for them. Their queriessometimes contained this class name as a result. These usersqueried StackSkim using the class name and selected the onemost relevant question. We believe that this behavior is rem-iniscent of how users often pick the first relevant result thatappears in a Google web search.

Copying code from the preview pane was difficult. By updat-ing the preview window to reflect the code bar under the cur-sor, we made it difficult for users to copy text from a code col-umn between two others. As users navigated to the preview

pane, it updated to focus on the text of the bars they passedthrough. One user (U4) recommeneded that we enable usersto click on answer regions to lock the source preview contentto that region. Several users (U2, U3, U4) commented on thedifficulty of selecting text from the preview menu.

DISCUSSIONStackOverflow and Google are already usable, functional in-terfaces for finding programming solutions. however, Stack-Skim’s unique strength arises when there are multiple ways tosolve a problem and many online resources describing how todo it. A breadth-first consideration could be helpful for a userto consider all viable alternatives before delving deeply intoone example. Viewing multiple related examples togethercould provide a more complete view of class use cases, setup,use and teardown. Our current study does not fully explorethese advantages. In the future, we plan to perform a larger-scale user study with programming tasks that require discov-ering new unfamiliar programming material.

We were surprised to see searchers use StackSkim likeGoogle search. Users entered a query, selected the singlemost relevant question from the questions list, inspecting sev-eral examples, and moved on to refine their query or use codefrom the current examples. StackSkim can return multiplerelevant questions related to a user’s query for which an-swers can be viewed side by side. For most users, however,we found that simultaneously examining answers to multiplequestions would require a shift in current search habits. Fu-ture versions of our tool can explore motivation techniquesto help users select multiple answers and view a breadth ofanswers.

Based on our preliminary study, we could save programmerstime by incorporating additional visual features into Stack-Skim. One user would have benefited from showing whichcode examples would compile in their IDE. Additional fea-tures including votes, estimated runtime, and configurableparameters, could be integrated into the column representa-tion to provide a more complete view of example usability.Users may find it helpful to be able to highlight other exam-ples similar to the lines they are currently previewing. We arealso interested in exploring how to incorporate question textinto answer bars and how to cluster the answers based on thequestion they answer.

Ultimately, we look to apply the StackSkim approach to re-sources beyond StackOverflow answers. By examining doc-ument elements of blog postings and tutorials, we could buildsimilar visual search results for the Google search index. Col-umn coloring could be modified to visually summarize struc-tured learning materials like technical textbooks. We plan tostudy how visualizations we produce could help learners inother domains discover more useful textual resources.

CONCLUSIONFor programmers searching for technical solutions, modernsearch engines fail to support easy assessment of code-to-textratio, common dependencies, and comparison of code exam-ples. In response, we present StackSkim, a visual search

Subject T1SS T1SO T2SS T2SO T3SS T3SO A1 A1 interface A2 A2 interface1 1:47 0:37 0:05 3:00 2:45 1:15 4:37 Skim 10:00 Overflow2 2:10 1:57 0:30 3:00 2:43 1:52 10:00 Overflow 10:00 Skim4 1:31 2:39 0:10 3:00 2:29 1:43 7:00* Skim N/A N/A

Table 1: Times recorded for performing code discovery and programming tasks. Each task was performed once with StackSkim(SS) and once with static StackOverflow search results (SO). Users were clearly able to discover the most common classes usedin examples much more quicky using StackSkim (T2). However, we believe that unfamiliarity of search paradigm and interfaceresulted in no improvement over the baseline in the other tasks. U3 was not able to particpate in these tasks.* U4 was given a time limit of 7:00 for the final task instead of the 10:00 prescribed for the other users.

tool to help programmers discover useful examples accord-ing to these tasks. Our tool leverages three visual represen-tations to help users find understandable programming solu-tions from search results. First, we represent code examplecontents as color-coded stacked blocks of code and text. Sec-ond, we rank classes and concepts by frequency and enablevisual annotation of locations in the source text where theseelements occur. Third, we provide a snippets bank for usersto store and compare a history of useful segments of code andtext. Through a 4-subject user study, we discovered that couldmore rapidly detect frequently used classes than with a base-line interface and that users believed the tool would be usefulto explore code examples in unfamiliar topics. In the future,we will explore how we can better encourage the breadth-firstsearch procedures enabled by StackSkim for users.

REFERENCES1. Aftandilian, E. E., Kelley, S., Gramazio, C., Ricci, N.,

Su, S. L., and Guyer, S. Z. Heapviz: Interactive heapvisualization for program understanding and debugging.In Proceedings of the 5th International Symposium onSoftware Visualization, SOFTVIS ’10, ACM (New York,NY, USA, 2010), 53–62.

2. Anderson, A., Huttenlocher, D., Kleinberg, J., andLeskovec, J. Discovering value from community activityon focused question answering sites: A case study ofstack overflow. In Proceedings of the 18th ACMSIGKDD International Conference on KnowledgeDiscovery and Data Mining, KDD ’12, ACM (NewYork, NY, USA, 2012), 850–858.

3. Bragdon, A., Reiss, S. P., Zeleznik, R., Karumuri, S.,Cheung, W., Kaplan, J., Coleman, C., Adeputra, F., andLaViola, Jr., J. J. Code bubbles: Rethinking the userinterface paradigm of integrated developmentenvironments. In Proceedings of the 32Nd ACM/IEEEInternational Conference on Software Engineering -Volume 1, ICSE ’10, ACM (New York, NY, USA,2010), 455–464.

4. Brandt, J., Dontcheva, M., Weskamp, M., and Klemmer,S. R. Example-centric programming: Integrating websearch into the development environment. InProceedings of the SIGCHI Conference on HumanFactors in Computing Systems, CHI ’10, ACM (NewYork, NY, USA, 2010), 513–522.

5. Brandt, J., Guo, P. J., Lewenstein, J., Dontcheva, M., andKlemmer, S. R. Two studies of opportunistic

programming: Interleaving web foraging, learning, andwriting code. In Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems, CHI ’09,ACM (New York, NY, USA, 2009), 1589–1598.

6. Coblenz, M. J., Ko, A. J., and Myers, B. A. JASPER: Aneclipse plug-in to facilitate software maintenance tasks.In Proceedings of the 2006 OOPSLA Workshop onEclipse Technology eXchange, eclipse ’06, ACM (NewYork, NY, USA, 2006), 65–69.

7. De Pauw, W., and Heisig, S. Zinsight: A visual andanalytic environment for exploring large event traces. InProceedings of the 5th International Symposium onSoftware Visualization, SOFTVIS ’10, ACM (New York,NY, USA, 2010), 143–152.

8. Eick, S. G. Graphically displaying text. Journal ofComputational and Graphical Statistics 3, 2 (June1994), 127–142.

9. Hearst, M. A. TileBars: Visualization of termdistribution information in full text information access.In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems, CHI ’95, ACMPress/Addison-Wesley Publishing Co. (New York, NY,USA, 1995), 59–66.

10. Hoffmann, R., Fogarty, J., and Weld, D. S. Assieme:Finding and leveraging implicit references in a websearch interface for programmers. In Proceedings of the20th Annual ACM Symposium on User InterfaceSoftware and Technology, UIST ’07, ACM (New York,NY, USA, 2007), 13–22.

11. HUNDHAUSEN, C. D., DOUGLAS, S. A., andSTASKO, J. T. A meta-study of algorithm visualizationeffectiveness. Journal of Visual Languages &Computing 13, 3 (June 2002), 259–290.

12. Islam, S. S., Krinke, J., and Binkley, D. Dependencecluster visualization. In Proceedings of the 5thInternational Symposium on Software Visualization,SOFTVIS ’10, ACM (New York, NY, USA, 2010),93–102.

13. Ko, A., Myers, B., and Aung, H. Six learning barriers inend-user programming systems. In 2004 IEEESymposium on Visual Languages and Human CentricComputing (Sept. 2004), 199–206.

14. Kong, N., Grossman, T., Hartmann, B., Agrawala, M.,and Fitzmaurice, G. Delta: A tool for representing andcomparing workflows. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems,CHI ’12, ACM (New York, NY, USA, 2012),1027–1036.

15. Mamykina, L., Manoim, B., Mittal, M., Hripcsak, G.,and Hartmann, B. Design lessons from the fastestq&#38;a site in the west. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems,CHI ’11, ACM (New York, NY, USA, 2011),2857–2866.

16. Pavel, A., Berthouzoz, F., Hartmann, B., and Agrawala,M. Browsing and analyzing the command-level structureof large collections of image manipulation tutorials.Tech. Rep. UCB/EECS-2013-167, EECS Department,University of California, Berkeley, Oct 2013.

17. Reiterer, H., Muler, G., Mann, T. M., and Handschuh, S.INSYDER &mdash; an information assistant forbusiness intelligence. In Proceedings of the 23rd AnnualInternational ACM SIGIR Conference on Research andDevelopment in Information Retrieval, SIGIR ’00, ACM(New York, NY, USA, 2000), 112–119.

18. Stylos, J., and Myers, B. Mica: A web-search tool forfinding API components and examples. In IEEESymposium on Visual Languages and Human-CentricComputing, 2006. VL/HCC 2006 (Sept. 2006), 195–202.

19. Torrey, C., Churchill, E. F., and McDonald, D. W.Learning how: the search for craft knowledge on theinternet. In Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems, ACM (2009),1371–1380.

20. Yang, J., Tao, K., Bozzon, A., and Houben, G.-J.Sparrows and owls: Characterisation of expert behaviourin StackOverflow. In User Modeling, Adaptation, andPersonalization, V. Dimitrova, T. Kuflik, D. Chin,F. Ricci, P. Dolog, and G.-J. Houben, Eds., no. 8538 inLecture Notes in Computer Science. SpringerInternational Publishing, Jan. 2014, 266–277.

21. Zhang, J., and Marchionini, G. Evaluation and evolutionof a browse and search interface: Relation browser++. InProceedings of the 2005 National Conference on DigitalGovernment Research, dg.o ’05, Digital GovernmentSociety of North America (Atlanta, Georgia, 2005),179–188.