Creating Visualizing and Exploring Knowledge Maps

Colorado ReedUC Berkeley

coloradoberkeleyedu

Figure 1 The knowledge map visualization and exploration system described in this paper reduces the number of visual edges for large knowledgemaps In addition it provides a number of interactive components to help users explore and understand these maps The left figure shows a knowledgemap (layered Sugiyama-style directed graph) of machine learning concepts The large number of crisscrossing edges makes the graph difficult tovisually parse The top right figure shows the same graph produced by our system Notice that the node locations are the same but a large number ofvisible edges have been replaced with ldquowisp edgesrdquo small dashed protruding lines from the source and target nodes that become full edges when theuser hovers on the wisp or the sourcetarget node The bottom right figure shows the focused mode that is displayed after clicking on a concept Thefocused mode shows the inlinks and outlinks of the targeted node and the nodes associated with these edges

ABSTRACTKnowledge maps are cognitive aids designed to help users ex-plore a body of knowledge in a nonlinear way Specificallya knowledge map is a graphical representation of knowledgechunks eg single-digit addition linked together by directededges that express prerequisite relationships In this paperwe describe a novel knowledge map creation visualizationand exploration system We employ edge reduction and in-teractive techniques to aid users in exploring large knowledgemaps Furthermore we describe a visual knowledge map ed-itor that allows users to create new knowledge maps asso-ciate related content with the map (eg textbook referencesof video lectures) and make local changes to large preexist-ing knowledge maps

INTRODUCTIONA knowledge map is a graphical representation of conceptsthat reveals the relationships and dependencies among the

concepts In this work a knowledge map is a directed graphin which the nodes are concepts and the edges specify prereq-uisite relationships between the concepts see Figure 1 Thisgraphical representation is a variant of the concept maps com-mon in educational psychology whereby the nodes are con-cepts and the edges are linking words that create a meaning-ful statement or proposition when combined with the nodesStated another way a knowledge map primarily serves as areference tool while a concept map serves as a learning toolThat is a knowledge map typically has associated contentwith each concept eg textbook references or video lecturethat a student would use to learn the material while a con-cept map summarizes a body of knowledge and is oftentimescreated by the student to help learn the material

In recent years several online education companies andschools have created knowledge maps to help students vi-sualize and navigate their content in a nonlinear way see

eg Khan Academy [3] Duo Lingo [2] and Metacademy[4]Khan Academy a nonprofit online educational company withover six million unique monthly visitors has a large (700+node) knowledge map that founder Salman Khan discussedon p 52 of [13]

[The knowledge map] became a core piece of the KhanAcademy software platform In stressing the connec-tions among subjects and giving learners a visual pictureof where theyrsquove been and where theyrsquore going we hopeto encourage students to follow their own pathmdashto moveactively up down and sideways wherever their imagi-nations lead

A problem with knowledge maps however is that the largenumber of relationships between the concepts produces alarge number of intertwined and crisscrossing edges in thegraph In turn it is difficult to interpret the conceptual re-lationships see Figure 2 for examples from Khan Academyand Metacademy These ldquohairballrdquo graphs are common innetwork visualizations and indeed there is a deep field ofresearch focused on visualizing networks eg see [7] Thehierarchical placements of the nodes and edges in knowledgemaps express the dependency structure within the body ofknowledge and for this reason many network visualizationtechniques will not work with knowledge maps as they ob-scure or break these relationships

In this work we create a novel interactive system for creat-ing visualizing and exploring knowledge maps On the cre-ation side we constructed a graphical user interface (GUI)for visually specifying concepts their relationships and as-sociated content On the visualization and exploration sideour system uses a layered (Sugiyama) directed graph place-ment algorithm [17] to initially place the nodes a topologi-cal sort to determine an initial set of visible edges (the topo-logical sort maintains the connectivity of the graph) an edgelength threshold to determine which remaining edges shouldbe shown and edge ldquowispsrdquo to indicate the absence of anedge These techniques greatly reduce the number of visibleedges in large knowledge graphs while maintaining the hier-archical and connectivity structure of the content

To further accommodate the goals of knowledge map userswe incorporated several interactive components that aid theuser in locating concepts of interest and understanding theirrelationships to other concepts In particular our system al-lows users to step through the dependency structure of a graphby sequentially visualizing contextual views of the concepts(the direct dependencies and outlinks) Furthermore we in-clude a number of hover and click operations that reveal thereason for the relationships and associated content present fora given concept

RELATED WORKIn this section we discuss three areas of previous research anddevelopment relevant to this paper online knowledge graphsystems visualizations of large directed graphs and tools forcreating knowledge maps

Online Knowledge GraphsThe knowledge map creation visualization and explorationsystem described in this paper was designed for Meta-cademymdashan open source platform for collaboratively cre-ating refining and sharing knowledge maps and associatedlearning material [4] Figure 2 (right) shows a Metacademyknowledge graph one of the two main components of Meta-cademy The other major component is a text-based view thatprovides the user with the following information for each con-cept a description of the concept goals for the learner theprerequisites of the concept learning resources for the con-cept and related (non-dependency) concepts Metacademy istypically used as a targeted learning resource whereby a usersearches for a particular concept and is then presented witha knowledge map with that concept as the sole leafmdashmostgraphs have 5-50 nodes and less than 100 edges

As mentioned in the introduction Khan Academy [3]mdasha freevideo lecture and exercise repositorymdashalso provides a knowl-edge map of their material Rather than showing target-basedcomponents of the graph Khan Academy shows users theirentire 700+ node knowledge map This knowledge map ismuch larger than any of Metacademyrsquos knowledge maps butthe techniques discussed in this work are applicable to theirgraph and we will format their content for our system in fu-ture work

Visualization and Exploration of Directed GraphsOur knowledge map visualization and exploration systemdraws on a number contributions from the graph visualizationcommunity In particular the authors of [18] developed an in-teractive visualization system for large networks eg threemillion nodes The authors focused on providing context-specific visualizations of large directed graphs and stated that

Although providing a structural overview of the graphis a laudable goal there are many cases where the useris simply not interested in a global view of the wholegraph but wants to solve a particular concrete task onthe graph instead

Rather than show all direct inlinks and outlinks as is donein several related systems [5 6] the authors devised a de-gree of interest (DOI) term for each node that they use todetermine which subset of contextual nodes to show the userSince individual nodes in knowledge maps have a relativelysmall degree it is typically possible to visualize the fullcontext of each node rather than a subset and we do notuse the DOI technique in this work The authors use wispedgesmdashtruncated inlinksoutlinks to a nodemdashto indicate thepresence of an edge without cluttering the visualization withlong crisscrossing edges We incorporate this technique intoour visualization (see the next section)

Figure 2 The left figure shows a section of the Khan Academy knowledge map the right figure shows a Metacademy knowledge map

There are a number of software packages and algorithms forvisualizing (large) directed graphs that inspired our system

bull the dot placement algorithm described in [12] is theunderlying placement algorithm for the popular graphvizC++ graph visualization software The dot algorithm isused in the dagre JavaScript graph placement library whichis used in this work (see the next item)

bull dagre [1] a JavaScript library for directed graph placementis used to generate node and edge placements in our knowl-edge map system

bull Dynasty [11] a system for browsing large directed graphswhere only a small portion of the graph is visible at anygiven time Dynasty inspired the ldquofocused moderdquo aspectof our system where a concept and its dependencies andoutlinks are visible at a given time

bull the transition animations in Open Graph Drawing frame-work [10] a large C++ library of algorithms for graphdrawing inspired the transition animations sequence andtiming used in this work

Furthermore Misue et alrsquos mental map preservation proper-ties for directed graphs [15] motivated our use of visualizationmodes rather than dynamic directed graphs

Finally many directed graph visualization systems transformthe data for visualization For instance hive plots [14] pro-duce a linear layout for grouping nodes by type and then ar-ranges the nodes along a radial axis Arranging knowledgemap concepts along a radial axis does not work well for a vi-sualization as the concepts commonly have relationships thatspan multiple levels and as a result the radial network be-comes polluted with edges that span most of the circumfer-ence Strictly hiding these types of dependencies masks theprerequisite complexity for various concepts and could de-ceive the user

Knowledge Map CreationAs of the time of this writing the author currently does notknow of any tools that are explicitly used for creating knowl-edge maps as defined in this paper Both Khan Academyand Metacademy use text-based systems eg html formsto specify the relationships in the graph However there is alarge number of tools for creating similar structures such as

diagrams mind maps and concept maps We briefly discussthese tools and their relevance to our system

Cmap tools [9] is a Java-based GUI for creating conceptmaps directed graphs where the nodes are concepts and theedges are linking words so that tracing through a path is akinto reading a sentence The Cmap tools GUI inspired our edi-tor as it has simple drag and click operations to construct thegraph The Cmap tool editor however does not have asso-ciated content with the graph and is not browser-based Fur-thermore the Cmap tools editor requires the user to manuallyplace the nodes and edges Our editor operates in a similarfashion but users have the ability to algorithmically arrangethe nodes and edges using the dagre javascript library Fi-nally our editor is browser-based while Cmap tools is a Javaapplet

There are countless diagramming tools such as gliffy1drawio2 and dia 3 Many of these of tools are polished in-tuitive and downright beautiful diagramming tools but theyare more general purpose than what is needed for knowledgemap creation Letting users choose the color and shape of thenodes the thickness of the edges etc detracts from the mainpurpose of specifying the concept relationships and providingassociated content

METHODOLOGYIn this section we describe the visualization and interactionmethods employed by our knowledge map system In par-ticular our system builds upon the Metacademy knowledgemap system and we have recently acquired the data for KhanAcademyrsquos knowledge map and plan to extend our system forthis data in the near future

Knowledge Map Creation and ExplorationFrom an informal usability study conducted with seven UCBerkeley students using the think-aloud protocol[16] and theMetacademy knowledge map system we identified the fol-lowing common objectives the students had when interactingwith the knowledge map4

1httpwwwgliffycomindex-gphp2httpwwwdrawio3httpsourceforgenetprojectsdia-installer4Specifically we observed the students interacting with relativelylarge (30-50 nodes) and small (circa 10 nodes) knowledge maps

eg Khan Academy [3] Duo Lingo [2] and Metacademy[4]Khan Academy a nonprofit online educational company withover six million unique monthly visitors has a large (700+node) knowledge map that founder Salman Khan discussedon p 52 of [13]

[The knowledge map] became a core piece of the KhanAcademy software platform In stressing the connec-tions among subjects and giving learners a visual pictureof where theyrsquove been and where theyrsquore going we hopeto encourage students to follow their own pathmdashto moveactively up down and sideways wherever their imagi-nations lead

A problem with knowledge maps however is that the largenumber of relationships between the concepts produces alarge number of intertwined and crisscrossing edges in thegraph In turn it is difficult to interpret the conceptual re-lationships see Figure 2 for examples from Khan Academyand Metacademy These ldquohairballrdquo graphs are common innetwork visualizations and indeed there is a deep field ofresearch focused on visualizing networks eg see [7] Thehierarchical placements of the nodes and edges in knowledgemaps express the dependency structure within the body ofknowledge and for this reason many network visualizationtechniques will not work with knowledge maps as they ob-scure or break these relationships

In this work we create a novel interactive system for creat-ing visualizing and exploring knowledge maps On the cre-ation side we constructed a graphical user interface (GUI)for visually specifying concepts their relationships and as-sociated content On the visualization and exploration sideour system uses a layered (Sugiyama) directed graph place-ment algorithm [17] to initially place the nodes a topologi-cal sort to determine an initial set of visible edges (the topo-logical sort maintains the connectivity of the graph) an edgelength threshold to determine which remaining edges shouldbe shown and edge ldquowispsrdquo to indicate the absence of anedge These techniques greatly reduce the number of visibleedges in large knowledge graphs while maintaining the hier-archical and connectivity structure of the content

To further accommodate the goals of knowledge map userswe incorporated several interactive components that aid theuser in locating concepts of interest and understanding theirrelationships to other concepts In particular our system al-lows users to step through the dependency structure of a graphby sequentially visualizing contextual views of the concepts(the direct dependencies and outlinks) Furthermore we in-clude a number of hover and click operations that reveal thereason for the relationships and associated content present fora given concept

RELATED WORKIn this section we discuss three areas of previous research anddevelopment relevant to this paper online knowledge graphsystems visualizations of large directed graphs and tools forcreating knowledge maps

Online Knowledge GraphsThe knowledge map creation visualization and explorationsystem described in this paper was designed for Meta-cademymdashan open source platform for collaboratively cre-ating refining and sharing knowledge maps and associatedlearning material [4] Figure 2 (right) shows a Metacademyknowledge graph one of the two main components of Meta-cademy The other major component is a text-based view thatprovides the user with the following information for each con-cept a description of the concept goals for the learner theprerequisites of the concept learning resources for the con-cept and related (non-dependency) concepts Metacademy istypically used as a targeted learning resource whereby a usersearches for a particular concept and is then presented witha knowledge map with that concept as the sole leafmdashmostgraphs have 5-50 nodes and less than 100 edges

As mentioned in the introduction Khan Academy [3]mdasha freevideo lecture and exercise repositorymdashalso provides a knowl-edge map of their material Rather than showing target-basedcomponents of the graph Khan Academy shows users theirentire 700+ node knowledge map This knowledge map ismuch larger than any of Metacademyrsquos knowledge maps butthe techniques discussed in this work are applicable to theirgraph and we will format their content for our system in fu-ture work

Visualization and Exploration of Directed GraphsOur knowledge map visualization and exploration systemdraws on a number contributions from the graph visualizationcommunity In particular the authors of [18] developed an in-teractive visualization system for large networks eg threemillion nodes The authors focused on providing context-specific visualizations of large directed graphs and stated that

Although providing a structural overview of the graphis a laudable goal there are many cases where the useris simply not interested in a global view of the wholegraph but wants to solve a particular concrete task onthe graph instead

Rather than show all direct inlinks and outlinks as is donein several related systems [5 6] the authors devised a de-gree of interest (DOI) term for each node that they use todetermine which subset of contextual nodes to show the userSince individual nodes in knowledge maps have a relativelysmall degree it is typically possible to visualize the fullcontext of each node rather than a subset and we do notuse the DOI technique in this work The authors use wispedgesmdashtruncated inlinksoutlinks to a nodemdashto indicate thepresence of an edge without cluttering the visualization withlong crisscrossing edges We incorporate this technique intoour visualization (see the next section)

Figure 2 The left figure shows a section of the Khan Academy knowledge map the right figure shows a Metacademy knowledge map

There are a number of software packages and algorithms forvisualizing (large) directed graphs that inspired our system

bull the dot placement algorithm described in [12] is theunderlying placement algorithm for the popular graphvizC++ graph visualization software The dot algorithm isused in the dagre JavaScript graph placement library whichis used in this work (see the next item)

bull dagre [1] a JavaScript library for directed graph placementis used to generate node and edge placements in our knowl-edge map system

bull Dynasty [11] a system for browsing large directed graphswhere only a small portion of the graph is visible at anygiven time Dynasty inspired the ldquofocused moderdquo aspectof our system where a concept and its dependencies andoutlinks are visible at a given time

bull the transition animations in Open Graph Drawing frame-work [10] a large C++ library of algorithms for graphdrawing inspired the transition animations sequence andtiming used in this work

Furthermore Misue et alrsquos mental map preservation proper-ties for directed graphs [15] motivated our use of visualizationmodes rather than dynamic directed graphs

Finally many directed graph visualization systems transformthe data for visualization For instance hive plots [14] pro-duce a linear layout for grouping nodes by type and then ar-ranges the nodes along a radial axis Arranging knowledgemap concepts along a radial axis does not work well for a vi-sualization as the concepts commonly have relationships thatspan multiple levels and as a result the radial network be-comes polluted with edges that span most of the circumfer-ence Strictly hiding these types of dependencies masks theprerequisite complexity for various concepts and could de-ceive the user

Knowledge Map CreationAs of the time of this writing the author currently does notknow of any tools that are explicitly used for creating knowl-edge maps as defined in this paper Both Khan Academyand Metacademy use text-based systems eg html formsto specify the relationships in the graph However there is alarge number of tools for creating similar structures such as

diagrams mind maps and concept maps We briefly discussthese tools and their relevance to our system

Cmap tools [9] is a Java-based GUI for creating conceptmaps directed graphs where the nodes are concepts and theedges are linking words so that tracing through a path is akinto reading a sentence The Cmap tools GUI inspired our edi-tor as it has simple drag and click operations to construct thegraph The Cmap tool editor however does not have asso-ciated content with the graph and is not browser-based Fur-thermore the Cmap tools editor requires the user to manuallyplace the nodes and edges Our editor operates in a similarfashion but users have the ability to algorithmically arrangethe nodes and edges using the dagre javascript library Fi-nally our editor is browser-based while Cmap tools is a Javaapplet

There are countless diagramming tools such as gliffy1drawio2 and dia 3 Many of these of tools are polished in-tuitive and downright beautiful diagramming tools but theyare more general purpose than what is needed for knowledgemap creation Letting users choose the color and shape of thenodes the thickness of the edges etc detracts from the mainpurpose of specifying the concept relationships and providingassociated content

METHODOLOGYIn this section we describe the visualization and interactionmethods employed by our knowledge map system In par-ticular our system builds upon the Metacademy knowledgemap system and we have recently acquired the data for KhanAcademyrsquos knowledge map and plan to extend our system forthis data in the near future

Knowledge Map Creation and ExplorationFrom an informal usability study conducted with seven UCBerkeley students using the think-aloud protocol[16] and theMetacademy knowledge map system we identified the fol-lowing common objectives the students had when interactingwith the knowledge map4

1httpwwwgliffycomindex-gphp2httpwwwdrawio3httpsourceforgenetprojectsdia-installer4Specifically we observed the students interacting with relativelylarge (30-50 nodes) and small (circa 10 nodes) knowledge maps

Figure 2 The left figure shows a section of the Khan Academy knowledge map the right figure shows a Metacademy knowledge map

There are a number of software packages and algorithms forvisualizing (large) directed graphs that inspired our system

bull the dot placement algorithm described in [12] is theunderlying placement algorithm for the popular graphvizC++ graph visualization software The dot algorithm isused in the dagre JavaScript graph placement library whichis used in this work (see the next item)

bull dagre [1] a JavaScript library for directed graph placementis used to generate node and edge placements in our knowl-edge map system

bull Dynasty [11] a system for browsing large directed graphswhere only a small portion of the graph is visible at anygiven time Dynasty inspired the ldquofocused moderdquo aspectof our system where a concept and its dependencies andoutlinks are visible at a given time

bull the transition animations in Open Graph Drawing frame-work [10] a large C++ library of algorithms for graphdrawing inspired the transition animations sequence andtiming used in this work

Furthermore Misue et alrsquos mental map preservation proper-ties for directed graphs [15] motivated our use of visualizationmodes rather than dynamic directed graphs

Finally many directed graph visualization systems transformthe data for visualization For instance hive plots [14] pro-duce a linear layout for grouping nodes by type and then ar-ranges the nodes along a radial axis Arranging knowledgemap concepts along a radial axis does not work well for a vi-sualization as the concepts commonly have relationships thatspan multiple levels and as a result the radial network be-comes polluted with edges that span most of the circumfer-ence Strictly hiding these types of dependencies masks theprerequisite complexity for various concepts and could de-ceive the user

Knowledge Map CreationAs of the time of this writing the author currently does notknow of any tools that are explicitly used for creating knowl-edge maps as defined in this paper Both Khan Academyand Metacademy use text-based systems eg html formsto specify the relationships in the graph However there is alarge number of tools for creating similar structures such as

diagrams mind maps and concept maps We briefly discussthese tools and their relevance to our system

Cmap tools [9] is a Java-based GUI for creating conceptmaps directed graphs where the nodes are concepts and theedges are linking words so that tracing through a path is akinto reading a sentence The Cmap tools GUI inspired our edi-tor as it has simple drag and click operations to construct thegraph The Cmap tool editor however does not have asso-ciated content with the graph and is not browser-based Fur-thermore the Cmap tools editor requires the user to manuallyplace the nodes and edges Our editor operates in a similarfashion but users have the ability to algorithmically arrangethe nodes and edges using the dagre javascript library Fi-nally our editor is browser-based while Cmap tools is a Javaapplet

There are countless diagramming tools such as gliffy1drawio2 and dia 3 Many of these of tools are polished in-tuitive and downright beautiful diagramming tools but theyare more general purpose than what is needed for knowledgemap creation Letting users choose the color and shape of thenodes the thickness of the edges etc detracts from the mainpurpose of specifying the concept relationships and providingassociated content

METHODOLOGYIn this section we describe the visualization and interactionmethods employed by our knowledge map system In par-ticular our system builds upon the Metacademy knowledgemap system and we have recently acquired the data for KhanAcademyrsquos knowledge map and plan to extend our system forthis data in the near future

Knowledge Map Creation and ExplorationFrom an informal usability study conducted with seven UCBerkeley students using the think-aloud protocol[16] and theMetacademy knowledge map system we identified the fol-lowing common objectives the students had when interactingwith the knowledge map4

1httpwwwgliffycomindex-gphp2httpwwwdrawio3httpsourceforgenetprojectsdia-installer4Specifically we observed the students interacting with relativelylarge (30-50 nodes) and small (circa 10 nodes) knowledge maps

1 visualize the overall complexity of the targeted conceptie the depth and breadth of the graph and the number ofedges and nodes

2 locate specific concepts within the knowledge map

3 identify the prerequisites and outlinks for a concept theprerequisites of its prerequisites outlinks of its outlinksetc

4 focus on a particular concept

5 explore related concepts not shown on the current graph(inclusion of new data)

Metacademyrsquos primary form of interaction is allowing usersto mark concepts they have learned and subsequently removethem from the graph When removing the concepts the Meta-cademy system naively recalculates the graph layout with-out taking into account the previous layout The system theninstantaneously flashes to the new knowledge graph withoutshowing the user the corresponding transitions see Figure 3This dynamic form of interaction does not directly addressany of the objectives expressed by the students in our usabil-ity test Furthermore according to Misue et alrsquos theory ofmental map preservation for dynamic directed graphs [15]this transition destroys the userrsquos mental map of the system asit violates the following three mental-map-preservation prop-erties

1 orthogonal ordering maintaining the relative directionsbetween the nodes (these updates can shift the relative di-rections see eg the ldquorandom variablesrdquo and ldquoexpectationand variancerdquo nodes in Figure 3)

2 proximity model maintaining the relative distance of thenodes (these updates can change the relative distance of thenodes see eg the ldquorandom variablesrdquo and ldquoexpectationand variancerdquo nodes in Figure 3)

3 topology model graphical objects within a relative regionshould stay within that region (these updates can changethe regional placement of nodes this is not present in thesupplied example but it tends to occur with larger graphs)

While letting users remove nodes and edges associated withlearned concepts helps reduce the number of objects in thegraph this form of interaction does not address the identifiedobjectives of the users and comes at the cost of nullifying theuserrsquos previous mental map From these observations we didnot include this functionality within our system

To address the visualization and exploration objectives abovewe created two modes within our system (1) an overviewmode in which the entire graph is shown and (2) a focusedmode in which a particular concept is shown along withits immediate dependencies and outlinks The goal of theoverview mode is to allow users to visualize the overall com-plexity of the targeted concept as well as quickly trace the

and associated content for 30-60 minutes while verbalizing theirthoughts These maps showed the full prerequisite structure for agiven concept in machine learning or probabilistic artificial intelli-gence

paths of various concepts (objectives 1 and 3) as well asquickly find specific concepts within the knowledge map andexplore related concepts by loading them into the currentgraph (objectives 2 and 5) The focused mode inspired bythe focused graph view from [11] address objective 4 Inthe following paragraphs we describe these modes in greaterdetail

Both Metacademy and Khan Academy knowledge maps havea large number of long crisscrossing edges that make it dif-ficult to trace the paths within the knowledge map We ad-dressed this problem in our system by employing the follow-ing algorithm to reduce the number of visible long crisscross-ing edges in our graph

1 remove all transitive edges from the graph

2 compute the graph layout using dagre

3 set visible-outlinkslarr empty

4 for each node n in the graphset visible-outlinkslarr visible-outlinkscup outlinks(n) with path length lt Lcup shortest-outlink(n)

5 for all edges e not in visible-outlinks display an edge wispprotruding from the source and target of e

This algorithm is used to help accurately convey the relation-ships between the concepts while reducing the edge satura-tion within the graph The motivating philosophy behind thistechnique is to show users edges when they are interested inseeing them while still providing enough edges to maintainthe proper structure of the graph

In step 1 we remove the transitive edges in the graph becausethey do not affect the relative prerequisite structure of thegraph That is if we have edges (AB) (BC) and (AC) thenremoving (AC) from the graph will not affect the ABC or-dering and from a pedagogical perspective it is a redundantedge In step 2 we compute the edge and node placements us-ing dagre though any graph placement algorithm can be usedduring this step

In step 4 we remove all edges exceeding a length thresholdL while ensuring that each node has at least one outlink (theshortest outlink) By including the shortest outlink for eachnode we maintain the visual connectivity of the graph afterhiding other long edges Naively truncating all edges couldinduce artificially separated clusters of concepts that would bedetermined by the graph placement algorithm rather than thecontent We note that with a small enough threshold L eachnode would have one outlink and the graph would become atree Therefore if we simply did not include the non-visibleedges in the graph placement algorithm it would be possibleto obtain a tree layout with zero edge crossings This tree lay-out however could visually invalidate the relative prerequi-site structure of the graph Using the previous example withnodes A B and C we could have a tree with visible edges(BC) and (AC) and invisible edge (BC) which would resultin visually placing nodes A and B on the same level

Figure 3 One of Metacademyrsquos primary forms of interaction is allowing users to remove concepts they know (left) the original graph (middle) thedimmed node is marked as known (right) the node is removed from the graph In removing the node the Metacademy system naively redeterminesthe location of the edges and nodes without taking into account their previous locations In the text we argue that this form of interaction nullifies theuserrsquos mental map and as a result we have not included this feature in our system

The edge wisps from step 4 give a simple visible indication ofthe number of prerequisites and outlinks for a given conceptand are used in the interactive components discussed belowFigure 4 shows a medium-sized Metacademy knowledge mapof 39 nodes rendered directly from dagre (74 visible edgeswith 61 visible edge crossings) and the same knowledge mapobtained using the above algorithm (38 visible edges with 0visible edge crossings)

We employ the following interactive tools and techniques tohelp the user explore and understand the knowledge map gen-erated using the visible-edge-reduction-algorithm

bull hovering over nodes highlights the inlinksoutlinks fromthat node as well as the source (target) of the inlink (out-link) and shows a set of clickable icons that the user canuse to eg show the content associated with the node

bull clicking on a node triggers a series of animations that tran-sitions to the focused mode for that node (see below)

bull clicking on an edge shows a pop-up justification for theedge

bull a contractible side menu gives users the ability to searchfor a concept in the graph

The ldquofocused moderdquo shows the user a selected concept aswell its dependencies and outlinks We use a series of ani-mations to contract and expand the graph when a user entersthe focused mode the node is highlighted then centered onthe screen then the dependencies and outlinks are translatedso that they are adjacent to the focused node while the othernodes and edges fade out This sequence is based on similartransitions in the Open Graph Drawing framework [10] Theanimations and interactive tools were implemented using thed3 JavaScript library [8]

In the focused mode a small information box appears atthe bottom of the screen and indicates the number of hid-den nodes and edges The entire animation sequence takesslightly longer than one second Clicking on an adjacent nodewithin focused mode moves that node into focus while click-ing on the focused node runs the sequence of animations in

reverse and returns the user to the full graph Figure 5 showsa visualization of focused mode and the associated transition

Knowledge Map CreationIn this subsection we describe our visual knowledge map cre-ator While this tool is independent of a particular platformits current implementation is aligned for the Metacademy sys-tem We designed this visual editor using the d3 JavaScriptlibrary [8] and only included features that benefitted the fol-lowing operations addremove concepts addremove rela-tionships between concepts (edges) addremove associatedcontent and preview the knowledge map using the techniquesdiscussed in the previous subsection

Figure 6 shows the knowledge map creator The main viewconsists of a large graph editing surface and a small toolboxThe user can perform the following editing operations

bull add node shift + click on the editing surface

bull remove node click on a node and press delete or backspace

bull move node click + drag node

bull change node title shift + click on the node

bull edit content associated with node click on the small circlethat appears when hovering the node

bull expandcontract a nodersquos dependencies (outlinks) click theplusminus icon at the top (bottom) of the node

bull add directed edge shift + click on the source node and dragto the target node

bull remove edge click on a edge and press delete or backspace

bull scale graph scroll

bull translate graph click + drag on the editing surface

In addition the toolbox shown in Figure 7 provides the userwith the ability to optimize the graph location using dagreclear all nodes and edges from the graph preview the graphusing the knowledge map exploration system described in this

Figure 4 The left figure shows the knowledge map obtained directly using the dagre output the right figure shows the knowledge map obtained usingour visible-edge-reduction algorithm

Figure 5 The left figure shows the highlighted edges and nodes when hovering over a specific node The right figure shows the ldquofocused moderdquo thatoccurs after clicking the node Clicking the focused node in the focused mode returns to the full-graph display while clicking the adjacent nodes placesthe focus on them

paper and downloadupload a graph tofrom a JSON5 repre-sentation

The search bar at the bottom of the toolbox gives users theability to add preexisting concepts to graph When a user addsa preexisting concept to the graph the inlinksoutlinks of thatnode are contracted by default so that eg 20-50 prerequi-site nodes are not immediately placed into the graph Whenexpandingcontracting nodes a small information box at thebottom of the screen displays the number of hidden nodes andedges

Figure 6 This figure displays the graph editor described in the text

Figure 7 This figure shows the annotated toolbox for the graph editor

When hovering over a node a small circle appears on theouter radius of the node If the user clicks the small circle thena content editor overlays the graphs with roughly a 08 alphavalue Figure 8 This content editor lets the user provide asummary for the concept as well as provide links to learningresources and justifications for the prerequisites Presentlythis form is hard-coded html but in the near future we wouldlike to create a simple interface for specifying the types dataassociated with the knowledge map In this way we hope tomake this system usable outside of Metacademy

DISCUSSIONThe Metacademy system was designed so that individuals invariegated disciplines could both contribute content and ex-plore preexisting content Until now this process involved

5httpwwwjsonorg

Figure 8 This figure shows the content-editing form that overlays theknowledge map creator This form allows users to associate content withthe graph

exploring large static graphs and editing text files The con-tributions from this paper will help users interactively explorethe knowledge maps and associated content Furthermoreusers can now contribute content via a visual knowledge mapcreator

From watching users interact with our system during aresearch-poster session we found that several users had hadcomments along the line of ldquovisualizing the edges isnrsquot nec-essary until yoursquove chosen a reference node and you want tosee itrsquos relationshipsrdquo While this certainly is not a formal us-ability study we found it encouraging the users organicallyagreed with one of the driving forces for this work We maypursue a formal usability study in the future

This work and its inevitable progressions and exten-sions can be found at httpwwwmetacademyorgThe exact system described in this paper can befound at httpgithubcommetacademymetacademy|applicationtreegraph|editor

FUTURE WORKThe Khan Academy knowledge map inspired this paper andin the coming weeks we plan to make our system compatiblewith their data As mentioned in the previous section a for-mal user study would help validate or nullify our conjecturethat our knowledge map exploration and creation tools helpusers efficiently explore and create educational material

Beyond the realm of visualization we would like to explorehow users interact with these graphs in an online educationalsetting Some interesting questions to answer include whatadvantages and disadvantages does a knowledge map systemhave over traditional content presentations such as a textbookor wiki page do users that create knowledge maps performbetter in the class (not a concept map as has been studiedcountless times in educational psychology) and to what ex-tent are large knowledge maps useful ie how big is too big

ACKNOWLEDGEMENTSColorado Reed would like to thank Kristin Stephens for herfantastic feedback throughout this project and Dan Allan andZach MacHardy for testing early versions of the knowledgemap creator

Figure 3 One of Metacademyrsquos primary forms of interaction is allowing users to remove concepts they know (left) the original graph (middle) thedimmed node is marked as known (right) the node is removed from the graph In removing the node the Metacademy system naively redeterminesthe location of the edges and nodes without taking into account their previous locations In the text we argue that this form of interaction nullifies theuserrsquos mental map and as a result we have not included this feature in our system

The edge wisps from step 4 give a simple visible indication ofthe number of prerequisites and outlinks for a given conceptand are used in the interactive components discussed belowFigure 4 shows a medium-sized Metacademy knowledge mapof 39 nodes rendered directly from dagre (74 visible edgeswith 61 visible edge crossings) and the same knowledge mapobtained using the above algorithm (38 visible edges with 0visible edge crossings)

We employ the following interactive tools and techniques tohelp the user explore and understand the knowledge map gen-erated using the visible-edge-reduction-algorithm

bull hovering over nodes highlights the inlinksoutlinks fromthat node as well as the source (target) of the inlink (out-link) and shows a set of clickable icons that the user canuse to eg show the content associated with the node

bull clicking on a node triggers a series of animations that tran-sitions to the focused mode for that node (see below)

bull clicking on an edge shows a pop-up justification for theedge

bull a contractible side menu gives users the ability to searchfor a concept in the graph

The ldquofocused moderdquo shows the user a selected concept aswell its dependencies and outlinks We use a series of ani-mations to contract and expand the graph when a user entersthe focused mode the node is highlighted then centered onthe screen then the dependencies and outlinks are translatedso that they are adjacent to the focused node while the othernodes and edges fade out This sequence is based on similartransitions in the Open Graph Drawing framework [10] Theanimations and interactive tools were implemented using thed3 JavaScript library [8]

In the focused mode a small information box appears atthe bottom of the screen and indicates the number of hid-den nodes and edges The entire animation sequence takesslightly longer than one second Clicking on an adjacent nodewithin focused mode moves that node into focus while click-ing on the focused node runs the sequence of animations in

reverse and returns the user to the full graph Figure 5 showsa visualization of focused mode and the associated transition

Knowledge Map CreationIn this subsection we describe our visual knowledge map cre-ator While this tool is independent of a particular platformits current implementation is aligned for the Metacademy sys-tem We designed this visual editor using the d3 JavaScriptlibrary [8] and only included features that benefitted the fol-lowing operations addremove concepts addremove rela-tionships between concepts (edges) addremove associatedcontent and preview the knowledge map using the techniquesdiscussed in the previous subsection

Figure 6 shows the knowledge map creator The main viewconsists of a large graph editing surface and a small toolboxThe user can perform the following editing operations

bull add node shift + click on the editing surface

bull remove node click on a node and press delete or backspace

bull move node click + drag node

bull change node title shift + click on the node

bull edit content associated with node click on the small circlethat appears when hovering the node

bull expandcontract a nodersquos dependencies (outlinks) click theplusminus icon at the top (bottom) of the node

bull add directed edge shift + click on the source node and dragto the target node

bull remove edge click on a edge and press delete or backspace

bull scale graph scroll

bull translate graph click + drag on the editing surface

In addition the toolbox shown in Figure 7 provides the userwith the ability to optimize the graph location using dagreclear all nodes and edges from the graph preview the graphusing the knowledge map exploration system described in this

Figure 4 The left figure shows the knowledge map obtained directly using the dagre output the right figure shows the knowledge map obtained usingour visible-edge-reduction algorithm

Figure 5 The left figure shows the highlighted edges and nodes when hovering over a specific node The right figure shows the ldquofocused moderdquo thatoccurs after clicking the node Clicking the focused node in the focused mode returns to the full-graph display while clicking the adjacent nodes placesthe focus on them

paper and downloadupload a graph tofrom a JSON5 repre-sentation

The search bar at the bottom of the toolbox gives users theability to add preexisting concepts to graph When a user addsa preexisting concept to the graph the inlinksoutlinks of thatnode are contracted by default so that eg 20-50 prerequi-site nodes are not immediately placed into the graph Whenexpandingcontracting nodes a small information box at thebottom of the screen displays the number of hidden nodes andedges

Figure 6 This figure displays the graph editor described in the text

Figure 7 This figure shows the annotated toolbox for the graph editor

When hovering over a node a small circle appears on theouter radius of the node If the user clicks the small circle thena content editor overlays the graphs with roughly a 08 alphavalue Figure 8 This content editor lets the user provide asummary for the concept as well as provide links to learningresources and justifications for the prerequisites Presentlythis form is hard-coded html but in the near future we wouldlike to create a simple interface for specifying the types dataassociated with the knowledge map In this way we hope tomake this system usable outside of Metacademy

DISCUSSIONThe Metacademy system was designed so that individuals invariegated disciplines could both contribute content and ex-plore preexisting content Until now this process involved

5httpwwwjsonorg

Figure 8 This figure shows the content-editing form that overlays theknowledge map creator This form allows users to associate content withthe graph

exploring large static graphs and editing text files The con-tributions from this paper will help users interactively explorethe knowledge maps and associated content Furthermoreusers can now contribute content via a visual knowledge mapcreator

From watching users interact with our system during aresearch-poster session we found that several users had hadcomments along the line of ldquovisualizing the edges isnrsquot nec-essary until yoursquove chosen a reference node and you want tosee itrsquos relationshipsrdquo While this certainly is not a formal us-ability study we found it encouraging the users organicallyagreed with one of the driving forces for this work We maypursue a formal usability study in the future

This work and its inevitable progressions and exten-sions can be found at httpwwwmetacademyorgThe exact system described in this paper can befound at httpgithubcommetacademymetacademy|applicationtreegraph|editor

FUTURE WORKThe Khan Academy knowledge map inspired this paper andin the coming weeks we plan to make our system compatiblewith their data As mentioned in the previous section a for-mal user study would help validate or nullify our conjecturethat our knowledge map exploration and creation tools helpusers efficiently explore and create educational material

Beyond the realm of visualization we would like to explorehow users interact with these graphs in an online educationalsetting Some interesting questions to answer include whatadvantages and disadvantages does a knowledge map systemhave over traditional content presentations such as a textbookor wiki page do users that create knowledge maps performbetter in the class (not a concept map as has been studiedcountless times in educational psychology) and to what ex-tent are large knowledge maps useful ie how big is too big

ACKNOWLEDGEMENTSColorado Reed would like to thank Kristin Stephens for herfantastic feedback throughout this project and Dan Allan andZach MacHardy for testing early versions of the knowledgemap creator

Figure 4 The left figure shows the knowledge map obtained directly using the dagre output the right figure shows the knowledge map obtained usingour visible-edge-reduction algorithm

Figure 5 The left figure shows the highlighted edges and nodes when hovering over a specific node The right figure shows the ldquofocused moderdquo thatoccurs after clicking the node Clicking the focused node in the focused mode returns to the full-graph display while clicking the adjacent nodes placesthe focus on them

paper and downloadupload a graph tofrom a JSON5 repre-sentation

The search bar at the bottom of the toolbox gives users theability to add preexisting concepts to graph When a user addsa preexisting concept to the graph the inlinksoutlinks of thatnode are contracted by default so that eg 20-50 prerequi-site nodes are not immediately placed into the graph Whenexpandingcontracting nodes a small information box at thebottom of the screen displays the number of hidden nodes andedges

Figure 6 This figure displays the graph editor described in the text

Figure 7 This figure shows the annotated toolbox for the graph editor

When hovering over a node a small circle appears on theouter radius of the node If the user clicks the small circle thena content editor overlays the graphs with roughly a 08 alphavalue Figure 8 This content editor lets the user provide asummary for the concept as well as provide links to learningresources and justifications for the prerequisites Presentlythis form is hard-coded html but in the near future we wouldlike to create a simple interface for specifying the types dataassociated with the knowledge map In this way we hope tomake this system usable outside of Metacademy

DISCUSSIONThe Metacademy system was designed so that individuals invariegated disciplines could both contribute content and ex-plore preexisting content Until now this process involved

5httpwwwjsonorg

Figure 8 This figure shows the content-editing form that overlays theknowledge map creator This form allows users to associate content withthe graph

exploring large static graphs and editing text files The con-tributions from this paper will help users interactively explorethe knowledge maps and associated content Furthermoreusers can now contribute content via a visual knowledge mapcreator

From watching users interact with our system during aresearch-poster session we found that several users had hadcomments along the line of ldquovisualizing the edges isnrsquot nec-essary until yoursquove chosen a reference node and you want tosee itrsquos relationshipsrdquo While this certainly is not a formal us-ability study we found it encouraging the users organicallyagreed with one of the driving forces for this work We maypursue a formal usability study in the future

This work and its inevitable progressions and exten-sions can be found at httpwwwmetacademyorgThe exact system described in this paper can befound at httpgithubcommetacademymetacademy|applicationtreegraph|editor

FUTURE WORKThe Khan Academy knowledge map inspired this paper andin the coming weeks we plan to make our system compatiblewith their data As mentioned in the previous section a for-mal user study would help validate or nullify our conjecturethat our knowledge map exploration and creation tools helpusers efficiently explore and create educational material

Beyond the realm of visualization we would like to explorehow users interact with these graphs in an online educationalsetting Some interesting questions to answer include whatadvantages and disadvantages does a knowledge map systemhave over traditional content presentations such as a textbookor wiki page do users that create knowledge maps performbetter in the class (not a concept map as has been studiedcountless times in educational psychology) and to what ex-tent are large knowledge maps useful ie how big is too big

ACKNOWLEDGEMENTSColorado Reed would like to thank Kristin Stephens for herfantastic feedback throughout this project and Dan Allan andZach MacHardy for testing early versions of the knowledgemap creator

paper and downloadupload a graph tofrom a JSON5 repre-sentation

The search bar at the bottom of the toolbox gives users theability to add preexisting concepts to graph When a user addsa preexisting concept to the graph the inlinksoutlinks of thatnode are contracted by default so that eg 20-50 prerequi-site nodes are not immediately placed into the graph Whenexpandingcontracting nodes a small information box at thebottom of the screen displays the number of hidden nodes andedges

Figure 6 This figure displays the graph editor described in the text

Figure 7 This figure shows the annotated toolbox for the graph editor

When hovering over a node a small circle appears on theouter radius of the node If the user clicks the small circle thena content editor overlays the graphs with roughly a 08 alphavalue Figure 8 This content editor lets the user provide asummary for the concept as well as provide links to learningresources and justifications for the prerequisites Presentlythis form is hard-coded html but in the near future we wouldlike to create a simple interface for specifying the types dataassociated with the knowledge map In this way we hope tomake this system usable outside of Metacademy

DISCUSSIONThe Metacademy system was designed so that individuals invariegated disciplines could both contribute content and ex-plore preexisting content Until now this process involved

5httpwwwjsonorg

Figure 8 This figure shows the content-editing form that overlays theknowledge map creator This form allows users to associate content withthe graph

exploring large static graphs and editing text files The con-tributions from this paper will help users interactively explorethe knowledge maps and associated content Furthermoreusers can now contribute content via a visual knowledge mapcreator

From watching users interact with our system during aresearch-poster session we found that several users had hadcomments along the line of ldquovisualizing the edges isnrsquot nec-essary until yoursquove chosen a reference node and you want tosee itrsquos relationshipsrdquo While this certainly is not a formal us-ability study we found it encouraging the users organicallyagreed with one of the driving forces for this work We maypursue a formal usability study in the future

This work and its inevitable progressions and exten-sions can be found at httpwwwmetacademyorgThe exact system described in this paper can befound at httpgithubcommetacademymetacademy|applicationtreegraph|editor

FUTURE WORKThe Khan Academy knowledge map inspired this paper andin the coming weeks we plan to make our system compatiblewith their data As mentioned in the previous section a for-mal user study would help validate or nullify our conjecturethat our knowledge map exploration and creation tools helpusers efficiently explore and create educational material

Beyond the realm of visualization we would like to explorehow users interact with these graphs in an online educationalsetting Some interesting questions to answer include whatadvantages and disadvantages does a knowledge map systemhave over traditional content presentations such as a textbookor wiki page do users that create knowledge maps performbetter in the class (not a concept map as has been studiedcountless times in educational psychology) and to what ex-tent are large knowledge maps useful ie how big is too big

ACKNOWLEDGEMENTSColorado Reed would like to thank Kristin Stephens for herfantastic feedback throughout this project and Dan Allan andZach MacHardy for testing early versions of the knowledgemap creator

REFERENCES1 dagre javascript library

httpsgithubcomcpettittdagre Accessed2013-12-01

2 Duolingo httpwwwduolingocom Accessed2013-12-01

3 Khan academy httpwwwkhanacademyorgAccessed 2013-12-01

4 Metacademy httpwwwmetacademyorg Accessed2013-12-01

5 Palantir httpwwwpalantirtechcom Accessed2013-12-01

6 Touchgraph httpwwwtouchgraphcomnavigatorAccessed 2013-12-01

7 Batagelj V and Mrvar A Pajekanalysis andvisualization of large networks Springer 2004

8 Bostock M Ogievetsky V and Heer J D3Data-driven documents IEEE Trans Visualization ampComp Graphics (Proc InfoVis) (2011)

9 Canas A J Hill G Carff R Suri N Lott JEskridge T Gomez G Arroyo M and Carvajal RCmaptools A knowledge modeling and sharingenvironment In Concept maps Theory methodologytechnology Proceedings of the first internationalconference on concept mapping vol 1 (2004) 125ndash133

10 Chimani M and Gutwenger C The open graphdrawing framework (ogdf) httpwwwogdfnet

11 Eisner J Kornbluh M Woodhull G Buse RHuang S Michael C and Shafer G Visual navigation

through large directed graphs and hypergraphs InProceedings of the IEEE Symposium on InformationVisualization (InfoVisrsquo06) PosterDemo Session(Baltimore Oct 2006) 116ndash117

12 Gansner E R Koutsofios E North S C and VoK-P A technique for drawing directed graphs SoftwareEngineering IEEE Transactions on 19 3 (1993)214ndash230

13 Khan S The one world schoolhouse Educationreimagined Hachette Digital Inc 2012

14 Krzywinski M Birol I Jones S J and Marra M AHive plotsrational approach to visualizing networksBriefings in Bioinformatics 13 5 (2012) 627ndash644

15 Misue K Eades P Lai W and Sugiyama K Layoutadjustment and the mental map Journal of visuallanguages and computing 6 2 (1995) 183ndash210

16 Nielsen J Evaluating the thinking-aloud technique foruse by computer scientists In Advances inhuman-computer interaction (vol 3) Ablex PublishingCorp (1993) 69ndash82

17 Sugiyama K Tagawa S and Toda M Methods forvisual understanding of hierarchical system structuresSystems Man and Cybernetics IEEE Transactions on11 2 (1981) 109ndash125

18 Van Ham F and Perer A Search show contextexpand on demand Supporting large graph explorationwith degree-of-interest Visualization and ComputerGraphics IEEE Transactions on 15 6 (2009) 953ndash960