reference coupling: an exploration of inter-project

Reference Coupling: An Exploration of Inter-projectTechnical Dependencies and their Characteristics

within Large Software Ecosystems

Kelly Blincoea,⇤, Francis Harrisonb, Navpreet Kaurb, Daniela Damianb

aUniversity of Auckland, New ZealandbUniversity of Victoria, BC, Canada

Abstract

Context: Software projects often depend on other projects or are developed

in tandem with other projects. Within such software ecosystems, knowledge

of cross-project technical dependencies is important for 1) practitioners under-

standing of the impact of their code change and coordination needs within the

ecosystem and 2) researchers in exploring properties of software ecosystems

based on these technical dependencies. However, identifying technical depen-

dencies at the ecosystem level can be challenging.

Objective: In this paper, we describe Reference Coupling, a new method that

uses solely the information in developers online interactions to detect technical

dependencies between projects. The method establishes dependencies through

user-specified cross-references between projects. We then use the output of this

method to explore the properties of large software ecosystems.

Method: We validate our method on two datasets — one from open-source

projects hosted on GitHub and one commercial dataset of IBM projects. We

manually analyze the identified dependencies, categorize them, and compare

them to dependencies specified by the development team. We examine the

types of projects involved in the identified ecosystems, the structure of the iden-

⇤Corresponding authorEmail addresses: [email protected] (Kelly Blincoe), [email protected] (Francis

Harrison), [email protected] (Navpreet Kaur), [email protected] (DanielaDamian)

Preprint submitted to Elsevier February 2, 2019

This is a post-peer-review, pre-copyedit version of an article published in Information and Software Technology. The final authenticated version is available online at: https://doi.org/10.1016/j.infsof.2019.03.005

© 2019. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/

tified ecosystems, and how the ecosystems structure compares with the social

behaviour of project contributors and owners.

Results: We find that our Reference Coupling method often identifies tech-

nical dependencies between projects that are untracked by developers. We de-

scribe empirical insights about the characteristics of large software ecosystems.

We find that most ecosystems are centered around one project and are inter-

connected with other ecosystems. By exploring the socio-technical alignment

within the GitHub ecosystems, we also found that the project owners social be-

haviour aligns well with the technical dependencies within the ecosystem, but

the project contributors social behaviour does not align with these dependencies.

Conclusions: We conclude with a discussion on future research that is en-

abled by our Reference Coupling method.

1. Introduction

Software is not developed in isolation anymore. Whether open source or

corporate-led, software development takes place within “a collection of software

projects which are developed and which co-evolve together in the same envi-

ronment”, and which are referred to as software ecosystems [1]. Within such

ecosystems, projects depend on one another [1], and yet awareness of such de-

pendencies is not trivial. Identifying technical dependencies to external projects

within the ecosystem is important for two main reasons: First, developers need

to understand how their tasks and code changes impact other projects and who

they need to coordinate their changes with at the ecosystem level [1, 2]. For

open source ecosystems in particular, it is also important for attracting new

contributors [1] since dependencies to other projects within an ecosystem are

more likely to attract attention. Newcomers might decide to join the project

based on a deeper knowledge of the structure of the system as well as of its exter-

nal dependencies. Second, information about software ecosystems and technical

dependencies within them enables further exploration and modeling of software

ecosystems, an area currently understudied in the software engineering litera-

2

ture [3].

However, identifying technical dependencies between projects on a large scale

has proven to be di�cult [4]. Existing static dependency analysis approaches

do not identify dependencies across projects. Methods for extracting exter-

nal dependencies from a project’s source code or configuration files have been

proposed [1, 4, 5, 6, 7, 8, 9, 10, 11], but these approaches limit the types of de-

pendencies detected to explicit relationships. Implicit relationships like depen-

dencies on web services, operating systems, or hardware are not always visible in

configuration files or source code. Further, methods that extract dependencies

from source code require large amounts of memory and computation time, so

they cannot be employed across a large set of projects [12]. Methods that are

applied to configuration or build files are not memory or computation-intensive,

but such files are not always available or accurate since not all projects use a

package manager.

Without a way to easily establish a comprehensive set of dependencies be-

tween projects, software practitioners are unable to quickly identify the external

dependencies of a software project or understand where their project sits within

a software ecosystem.

In this paper, we propose a new method, Reference Coupling, to detect cross-

project dependencies that leverages solely the information in the developers’ so-

cial interactions. The social aspects of a project and its surrounding ecosystem

significantly influence the way in which the software project will evolve over

time [13], and they cannot be ignored in the development of models, guide-

lines and best practices for the analysis and maintenance of software ecosys-

tems’ health [14]. Reference Coupling mines the references to other projects

that developers make in their online interaction (referred to as cross-references

henceforth). We validated the method in identifying true cross-project technical

dependencies by using it in two large datasets of GitHub and IBM projects and

comparing its results with manually identified cross-references in each dataset.

We found that the Reference Coupling method does identify technical depen-

dencies between projects, and we describe several properties of the ecosystems

3

identified using this method. Reference coupling represents a significant novel

complement to the other existing, but insu�cient, code-based or configuration

file-based methods to identify external dependencies within a project’s ecosys-

tem. Our method identifies ecosystems of projects within the same organization,

and also outside of the organization for which there are technical dependencies,

often hidden or even unknown to developers within a project.

Having identified external technical dependencies to projects in our datasets,

we further used our Reference Coupling method with a popular community

detection algorithm [15] to identify ecosystems across all GitHub-hosted projects

and explore socio-technical aspects within these ecosystems. We found that the

developers’ socio-technical behavior within GitHub project ecosystems di↵ers

between the project owners and actual contributors. Our analysis illustrates the

potential for further analysis of software ecosystems’ health, something di�cult

to assess given the dynamic nature of relationships within the ecosystem, as well

as the lack of a centralized management structure for overseeing the ecosystem’s

health and survival, most often typical of open source projects [14].

Our previous conference publication reported on some elements of this work

[16]; however, this paper introduces numerous extensions to our work. Specifi-

cally this paper extends our previous publications by:

• describing how to utilize the Reference Coupling method on a wide variety

of software tools (compared to just GitHub in the conference publication).

• validating the Reference Coupling approach on a new dataset from a pro-

prietary ecosystem of IBM projects.

• providing a more detailed understanding of the types of dependencies that

are captured by the Reference Coupling method through the analysis and

categorization of dependencies from the IBM ecosystem and a more de-

tailed analysis of the dependencies on GitHub.

• extending the discussion to further describe how the Reference Coupling

method can be used to support software developers and software engineer-

4

ing researchers.

The paper has also been significantly restructured to better describe our research

methods, development, validation and application of the Reference Coupling

technique.

The rest of the paper is structured as follows: Section 2 provides an overview

of related work in software ecosystems and dependency conceptualizations. The

Reference Coupling method is described in Section 3. Our research methods

are presented in Section 4. Our results are presented in Section 5. In Section 6,

we summarize our findings and discuss open questions for future research. We

provide a brief conclusion in Section 7.

2. Related Work

The term software ecosystem (SECO) has emerged as a paradigm to un-

derstand the dynamics and heterogeneity in collaborative software engineering.

Unlike natural ecosystems, however, there is no common definition for SECO.

Two di↵erent perspectives on SECOs have been identified in the literature,

namely business-centric and platform-centric [3]. The business-centric defini-

tion refers to the holistic, business-oriented perspective of a SECO as a network

of actors, organizations and companies [17, 18, 19]. The platform-specific per-

spective emphasizes the social and technical aspects of a set of software projects,

technical platforms, and communities, in line with work of [20, 21]. In our work,

we take a platform-specific perspective to leverage and study the socio-technical

relationships within ecosystems. The ecosystems we consider are not limited to

only those projects in the same organization, but also include projects outside

of the organization for which there are technical dependencies.

The software ecosystems that received the most attention in previous liter-

ature include those around Eclipse (e.g. [6, 22]), Ruby on Rails (e.g. [23, 24]),

Apache (e.g. [7, 9]), and GNOME (e.g. [13]). Notable research developments

also exist in the area of frameworks for analysis of Open Source Software Ecosys-

tems (OSSECOs) (e.g. [21]), OSSECO health measurement (e.g. [25]), and tools

5

for visualizing OSSECO projects (e.g. [26]).

Recent extensive literature surveys show a growing interest in studies of

ecosystems, both in the domain of proprietary [27] as well as open source soft-

ware development [3]. In the proprietary software space, the focus has been on

the organizational and business aspects of the ecosystems, with a clear lack of

deeper investigations of technical and collaborative aspects of work [27]. In the

open source software ecosystems space, the pressing research challenges include

the development of methods and tools for the ecosystem modelling and analysis,

socio-technical theories to explain the interplay between the social and technical

system within ecosystems, as well as the diagnostic and monitoring of ecosystem

quality and health [3].

An important step in this direction lies with methods for the identification

of a project’s external technical dependencies within its ecosystem, in order to

study its structure and socio-technical aspects. Analysis of a project’s source

code is a common technique to identify technical dependencies within a project

(intra-project). However, these techniques do not scale up well to identify depen-

dencies between projects (inter-project). Lungu et al. [5] describe several meth-

ods for extracting inter-project dependencies by considering external method

and class calls in a project’s source code. However, when investigating a large

number of projects, obtaining the source code for every project is not always

feasible. Collecting source code data across an entire versioning system would

require multiple Terabytes of data and more than a year in processing time [12].

Ossher et al. [4] introduced a technique that analyzes import statements in Java

source code to resolve inter-project dependencies. Businge and Serebrenik [6]

employ a similar technique in their study of the Eclipse ecosystem. However, this

technique still requires obtaining a large amount of source code and, therefore,

requires a large amount of memory. These techniques, therefore, are limited in

the number of projects that can be studied.

Previous studies have also proposed ways to identify technical dependencies

without relying on analysis of source code. One method is to identify technical

dependencies by examining declared dependencies from a project’s configuration

6

files or its build files from a dependency management tool like Maven [1, 7, 8,

9, 10, 11]. However, not all projects declare dependencies in configuration files

or employ a dependency manager, and, even for those that do, the data can be

missing. Bavota et al. [9] found that this information was missing in 37% of

releases in a study of the Apache project. Syeed et al. [23] extracted metadata on

inter-project dependencies from the published specifications at rubygems.org in

their study of the Ruby on Rails ecosystem. However, the specified dependencies

may be out of date and the approach is specific to only projects that publish

dependency specifications.

Considering the social aspects of software ecosystems are also important [3].

The work of Mens and his colleagues highlights the role that social aspects

play in the future understanding and development of tools, prediction models,

guidelines and best practices that allow ecosystem communities to improve upon

their current practices [13]. Several other studies [28, 29] have explored ways to

detect social connections in software ecosystems. These studies used community

detection algorithms to detect communities across GitHub projects focusing on

relationships between developers.

We use technical dependencies for community detection since the structure

of an ecosystem is defined by its technical dependencies [5]. Thung et al. [30]

constructed similar project-to-project networks for GitHub-hosted projects. In

their networks, edges between projects represent a single developer contributing

to both projects. Since developers can often work on multiple independent

projects, sharing developers is not an indication of a technical dependency and

their network is more of a social perspective.

In our approach, we propose a method for automatic identification of techni-

cal dependencies that does not rely on analyzing source code, but takes advan-

tage of the cross-references that can be made in developers’ social interactions.

These cross-references are user-specified links between a pair of projects. They

are made in comments on work items, pull requests, issues, and commits as de-

velopers coordinate and manage their work dependencies. With these identified

technical dependencies we were then able to conduct a socio-technical analy-

7

sis of the behavior of di↵erent project members, namely project owners and

contributors in the GitHub ecosystem.

3. Reference Coupling

To identify dependencies between projects, we relied on comments made by

the developers within one project’s tasks, issues, pull requests and commits that

cross-reference another project. Modern collaborative software development

tools make it easy for developers to create links between projects within their

comments. We call this conceptualization of dependencies between projects

Reference Coupling.

To develop the Reference Coupling method, we manually examined cross-

references between projects on several software development tools to under-

stand how they could be automatically extracted. We examined comments that

cross-reference other projects on the following, popular open source software

code hosting platforms, forges, and issue trackers: GitHub, GitLab, BitBucket,

SourceForge, and Jira. We also examined cross-references in a proprietary soft-

ware ecosystem by examining the comments in an IBM set of products that

together form the Rational solution for Collaborative Lifecycle Management

(CLM).

We found that all of the open source software code hosting platforms, forges,

and issue trackers we examined (GitHub, GitLab, BitBucket, SourceForge, and

Jira) employ Markdown languages, which allow plain text to be formatted in

a lightweight way. When a Markdown language is employed and a user cross-

references another project in a comment using the appropriate syntax, a link to

the other project is automatically created, making it easier to navigate between

the projects. Due to the adoption of these Markdown languages, cross-references

to other projects are created in a standard format to enable these automatic

links to be created. GitHub, Bitbucket, and GitLab all extend the Common-

Mark specification [31], which was introduced to standardize markdown imple-

mentations. Jira and SourceForge, use their own Markdown languages.

8

Table 1: Syntax of Cross-References

Tool Link ToSyntax

ExampleProject Identifier Artifact Type Artifact ID

GitHub,Issues/Pull Requests OWNER/PROJECT # NUMBER rails/rails#123

GitLab,

Bitbucket, andCommits OWNER/PROJECT @ SHA twbs/bootstrap@6e2a82

Jira

SourceForge IssuesPROJECT[/SUBPROJECT] bugs NUMBER allura:bugs:#123

PROJECT[/SUBPROJECT] features #NUMBER allura:features:#123

Commits PROJECT[/SUBPROJECT] code SHA allura:code:3b9d48

IBM CLM Work Item

— [work] item NUMBER work item 123

— task NUMBER task 456

— story NUMBER story 789

— defect NUMBER defect 123

— URL NUMBER https://jazz.net/jazz/resource/itemName/

com.ibm.team.workitem.WorkItem/123

*Text in [ ] is optional

The one proprietary software tool ecosystem we reviewed, the IBM CLM

suite, did not employ a Markdown language. However, it still allowed explicit

links to be created between projects using a graphical user interface. When

entering text in a comment, users can click on a button that will allow them to

insert a link to a work item. This will open a window which will allow the user

to search for the work item(s) they wish to link to.

For all of these tools, whether they employ a Markdown language or not,

the format of the cross-references in the comment follows the same high-level

pattern with an optional project identifier followed by the artifact type and

the artifact identifier. The project identifier is optional in the cases that all

projects in the ecosystem share the same instance of the project management

tool, resulting in unique artifact identifiers across all of the projects, as is the

case in the IBM CLM ecosystem. Given the commonality of the syntax across all

of these tools, such cross-references can be automatically extracted. Thus, our

Reference Coupling method identifies dependencies by considering the following

pattern in comments:

<project identifier (optional) ><artifact type ><artifact identifier >

The detailed syntax for the various tools and artifact types is shown in Table 1.

If this method were to be implemented in a tool, these cross-references could

9

be automatically stored in a separate database table when they are created,

making the detection of dependencies nearly real-time. Post-hoc analysis, where

cross-references have not been previously extracted and stored, could be done

in O(n) time where n is the number of comments to be analyzed since one

computation is required for each comment to examine whether there is a cross-

reference to another project.

4. Research Methodology

4.1. Research Questions

We validated the Reference Coupling method on two datasets — one from

open-source projects hosted on GitHub and the one commercial dataset of IBM

projects. Our validation was guided by the following research questions:

RQ1a: Does the Reference Coupling method identify inter-project technical de-

pendencies on GitHub issue, pull request, and commit comments? If so, what

are the characteristics of these dependencies?

RQ1b: Does the Reference Coupling method identify inter-project technical de-

pendencies on IBM work item comments? If so, what are the characteristics of

these dependencies?

We then explored the characteristics of as well as socio-technical alignment

within the identified GitHub ecosystems by asking: RQ2: What ecosystems

exist across GitHub-hosted projects and what is their structure?, and RQ3: Do

the project owners’ and contributors’ social behaviours align with the technical

dependencies?

4.2. Research Setting and Data Collection

To answer our research questions, we conducted an analysis of the comments

and cross-references in the GitHub and IBM CLM projects.

4.2.1. GitHub

We obtained data from the GHTorrent [32] project, which provides a mirror

of the GitHub API data. GHTorrent obtains its data by monitoring and record-

10

ing GitHub events as they occur. We used the MySQL 2014-04-02 dataset to

obtain information on the projects since this paper extends our previous con-

ference publication which used this dataset [16]. This dataset contains data on

2,399,526 repositories, 3,426,046 users, and their events — including commits,

issues, pull requests and comments. We define a project as a repository and all

of its forks as recommended by Kalliamvakou et al. [33].

Since the MySQL database contains only the first 256 characters of com-

ments, we obtained all comments from GHTorrent’s main MongoDB server in

May 2014. The MongoDB contains the full text of all comments. These com-

ments were downloaded and stored in a PostgreSQL database for analysis. No

pre-processing was needed.

Using our Reference Coupling method, we identified 89,784 comments in

the GitHub data with a cross-reference to another project1. There are 29,018

repositories (18,533 unique projects when forks are considered) that make a

cross-reference to another project. While this is only a small portion of the

total number of repositories in our dataset, this is expected since Kalliamvakou

et al. [33] have found that the majority of the projects on GitHub are personal

and inactive.

4.2.2. IBM Collaborative Lifecycle Management (CLM)

We collected data from the products in the IBM Rational solution for Collab-

orative Lifecycle Management (CLM). CLM brings together requirements man-

agement, quality management, change and configuration management, project

planning and tracking on common uniform platform. CLM consists of number

of products including Rational Team Concert (RTC), Rational Quality Man-

ager (RQM), Rational DOORs Next Generation (DNG), Rational Requirement

Composer (RRC), Rational Software Architect (RSA), Rational Rhapsody and

Rational insight. The IBM CLM ecosystem is broken into 16 distinct projects.

1These cross-references and the scripts used to identify them are available in a replication

package at https://doi.org/10.5281/zenodo.2555526

11

MethodValidation

MethodApplication

RQ1a:GitHub RQ1b:IBM

TwoCoders

Manualanalysisof200randomcross-references•  Technicaldependency?•  Ifyes,typeofdependency?

CompareIBMDeveloperspecifieddependenciestocross-references

Manualanalysisof111cross-referencesnotdeclaredasdependencybydevs

RQ2 RQ3

NKHBMMBH�NFUFPS�CPPUTUSBQ��

ZBCBXPDL�CPPUTUSBQ�TBTT�SBJMT

UXCT�CPPUTUSBQUXJUUFS�SFDFTT

UPED�UPED�CPPUTUSBQ'PSU"XFTPNF�'POU�"XFTPNF

VR�FSFTFBSDI�BPSSB

OFUEOB�CPPUTUSBQ�DEO

.BY$%/�CPPUTUSBQ�DEO

EBSU�MBOH�EBSUMBOH�PSH

SPVHF��'POU�"XFTPNF

Usecross-referencestoconstructatechnicaldependencynetwork

IdentifyecosystemsusingLouvaincommunitydetectionmethod[11]

Characterizeecosystemsby:•  Manualanalysisofvisualizations•  Computenetworkstatistics•  ExamineprojectREADMEfiles

Constructproject-to-projectnetworksbasedonthe“following”and“starring”activityoftheprojectownersandcontributorsCompareedgeweightsofthesesocialnetworkswiththeedgeweightsfromthedependencynetwork

Figure 1: Summary of Research Methods

For each CLM project, we collected data on all work items created from

2005 to 2015. A work item is a task or an issue that must be attended to

during development. The data was downloaded in XML format from the IBM

REST API and converted into JSON and stored in PSQL tables. For each work

item, we collected the metadata, history, and comments. No pre-processing was

done on the data. The dataset consisted of 3,009 work items with a total of

17,708 comments. There were 635 cross-references to another project within

those comments. All projects contain cross-references.

4.3. Research Methods

An overview of our research methods to answer each research question are

shown in Figure 1 and described in detail in the following subsections.

12

4.3.1. Reference Coupling: Method Validation

To validate the Reference Coupling method, we used the method to identify

cross-references between projects in both GitHub and the IBM CLM products.

We automatically extracted these cross-references with pattern matching using

Java Regular expressions [34]. Since we are interested only in relationships

between projects, we filtered the cross-references to ignore references within the

same project.

RQ1a: Does the Reference Coupling method identify inter-project technical

dependencies on GitHub issue, pull request, and commit comments? If so, what


To verify that the cross-references to other projects made in comments on

GitHub identified through the Reference Coupling method are a valid concep-

tualization of dependencies, we examined 200 random comments which were

classified as dependencies using the Reference Coupling method since they cross-

referenced another project. Di↵erent types of comments may be made on di↵er-

ent artifacts. To ensure our analysis included various types of cross-references

that may occur, we ensured our randomly selected 200 comments were equally

distributed across each of the following relationships: 1) commit comment cross-

references another commit, 2) commit comment cross-references an issue or pull

request, 3) an issue or pull request comment cross-references a commit, and 4)

an issue or pull request comment cross-references another issue or pull request.

Thus, there were 50 random comments selected from each of these types of

cross-references.

This manual content analysis [35] was performed by two people familiar with

software development practices. They classified a comment as a technical de-

pendency if the comment described a work dependency, either direct or indirect,

between the two projects. For each dependency, they also noted if the depen-

dency was direct (between the two projects) or indirect (both projects depend

on a third project).

The same two people further examined the comments that were classified as

13

technical dependencies to identify the types of dependencies that are identified

using the Reference Coupling method. The dependencies were classified using

common dependency types that can be declared in issue tracking systems:

• Duplicate: the issue/commits on the two projects are duplicates of each

other. Within a project, duplicate issues would describe the same problem.

Across projects, an example of a duplicate issue could be both projects

have created issues to deal with a breaking API change from a shared

dependency.

• Blocking: an issue/commit on one project is blocking work in the other

project. For example, one project is waiting for the other project to release

a promised API change before they can finalize a new feature that will be

enabled by that API change.

• Resolving: an issue/commit on one project resolves an issue in the other

project. For example, one project had security issues which it has inherited

from a project it depends on. Once the other project fixes its security

issues, the issue will be resolved in the dependent project as well.

• A↵ecting: an issue/commit on one project is impacted by an issue/commit

on the other project. In other words, changes need to be made in the first

project due to changes made in the other project. For example, a project

deprecates an old API which would cause any projects using that old API

to update to a more recent API.

When a dependency did not fit one of these categories, open coding was used

to identify the type of dependency [36]. The cross-references that did not match

one of the pre-defined dependency types were reviewed and conceptually similar

comments were grouped into categories. This resulted in two new dependency

categories being introduced, Leveraging and Updating, which are described in

Section 5. Each coder independently did the manual analysis and coding. Then

the two coders met to discuss their results and try to come to a consensus.

The coders were able to come to a consensus for all items after this discussion,

14

Table 2: Inter-coder Reliability: Kohen’s KappaInitial Agreement Initial Cohen’s Kappa Final Agreement Final Cohen’s Kappa

Existance of Dependency 99% 0.828 100% 1

Direct/Indirect 97% 0.807 100% 1

A↵ecting 88.5% 0.678 100% 1

Blocking 98.5% 0.911 100% 1

Duplicate 96.5% 0.895 100% 1

Leveraging 98.5% 0.660 100% 1

Resolving 92% 0.826 100% 1

Updating 98.5% 0.816 100% 1

resulting in 100% agreement. Table 2 shows the inter-coder reliability using

Kohen’s Alpha for each of the categories for the intial independent coding and

the final agreed upon codes. Cohen’s alpha was calculated using ReCal [37].

RQ1b: Does the Reference Coupling method identify inter-project technical

dependencies on IBM work item comments? If so, what are the characteristics

of these dependencies?

To validate the Reference Coupling method identifies true technical depen-

dencies between the IBM projects, we are able to compare the dependencies

identified using the Reference Coupling method to the developer declared de-

pendencies. For each work item, the metadata contains information on depen-

dencies between the work items captured by the developers along with the type

of dependency. IBM developers can choose between 26 established dependency

classifications such as Depends On, Blocks, Duplicate Of, and Resolves.2 Of

the 26 classification, only 14 are used by the IBM developers in our dataset.

These 14 dependency classifications are described in Table 3. We grouped the

classifications into types, since there are pairs of classifications which represent

reciprocal relationships.

We analyze these developer specified dependencies to see how many were

also identified by the Reference Coupling method. In addition, we also manu-

ally analysed the cases where the Reference Coupling method identified a depen-

dency, but the IBM developers did not indicate the dependency within the work

2https://jazz.net/help-dev/clm/index.jsp?topic=%2Fcom.ibm.team.concert.sdk.doc%2Ftopics%2Fr link domains.html

15

Table 3: Types of Dependencies

Dependency Type Classification Description

BlockingBlocks The work item blocks work item X

Depends on The work item depends on work item X

ResolvingResolves The work item resolves work item X

Resolved by The work item is resolved by work item X

DuplicateDuplicated by The work item is duplicated by work item X

Duplicate of The work item is a duplicate of work item X

A↵ectingA↵ected by Defect A work item is a↵ected by a defect

A↵ects Plan Item The work item impacts plan item X

Parent/ChildParent The work item is a parent of work item X

Children The work item is a child of work item X

RelatedRelated The work item has a general relationship with work

item X

Related Change Request The work item is related to a change request item

PlanningContributes to The work item contributes to work item X

Tracks The work item tracks work item X

item. We analysed these cases to determine if the Reference Coupling method

identifies cases of true technical dependencies that had not been marked as such

by the IBM developers. In the case that the manual analysis revealed a technical

dependency, we also categorized the type of dependency using the same cate-

gories as used by the IBM developers to understand what types of dependencies

Reference Coupling captures.

4.3.2. Reference Coupling: Method Application

RQ2: What ecosystems exist across GitHub-hosted projects and what is their

structure?

To illustrate the applicability of our method, we constructed a network of

the technical dependency relationships established through Reference Coupling

as described in Section 3. The Dependency Network is defined as a directed

graph Gd =< V,E >. The set of vertices, denoted by V , is all GitHub projects

involved in at least one cross-reference. There are 18,533 projects in this set.

The set of edges, denoted by E, is a set of node pairs E(V ) = {(x, y)|x, y 2

V }. If the project represented by node xi cross-referenced the project repre-

16

sented by node yj , there is a directed edge from xi to yj . The weight of each edge

is the count of cross-references for the pair of projects. We filtered the edges

to only consider dependencies between nodes if the pair of projects have been

cross-referenced two or more times to capture only the stronger dependencies.

It is important to note that this directed graph captures the direction of

the cross-referencing comments and not the direction of the dependencies that

those comments imply. A project could cross-reference another project because

it is blocked by that project or because it is blocking that project. The nuances

of the dependency direction are not captured by our method.

To identify ecosystems across projects hosted on GitHub, we used the pop-

ular Louvain community detection method [15] on the Dependency Network

established through Reference Coupling. The Louvain method is a greedy opti-

mization method that aims to partition a network into communities of densely

connected nodes and optimize the modularity of the network. Modularity is de-

fined as “the number of edges falling within [communities] minus the expected

number in an equivalent network with edges placed at random [38].” High mod-

ularity scores indicate that there are dense connections within the communities

but sparse connections across communities, showing that an optimal solution

has been found. When high modularity scores are obtained, the communities

have significant real-world meaning [15]. The Louvain method is comprised of

two steps. It first optimizes modularity locally by looking for small communi-

ties. Then it aggregates the nodes in each small community and builds a new

network with these aggregated nodes. It iterates on these two steps until the

modularity is maximized. The Louvain method outperforms all other commu-

nity detection methods in terms of both the modularity that is achieved and

the computation time [15].

In our network, the identified communities represent sets of projects densely

connected by technical dependencies. Since dependencies that exist between

projects define the structure of an ecosystem [5], these communities represent

software ecosystems.

To identify properties of the identified ecosystems, we:

17

• Analyzed visualizations of the Dependency Network. Visualizations of

each ecosystem detected by the Louvain community detection method (as

described above) were manually reviewed. We used the Gephi [39] graph-

ing tool to create these visualizations. One of the authors inspected these

visualizations of the network to visually identify patterns. The identified

patterns were cross-checked by two of the other authors.

• Computed network statistics for each of the ecosystems, such as in-degree

and out-degree of the nodes.

• Examined the types of projects involved in the ecosystems by reviewing

the GitHub README files of the most well-connected project node in

each of the ecosystems.

RQ3: Do the project owners’ and contributors’ social behaviours align with

the technical dependencies?

To complement our investigation of technical dependencies and connect-

edness of projects in GitHub, we also sought to understand the social be-

haviour of project owners/contributors in relation to the ecosystems we identi-

fied. We studied two of GitHub’s social relationships, following users and star-

ring projects. On GitHub, users can follow other users to receive notification

on their activity and star a repository to bookmark it or indicate interest in the

project. To understand how the social behaviour of project owners/contributors

relates to the identified ecosystems, we examine the alignment between social

and technical connections between the projects.

To answer RQ3, we construct project-to-project networks based on the fol-

lowing and starring activity of the project owners and contributors.

Project Owners. We constructed two networks using the following and star-

ring relationships by considering the actions of the project owners. The Owner

Stars Network, Gos =< V,E >, and the Owner Follows Network, Gof =<

V,E >, are both undirected graphs whose set of vertices is all GitHub projects

involved in at least one cross-reference. For the Owner Follows Network, there

is an edge from nodes xi to yj if the owner of project xi follows the owner of

18

project yj . There is an edge from xi to yj in the Owner Stars Network if an

owner of any project in our dataset has starred both project xi and project yj .

Project Contributors. We constructed two additional networks using these

following and starring relationships by considering the actions of the project

contributors (users who have made commits on the project or are members

of the project). The Contributor Stars Network, Gcs =< V,E >, and the

Contributor Follows Network, Gcf =< V,E >, are also undirected graphs whose

set of vertices is all GitHub projects involved in at least one cross-reference. The

Contributor Follows Network has an edge from nodes xi to yj if a contributor of

project xi follows a contributor of project yj . The Contributor Stars Network

has an edge from xi to yj if a contributor to any project in our dataset has

starred both project xi and project yj .

To compare the social connections with the technical dependencies, we com-

pare the edge weights of these two networks with the edge weights of the De-

pendency Network constructed to answer RQ2. Pearson correlations were used

since the data was normally distributed. The edge weights of these networks

represent the following:

• Dependency Network Gd: Number of technical dependencies, measured

through Reference Coupling, between the two project nodes.

• Owner Follows Network Gof : 0 if neither project owner follows the other,

1 if one project owner follows the other project owner, and 2 if both project

owners follow each other.

• Owner Stars Network Gos: Number of project owners who have starred

both projects.

• Contributor Follows Network Gcf : Number of contributors with following

relationships for the pair of projects.

• Contributor Stars Network Gcs: Number of project contributors who have

starred both projects.

19

5. Results

5.1. Method Validation on GitHub Data

RQ1a: Does the Reference Coupling method identify inter-project technical

dependencies on GitHub issue, pull request, and commit comments? If so, what


Of the 200 examined cross-references, we obtained 96.5% precision as 193

were found by the two manual coders to be true technical dependencies. We are

unable to calculate recall of our method since we do not have a ground truth

set of all dependencies.

Of these 193, 176 (91%) were direct dependencies between the two projects

and 17 (9%) were indirect dependencies where the two projects both depend on

a third project.

Dependency between the two projects. The most common type of dependency

found was a direct technical dependency between the two projects. An example

of a direct technical dependency is when an issue created in one project depends

on a fix/update in another project. Another example is when a project needs

to be updated based on changes made in another project.

Below we provide three examples of cross-reference comments that are in-

dicative of direct technical dependencies between the two projects. Project

names follow the pattern user/repository where user is the owner’s GitHub lo-

gin and repository is the name of the project repository.

Issue #449 on the sensu/sensu project describes an issue that is the result

of the interaction between the sensu/sensu code and the ruby-amqp/amq-client

library. The comment references a commit on the ruby-amqp/amq-client that

fixes the issue.

“I verified that the problem is still the one referenced in ruby-amqp/amq-

client#14. This fix is not merged with amq-client’s ‘0.9.x-stable’

branch. This is why I am still hitting it. The commit ruby-amqp/amq-

client@60f1c59 is the fix but it resides only in the master branch.”

20

Issue #8 on the tsujigiri/axiom project notes that changes must be made to

the code base to allow an upgrade to the latest release of the ninenines/cowboy

project.

“Upgrade Cowboy: After Cowboy 0.6.1 Cowboy’s http req record was

made opaque and can not be used directly anymore. I didn’t really

have the time yet to look into it, but it looks like we just need to

remove all references to the record from the documentation and add

directions on how to access cowboy req:req() via the cowboy req func-

tions. See ninenines/cowboy#266 and ninenines/cowboy#267.”

Commit 81bbbec21c04b6392f6892f7735243387d295337 on the joyent/node

project closes isaacs/node-graceful-fs issue #6, which describes a problem in

the isaacs/node-graceful-fs code stemming from the use of joyent/node. GitHub

allows automatic closure of issues through commit comments, even when the

commit is in a di↵erent repository.3

“This fixes isaacs/node-graceful-fs#6.”

Both projects depend on a third project. We also identified some cases

where the comments describe a dependency on a third project that is not cross-

referenced. For example, everzet/capifony’s pull request #376 cross-references

composer/composer’s issue #1453, but the problem stems from the use of the

symfony/symfony project. After identifying the source of the problem, a new is-

sue (#411) is created on the everzet/capifony project that identifies the changes

that need to be made to the way the symfony environment is set so that the

composer/composer code executes correctly.

“As described in #376 capifony should execute composer with the

right symfony environment set. Currently, with --no-scripts op-

tion removed in #376, composer is always executing symfony scripts

with default dev environment.”

3https://github.com/blog/1439-closing-issues-across-repositories

21

Table 4: Types of Dependencies Identified by Reference Coupling

Dependency Type Count (Ratio)

Resolving 74 (38.3%)

Duplicate 43 (22.3%)

A↵ecting 43 (22.3%)

Blocking 18 ( 9.3%)

Updating 9 ( 4.7%)

Leveraging 6 ( 3.1%)

5.1.1. Dependency Categories

Most (178 of 193 or 92%) of the dependencies were able to be assigned to

one of the four pre-existing dependency types (duplicate, blocking, resolving or

a↵ecting). The remaining cross-references were examined and open coding was

used to generate additional dependency categories. Only two new categories

were created by the two manual coders, both of which are specific to cross-

project dependencies:

• Leveraging: the two projects depend on a third project and both experi-

ence the same issue due to this shared dependency. One of the projects

leverages a solution to this problem that has been generated by the other

project.

• Updating: one of the projects depends on the other project and is updating

to a more recent version of the other project.

Table 4 shows the breakdown of how often each dependency type appeared

in our data. Resolving is the most common dependency type.

Answer to RQ1a: The Reference Coupling method does identify inter-

project technical dependencies on GitHub pull request, issue and commit

comments; 96.5% of manually analyzed comments revealed technical de-

pendencies between the cross-referenced projects. Of these, most are direct

22

Table 5: Types of Dependencies Not Identified by IBM Developers.

Dependency Type Count

Duplicate 57

Related 42

Parent/Child 28

A↵ecting 10

Blocking 6

Planning 2

Resolving 1

dependencies between the two projects, but some are a shared dependency

on a third project. We further classified these dependencies and found that

the Reference Coupling method identifies a variety of dependency types.

The most common type of dependency identified is Resolving: where a

change in one project resolves an issue in the other project.

5.2. Validating Method on IBM Data

RQ1b: Does the Reference Coupling method identify inter-project technical

dependencies on IBM work item comments? If so, what are the characteristics

of these dependencies?

The Reference Coupling method identified dependencies between 2108 pairs

of work items, 635 of which are inter-project dependencies. Of the inter-project

dependencies, 146 (23%) were marked as technical dependencies by the IBM

developers. Table 5 shows the types of dependencies that were both identified

by Reference Coupling and the IBM developers. The most common types of

dependencies that were identified by both the Reference Coupling method and

the IBM developers are duplicates, related, and parent/child relationships.

However, there were 489 inter-project dependencies identified by the Ref-

erence Coupling method which were not marked as dependencies by the IBM

23

developers. To validate that our Reference Coupling method was, in fact, identi-

fying dependencies, despite the fact that not all were marked as such within the

repository, two coders familiar with software development practices reviewed

a random sample of 111 of the 489 work items and manually assessed if the

Reference Coupling method identified a valid technical dependency. The two

coders achieved a 95% inter-coder reliability. They discussed any cases where

their coding did not align and came to a concenus.

Of the 111 work items, all were found to have dependencies with other

projects by the coders. The coders also categorized these 111 dependencies using

the dependency types in Table 3. They also used an additional type (Unknown)

to label dependencies that did not easily fit into one of the 26 pre-established

dependency types.

Table 6 shows the types of dependencies that were evident among the pairs

of work items. The most common dependency type that was identified by our

Reference Coupling method but was not marked as a dependency by the IBM

developers was ‘A↵ected by Defect’. For example, this comment describes how a

defect, which is associated to a di↵erent project, caused problems on the current

project.

“The Validate is not defined error from comment 2 was fixed in defect

162650.”

Another common dependency type identified by Reference Coupling but not

flagged as a dependency by the developers is ‘Related’. For example,

“This is related to item 125838.”

The links between projects are not as clearly seen in the textual comments

in IBM CLM, since the project associated with the work item is only available

in the metadata on the work item. In these cases, it would be quite easy for

developers to be unaware that a coordination need with another project exists

since the fact that these work items are part of another project is not apparent.

24

Table 6: Types of Dependencies Not Identified by IBM Developers.

Dependency Type Count

A↵ecting 73

Related 30

Planning 1

Blocking 1

Unknown 6

Answer to RQ1b: The Reference Coupling method does identify inter-

project technical dependencies on IBM work item comments. It identifies a

wide variety of dependency types. The most common type of dependency

that is found with the Reference Coupling method but missed by the IBM

developers is the ‘A↵ecting’ category.

5.3. Applying Reference Coupling to identify and examine GitHub ecosystems

5.3.1. Ecosystem Identification

RQ2: What ecosystems exist across GitHub-hosted projects and what is their

structure?

Figure 3 shows the full Dependency Network, though for visibility we only

display nodes with degree of 3 or greater. As visible on the graph, most of

the nodes (10,484 of 18,533 projects or 57%) are part of the largest connected

component (commonly referred to as the giant component [40]), which is the

largest subgraph in which every node is connected to every other node by some

path. The connected components isolated from the giant component are primar-

ily comprised of same owner communities in which all nodes in the connected

component are projects owned by the same GitHub user or organization. For

example, the second largest connected component is comprised of 65 nodes, of

which, all but two are owned by GitHub user deathcap.4 Most of the nodes

4https://github.com/deathcap

25

0500

1000

1500

2000

2500

With Outliers

12

34

56

78

Outliers Removed

Figure 2: Number of unique cross-referenced projects for all projects that make at least one

cross-reference to another project on GitHub.

isolated from the giant component are connected to only a small number of

nodes. In fact, 75% of nodes not in the giant component are connected to only

one other node.

Figure 2 shows boxplots for the number of unique cross-references for each

project which cross-references at least one other project. These boxplots show

that while some projects make cross-references to many other projects, most

projects have only a small number of other projects which they cross-reference.

Since we are most interested in studying the popular GitHub ecosystems, we

focus our analysis on the interconnected part of the network or the giant com-

ponent. Figure 4 shows the giant component. The color of the nodes represent

26

Figure 3: All GitHub projects with cross-references. The largest connected component (or

giant component) is easily identified as the well-connected subgraph appearing in the center

of the graph.

communities as detected by the Louvain method. We obtained a modularity

score of 0.913 (out of a possible range of 0 to 1). This high modularity score

indicates that the detected communities are much more tightly connected by

technical dependencies than would appear in a random graph.

There were 43 ecosystems identified in this network. Nodes are sized ac-

cording to their authority to display the nodes that are more prominent in each

ecosystem. When a node has a high number of cross-reference relationships

pointing to it, it has a high authority value [41]. Table 7 shows the most well-

connected project node (highest Authority value) in each of the ecosystems.

Properties of GitHub Ecosystems

Ecosystems revolve around one central project. As depicted in Figure 4, each

ecosystem appears to revolve around one main project. In Table 7, the most

well-connected project node in each ecosystem is listed along with a description

27

BTUSPQZ�BTUSPQZ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE%C

KRVFSZ�KRVFSZ�XQ�DPOUFOU

USPPQKT�USPPQKT�CSPXTFS

NJUDIFMMI�WBHSBOU

HFUMBOUFSO�MBOUFSO�VJ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE5FTU

USPPQKT�USPPQKT�CVOEMF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE$POGJH

[FOEGSBNFXPSL�$PNQPOFOU@;FOE%PN

QOPSNBO�PQFOTUSFFUNBQ�DBSUP

[FOEGSBNFXPSL�$PNQPOFOU@;FOE)UUQ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE7JFX

GBDFCPPL�IIWN

6OJPO0G3"%�GSBNFXPSL

KRVFSZ�RVOJU

)PNFCSFX�MJOVYCSFX

BMFYNNN�KTPO�TDIFNB

TIJSBZVLJ�PQFOTUSFFUNBQ�XFCTJUF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE7FSTJPO

.BKLM��OFUUF

EBWJEEBWJT�LBUFMMP�DMJ


SFGJOFSZ�SFGJOFSZDNT

ENBTDJB�3FNJY$IBSU

[DCFO[�DISPNJVN

VTF�JOJU�JOJU

5PP5BMM/BUF�OPEF�HZQ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE-PBEFS

FDPNGF�FEQ�CVJME

PXODMPVE�BQQT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE4FTTJPO

USBWJT�DJ�USBWJT�DJ�HJUIVC�DPN

+PTIVB�"OEFSTPO�EQM

NBDHJUWFS�NBDHJUWFS

USBWJT�DJ�USBWJT�DPPLCPPLT

TDPUUHPO[BMF[�KRVFSZ

TZNGPOZ�%PDUSJOF#SJEHF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE$BDIF

[BSHPOZ�SVTU�GVTF

TUFWFOHK�1Z1MPU�KM

LPSBLUPS�TUFBN�DPOEFOTFS�KBWB

QBESJOP�QBESJOP�EPDT

+VMJB-BOH�KVMJB

EPDUSJOF�%PDUSJOF.POHP%##VOEMF

QMMJN�QZTZOQIPU

GC��IUNMQBSTFS�

KBXTIPPBI�IPNFCSFX

[FOEGSBNFXPSL�$PNQPOFOU@;FOE$PEF

USPPQKT�USPPQKT�DPSF

NBSDK�1SPQFM�

DIBQNBOC�DMPVECJPMJOVY

TQSFF�TQSFF@HBUFXBZ

KSCVSLF�S�KT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE'PSN

DIBQNBOC�CDCJP�OFYUHFO�WN

[FOEGSBNFXPSL�$PNQPOFOU@;FOE%FCVH

LBSNJ�SVCZHFNT�BXT

PQEFNBOE�EFJT�DPPLCPPL

[FOEGSBNFXPSL�$PNQPOFOU@;FOE$POTPMF

TZNGPOZ�TZNGPOZ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE%J

BTTBG�WBOJUZ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE6SJ

TDBNCSB�EFWJTF@JOWJUBCMF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE4FSWJDF.BOBHFS

TOBSGFE�BDUJWJUZTUSFBNT�VOPGGJDJBM

XIT�DISPNJVN�TSD

NPOHPJE�NPQFE

[FOEGSBNFXPSL�$PNQPOFOU@;FOE4UEMJC

QKBDLTPO��UIFNF�HD�JOUSBOFU

CBTIP�SJBL@LW

,BUFMMP�LBUFMMP�DMJ

KBTILFOBT�DPGGFF�TDSJQU

LFWJO�TNFUT�BOHVMBS�CMFFE

MFPCBMUFS�RVOJUKT�DPN

[FOEGSBNFXPSL�[G�

XFU�CPFX�UIFNF�CBTF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE.WD

)PNFCSFX�IPNFCSFX

FMBTUJDTFBSDI�FMBTUJDTFBSDI�SVCZ

+VMJB-BOH�KVMJBMBOH�HJUIVC�DPN

MPTUJTMBOE�GBSBEBZ

USBWJT�DJ�USBWJT�CVJME

&SJD%VOTXPSUI�UIFNF�HD�JOUSBOFU

XFCQBDL�MFTT�MPBEFS

BMSSB�TFSWFS�DPOGJHT

LBDIJDL�SVCJOJVT

USBWJT�DJ�USBWJT�DPSF

NBDHJUWFS�MJC(JU8SBQ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE+TPO

*SPO-BOHVBHFT�NBJO

EDFTUBSJ�5FYU4FDVSF�J04

HBSFUIS�WBHSBOUCPYFT�IFSPLV

[FOEGSBNFXPSL�$PNQPOFOU@;FOE#BSDPEF

USPPQKT�USPPQKT�EBUB

XZVFOIP�CBDLHSJE�GJMUFS

OBSFTIW�IIWN

CVOEMFS�CVOEMFS

OJDLM��DPNQPTFS

[FOEGSBNFXPSL�$PNQPOFOU@;FOE1BHJOBUPS

LPSBLUPS�TUFBN�DPOEFOTFS�SVCZ

MPTUUJNF�TUFBN�DPOEFOTFS�QIQ

NBQCPY�NJMMTUPOF

IBDLFO�JO�QVQQFU�IBDLFOJO

XBZOFFTFHVJO�SWN

[FOEGSBNFXPSL�$PNQPOFOU@;FOE"VUIFOUJDBUJPO

OPGMP�OPGMP�VJ

OVMMUPLFO�MJCHJU�

DIBQNBOC�IPNFCSFX�DCM

TZNGPOZ�DNG�.FOV#VOEMF

TZNGPOZ�'SBNFXPSL#VOEMF

CPSMBOE�JSPOMBOHVBHFT�NBJO

[FOEGSBNFXPSL�$PNQPOFOU@;FOE5BH

UIFLJE�YQ�GSBNFXPSL

JOEJSFDU�FNCFS�SBJMT

I�CQ�TFSWFS�DPOGJHT�BQBDIF

NJSBHF�PQBN�SFQPTJUPSZ

SPHFSXBOH�OPEF

0DSBNJVT�EPDUSJOF�

QMFOU[�KSVCZ@SFQPSU

[FOEGSBNFXPSL�$PNQPOFOU@;FOE4FSJBMJ[FS

FVSPEFW�TZNGPOZ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE9NM3QD

$PDPB1PET�$PSF

TBUZS�DPDP

ZZY��OQN

YQ�GSBNFXPSL�SFNPUF

SBEBS�TQSFF@HBUFXBZ

USPPQKT�USPPQKT

IIBUUP�BVUPQFQ�

HFUMBOUFSO�MBOUFSO�DPOUSPMMFS

3FE8PMWFT�KRVFSZ�XQ�DPOUFOU

DISJT�NPSHBO�SVTU�IUUQ

KPZFOU�OPEF

USBWJT�DJ�EPDT�USBWJT�DJ�DPN

XBZOFFTFHVJO�SWN�UFTU

[FOEGSBNFXPSL�$PNQPOFOU@;FOE.FNPSZ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE.PEVMF.BOBHFS

4-BLT�4-BLT�#MPH

BMQIBHPW�TUBUJD

XFU�CPFX�UIFNF�HDXV�GFHD

.JDBI$IBMNFS�SVTU�GVTF

HSVOUKT�HSVOU�DPOUSJC�SFRVJSFKT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE'JMFI�CQ�TFSWFS�DPOGJHT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE-PH

YQ�GSBNFXPSL�YQ�GSBNFXPSL

QBSOEU�EFDPSBUPST

[FOEGSBNFXPSL�$PNQPOFOU@;FOE-EBQ

JTBBDT�OQN

[FOEGSBNFXPSL�$PNQPOFOU@;FOE*OQVU'JMUFS

HSVOUKT�HSVOU�DPOUSJC�DPNQBTT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE/BWJHBUJPO

#FIBU�.JOL4FMFOJVN�%SJWFS

DPTN�KTIJOUSC

0DSBNJVT�DPNNPO

JFMHOBX�FEQ�EPDUPS

ZBIPOEB�PSBDMF�FOIBODFE

TGFSJL�SBJMT@BENJO

MPWBOXVCJOH�S�KT

HSFHCFMM�BDUJWF@BENJO

1PMZNFS�QSPKFDUT

QKBDLTPO��UIFNF�HDXV�GFHD

TQSFF�TQSFF

USBWJT�DJ�USBWJT�BQJ

USPPQKT�USPPQKT�VUJMT

WUTWBOH�HSVOU�DPOUSJC�OPEFVOJU

3FTU,JU�3FTU,JU

UFOEFSMPWF�QTZDI

[FOEGSBNFXPSL�$PNQPOFOU@;FOE*��O

XFCQBDL�KBEF�MPBEFS

CBTIP�SJBL@BQJ

0$BNM1SP�PQBN�SFQPTJUPSZ

SBEBS�TQSFF

MFPCBMUFS�KRVFSZ�XQ�DPOUFOU

%JF#VDIF�PQFOTUSFFUNBQ�XFCTJUF

CBKUPT�MJCVW

TZNGPOZ�TZNGPOZ�EPDT

4QBSLT�$SFBUJWF�-JNJUFE�BOHVMBS�KT

PQFOJE�SVCZ�PQFOJE

SBJMT�SBJMT

HFUMBOUFSO�MBOUFSO@BXT

NJDIBFMGJDBSSB�$PGGFF4DSJQU3FEVY

CPPUTUSBQPOMJOF�HPMMVN�MJC

BWTN�NJSBHF

FDPNGF�FEQ�EPDUPS

IFOSJLIPEOF�USBWJT�DPSF

KPZFOU�MJCVW

8J[DPSQ�EJTUSJCVUF

EPDUSJOF�NPOHPEC

NPOHPJE�NPOHPJE�TJUF

IVNEFEVN�IPNFCSFX

DVQ�PG�HJSBG�TZNGPOZ

[FOEGSBNFXPSL�;'�1BDLBHF

NJDIBFMGJDBSSB�DPNNPOKT�FWFSZXIFSF

NBQOJL�OPEF�NBQOJL

)PNFCSFX�IPNFCSFX�TDJFODF

OVLPNFFU�DPXPSGJOH

LTXFECFSH�HSVOU�KRVFSZ�DPOUFOU

+VMJB4UBUT�%BUB"SSBZT�KM

QKBDLTPO��UIFNF�PHQM

,BUFMMP�LBUFMMP

NPOP�NPOPNBD

ECBTIGPSE�NJNPTB�DMJFOU�KBEF�TUBUJD

PXODMPVE�EPDVNFOUBUJPO

[FOEGSBNFXPSL�$PNQPOFOU@;FOE7BMJEBUPS

BN��8FC&TTFOUJBMT��

KSVCZ�KSVCZ

DPNQPTFS�HFUDPNQPTFS�PSH

[FOEGSBNFXPSL�$PNQPOFOU@;FOE'FFE

SCSBOTPO�OPEF�GGJ

NQBQJT�SVCZHFNT�CVOEMFS

HFPSHFUIPNBT�OPEF�SFTUJGZ

1PMZNFS�1PJOUFS(FTUVSFT

BWTN�IPNFCSFX

IFVFSNI�DMPVECJPMJOVY

EDKPOFT�(BEGMZ�KM

NBQOJL�NBQOJL�QBDLBHJOH

OUPNLB�CPMU

GBCQPU�5XJH

1PMZNFS�MBCT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE'JMUFS

.JUIHPM�OPEF�QSF�HZQ

NBSJKOI�UFSO

[JNCBUN�IPNFCSFX�WFSTJPOT

EPDUSJOF�DPNNPO

LNDBMMJTUFS�TFSWP

4VQBDPDP�HFOFSBUPST

NBSJKOI�$PEF.JSSPS

ZJJ��DIJOFTJ[BUJPO�ZJJ��[I�DO

UIFLJE�YQ�DPOUSJC

TKBDLNBO�IPNFCSFX

)PNFCSFX�IPNFCSFX�WFSTJPOT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE.BJM

ECBTIGPSE�NJNPTB�SFRVJSF

BOHVMBS�VJ�VJ�NBQ

STJN�PSBDMF�FOIBODFE

STQFD�STQFD�SBJMT

SLI�EQM

GVTTM�QBTTNF

HFOFTJT�XPSEQSFTT

NYDM�IPNFCSFX

BSUIVSTDISFJCFS�SVHHFE

YQ�GSBNFXPSL�DPSF

YQ�GSBNFXPSL�SFTU

IDBUMJO�TBTT�TQFD

KRVFSZ�KRVFSZ�VJ

DIBODBODPEF�SBJMT

NFSL�$JUBEFM

DPXCPZ�HSVOU

ECBTIGPSE�MPHNJNPTB

HFUMBOUFSO�MBOUFSO�DPNNPO

NBQOJL�NBQOJL

JQZUIPO�JQZUIPO

'BCJFO1FOOFRVJO�TDBMBTUJD

SBEBS�TQSFF@BVUI@EFWJTF

XFCQBDL�XFCQBDL

[KIJQIPQ�USPPQKT�DPSF

FDPNGF�FEQ�NJOJGZ

TQSFF�TQSFF@SFWJFXT

$PDPB1PET�HVJEFT�DPDPBQPET�PSH

KRVFSZ�CVHT�KRVFSZ�DPN

TZNGPOZ�.POPMPH#VOEMF

BTUSPQZ�QBDLBHF�UFNQMBUF

CSBEU�XQ�NJHSBUF�EC

LFWWB�JNBHF�NJO

BTUSPQZ�QIPUVUJMT

#FIBU�.JOL&YUFOTJPO

KRVFSZ�CMPH�KRVFSZ�DPN�UIFNF

HSVOUKT�HSVOU�DPOUSJC�KTIJOU


BEIFBSTJPO�BEIFBSTJPO�J��O

4DJ5PPMT�JSJT�UFTU�EBUB

HSVOUKT�HSVOU�DPOUSJC�VHMJGZ

DVDVNCFS�NVMUJ@UFTU

QXOBMM�SBJMT��

LFOOFUISFJU[�HSFRVFTUT

TZNGPOZ�*DV

SPHFSXBOH�CMJOL

I�CQ�BOU�CVJME�TDSJQU

CBTIP�SJBL@FOTFNCMF

COPPSEIVJT�OPEF�JDPOW

I�CQ�OPEF�CVJME�TDSJQU

EBWJEBWEBW�/BNFE"SSBZ

QPSBEB�NJEEMFNBO�BVUPQSFGJYFS

STQFD�STQFD�JUT

WUKOBTI�0%&�KM

KDL�VIEM

$PDPB1PET�DPDPBQPET�JOUFHSBUJPO�TQFDT

0DSBNJVT�0DSB%J$PNQJMFS

PDBNMMBCT�PQBNGV

[FOEGSBNFXPSL�[G��EPDVNFOUBUJPO

NSFJE�NP[�T�GVOOFM

DBNCSJEHF�IFBMUIDBSF�EPDLFSGJMFT

BOM�LJUDIFO�HDF

JTBBDT�SFBE�JOTUBMMFE

SVCJOJVT�SVCJOJVT�BTU

+VMJB-BOH�KVMJB�JQZUIPO

JTBBDT�DMVTUFS�NBTUFS

0DSBNJVT�0DSB4FSWJDF.BOBHFS

UJBOPO�EPDLFSGJMFT

DPXCPZ�OPEF�FYJU

+VMJB-BOH�*UFSBUJWF4PMWFST�KM

BMFYDSJDIUPO�MJCVW

TZNGPOZ�5XJH#VOEMF

U[JOGP�U[JOGP�EBUB

KMBXT�+-"EESFTT#PPL

KSPXFCPZ�SVTU�NVTUBDIF

)PNFCSFX�IPNFCSFX�IFBEPOMZ

KRVFSZ�RVOJU�SFQPSUFS�KVOJU

STQFD�STQFD�NPDLT

BMQIBHPW�DJ�QVQQFU

KIFMXJH�UFDIOPTPSDFSZ�OFU

I�CQ�TFSWFS�DPOGJHT�OHJOY

KRVFSZ�HSVOU�KRVFSZ�DPOUFOU

HFUMBOUFSO�MBFQSPYZ

TFCBTUJBOCFSHNBOO�QIQVOJU�EPDVNFOUBUJPO

YQ�GSBNFXPSL�YQ�SVOOFST

#FIBU�.JOL4BIJ%SJWFS

KTEPD��KTEPD��HJUIVC�DPN

USBWJT�DJ�USBWJT�UBTLT

ECBTIGPSE�NJNPTB�UFTUFN�SFRVJSF

ZBBVJF�DMJWFS

QSPHSJVN�QMVHJOIPPL

HSVOUKT�HSVOUKT�DPN

XQ�DMJ�XQ�DMJ�HJUIVC�DPN

XQ�DMJ�QBDLBHF�JOEFY

DPNQPTFS�JOTUBMMFST

NBETLSJTUFOTFO�$TT4PSUFS

HSVOUKT�HSVOU�DPOUSJC�KBEF

4DJ5PPMT�JSJT�DPEF�HFOFSBUPST

HET�PQFSBUJPOT�QVQQFU�FMBTUJDTFBSDI

ZFPNBO�HSVOU�VTFNJO

BEPCF�CSBDLFUT�TIFMM

JBOXIJUF�PSN@BEBQUFS

ULOFSS�WBHSBOU�QMVHJO�CVOEMFS

,BUFMMP�LBUFMMP�JOTUBMMFS

LCIPNFT�HPPHMF�NVTJD�NBD

(FFL0O$PGGFF�TQSFF@QSPEVDU@TPSU

[FOEGSBNFXPSL�;FOE0"VUI

SMJEXLB�ZBQN

TZNGPOZ�)UUQ'PVOEBUJPO

TZNGPOZ�%PN$SBXMFS

DPSEPWBM�TZNGPOZ�TUBOEBSE

+FSFNZ-FUBOH�SVTU�TGNM

TFCBTUJBOCFSHNBOO�ECVOJU

EPDUSJOF�%PDUSJOF03..PEVMF

3JPU(BNFT�WBHSBOU�CFSLTIFMG

TPOBUB�QSPKFDU�4POBUB%PDUSJOF1IQDS"ENJO#VOEMF

BMQIBHPW�QBOPQUJDPO

QSPQFMPSN�QSPQFMPSN�HJUIVC�DPN

QSPQFMPSN�1SPQFM#VOEMF

)PNFCSFX�IPNFCSFX�EVQFT

SBJMT�TBTT�SBJMT

LSJTXBMMTNJUI�BTTFUJD

XFU�CPFX�XFU�CPFX�TUZMFHVJEF

STQFD�STQFD�DPSF [FOEGSBNFXPSL�$PNQPOFOU@;FOE4PBQ

BSJBUFNQMBUFT�BSJBUFNQMBUFT

ZFPNBO�ZFPNBO

LBSNB�SVOOFS�LBSNB

FNCFSKT�XFCTJUF

HFUMBOUFSO�MBOUFSO

XQ�DMJ�XQ�DMJ

NCPTUPDL�E�

WJTJPONFEJB�FYQSFTT

BEPCF�CSBDLFUT

USBWJT�DJ�USBWJT�DJ

DBSMIVEB�CVOEMFS

"'/FUXPSLJOH�"'/FUXPSLJOH

$PDPB1PET�4QFDT

LFOOFUISFJU[�SFRVFTUT

PXODMPVE�NJSBMM

GMPSJ�KTPO

,OQ-BCT�,OQ.FOV#VOEMF

ZFPNBO�HFOFSBUPS�BOHVMBS

KRVFSZ�XFC�CBTF�UFNQMBUF

USBWJT�DJ�EQM

$PDPB1PET�9DPEFQSPK

LFOOFUISFJU[�IUUQCJO

HNB�OFTUB

$PDPB1PET�$PDPB1PET

UXCT�CPPUTUSBQ

PDBNM�PQBN

HPMMVN�HPMMVN�MJC

HSVOUKT�HSVOU

FNCFSKT�FNCFS�KT

HSFBTFNPOLFZ�HSFBTFNPOLFZ

CPXFS�CPXFS

TFODIBMBCT�DPOOFDU

KPTFHPO[BMF[�IPNFCSFX�QIQ

CBTIP�CBTIP@EPDT

BOHVMBS�VJ�CPPUTUSBQ

NP[JMMB�SVTU

LPSBLUPS�TUFBN�DPOEFOTFS

$POTUFMMBUJPO�FTDPEFHFO

CBTIP�FMFWFMEC

PXODMPVE�DBMFOEBS

TZNGPOZ�"TTFUJD#VOEMF

TIBNB�HB[F

CBTIP�SJBL�FSMBOH�DMJFOU

TZNQZ�TZNQZ�CPU

QBESJOP�QBESJOP�GSBNFXPSL

MJCHJU��MJCHJU�

'SJFOET0G4ZNGPOZ�'046TFS#VOEMF

NJEEMFNBO�NJEEMFNBO

4IPQJGZ�BDUJWF@NFSDIBOU

NQFSIBN�TJEFLJR

FNCFSKT�EBUB

NBETLSJTUFOTFO�8FC&TTFOUJBMT��

NPKPNCP�KFLZMM

SVCZHFNT�SVCZHFNT

EPDVNFOUDMPVE�CBDLCPOF

OFY��TBTT

MJCHJU��SVHHFE

MJCHJU��PCKFDUJWF�HJU

NJSBHF�NJSBHF�XXX

.BD3VCZ�.BD3VCZ

KSVCZ�KSVCZ�SBDL

TFCBTUJBOCFSHNBOO�QIQVOJU�NPDL�PCKFDUT

DBPMBO�BTZOD

KPZFOU�IUUQ�QBSTFS

HSVOUKT�HSVOU�DPOUSJC�DPGGFF

(SPVQ5BMFOU�FQG

ZJJTPGU�ZJJ�

CFSLTIFMG�CFSLTIFMG

PDBNM�PQBN�SFQPTJUPSZ

-FBSO#PPTU�NPOHPPTF

DISJTFQQTUFJO�DPNQBTT

4ZMJVT�4ZMJVT

KOJDLMBT�DBQZCBSB

TFCBTUJBOCFSHNBOO�QIQVOJU

TJOBUSB�TJOBUSB�HJUIVC�DPN

KNSFJEZ�HSVOU�CSPXTFSJGZ

LBSNB�SVOOFS�HSVOU�LBSNB

CBTIP�OPEF@QBDLBHF

DFMMVMPJE�SFFM

CBTIP�SJBL@TFBSDI

POFDMJDL�SVCZJOTUBMMFS

BEPCF�CSBDLFUT�BQQ

HSVOUKT�HSVOU�DPOUSJC�DPQZ

BEPCF�CSBDLFUT�FEHF�XFC�GPOUT

FDPNGF�FEQ

TQSFF�TQSFF@TPDJBM

SVCJOJVT�SVCJOJVT

QSZ�QSZ

DPXCPZE�UIFSVCZSBDFS

TZNGPOZ�DNG�$PSF#VOEMF

CBTIP�ZPLP[VOB

TZNGPOZ�DNG�4JNQMF$NT#VOEMF

1PMZNFS�1PJOUFS&WFOUT

KRVFSZ�TJ[[MF

LOPDLPVU�LOPDLPVU

BMQIBHPW�TNBSU�BOTXFST

EPDUSJOF�%PDUSJOF.PEVMF

TPOBUB�QSPKFDU�4POBUB"ENJO#VOEMF

+VMJB-BOH�*+VMJB�KM

+VMJB0QU�+V.1�KM

TFOTJPMBCT�4FOTJP%JTUSJCVUJPO#VOEMF

HFPBENJO�NG�HFPBENJO�

KRVFSZ�KRVFSZ

TZNQZ�TZNQZ

OPGMP�OPGMP

STUVEJP�HHWJT

TXDBSQFOUSZ�CD

MJCHJU��QZHJU�

MBVSFOUK�TMJNFSKT

SVCZHFNT�SVCZHFNT�PSH

UXJUUFS�SFDFTT

TQBSLMFNPUJPO�OPLPHJSJ

QIJO[F�IPNFCSFX�DBTL

KSVCZ�XBSCMFS

MJCHJU��MJCHJU�TIBSQ

HVBSE�HVBSE�STQFD

KPTIENJMMFS�OH�CPJMFSQMBUF

NPOHPEC�OPEF�NPOHPEC�OBUJWF

QZQB�QJQ

I�CQ�IUNM��CPJMFSQMBUF

NP[JMMB�TFSWP

CJDLFSTUPGG�[G��EPDVNFOUBUJPO

UIF�HSJE�UIF�HSBQI

KRVFSZ�BQJ�KRVFSZNPCJMF�DPN

KOJDLMBT�UVSOJQ

BMQIBHPW�HPWVL@DPOUFOU@NPEFMT

UPED�UPED�CPPUTUSBQ

CDBSEBSFMMB�DMJFOU@TJEF@WBMJEBUJPOT

GPH�GPH

'PSU"XFTPNF�'POU�"XFTPNF

GBDFCPPL�IJQIPQ�QIQ

FDPNGF�FEQ�DPSF

PXODMPVE�DPSF

#FIBU�.JOL#SPXTFS,JU%SJWFS

UPNCI�EFJT�DPPLCPPL

BMQIBHPW�GSPOUFOE

1PMZNFS�QPMZNFS�VJ�FMFNFOUT

YQ�MBOH�DPNQJMFS

OPGMP�EBUBGMPX�OPGMP

$POTUFMMBUJPO�FTNBOHMF

LPSBLUPS�TUFBN�DPOEFOTFS�QIQ

KRVFSZ�BQJ�RVOJUKT�DPN

USBWJT�DJ�USBWJT�BTTFUT

HSVOUKT�HSVOU�DPOUSJC�IBOEMFCBSTMBWFSEFU�OPEF�GJCFST

QBVMDCFUUT�.PEFSO)UUQ$MJFOU

TQBDFUFMFTDPQF�1Z'*54

XFU�CPFX�XFU�CPFX�YTM

QBESJOP�QBESJOP�SFDJQFT

COPHVDIJ�IPPLT�KT

FMJB�BDUJWFBENJO�NPOHPJE

MJOEBIVB�/VNFSJD&YUFOTJPOT�KM

EPUDMPVE�TUBDLCSFX

NJOSL�JQZUIPO@FYUFOTJPOT

OQN�SFBE�JOTUBMMFE

N[BCBMVFW�HSVTU

TXJGUNBJMFS�TXJGUNBJMFS

EFWFMPQNFOUTFFE�OPEF�TRMJUF�

ESVCMJD�DTT�NPEBM

CBTIP�SJBL�SVCZ�DMJFOU

GSBDUJPO�GSBDUJPO

CFSLTIFMG�WBHSBOU�CFSLTIFMG

IFBSUTFOUXJOFE�FNCFS�BVUI

SBJMT�TUSPOH@QBSBNFUFST

EPDVNFOUDMPVE�VOEFSTDPSF

BEIFBSTJPO�QVODICMPDL

IDBUMJO�MJCTBTT

CBTIP�HJEEZVQ

HVBSE�MJTUFO

SZBONDHSBUI�UXZUIPO

USBWJT�DJ�USBWJT

DMPVEIFBE�MFTT�KT

DISJT�NPSHBO�ODVSTFT�ST

DPNQPTFS�DPNQPTFS

.PEFSOJ[S�.PEFSOJ[S

-FBGMFU�-FBGMFU�ESBX

TBHIVM�QZVW

HBNNBQZ�HBNNBQZ

STQFD�STQFD�FYQFDUBUJPOT

CBTIP�SJBL@DPOUSPM

CBTIP�DVUUMFGJTI

USJOJEBE�USJOJEBE

CIPMMJT�NBSVLV

QBVMNJMMS�DIPLJEBS

BSJZB�QIBOUPNKT

NBUQMPUMJC�NBUQMPUMJC

SPHFSXBOH�OPEF�XFCLJU

JQZUIPO�OCDPOWFSU

TDINJUUKPI�TFSJBMJ[FS

NBQCPY�NBQCPY�KT

BMQIBHPW�HET�BQJ�BEBQUFST

KPOMFJHIUPO�QPMUFSHFJTU

SBJMT�BDUJWFSFTPVSDF

LDINDL�WJN�DPGGFF�TDSJQU

CVOEMFS�CVOEMFS�TJUF

DBTLSPPN�IPNFCSFX�GPOUT

SVCZHFNT�SVCZHFNT�BXT

HSBWJUZTUPSN�PQFOTUSFFUNBQ�DBSUP

SBJMT�TQSPDLFUT�SBJMT

TDJLJU�MFBSO�TDJLJU�MFBSO

.BZIFN:%(��DIBO�Y

4DJ5PPMT�JSJT

4MFY"YUPO�SFRVJSF�IBOEMFCBST�QMVHJO

BOHVMBS�BOHVMBS�KT

-FBSO#PPTU�TPDLFU�JP

KBTILFOBT�CBDLCPOF

TDINJUUKPI�+.44FDVSJUZ&YUSB#VOEMF

HFOFNV�(FOFNV'PSN#VOEMF

CBTIP�SJBL@UFTU

FNCFSKT�FNCFS�EFW

TDINJUUKPI�+.4*��O3PVUJOH#VOEMF

KHN�QBOEPD

[FOEGSBNFXPSL�[G�XFC

TUFGBOQFOOFS�FNCFS�BQQ�LJU

TTUFQIFOTPO�TQSPDLFUT

1BSBMMFMT�WBHSBOU�QBSBMMFMT

MJGP�EPDSBJMT

CK[�HMGX�ST

TUBUTNPEFMT�TUBUTNPEFMT

XFU�CPFX�XFU�CPFX

HFFNVT�FYDPO

TZNGPOZ�DNG�3PVUJOH#VOEMF

I�CQ�NPCJMF�CPJMFSQMBUF

-FBGMFU�-FBGMFU�NBSLFSDMVTUFS

SFRVFTUT�SFRVFTUT�PBVUIMJC

PTNMBC�PQFOTUSFFUNBQ�VQDPNJOH�GFBUVSFT

EPUDMPVE�EPDLFS

TUSPOHMPPQ�MPPQCBDL

TXDBSQFOUSZ�TJUF

DPNQPOFOU�CVJMEFS�KT

EPUDMPVE�EPDLFS�SFHJTUSZ

QJWPUBM�QJWPUBM@XPSLTUBUJPO

KTIJOU�KTIJOU

BMFTTBOESP��IPNFCSFX�QIQ

EPVHMBTEVUFJM�VJ�VUJMT

COPPSEIVJT�MJCVW

TOBSGFE�XFCVUJM

HSVOUKT�HSVOU�DPOUSJC�TBTT

BSFESJEFM�IUNM�

TXDBSQFOUSZ�CPPU�DBNQT

USBWJT�DJ�CMPH�USBWJT�DJ�DPN

HET�PQFSBUJPOT�QVQQFU�HSBQIJUF

USPPQKT�USPPQKT�KRVFSZ

HSVOUKT�HSVOU�DPOUSJC

USVTUNBTUFS�HPGMPX

HSVOUKT�HSVOU�DPOUSJC�JOUFSOBM

YQ�GSBNFXPSL�YQ�MBOHVBHF

YQ�GSBNFXPSL�YQ�DPOUSJC

1PMZNFS�UPEPNWD

YGPSUZ�WBHSBOU�ESVQBM

USPPQKT�USPPQKT�SFRVJSFKT

DPNQPTFS�TBUJT

KBOFTUSFFU�DPSF

HSVOUKT�HSVOU�DPOUSJC�KTU

DPSFPT�DPSFPT�WBHSBOU

CFSLTIFMG�TPMWF

STQFD�STQFD�TVQQPSU

TVCTUBDL�OPEF�NLEJSQ

$PDPB1PET�DPDPBQPET�EPXOMPBEFS

JTBBDT�SFBE�QBDLBHF�KTPO

EPDUSJOF�%PDUSJOF#VOEMF

OQN�OQN�SFHJTUSZ�NPDL

TZNGPOZ�DNG�3PVUJOH"VUP#VOEMF

JQZUIPO�JQZUIPO�DPNQPOFOUT

SVTU�MBOH�SGDT

ZFPNBO�HFOFSBUPS�XFCBQQ

CBTIP�CJUDBTL

NBDPVSOPZFS�UIJO

GHSFIN�WBHSBOU�MYD

TZTUFNFE�J%

CBTIP�SJBL�QZUIPO�DMJFOU

HL[�-JWF4DSJQU

SBDL�SBDL

BOESFX�OPEF�TBTT

TFCBTUJBOCFSHNBOO�QIQ�DPEF�DPWFSBHF

ZFPNBO�HFOFSBUPST

PQFOTUSFFUNBQ�PQFOTUSFFUNBQ�XFCTJUF

DBTLSPPN�IPNFCSFX�WFSTJPOT

-JHIU5BCMF�-JHIU5BCMF

NPOHPJE�NPOHPJE


ULG�FNBDT�JQZUIPO�OPUFCPPL

USBWJT�DJ�USBWJT�XFC

0$BNM1SP�PQBN

NBQCPY�UJMFNJMM

NJSBHF�NJSBHF

BMQIBHPW�HPWVL@DPOUFOU@BQJ

FSOJF�SBOTBDL

3JPU(BNFT�SJEMFZ

TPOBUB�QSPKFDU�4POBUB%PDUSJOF03."ENJO#VOEMF

NBQCPY�DBSUP

FNCFSKT�FNCFS�SBJMT

FMBTUJDTFBSDI�DPPLCPPL�FMBTUJDTFBSDI

TQSFF�EFGBDF

SVCZ�BNRQ�CVOOZ

XFU�CPFX�($8FC

GSFFSVOOJOHUFDIOPMPHJFT�TQSFF@SFWJFXT

CBTIP�SJBL

1PMZNFS�EPDT

SBJMT�BSFM

QSPHSJVN�EPLLV

OPEFKJUTV�OPEF�IUUQ�QSPYZ

BJSCMBEF�QBQFS@USBJM

JWBZOCFSH�TFMFDU�

NBUIOFU�NBUIOFU�OVNFSJDT

0DSBNJVT�1SPYZ.BOBHFS

KUSJMFZ�4UBS$MVTUFS

MFTT�MFTT�KT

TFBODSJCCT�XFCNBDIJOF�SVCZ

PXODMPVE�DPOUBDUT

DMVUDITLJ�DPGGFFMJOU

TXJH�TXJH

KFLZMM�KFLZMM

TFOTJP�4FOTJP%JTUSJCVUJPO#VOEMF

TTUFQIFOTPO�SCFOW

KRVFSZ�QMVHJOT�KRVFSZ�DPN

QSPQFMPSN�1SPQFM�

TFBUUMFSC�NJOJUFTU

BSBTIQBZBO�BQQJSBUFS

(PPHMF$MPVE1MBUGPSN�HTVUJM

SBJMT�BQJ�BDUJWF@NPEFM@TFSJBMJ[FST

/BNJ�%PD�-JWF4DSJQU

[FOEGSBNFXPSL�$PNQPOFOU@;FOE.JNF

BQNBTFMM�WBQJT

SPHFSXBOH�8FC,JU@USJNNFE

COPPSEIVJT�OPEF

TGFSJL�U

GHSFIN�WBHSBOU�DBDIJFS

UNBUJMBJ�WBHSBOU�QSPYZDPOG

EPDUSJOF�%PDUSJOF.POHP0%..PEVMF

TJNPOTUFS�."5�KM

KPOOPS�NJDSPGMP

BMQIBHPW�CVTJOFTT�TVQQPSU�GJOEFS

1PMZNFS�QMBUGPSN

HFUMBOUFSO�XXX�HFUMBOUFSO�PSH

TZNGPOZ�DNG�5FTUJOH

BMQIBHPW�USBOTGPSNBUJPO�EBTICPBSE

PXODMPVE�NFEJB

BTUSPQZ�DPPSEJOBUFT�CFODINBSL

MJCHJU��MJCHJU��HJUIVC�DPN

IBNM�IUNM�IBNM

HIFEP�Q��(JU�3BX

KXBSLFOUJO�OPEF�NPOLFZ

BIFDLNBOO�NRVFSZ

EFWFMPQNFOUTFFE�OPEF�CMFOE

GPH�GPH�DPSF

KFLZMM�IFMQ

OVNQZ�OVNQZEPD

JTBBDT�OQNDPOG

CPPU�EPDLFS�CPPU�EPDLFS�DMJ

SBJMT�KPVSOFZ

SVNCMFMBCT�BTTFU@TZOD

6ODPNNPO�9JU

TIB[PX�VSMMJC�

DIBQMJOKT�DIBQMJO

3VCZ.POFZ�NPOFZ

BEBNGJTL�-JUUMF1SPYZ

SPXBOK�HJUY

QJWPUBM�TQSPVU�TQSPVU

[FOEGSBNFXPSL�$PNQPOFOU@;FOE4FSWFS

UNQWBS�KTEPN

OFDPMBT�OPSNBMJ[F�DTT

TDINJUUKPI�+.44FSJBMJ[FS#VOEMF

PDBNMMBCT�SXP�DPNNFOUT

BEIFBSTJPO�BEIFBSTJPO

OPEFKJUTV�GPSFWFS

ZFPNBO�ZFPNBO�JP

CSVODI�CSVODI

GOJDIPM�DIFG�SWN

TZNGPOZ�TZNGPOZ�TUBOEBSE

PXODMPVE�NVTJD

,BUFMMP�IBNNFS�DMJ�LBUFMMP

NBQCPY�OPEF�TRMJUF�

SFBDUQIQ�SFBDU

OPEF�JOTQFDUPS�OPEF�JOTQFDUPS

BHSPCCJO�VDMBTT

WJTJPONFEJB�KBEF

KTEPD��KTEPD

TVM�EMTT�TQPUMJHIU

4ZMJVT�4ZMJVT�%PDT

ZJJTPGU�ZJJ

-FBGMFU�-FBGMFU

TQSFF�TQSFF@BDUJWF@TIJQQJOH

NTZTHJU�NTZTHJU

DPNQPTFS�QBDLBHJTU

DCPEFO�3BUDIFU

NJEEMFNBO�NJEEMFNBO�CMPH

4FMEBFL�NPOPMPH

EPDUSJOF�EPDUSJOF�

SVCZTQFD�SVCZTQFD

CFTUJFKT�MPEBTI

JOUSJEFB�NVMUJ@KTPO

TZNGPOZ�DNG�TZNGPOZ�DNG

KRVFSZ�MFBSO�KRVFSZ�DPN

[FOEGSBNFXPSL�[G�

-FBSO#PPTU�TUZMVT

TZNGPOZ�DNG�$SFBUF#VOEMF

+BNFT.(SFFOF�RVOJU

TMFFQFS�HFOFSBUPS�CBDLCPOF

USBWJT�DJ�USBWJT�CPYFT

PXODMPVE�BQQGSBNFXPSL

MBVUJT�VHMJGJFS

IBMPTUBUVF�EJGG�MDT

QIQTQFD�QIQTQFD

DVDVNCFS�DVDVNCFS�IUNM

.PEFSOJ[S�NPEFSOJ[S�DPN

1PMZNFS�NPSF�FMFNFOUT

KSCVSLF�SFRVJSF�DT

TVCTUBDL�CSPXTFS�QBDL

HSVOUKT�HSVOU�DPOUSJC�XBUDI

:PSJDL1FUFSTF�PHB

HSVOUKT�HSVOU�DPOUSJC�DPNQSFTT

HSVOUKT�HSVOU�DPOUSJC�RVOJU

SFRVJSFKT�YSBZRVJSF

ZFPNBO�HSVOU�SFHBSEF

.PEFSOJ[S�HSVOU�NPEFSOJ[S

ZPMB�EFNBOET

OJDPVMBK�WBHSBOU�NBWFO�QMVHJO

PXODMPVE��SEQBSUZ

1ZCPOBDDJ�QPMJBTUSP

*BJO/;�.BUI1SPH�KM

+VMJB-BOH�0%&�KM

UIPNBT�NDEPOBME�CPPUTUSBQ�TBTT

KLCS�IUUQJF

FSOJF�TRVFFM

FMJYJS�MBOH�FMJYJS�MBOH�HJUIVC�DPN

PXODMPVE�NP[JMMB@TZOD

DPNQPOFOU�DPNQPOFOU

IBEMFZ�EFWUPPMT

KRVFSZ�KRVFSZ�NPCJMF

UJFOOPV�OFTUFE@GPSN

BMQIBHPW�HPWFSONFOU�TFSWJDF�EFTJHO�NBOVBM

SEPD�SEPD

CBTIP�XFCNBDIJOF

SBJMT�EPDSBJMT

HJUFYUFOTJPOT�HJUFYUFOTJPOT

NJTIPP�6HMJGZ+4�

EPDUSJOF�QIQDS�PEN

JOLMJOH�4VCMJNJOBM

TUFFWF�CPPU�EPDLFS

QZQB�WJSUVBMFOW

NCPTUPDL�UPQPKTPO

ESBQFSHFN�ESBQFS

TUPG�%PDUSJOF.POHP%##VOEMF

QMBUBGPSNBUFD�EFWJTF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE1SPHSFTT#BS

CPUP�CPUP

JNBUIJT�PDUPQSFTT

EPDUSJOF�NPOHPEC�PEN

BEBNW�IPNFCSFX�BMU

PXODMPVE�EPDVNFOUT

[FOEGSBNFXPSL�;FOE%FWFMPQFS5PPMT

TJNPOTUFS�%BUB'SBNFT�KM

BMQIBHPW�EFTJHO�QSJODJQMFT

QIPFOJYTPOH��IPNFCSFX�QIQ

ECBTIGPSE�NJNPTB

NTZTHJU�HJU

ZFPNBO�HFOFSBUPS�CBDLCPOF

CPPU�EPDLFS�CPPU�EPDLFS

4JNPO4BQJO�TFSWP

FSJDBN�TVTZ

TXDBSQFOUSZ�XFCTJUF

TQSFF�TQSFF@BVUI@EFWJTF

SWN�SWN�TJUF

ECBTIGPSE�NJNPTB�MJWF�SFMPBE

HJMEFHPNB�USBWJT�DPPLCPPLT

BMQIBHPW�SVNNBHFS

PDBNMMBCT�PDBNM�DUZQFT

DPXCPZE�MJCW�

OPGMP�GCQ

KEPSO�TRM�GPSNBUUFS

UNN��IUUQ@QBSTFS�SC

UPSBOC�FNCFS�EBUB�EKBOHP�SFTU�BEBQUFS

KRVFSZ�KRVFSZ�DPN

YEJTTFOU�LBSNB�CSPXTFSJGZ

CGMBE�DIFG�EPDLFS

HV[[MF�HV[[MF�

KHN�QBOEPD�UFNQMBUFT

ECBTIGPSE�NJNPTB�CPXFS

CSVODI�VHMJGZ�KT�CSVODI

NJEEMFNBO�NJEEMFNBO�TQSPDLFUT

'PSCFT-JOEFTBZ�DPNQPOFOU�XFCTJUF

SWN�SWN�CJOBSZ

SVCZHFNT�CVOEMFS�BQJ

TFCDSP[FU�LJTT�E

NJTUZEFNFP�UJHFSCSFX

ZZVV�QZFOW

PSHTZOD�BDUJWF@JOUFSBDUJPO

XLIUNMUPQEG�XLIUNMUPQEG

UIFLJE�YQ�FYQFSJNFOUT

JDIVBO�CPXFS�BOHVMBS�MBUFTU

DVDVNCFS�HIFSLJO

EPDUSJOF�ECBM

CBTIP�SJBL@DPSF

BOTJCMF�BOTJCMF

MTFHBM�ZBSE

OHCQ�OH�CPJMFSQMBUF

WJTJPONFEJB�FYQSFTTKT�DPN

CMBLFXBUUFST�5SBOTJUJPO,JU

SWN�HFN�XSBQQFST

KRVFSZ�EPXOMPBE�KRVFSZVJ�DPN

LBSNJ�UJSF

.BUUIFX.VFMMFS�DIFFSJP

8JO3C�WBHSBOU�XJOEPXT

QSPKFDUIZESB�TVGJB

[FOEGSBNFXPSL�;FOE4LFMFUPO"QQMJDBUJPO

DBQOSFHFY�TQSFF@BVUI@EFWJTF

KRVFSZ�KRVFSZNPCJMF�DPN

HVMQKT�HVMQ

IPUPTN�)%.�$BSUP$44

&YFSDJTF�'02&MBTUJDB#VOEMF

'SJFOET0G4ZNGPOZ�'043FTU#VOEMF

HPMMVN�HPMMVN

NJTIPP�6HMJGZ+4

NBQCPY�XBY

NBQOJL�NBQOJL�SFGFSFODF

QIQ�QIQ�TSD

QBESBJD�NPDLFSZ

[FOEGSBNFXPSL�$PNQPOFOU@;FOE&TDBQFS

4IBVO,�OPEF�TPBQ

DBEFODFNBSTFJMMF�SVTU�QDSF

ZBIPOEB�SBJMT�EFW�CPY�SVOT�PSBDMF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE$SZQU

NBSJKOI�UFSO@GPS@WJN

GC��$44TFMFDU

UFSJOKPLFT�HVMQ�VHMJGZ

IFHFNPOJD�DBUIBSTJTBJL��RB�UPPMT

TIBNB�HSVOU�IVC

TZNGPOZ�4XJGUNBJMFS#VOEMF

KJNIFTUFS�LOJUS#PPUTUSBQ

GSFESJL�KPIBOTTPO�NQNBUI

+VMJB-BOH�$BJSP�KM

KRVFSZ�KRVFSZ�SFMFBTF

O�L��DBTQFSKT

WPKUBKJOB�UFTUBDVMBS

LBSNJ�SFUJSF

BMQIBHPW�QVCMJTIFS

#FIBU�.JOL

K[BFGGFSFS�KRVFSZ�WBMJEBUJPO

TVCTUBDL�OPEF�CSPXTFSJGZ

HJUIVC�NBSLVQ


SPCPHVJDF�SPCPHVJDF

WJTJPONFEJB�OJC

NBSLFWBOT�ESBHPOGMZ

QPTUSBOL�MBCT�HPMJBUI

BNBUTVEB�LBNJOBSJ

CCMJNLF�XFCNPDL

PDMJOU�PDMJOU

HJU�UGT�HJU�UGT

SFGJOFSZ�SFGJOFSZDNT�CMPH

NJUDIFMMI�WBHSBOU�BXT

KRVFSZ�BQJ�KRVFSZVJ�DPN

HSBWJUZTUPSN�NBQOJL�SFGFSFODF

DSFBUJPOJY�OWN

BMQIBHPW�HPWVL@OFFE@BQJ

BMQIBHPW�USBWFM�BEWJDF�QVCMJTIFS

TUSPOHMPPQ�MPPQCBDL�XPSLTQBDF

QBVMNJMMS�CSVODI�XJUI�DIBQMJO

EPDUSJOF�BOOPUBUJPOT

TVCTUBDL�NPEVMF�EFQT

HSVOUKT�HSVOU�JOJU

CBTIP�SJBL�FSMBOH�IUUQ�DMJFOU

KOS�KOS�QPTJY

DSBJHDJUSP�S�USBWJT

TUFWFOHK�1Z$BMM�KM

EJBTQPSB�EJBTQPSB�QSPKFDU�TJUF

TGBDLMFS�SVTU�QPTUHSFT

NPOHPEC�NPOHP�QIQ�ESJWFS

EJTDPVSTF�EJTDPVSTF

TPOBUB�QSPKFDU�4POBUB.FEJB#VOEMF

NBQOJL�DBSUP�QBSTFS

ZFPNBO�HFOFSBUPS

DVDVNCFS�DVDVNCFS�KWN

NJLFZ��YQ�GSBNFXPSL

XZVFOIP�CBDLHSJE

JWBOPW�JQZUIPO

BMFYBJULFO�HVBSE�LPOBDIB

TQSFF�TQSFF@J��O


NEFWJMT�OPEF�KTDT

PXODMPVE�OFXT

TFOTJP�4FOTJP(FOFSBUPS#VOEMF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE.BUI

HSVOUKT�HSVOU�DPOUSJC�JNBHFNJO

TZNGPOZ�DNG�TZNGPOZ�DNG�EPDT

+VMJB4UBUT�%JTUSJCVUJPOT�KM

DBBSMPT��CPPUTUSBQ�TBTT

ENBUIJFV�USBWJT�XFC

GBCQPU�4JMFY

5X1�EJSFDUPSZ@XBUDIFS

ZFPNBO�ZP

BMQIBHPW�MJDFODF�GJOEFS

ZZVV�QZFOW�WJSUVBMFOW

BOHVMBS�VJ�BOHVMBS�VJ�HJUIVC�DPN

KTPO�BQJ�KTPO�BQJ

CPXFS�CPXFS�HJUIVC�JP

PQFOMBZFST�PQFOMBZFST�HJUIVC�JP

MLDBNQCFMM�CSBDLFUT�SVMFS

BEPCF�SFTFBSDI�UIFTFVT

DGKFEJNBTUFS�CSBDLFUT�KTIJOU

)PNFCSFX�IPNFCSFX�CJOBSZ

NSPUI�MPMDPNNJUT

UIPVHIUCPU�TVTQFOEFST

JQZUIPO�OCWJFXFS

XQ�DMJ�CVJMET

XPKTNPM�XQ�DMJ�HJUIVC�DPN

)BSMBO)�%BUB'SBNFT�KM

HJCJBOTLZ�*)BTLFMM

U1M�DI�TZNGPOZ�EPDT

OPEFKJUTV�IBJCV�DBSBQBDF

TFOTJP�4FOTJP'SBNFXPSL&YUSB#VOEMF

ECBTIGPSE�NJNPTB�TFSWFS

ZPIBOCPOJGBDF�EKBOHP�MFBGMFU�TUPSBHF

TFOTJPMBCT�TFDVSJUZ�BEWJTPSJFT

HSVOUKT�HSVOU�DPOUSJC�IUNMNJO

BMQIBHPW�GFFECBDL

TUSPOHMPPQ�MPPQCBDL�BOHVMBS

CFOCBMUFS�XPSEQSFTT�QMVHJO�UFTUT

TIUZMNBO�OPEF�CSPXTFS�SFTPMWF

FDMJQTFTPVSDF�KTIJOU�FDMJQTF

BTUSPQZ�BTUSPQZ�XFCTJUF

KFLZMM�KFLZMM�JNQPSU

$JSDBEJP�DJSDBEJP�JTTVFT

+VMJB-BOH�MJCVW

+VMJB%41�%41�KM

SBJMTJOTUBMMFS�SBJMTJOTUBMMFS�OJY

UIPVHIUCPU�TIPVMEB

PQFOMBZFST�PM�

GFMJYHF�OPEF�GPSNJEBCMF

TQSFF�TQSFF�HVJEFT

HSVOUKT�HSVOU�DPOUSJC�TUZMVT

UZQIPFVT�UZQIPFVT

PQEFNBOE�EFJT

DFMMVMPJE�DFMMVMPJE

UIPVHIUCPU�TIPVMEB�NBUDIFST

EPUDMPVE�EPDLFS�QZ

KSCVSLF�SFRVJSFKT

DVDVNCFS�DVDVNCFS

KVTUJOGSFODI�GPSNUBTUJD

WDS�WDS

BOESFOBSDIZ�LSZQZ

CPXFS�EFDPNQSFTT�[JQ

CBSNTUSP�XFC�BOJNBUJPOT�UPPMT

IFOSJLIPEOF�USBWJT�DPPLCPPLT

4-BLT�8FC&TTFOUJBMT��

BEBN�HFHPUFL�TZNGPOZ

[DCFO[�OPEF

QKBDLTPO��($8FC

KQDBNBSB�KSVCZ

XZVFOIP�CBDLHSJE�TFMFDU�BMM

U6S(�O�ODPOG

SPCFSULPXBMTLJ�OPSNBMJ[F�QBDLBHF�EBUB

XFU�CPFX�UIFNF�PHQM

BWTN�NJSBHF�XXX

4UFNFSEJOL*5�4FOTJP'SBNFXPSL&YUSB#VOEMF

BDIFNQJPO�TJNQMF@GPSN

ZJJTPGU�ZJJ��GSBNFXPSL

XJMMUIBNFT�BOTJCMF

MJCFSGB�FSGB

OVMMUPLFO�MJCHJU�TIBSQ

QPSUFSKBNFTK�JQZUIPO

ESFXXFMMT�MPMTSW

DPXCPZ�XFTCPT

MVLFTBSOBDLJ�USBWJT�XFC

ECBTIGPSE�NJNPTB�JDFE�DPGGFFTDSJQU

XLJOH�FNBJM�JTTVF�SFQMZ�UFTUJOH

FOSJDPTBEB�*SPO-BOHVBHFT

)PNFCSFX�IPNFCSFX�CPOFZBSE

"HJM5FD�TQSFF@GBODZ

FDPNGF�FEQ�BEE

XFU�CPFX�UIFNF�HD�JOUSBOFU

4ZOP$PNNVOJUZ�TQLTSD

HSVOUKT�HSVOU�MJC�QIBOUPNKT

UFTU�VOJU�UFTU�VOJU�BDUJWFTVQQPSU

FDPNGF�FEQ�QSPKFDU

XFCQBDL�XFCQBDL�HJUIVC�DPN

HMPCBMJ[F�HMPCBMJ[F�WFSTJPOJOH

WJODFOU�IVHPU�J5F.-

+BNFT.(SFFOF�RVOJU�BTTFSU�IUNM

NFTLZBOJDIJ�IJSFGJSF�SFTPVSDF

SVCZTM�SVCZTM�TPDLFU

BTUSPQZ�BTUSPQZ�BQJ

PCTQZ�TBOECPY

TQSPVUTPDJBM�FT��JNQPSU�WBMJEBUF

HSVOUKT�HSVOU�DPOUSJC�MJWFSFMPBE

PXODMPVE�CPPLNBSLT

QMFQF�QHNBQDTT

KQFUB[[P�EJOE

GBCQPU�TQIJOY�QIQ

XFCQBDL�FOIBODFE�SFRVJSF

QBP�.POBET�KM

QFUFSGMZOO�CSBDLFUT�DPNNBOET�HVJEF

U[JOGP�U[JOGP

SBCCJU�TIPDLFS�SBCCJUFS

KRVFSZ�DPOUSJCVUF�KRVFSZ�PSH

TZNGPOZ�DNG�.FEJB#VOEMF

BEPCF�$PEF.JSSPS�

OQN�OQNKT�PSH

BWTN�PDBNM�HJUIVC

IFBEJVT�UISFBE@TBGF

PTNMBC�FEJUPS�JNBHFSZ�JOEFY

DSBVF�$SBVF'PSN'MPX#VOEMF

GSFFSBOHF�NPDIB

KDSPDIPMM�QFQ�

+VMJB-BOH�.&5"%"5"�KM

GGJ�GGJ

KSVCZ�BDUJWFSFDPSE�KECD�BEBQUFS

+PTIVB�"OEFSTPO�USBWJT�DPSF

XFCQBDL�FOIBODFE�SFTPMWF

TWFOGVDIT�HMPCBMJ[F�

FEHFDBTF�&$4MJEJOH7JFX$POUSPMMFS

#MVF4QJSF�%VSBOEBM

JWBOPW�WJN�JQZUIPO

NQFSIBN�EBMMJ

NSVCZ�NSVCZ

EBOF[�TZNGPOZ�EPDT

QZEBUB�QBOEBT

KRVFSZ�BQJ�KRVFSZ�DPN

QBTTZ�HFOFSBUPS�DBOKT

USPPQKT�USPPQKT�UPEPT

JWBOPW�NBUQMPUMJC

FOOPWB�QPTUNBSLEPXO

QIQ%PDVNFOUPS�3FGMFDUJPO

QPMZGSBDUBM�BUIMFUJD

$BO$BO$PNNVOJUZ�DBODBODBO

TUFGBOQFOOFS�FNCFS�JOGMFDUPS

HSBZEPO�SVTU�XXX

KPZFOU�OPEF�XFCTJUF

SBJMT�DBDIF@EJHFTUT

SZBOC�OFTUFE@GPSN

FMBTUJDTFBSDI�FMBTUJDTFBSDI

SZBOC�DBODBO

YQ�GSBNFXPSL�SGD

NOF�UPPMT�NOF�QZUIPO

IBNM�IBNM

OBIPMZS�EPHHZCBH

PQFOMBZFST�PQFOMBZFST

FJOBSPT�XT

BMQIBHPW�TJHOPOPUSPO�

TDIJTBNP�WBHSBOU�PNOJCVT

NBQCPY�NBSLFST�KT

UJNIPMZ�)%'��KM

TUSPOHMPPQ�TUSPOH�SFNPUJOH

#FIBU�4BIJ$MJFOU

JTBBDT�OPQU

QBZMPHJD�QJQ�BDDFM

JTBBDT�OPEF�HSBDFGVM�GT

E��E��QMVHJOT

HBNNBQZ�UFWQZ

BZPTFD�BENJOGVM

LBSCBSDDB�%BUFUJNF�KM

NDVBESPT�IPNFCSFX�IIWN

NBSJPOFUUFKT�CBDLCPOF�NBSJPOFUUF

EFOOJTSFJNBOO�JPDUPDBU

HMPCBMJ[F�HMPCBMJ[F

MZSJYY�TZNGPOZ

JHSJHPSJL�FN�IUUQ�SFRVFTU

FWFOUNBDIJOF�FWFOUNBDIJOF

NJEEMFNBO�NJEEMFNBO�HVJEFT

TFOTJPMBCT�4FOTJP'SBNFXPSL&YUSB#VOEMF

OPGMP�OPGMP�SVOUJNF�CBTF

FDPNGF�FEQ�XFCTFSWFS

XIJU��XXX�HJUUJQ�DPN

+VTUJO5VMMPTT�[FSPNR�OPEFLBOTP�LBOTP

3JPU(BNFT�CFSLTIFMG

UFDIOPXFFOJF�GBSBEBZ

KRVFSZ�UFTUTXBSN

XZDBUT�UIPS

QKBDLTPO��UIFNF�CBTF

NJOSL�JQZUIPO

UJBHPIJMMFCSBOEU�CVJMET

BMQIBHPW�TMJNNFS

TJMFYQIQ�4JMFY�8FC1SPGJMFS

"TRVFSB�FMBTUJDTFBSDI�SBLF�UBTLT

#SFX5FTU#PU�IPNFCSFX

BTUSPQZ�BTUSPRVFSZ

EFGVOLU�IVC

TFCBTUJBOCFSHNBOO�QIQVOJU�TFMFOJVN

EJBTQPSB�EJBTQPSB

NBSJKOI�BDPSO

OPGMP�OPGMP�SVOUJNF

1PMZNFS�QPMZNFS

ZFMMBQVIBSJ�XFC�CBTF�UFNQMBUF

TFOO[B�$IBTTJT

KRVFSZ�RVOJUKT�DPN

CBTIP�SJBL@QC

QSPHSJVN�CVJMETUFQ

SPCSJDI�PSDIFTUSBUPS

WKFVY�K%BUB7JFX

HJUIVC�QBHFT�HFN

HJUIVC�HJUIVC�TFSWJDFT

[FOEGSBNFXPSL�$PNQPOFOU@;FOE5FYU

TUVDLJ�EPDLFS�DZBOPHFONPE

TBNPIU�PQBN�SFQPTJUPSZ

[JNCBUN�SBWFO�SVCZ

NT�BUJ�EPDJMF

4JS7FS�VMUJTOJQT

YJBOZJ�0QFO#-"4

CSJBONBSJP�NZTRM�

WNH�SFEDBSQFU

TIBNB�HSVOU

NBMGBVY�IUUQ�QBSTFS

UFOFSE�$PGGFF4DSJQU3FEVY

BHSPCCJO�BDUJWF@BENJO

KMFSUMF�HFOFSBUPS�FNCFS

KOVOFNBLFS�UXJUUFS

HNB�OFTUBDNT�DPN

KRVFSZ�KRVFSZ�PSH

HVMQKT�HVMQ�VUJM

CPMU�CPMU�EPDT

LJ[V�CFNUP

3BQPU03�$POTPMF#VOEMF

CFONJMMT�SPCPUTLJSU

ECBTIGPSE�NJNPTBKT�DPN

OFTRVFOB�SBCM

EQC��TZNGPOZ�EPDT

DVDVNCFS�DVDVNCFS�SBJMT

/JDL2J;IV�ED�KT

TDJQZ�TDJQZ

STUVEJP�TIJOZ

TTUFQIFOTPO�SVCZ�CVJME

LZLZFW�BMNPOE

JTBBDT�OQN�XXX

HFPBENJO�NG�DITEJ�

CBTIP�SJBL@DT

1PMZNFS�4IBEPX%0.

[FOEGSBNFXPSL�$PNQPOFOU@;FOE1FSNJTTJPOT"DM

QSPUPNPVTF�IPNFCSFX�WFSTJPOT

��$SFBUJWF�TQSFF@CPPUTUSBQ@GSPOUFOE

BOESJPOJ�.1'*�KM

0CWJPVT�QIBOUPNKT

DUBMLJOHUPO�OPEF�BSDIJWFS

#FIBU�.JOL4FMFOJVN%SJWFS

LBSUFOB�1SPK�-FBGMFU

[FOEGSBNFXPSL�;'5PPM

[FSFCVCVUI�PQFOTUSFFUNBQ�DHJNBQ

SBGBFMGSBODB�TJNQMF@GPSN�CPPUTUSBQ

UIFEBSLPOF�SBJMT�EFW�CPPTU

IIWN�QBDLBHJOH

BEPCF�CSBDLFUT�JP

PXODMPVE�HBMMFSZ

SBJMT�QSPUFDUFE@BUUSJCVUFT

GBDFCPPL�UPSOBEP

FDPNGF�FEQ�CDT

QSPQFMPSN�1SPQFM

TOLJOBSE�TUFBN�DPOEFOTFS�KBWB

OBOPD�OBOPD

1Z5BCMFT�1Z5BCMFT

HVBSE�HVBSE

DZUIPO�DZUIPO

F[UJFSOFZ�UFSO

4ZMJVT�4ZMJVT3FTPVSDF#VOEMF

IDBUMJO�TBTTD

DFCF�NBSLEPXO

KMPHTEPO�QIQ�DMJ�UPPMT

NBETLSJTUFOTFO�[FODPEJOH

BEPCF�CSBDLFUT�SFHJTUSZ

+VMJB-BOH�;.2�KM

OQN�OQN

SFBMXPSMEPDBNM�FYBNQMFT

FCSZO�FNCFS�NPEFM

OVNQZ�OVNQZ

ZPIBOCPOJGBDF�-FBGMFU�4UPSBHF

SMJOFIBO�EPUGJMFT

TIBXOCPU�BJHIU

ZFPNBO�HFOFSBUPS�LBSNB

WNH�DMBS

TNPDLMF�TNPDLMF

MBLUFL�K2VFSZ�4NBSU�"VUP�$PNQMFUF

[FSPNR�QZ[NR

6OJPO0G3"%�MJUIJVN

TGFSJL�UXJUUFS

4IPQJGZ�MJRVJE

ZZVV�IPNFCSFX

0DSBNJVT�NPOHPEC�PEN

BMQIBHPW�NBTMPX

,OQ-BCT�,OQ.FOV

1PMZNFS�)5.-*NQPSUT

1PMZNFS�QPMZNFS�FMFNFOUT

BHOBU�OPEF@NEOT

/7�$440.

TRVBSF�DSPTTGJMUFS

NJLFBM�SFRVFTU

PXODMPVE�BOESPJE

TJNQMFQJF�TJNQMFQJF

'SJFOET0G4ZNGPOZ�'04$PNNFOU#VOEMF

QSPUPCPY�QSPUPCPY�XFC

TZNGPOZ�DNG�$POUFOU#VOEMF

BMQIBHPW�XIJUFIBMM

TZNGPOZ�'PSN


NBSTVCPTT�ZJJ�

KGJSFCBVHI�PQFOTUSFFUNBQ�XFCTJUF

PTNMBC�PQFOTUSFFUNBQ�XFCTJUF

[FOEGSBNFXPSL�$PNQPOFOU@;FOE&WFOU.BOBHFS

[FOEGSBNFXPSL�$PNQPOFOU@;FOE$BQUDIB

SVCZ�SVCZ

TVQFSEXFFCJF�NPOHPEC�PEN

JBOHSFFOMFBG�CSVODI�XJUI�DIBQMJO

DVDVNCFS�DVDVNCFS�SVCZ�DPSF

FDPNGF�FEQ�QBDLBHF

UIFUBMMHSBTTOFU�UIFUBMMHSBTT�OFU

U��DI�VSMMJC�

DMEXBMLFS�IJSC

NPOHPJE�NPOHPJE�HJUIVC�DPN

QIQ##�#MPH�QIQ##�#MPH�GPS��

EBWJEEBWJT�LBUFMMP

DIFWFY�IPNFCSFX

BTUSPQZ�DDEQSPD

NDBWBHF�OPEF�GBTU

[FOEGSBNFXPSL�$PNQPOFOU@;FOE1FSNJTTJPOT3CBD

XFJFSPQIJOOFZ�;FOE4LFMFUPO"QQMJDBUJPO

OJTIJHPSJ�OZBODBU�QIQVOJU�SFTVMUQSJOUFS

USBWJT�DJ�USBWJT�XPSLFS

TN�TN�MJCSBSJFT

+VMJB4UBUT�(-.�KM

K%BUB7JFX�K#JOBSZ

/BNJ�%PD�DPDP

ZPSBI�MJCHJU�

UXBMQPMF�TBTT

MF�QIBSF�#B[JOHB+T5SBOTMBUJPO#VOEMF


UIFLJE�CFIBWF

HJMEFHPNB�USBWJT�CVJME

USPOEO�MJCVW

CTNS�SVCZ�USBWJT�CPYFT

BWTN�NJSBHF�UDQJQ

BMQIBHPW�QSPUPUZQJOH

E��E��HFP�QSPKFDUJPO

STQFD�STQFD�BDUJWFNPEFM�NPDLT

EJTDPQSPKFDU�PEJTDP

USBWJT�DJ�USBWJT�JNBHFT

XFU�CPFX�CPPLNBSL

ZJJFYU�USBTI�CJO�CFIBWJPS

BTUSPQZ�BTUSPQZ�"1&T

SVCJOJVT�SVCJOJVT�NFMCPVSOF

JTBBDT�PTFOW

GWEN�OPEFKT�ZPVUVCF

TZNGPOZ�'JMFTZTUFN

ZFMMPX��GPSFJHOFS�NBUDIFS

)PNFCSFX�IPNFCSFX�HBNFT

MTFHBM�SVCZEPD�JOGP

TJOBUSB�TJOBUSB�DPOUSJC

SLI�SBDL�QSPUFDUJPO

S[F[FTLJ�ZPLP[VOB

UFSPQB�CVJME�ZPVS�PXO�BOHVMBSKT

NFSZO�OPSNBMJ[F�QBDLBHF�EBUB

TZNGPOZ�DNG�3PVUJOH

SVGMJO�&MBTUJDB

4DJ5PPMT�DBSUPQZ

TPOBUB�QSPKFDU�4POBUB6TFS#VOEMF

FSFHPO�SBDD

JMQBJKJO�HFOFSBUPS�DISPNFBQQ

GTIBSQ�'4IBSQ�%BUB

&QQ0�SPMJGZ

TOBSGFE�CSJEHZ

HL[�QSFMVEF�MT

UZQIPFVT�FUIPO

NBQCPY�UJMFMJWF�NBQOJL

UPJWPI�%FCVH�KM

QIVTJPO�CBTFJNBHF�EPDLFS

OQN�SFBE�QBDLBHF�KTPO

+VMJB-BOH�%BUB4USVDUVSFT�KM

KRVFSZ�KRVFSZVJ�DPN

JTBBDT�OQN�SFHJTUSZ�NPDL

FDPNGF�FEQ�MJOU

DIBQNBOC�CDCJP�OFYUHFO

HIBSMBO�QSPQFMPSN�HJUIVC�DPN

OPSNBO�GSJFOEMZ@JE

TOST�TPOPSPVT

CSBOEPO�GSZTMJF�FNCFS�SFTU�DPGGFF

XWFOHFO�GPPETPGU

DBSTPNZS�SCFOW�CVOEMFS

OJQZ�1Z4VSGFS

4ZMJVT�4ZMJVT�4UBOEBSE

7BMMPSJD�:PV$PNQMFUF.F

BOHVMBS�VJ�VJ�SPVUFS

IBDLFO�JO�XFCTJUF

BOHVMBS�VJ�BOHVMBS�VJ

+VMJB4UBUT�%BUB'SBNFT�KM

BJEBI[�SJBL@DPSF

KWQFSSJO�EBUF@WBMJEBUPS

PIMFS��PK

HV[[MF�HV[[MF

ETD�DPDP

#BO[BJ.BO�USBWJT�BQJ

NFTLZBOJDIJ�CBDLVQ

.BH-FW�NBHMFW

DVDVNCFS�DVDVNCFS�KT

CBTIP�SJBLOPTUJD

TUSPOHMPPQ�TMT�TBNQMF�BQQ

EPDUSJOF�NPOHPEC�PEN�EPDVNFOUBUJPO

KSCVSLF�BMNPOE

4IPQJGZ�BDUJWF@GVMGJMMNFOU

JQZUIPO�DPOUSJC�*1ZUIPO�OPUFCPPL�FYUFOTJPOT

USPPQKT�USPPQKT�DPOUSJC�CSPXTFS

HSVOUKT�HSVOU�DPOUSJC�DMFBO

BTTBG�[PNCJF

PXODMPVE�DIBU

NBETLSJTUFOTFO�WTXFCFTTFOUJBMT�DPN

FCSZO�CBDLCVSOFS�KT

TQSFF�TQSFF@GBODZ

�VQ�MBC�'04.FTTBHF#VOEMF

KN��EN�EFWJTF

TUBBCN�1SPQFM�

BLFOFP�QJN�DPNNVOJUZ�TUBOEBSE

MVTJT�DIFG�LJCBOB

TQBDFUFMFTDPQF�QZTZOQIPU

MVJTMBWFOB�SBLF�DPNQJMFS

TQSJOHNFZFS�OPEF�QSF�HZQ

PDBNM�PQBN�XFC

QSPUPCPY�QSPUPCPY

SBJMT�DPGGFF�SBJMT

HPUDIB�JQEC

TQSFF�TQSFF@XJTIMJTU

LBSNJ�UJSF�DPOUSJC

OPCV�NSVCZ

0DSBNJVT�BOOPUBUJPOT

ZBUTLFWJDI�HSVOU�CPXFS�UBTL

DBNQUPDBNQ�D�DHFPQPSUBM

UPNB[�BQQMFEPD

LFWJOXBMMBDF�RFNV�EPDLFS

,PEPXB�-JHIU�5BCMF�1MBZHSPVOE

UXJUUFS�CPPUTUSBQ

GMZCPBSEFS�"JS04

HFUUBMPOH�LSBNEPXO

UFTU�VOJU�UFTU�VOJU

QIQTQFD�QSPQIFDZ

DSBCUX�SVTU�CJOEHFO

GSVVY�TBCSF�EBW

BVUP[JNV�KVMJB@[I@DO

UJMEFJP�SPVUFS�KT

#FIBU�.JOL;PNCJF%SJWFS

NP[JMMB�SIJOP

LFWJOZLDIBO�LOJGF�KPZFOU

NBQCPY�OPEF�QSF�HZQ

CPCUIFDPX�QTZTI

NBDHJUWFS�%JGG7JFXFS

QMBUBGPSNBUFD�TJNQMF@GPSN

5PP5BMM/BUF�OPEF

XFCCK��QVQQFU�GJUCJU

TZNGPOZ�7BMJEBUPS

JNM�IPNFCSFX�TDJFODF

IFOSJLIPEOF�UFTU�USBWJT�UFTU

"MBNP[�EPDSBJMT

SJYCFDL�CPMU

,SPJTTF�SVTU�NVTUBDIF

,SPJTTF�SVTU�IUUQ

BSR�Y�HFNJOJ

NCSVCFDL�TFSWP

UIFLJE�YQ�MBOHVBHF

3JPU(BNFT�CVGG�DPOGJH

CEPXMJOH�KRVFSZ�VJ

0DSBNJVT�%PDUSJOF.PEVMF

USJDLOPUFT�FNCFS�KT

EJHJUBMLBP[�DPNQPTFS

JFMHOBX�FEQ

UPNIVHIFT�PQFOTUSFFUNBQ�XFCTJUF

UJBOPO�CSFX

ZBOOJLT�MJOVYCSFX

BN��$TT4PSUFS

BMQIBHPW�QMBOOFS

TFOTJPMBCT�4JMFY�$POOFDU

BWTN�PDBNM�DPIUUQ

KRVFSZ�FWFOUT�KRVFSZ�PSH

ZFPNBO�HFOFSBUPS�NPDIB

LBOTP�DPVDIUZQFT

TDPUUHPO[BMF[�HSVOU�XPSEQSFTT

HSVOUKT�HSVOU�DPOUSJC�ZVJEPD

*DFDBWF4UVEJPT�BSDIFS

RRVFVF�IUNM�DIBO

0DSBNJVT�-B[Z.BQ

KBOFTUSFFU�CJO@QSPU

0QFO6TFS+T�0QFO6TFS+4�PSH

T�NBHF�SVTI

KRVFSZ�EFNPT�KRVFSZNPCJMF�DPN

QFUFSGMZOO�TWH�QSFWJFX

TZNGPOZ�4FSJBMJ[FS

SVCZ�MMWN�SVCZ�MMWN

DFMMVMPJE�EDFMM

CSZBOXC�DIFG�BSL

NPOHPJE�PSJHJO

WFSTHVJ�'PSN

,FOU#FDL�KVOJU

ECBTIGPSE�NJNPTB�JNQPSU�TPVSDF

6OJPO0G3"%�NBOVBM

EBMJXBMJ�GPSUVOF

ESVCMJD�I�CQ�TUZMVT

NJLFM�NBJM

XFCQBDL�XPSLFS�MPBEFS

EBIMCZL�MJCHJU�TIBSQ

GPSSFTUP�EBUBGMPX�OPGMP

LBSUJLQSBCIV�BDUJWJUZTUSFBNT�VOPGGJDJBM

CFTBOFL�1SFTFOUFS5FTUFS

1ZUIPO$IBSNFST�QZUIPO�GVUVSF

HSVOUKT�HSVOU�DMJ

BOHVMBS�BOHVMBS�EBSU

PCTQZ�PCTQZ

TUFSFPCPPTUFS�FYFDKT

KFSFNZ�NBJM

CVOEMFS�CVOEMFS�GFBUVSFT

BSUTZ�HBSOFS

WPMPLP�TEPD

"OHSZ-BXZFS�SVTU�TEM�

DBNQUPDBNQ�DHYQ

PQFOMBZFST�DMB

BOHVMBS�VJ�VJ�VUJMT

TUSPOHMPPQ�MPPQCBDL�EBUBTPVSDF�KVHHMFS

USPPQKT�USPPQKT�DPNQPTFS

DPNQPOFOU�CVJMEFS��KT

DFMMVMPJE�DFMMVMPJE�JP

0$BNM1SP�PQBN�XFC

NPOP�NPOP

BEPCF�CSBDLFUT�QIPOFHBQ

"EEFQBS�FNCFS�UBCMF

CSTPO�SVTU�TEM

DKEFMJTMF�DKEOT

EBWJECLFNQ�OPEF�HZQ

ECBTIGPSE�NJNPTB�DPQZ

QFFSJHPO�BMBNJE

CSJBOD�OPEF�QPTUHSFT

TJOBUSB�TJOBUSB

XFCQBDL�HSVOU�XFCQBDL

NJSBHF�NJSBHF�UDQJQ

NBQCPY�UJMFMJWF�WFDUPS

TJMFYQIQ�4JMFY

0DSBNJVT�IJQIPQ�QIQ

DISNPSJU[�IPNFCSFX�CJOBSZ

ECBTIGPSE�NJNPTB�TFSWFS�SFMPBE

CBTIP�TUBODIJPO

CBTIP�SJBL@QJQF

.BOHF�SPBEJF

NJUDIFMMI�WBHSBOU�SBDLTQBDF

PYQFDL�TQSFF

JMPWFJUBMZ�TQSFF@WPMVNF@QSJDJOH

HBHFSO�VSMMJC�

SPCPUPMPHZ�HB[FCP@ZBSQ@QMVHJOT

ZBLB[�FMBTUJDTFBSDI�BDUJPO�VQEBUFCZRVFSZ

XBOEFSWJFX�OPEF�UBQ

ZFPNBO�ZFPNBO�HFOFSBUPS�MJTU

SFBMXPSMEPDBNM�TDSJQUT

USJDLOPUFT�FNCFS�EBUB

ESVCMJD�NPCJMF�CPJMFSQMBUF

TUPG�.JOL#SPXTFS,JU%SJWFS

ETIFFUT�PDBNM�TPEJVN

ULFMMFO�OPEF�MJGUPGG

/6#*$�ODT@NEFT

KSVCZ�KSVCZ�MBVODIFS

TMBOH��KBEF�CPPL

NBQOJL�NBQOJL�TVQQPSU

SPTTBOU�JQZDBDIF

UJNNZXJM�HSVOU�CPXFSDPQZ

TZNGPOZ�8FC1SPGJMFS#VOEMF

STQFD�STQFD�EFW

SXFOH�QSZ�SBJMT

TJLBDIV�SBJMT

IFSPLV�IFSPLV�CVJMEQBDL�OPEFKT

TBMUTUBDL�TBMUZ�WBHSBOU

DSBGUTNFO�EBE

BSJBUFNQMBUFT�VTFSNBOVBM

+VMJB4UBUT�4UBUT#BTF�KM

KMPOH�TBTT�UXJUUFS�CPPUTUSBQ

LPG�OPEF�RVOJU

4DJ5PPMT�CJHHVT

EUBP�TBGF@ZBNM

COPPSEIVJT�OPEF�CVGGFSUPPMT

JOEJSFDU�IBNM�SBJMT

OQN�OQN�XXX

GBCQPU�1JNQMF

EFWFMPQNFOUTFFE�CPOFT

ECBTIGPSE�NJNPTB�MFTT

TUBSTRVBSF�#V[[

PQFOTUSFFUNBQ�NPE@UJMF

BMQIBHPW�HPWVL@GSPOUFOE@UPPMLJU

UIFIPHGBUIFS�CSBDLFUT�DPEF�GPMEJOH

OFNFSMF�SFE@UJOU

IFOSJLIPEOF�USBWJT�BTTFUT

I�CQ�IUNM�CPJMFSQMBUF�DPN

NBDHJUWFS�.BD(JUWFS.PEVMFT

NFFNPP�EBUBGMPX

[FSPNR�[FSPNR��Y

TZNGPOZ�$POTPMF

SPEZIBEEBE�BOHVMBS�KT

NZJOU�QFSDFQUVBMEJGG

SPNBO[FOLB�DMPVECJPMJOVY

TZNQZ�TZNQZ�MJWF

CBTIP�SJBL�QIQ�DMJFOU

ZVKJOBLBZBNB�USBOTQFD

DVKPKT�SFTU

QKBDLTPO��XFU�CPFX

TUFSFPCPPTUFS�KTIJOUSC

CPXFS�SFHJTUSZ

TDSBQFSXJLJ�UBOH

EPDUPSKOVQF�DPPLCPPL�FMBTUJDTFBSDI

GSFESJLCPOBOEFS�LBSNB

OPGMP�OPGMP�HJUIVC�JP

ECBTIGPSE�NJNPTB�TBTT

ZBLB[�FMBTUJDTFBSDI�RVFSZ�JOEJDFT��

CKFBOFT�TRVBTI�XFC

SPSZL�JQZUIPO�DMVTUFS�IFMQFS

KPOBUBDL�QPMZBNPSPVT

NBQCPY�UN�

HBMZN[IBO�DPNQPTFS

CIPMMJT�NJEEMFNBO�TQSPDLFUT

SVCZ�LPSFB�SVCZ�LPSFB�HJUIVC�JP

FDPNGF�FEQ�UFTU

XFCQBDL�DPSF

WNFVSJTTF�OPEF�QIBOUPN

UIFLJE�TUPNQ

QIQTQFD�QIQTQFD��TJUF

NJDIBFMGJDBSSB�DTDPEFHFO

JWBOPW�*)BTLFMM

NBQCPY�UJMFNJMM�XJO�MBVODIFS

TQBHBMMPDP�FN�UXJUUFS

1PMZNFS�UPPMLJU�VJ

HSVOUKT�HSVOU�DPOUSJC�OPEFVOJU

XQ�DMJ�TFSWFS�DPNNBOE

HSFHCFMM�BSCSF

HMBT[JH�4;5FYU7JFX

MF��CSBDLFUT�IUNM�TLFMFUPO

LFWBSDI�TZNGPOZ�EPDT

BMFYDSJDIUPO�SVTU

GSP[FOJDF��ZFPNBO�HFOFSBUPS�MJTU

TDINJUUKPI�+.4%FCVHHJOH#VOEMF

SPHFSXBOH�DISPNJVN�TSD

DSJTUJBOPTJTUFNBT�QIQVOJU�EPDVNFOUBUJPO

I�CQ�TFSWFS�DPOGJHT�HBF

JOEVUOZ�MJCVW

DBQEFWD�GMZDIFDL

&SJD%VOTXPSUI�UIFNF�HDXV�GFHD

CPMU�CPMU

SUGE�SFBEUIFEPDT�PSH

,BUFMMP�SVODJCMF

TQSFF�TQSFF�QSPEVDU�BTTFNCMZ

EPDUSJOF�%PDUSJOF1)1$3#VOEMF

NMFUUFSMF�SVTU�IUUQ

ECBTIGPSE�NJNPTB�DPNCJOF

OJDLTQ�HFOFSBUPS�CCC

SJDIBSEIJOLBNQ�CPMU�EPDT

HFUMBOUFSO�TUBUTIVC

FMBTUJDTFBSDI�LJCBOB

KPCMJC�KPCMJC

SWBHH�OBO

SPCJO��DPGGFF�SBJMT

B'BSLBT�IUNM�TIJW

QJVDDJP�BSJBUFNQMBUFT

T��U�PXODMPVE

�DIBO��DIBO�+4

WUTWBOH�HSVOU�DPOUSJC�VHMJGZ

KJOHXFOP�NTHQBDL@SBJMT

UIF�HSJE�UIF�CFIBWJPS

UBONBZLN�%BUB'SBNFT�KM

OPWBMJT�CJUDBTL

LSMNMS�S�USBWJT

KNJLPMB�NPOHP�QIQ�ESJWFS

HPSEPOTMPOEPO�'046TFS#VOEMF

ECBTIGPSE�NJNPTB�MJOU

DMBFT��TZNGPOZ�TUBOEBSE

DIBQNBOC�HFNJOJ

&SJD%VOTXPSUI�($8FC

ECBTIGPSE�NJNPTB�NJOJGZ�DTT

BESJEV��SVTU�UVUT

QSZ�QSZ�DPPMMJOF

4DJ5PPMT�JSJT�TBNQMF�EBUB

*ODVCBJE�CBBSETLFFSEFS

FNCBSLNPCJMF�BOESPJE�NBWFO�FYBNQMF

OKY�CSBDLFUT�CPXFS

GFBUVSJTU�TQBXO�DNE

SPCPUPMPHZ�ZBSQ

NP[JMMB�TFSWP�SVTU�NP[KT

ECBTIGPSE�NJNPTB�NJOJGZ

TLJWWJFT�BOHVMBS�KT

[FSPNR�MJC[NR

JTBBDT�MJCVW

BDEIB�PQFOTFBESBHPO

TDINJUUKPI�+.4%J&YUSB#VOEMF

1PMZNFS�$VTUPN&MFNFOUT

QVQQFUMBCT�QVQQFUMBCT�NPOHPEC

HJMEFHPNB�USBWJT�JNBHFT

TUFWFMVTDIFS�TBTT�CSVODI

ECBTIGPSE�NJNPTB�DPGGFFTDSJQU

CSBJOTUPSN�JQZUIPO�DMVTUFS�IFMQFS

JOUSJEFB�UXFFUTUSFBN

QPJTF�QZUIPO

QSPKFDUCMBDLMJHIU�CMBDLMJHIU

OBEFSNBO�DPNQPTFS

EPUMFTT�EF�WBHSBOU�WCHVFTU

MVJTMBWFOB�SC�SFBEMJOF

NQBQJT�FYFDVUBCMF�IPPLT

WBOHEGBOH�HJU

NCPTUPDL�TNBTI

SMBOF�HMGX�ST

NBUUXJMEJH�TJOBUSB�HJUIVC�DPN

NJDIBFMGJDBSSB�FTGV[[

BWTN�NJSBHF�QMBUGPSN

UIPSJO�SFENJOF@MEBQ@TZOD

BBSPOCVTIOFMM�HFOFSBUPS�UNQSPKFDU

TUSPOHMPPQ�TUSPOH�TVQFSWJTPS

ECBTIGPSE�TLFMNJNPTBNJSBHF�PQBN�SFQP�EFW

SBONPDZ�HVBSE�SBJMT

MPMBEJSP�3&1-�KM

K[BFGGFSFS�OPEF�UFTUTXBSN

SPCFSULPXBMTLJ�PTFOW

UPQB[QSPKFDU�UPQB[

CSJBONBSJP�ZBKM�SVCZ

YGPSUZ�DIFG�ESVQBM

KCDSBJM�)BZXJSF

GFOHNL��OPEF�HZQ

KOJDLMBT�DFMMVMPJE�[NR

JOUJNJEBUF�HFOFSBUPS�XFCBQQ

CBTIP�SJBL@EU

BEPCF�CSBDLFUT�VQEBUFT

JTBBDT�OQNKT�PSH

MBTUHVFTU�5XJH�FYUFOTJPOT

DFESJDMPNCBSEPU�1SPQFM

ECBTIGPSE�NJNPTB�XFC�QBDLBHF

FWBOMVDBT�OQN�SFHJTUSZ�NPDL

TUBUCJU�DPNQPTJUF@QSJNBSZ@LFZT

QFUFS�NVSBDI�HJUIVC

OFU�TTI�OFU�TTI

+BCC3�+BCC3

EPUDMPVE�HPSEPO

SVCZHFNT�HVJEFT

KUSJMFZ�TDQ�QZ

NBHJDMBCT�BMDIFNZ@DNT

STUBDSV[�KT�DPGGFF

LCPOE�TZNGPOZ�TUBOEBSE

CBTIP�MFWFMEC

EBWFZ��DISPNJVN

MFBGP�TDTTQIQ

TOBSGFE�PBVUI�ESPQJOT

MZSJEJPVT�IPNFCSFX

JWBOUTFQQ�KFLZMM

,J/H.B3�MJCVW

ETIFFUT�PQBN�SFQPTJUPSZ

OFUK�IPNFCSFX�CPOFZBSE

EBOJFMIPMNFT�TZNGPOZ�EPDT

1BVM,JOMBO�ZFPNBO�JP

SPMBOEXBMLFS�IPNFCSFX�DBTL

TQSBJOUT�HJU�UGT

FVHFOFP�CSBDLFUT

TZNGPOZ�DNG�4FBSDI#VOEMF

NFHJFT�PCTQZ

/FUQPTJUJWF�OEFQMPZ

8PVUFS+�TZNGPOZ

BOBSDIJWJTU�BSDIJWFTTQBDF

FDPNGF�FEQY�VC�SJB

H�Q�CMPDLT

SPTB�BCG�WBHSBOUCPYFT�IFSPLV

QBVM��4NBMMFTU�'FEFSBUFE�8JLJ

NJOSL�[HVJEF

UIFLJE�YQ�SVOOFST

0DSBNJVT�QIQDS�PEN

BN��[FODPEJOH

SVCZ�BNRQ�SVCZCVOOZ�JOGP

[HSBNBOB�SPVUFS�KT

KPIONEPOBIVF�OQNKT�PSH

OJDLTQ�HFOFSBUPS�BOHVMBS

DBCMFHSBN�HVBSE�SBJMT

USPPQKT�USPPQKT�OFU

OPEFKJUTV�GPSFWFS�NPOJUPS

HSFHLBSF�KFLZMM

TZNGPOZ�&WFOU%JTQBUDIFS

LNDBMMJTUFS�IUNM�

LBTQFSJTBHFS�WBOJMMB�CPPUTUSBQ

FDPNGF�FEQ�DPEFHFO

KNFBT�HSVOU

NIJU[B�1SPQFM�

QJHP[�NQW

NPOP�NPOP�UPPMT

TJNPOTUFS�)%'��KMTMFVUILJU�TMFVUILJU

NBDHJUWFS�MJC)FBWFO

TQBDFUFMFTDPQF�TUTZOQIPU

TBNPIU�IUDBNM

3FTQFDU�-PBEFS

TDIXFJLFSU�GQJOH

NBQCPY�DBSNFO

ESCSBJO�OFU�IUUQ�QFSTJTUFOU

NBSJKOI�UFSO@GPS@TVCMJNFTKBDLNBO�BCZTT

3VCFO7FSCPSHI�/��KT

NPKPNCP�HJUIVC�GMBWPSFE�NBSLEPXO

KRVFSZ��EFW�TVNNJU

EPDUSJOF�NJHSBUJPOT

BMQIBHPW�QVQQFU�KFOLJOT

HJU�HJU

NBSJKOI�$PEF.JSSPS�

)JQ#ZUF�NPUJPO�DPDPBQPET

TUSFBN�VUJMT�SBX�CPEZ

0SBOHF�0QFO4PVSDF�:"$BTTBOESB1%0

6ODPEJO�CZQBTT

ENBKEB�QFHKT

TFBUUMFSC�NJOJUFTU�CBDPO

EBCFB[�QMZ

BFPMVT�JODVCBUPS�CVOEMFS@FYU

JTBBDT�SJNSBG

XLIUNMUPQEG�RU

DBSMIVEB�CVOEMFS�TJUF

DMPVEIFBE�MFTTDTT�PSH

OKI�SFETUPSF

LFJUIX�NPTI

GMZFSI[N�TXJUDI@VTFS

KOS�KOS�DPOTUBOUTKSVCZ�CZUFMJTU

NBQCPY�UJMFMJWF�KT

TVTFTUVEJP�TUVEJP�IFMQ

SQGMPSFODF�FNCFS�RVOJU

NTZTHJU�(JU�$IFFUBI

DIPCJF�QIQ�VW

+VMJB-BOH�PQFOMJCN

XLIUNMUPQEG�XLIUNMUPQEG�HJUIVC�JP

EFWFMPQNFOUTFFE�OPEF�HFU

OPNJEEMFOBNF�MPH�KT�OPEF

EBIMCZL�QPTI�HJU

PDBNM�DBNMQ�KPTI�CSFX�QJQ

MPPQK�KRVFSZ�UPLFOJOQVU

SFTPMWF�SFGJOFSZDNT�CMPH

NP[JMMB�TFSWP�SVTU�IUUQ

JPOSPDL�DBDIFDPOUSPM

EJTUMFS�TZOUBY

NZSPONBSTUPO�WDS

0QFO4USFFU.BQ�J%

CFSHJF�EOPEF�QIQ

KMPOH�TBTT�CPPUTUSBQ

[G�GS�[GS�PBVUI��TFSWFS�NPEVMF

TLBFT�SWN�QBUDITFUT

GHSFIN�CJOEMFS

*DFDBWF4UVEJPT�XPPEIPVTF

TEFHVUJT�[FQIZSPT

SPNBOCTE�GBTU�TUFNNFS

QBSOEU�TFP@NFUB

LS�QUZ

CBTIP�FSMBOH@KT

QFUFSNJDIBVY�NBSJB

SBZP�YNQQ

TQBTUPSJOP�SBJMT�BQJ

TVCTUBDL�OPEF�EFUFDUJWF

YBWJFSTIBZ�STQFD�GJSF

NP[JMMB�TFSWP�SVTU�HFPN

NCPTUPDL�VT�BUMBT

UFDIOJDBMQJDLMFT�STQFD�TQJFT

WJTJPONFEJB�FYQSFTTP

[FETIBX�NPOHSFM�

DNVT�DNVT

MSB�NBDLVQ

CBTIP�MBHFS

UFOEFSMPWF�SBDD

CFOMFNBTVSJFS�TUPSNGT

VMUSBWJEFP�LWB[BBS

WBLBUB�KTUSFF

'SJFOET0G4ZNGPOZ�'04'BDFCPPL#VOEMF

DPEBIBMF�CDSZQU�SVCZ

MJC-"4�MJC-"4

USBWJT�DJ�USBWJT�SVCJFT

$POUJOVVN*0�BOBDPOEB�FD�

3JPU(BNFT�CFSLTIFMG�BQJ

OJQZ�OJUJNF

UBMEDSPGU�BTDJJUBCMF

OPTF�EFWT�OPTF

T�V�$BJSP

JTBBDT�JOJU�QBDLBHF�KTPO

KCBSOFUUF�JTPMBUF

CSBJOTQFD�FOVNFSJ[F

.D/FUJD�1)1;JQ4USFBNFS

DPQJPVTGSFFUJNF�IJUJNFT

UIPNBTCPZU�EFGFBUVSFJGZ

BOCB�SIJOP

1SPNZ-01I�QJBOPCBS

NBSDPEBOJFM�NEGSBNFE

HFPQIZTJDT�.P1B%

W��W�

BOHVMBS�VJ�BOHVMBS�VJ�EPDT

QBMBOUJS�UTMJOU

TFFCFFT�OPEF��

CBTIP�TJEFKPC

XQSFVM�OPEF�GPSL

NQBQJT�CVOEMFS�VOMPBE

BOESFBDSFNBTDIJ�4QBUJBM%#,JU

BWTN�NFMBOHF

OPGMP�OPGMP�HFTUVSFT

KTIJOU�TJUF

4IPQJGZ�BDUJWF@VUJMT

DPXCPZ�OPEF�GJOEVQ�TZOD

KUT�THB

SUPNBZLP�QPTJY�TQBXO

IBMM��BOHVMBS�VJ�CPPUTUSBQ

NBUIJF�CSPLFO@DVDVNCFS@EFNP

KSVCZ�KSVCZ�PTTM

QIMJQQFS�DIFG�NPOJU

NBEFCZNBOZ�TJS�USFWPS�KT

Figure 4: Ecosystems in the largest connected component of GitHub-hosted projects. Project

names follow the pattern user/repository where user is the owner’s GitHub login and repository

is the name of the project repository.

of the project, the number of stars the project has, the size of the associated

ecosystem, and the node’s degree. Each of these projects has a higher in-degree

than out-degree with the exception of the mxcl/homebrew project. On the

other hand, low-degree project nodes are four times as likely to be dependent

on another project than they are to have a project depend on them. This

shows that ecosystems are being formed around a central project with the other

projects in the ecosystem mostly depending on that central project. This results

28

Table 7: Ecosystems in GitHub. Details of the most well-connected node in each ecosystem.Project Description Stars Ecosystem Size Degree (in,out)

joyent/node Framework 39,373 10.08% 69 (53,16)

symfony/symfony Framework 10,985 8.46% 93 (53,40)

rails/rails Framework 29,744 7.92% 93 (65,28)

JuliaLang/julia Programming Language 5,531 6.74% 51 (35,16)

rubygems/rubygems Package Manager 1,304 6.04% 22 (14,8)

mxcl/homebrew Package Manager 13,723 3.94% 48 (21,27)

zendframework/zf2 Framework 5,841 3.88% 72 (65,7)

travis-ci/travis-ci Development Tool (Continuous Integration Platform) 3,693 3.50% 70 (54,16)

wet-boew/wet-boew Framework 688 3.34% 19 (15,4)

twbs/bootstrap Framework 41,828 3.29% 9 (9,0)

dbashford/mimosa Development Tool (Browser development) 472 2.43% 25 (20,5)

h5bp/html5-boilerplate Framework 31,926 2.37% 19 (15,4)

mitchellh/vagrant Framework 9,274 2.10% 23 (15,8)

libgit2/libgit2 Library 5,161 2.05% 20 (11,9)

Behat/Mink Development Tool (Testing) 673 1.99% 13 (9,4)

OCamlPro/opam Package Manager 118 1.89% 9 (8,1)

basho/riak Database 2,520 1.83% 27 (18,9)

Polymer/polymer Library 8,787 1.83% 16 (11,5)

mapnik/mapnik Development Tool (Toolkit for developing mapping applications) 1,003 1.78% 20 (12,8)

mozilla/rust Programming language 5,604 1.78% 36 (29,7)

alphagov/static Other (GOV.UK static files/resources) 67 1.73% 13 (10,3)

adobe/brackets Development Tool (code editor) 23,921 1.46% 26 (16,10)

CocoaPods/CocoaPods Development Tool (dependency manager) 5,711 1.46% 14 (9,5)

yeoman/yeoman Development Tool (web development tools) 7,246 1.46% 18 (13,5)

angular/angular.js Framework 42,950 1.40% 12 (8,4)

dotcloud/docker Development Tool (application container engine) 14,270 1.35% 24 (19,5)

emberjs/ember.js Framework 14,185 1.29% 20 (12,8)

owncloud/core Other (personal cloud storage tool) 3,222 1.19% 26 (13,13)

typhoeus/typhoeus Library 2,465 1.19% 6 (4,2)

facebook/hhvm Other (Virtual machine) 11,506 1.08% 15 (10,5)

celluloid/celluloid Framework 2,855 0.86% 9 (6,3)

xp-framework/rfc Framework 0 0.86% 16 (14,2)

rogerwang/node-webkit Framework 19,737 0.86% 16 (11,5)

ecomfe/edp Development Tool (front-end development platform) 264 0.86% 18 (15,3)

kennethreitz/requests Library 13,812 0.81% 13 (10,3)

documentcloud/underscore Library 7,135 0.81% 6 (4,2)

middleman/middleman Development Tool (website generator) 4,179 0.75% 8 (5,3)

elasticsearch/elasticsearch Other (search and analytics tool) 10,700 0.70% 11 (11,0)

chapmanb/bcbio-nextgen Other (RNA-seq analysis tool) 173 0.59% 10 (9,1)

wp-cli/wp-cli Development Tool (command line interface for WordPress) 1,968 0.59% 13 (9,4)

cucumber/cucumber Development Tool (Testing) 5,142 0.49% 7 (4,3)

jsdoc3/jsdoc Development Tool (API documentation generator) 2,909 0.49% 6 (3,3)

propelorm/Propel Development Tool (Object-Relational Mapping) 893 0.49% 7 (7,0)

in a star pattern. The twbs/bootstrap ego network (Figure 5) clearly depicts

this pattern within the graph.

Predominant type of ecosystems is software development support. Interest-

ingly, nearly all of the ecosystems are centered around projects whose purpose

29



UXCT�CPPUTUSBQUXJUUFS�SFDFTT

UPED�UPED�CPPUTUSBQ'PSU"XFTPNF�'POU�"XFTPNF






Figure 5: twbs/bootstrap Ego Network. Portraying a sample star pattern in the network.

Table 8: Ecosystem Types. Nearly all support software development.

Type Count

Software Development Tool 14

Framework 13

Library 5

Package Manager 3

Programming Language 2

Database 1

Other 5

is to support software development, such as frameworks, libraries and program-

ming languages. In fact, as shown in Table 8, of the 43 ecosystems, there are

only 5 whose purpose is not to support software development. The 14 software

development tools include a testing tool, a continuous integration platform, and

an API documentation generator. The type of each ecosystem is also shown in

Table 7.

Ecosystems are interconnected. The graph in Figure 3 shows two types of

communities that occur in GitHub-hosted projects, those that are part of the

largest connected component and those that are isolated from the largest con-

30

nected component. The majority of project nodes, 10,484 or 57%, are involved

in the largest connected component, indicating that many ecosystems are con-

nected to each other across the projects in our Dependency Network. The next

biggest connected component in the graph is only 65 nodes indicating that the

ecosystems that are isolated are small and have not attracted public attention.

Figure 4 displays the interconnected part of the network, and the connec-

tions between the ecosystems are apparent. As an example, Figure 6 shows

the rubygems/rubygems ego network, clearly depicting its connection to the

rails/rails project. This is not surprising, since the rubygems project is a pack-

age management framework for the Ruby programming language and rails/rails

is a web application framework written in Ruby. There is a direct connection

between the rubygems/rubygems and rails/rails nodes. In addition, there are

projects, like carlhuda/bundler and airblade/paper trail, which connect the two

projects.

Answer to RQ2: The Reference Coupling method can be used to

identify and examine ecosystems. The predominant type of ecosystems

on GitHub is software development support. Ecosystems tend to revolve

around one central project and be interconnected to other ecosystems.

5.3.2. Investigation of Socio-Technical Alignment within the Ecosystems

RQ3: Do the project owners’ and contributors’ social behaviours align with

the technical dependencies?

Project Owners: Table 9 shows strong, positive correlations between the

technical dependencies and the social behaviour of the owners. Along with these

strong correlations, Figure 7 shows a pronounced star pattern in the Owner

Follows Network. This indicates that the project owners in an ecosystem tend

to follow the owner of the central repository.

Project Contributors. As shown in Table 10, the social behaviour of project

contributors does not align with the technical dependencies. This indicates

31

CVOEMFS�CVOEMFS

XBZOFFTFHVJO�SWN

SBJMT�SBJMT

KSVCZ�KSVCZ

STQFD�STQFD�SBJMT

HET�PQFSBUJPOT�QVQQFU�FMBTUJDTFBSDI

DBSMIVEB�CVOEMFS

SVCZHFNT�SVCZHFNT

POFDMJDL�SVCZJOTUBMMFS

SVCJOJVT�SVCJOJVT

SVCZHFNT�SVCZHFNT�PSH

KSVCZ�XBSCMFS

BJSCMBEF�QBQFS@USBJM

SEPD�SEPDIFBEJVT�UISFBE@TBGF

GSFFSBOHF�NPDIB

HMPCBMJ[F�HMPCBMJ[F

XIJU��XXX�HJUUJQ�DPN

NT�BUJ�EPDJMF

SVCZ�SVCZ

XFCCK��QVQQFU�GJUCJU

CVOEMFS�CVOEMFS�GFBUVSFT

UIPSJO�SFENJOF@MEBQ@TZOD

SVCZHFNT�HVJEFT

KCBSOFUUF�JTPMBUF

Figure 6: rubygems/rubygems Ego Network. Portraying connections between ecosystems.

Table 9: Project Owners: correlations between technical dependencies and social behaviour.

Pearson Correlation p-value

Technical Dependencies and Following 0.91 <0.001

Technical Dependencies and Stars 0.79 <0.001

that, while the project owners seem to follow the right people and are aware of

the right projects based on the technical dependencies that exist in the ecosys-

32

Figure 7: The Owner Follows Network, Gof .

Table 10: Project Contributors: correlations between technical dependencies and social be-

haviour.

Pearson Correlation p-value

Technical Dependencies and Following 0.0002 0.98

Technical Dependencies and Stars 0.001 0.88

tem, the social behaviour of project contributors is not aligned with project

dependencies.

Figure 8 shows the Contributor Follows Network. As shown, the structure

is quite di↵erent than the Dependency Network. Communities do not have one

central project and the network is much more densely connected.

Answer to RQ3: The Reference Coupling method can be useful for

other investigations of software ecosystems where identification of inter-

project technical dependencies is needed. We investigated social behaviour

in software ecosystems. We found that the project owners social behaviours

33

Figure 8: The Contributor Follows Network, Gcf .

do align with the technical dependencies, but the project contributors social

behaviours do not align with the technical dependencies.

6. Discussion

The method we proposed in this paper, Reference Coupling, identifies cross-

references to other projects within a project’s ecosystem in the comments made

by developers on project artifacts like issues, commits, pull requests or work

items. We showed that the method outputs are a valid conceptualization of

technical dependencies by analyzing the content of these cross-references and

comparing the cross-references to dependency relationships identified by the

development team.

Our method adds to the important, but scarce, research that leverages the

social aspects of work within software ecosystems [13]. Reference Coupling de-

tects technical dependencies that may not manifest themselves in source code by

34

identifying tasks, issues, pull requests, or commits that rely on another project,

and, therefore, it can identify dependencies not identified by other methods.

Our results show that the cross-references identify many di↵erent types of de-

pendencies including duplicates, a↵ecting, blocking, and related relationships.

Our method is analogous to the logical coupling method that detects de-

pendencies within a project proposed by Gall et al. [42] except at the ecosys-

tem level. Where logical coupling detects dependencies when artifacts have

been worked on together, our method detects dependencies when issues, pull

requests or commits have been worked in conjunction with another project (as

evidenced through user-specified cross-references). Thus, the dependencies es-

tablished through our method are those that are logical.

Limiting the detected dependencies to those that are logical is important

when using those dependencies to identify ecosystems. Methods that detect

technical dependencies between projects through analysis of code or configu-

ration files may not be best suited for identifying software ecosystems. For

example, when one project uses another project, it does not necessarily mean

the two software projects are evolving together in the same environment, espe-

cially when the dependency is to an established, o↵-the-shelf software package.

Thus, identifying all relationships that manifest in the source code or configura-

tion files may result in dependencies that are not important for the identification

of ecosystems.

For the GitHub projects, we used Reference Coupling to identify and ex-

plore ecosystems. To do this, we used a popular community detection algo-

rithm [15] on the dependency network, which identifies clusters of nodes densely

connected by technical dependencies. These detected communities represent

software ecosystems. Through analysis of the resulting ecosystems found in

GitHub-hosted projects, we showed that the ecosystems are centered mostly

around projects that support software development through developing frame-

works and toolkits. The predominant structure of the ecosystems is a star where

one central project is the hub of the ecosystem.

Our method also allows for the identification of dependencies across ex-

35

tremely large sets of projects. For example, we ran our method on all public

projects hosted on GitHub. Other methods that detect dependencies are limited

to analyzing a given project or set of projects. Analyzing the dependencies of

a popular project through its source code or configuration files to identify its

ecosystem would not identify projects that rely on that project. We saw that

most ecosystems across the GitHub-hosted projects are centered around one

main project and many projects depend on that project without a reciprocal

relationship. These relationships would be missed if only the dependencies of

the main project were studied to identify its ecosystem. Since the ecosystems

are not always well-defined, it would be impossible to know which other projects

to consider for analysis. Thus, our method is better suited to identifying ecosys-

tems since it is not limited in the number of projects it can analyze.

6.1. A Research Agenda

The ability to easily identify technical dependencies between a large set of

projects opens the door for many interesting avenues of research.

Socio-technical analysis. Studies that have attempted to study how commu-

nication aligns with dependencies across projects have been limited to studying

well-defined ecosystems where dependency information is publicized in some

way. For example, dependencies can be made available through a project’s

configuration files, build files, or through publicly available dependency specifi-

cations [1, 7, 8, 9, 10, 23, 11]. Our method allows the identification of technical

dependencies more broadly across projects and opens the door to continuing

the study of socio-technical alignment across a larger set of projects and their

stakeholders.

In this study, we found that when dependencies exist between a pair of

projects, the project owners tend to be following the owner of the other project.

Conway was the first to describe the possibility of an alignment between social

connections and technical dependencies in software engineering projects, com-

monly referred to as Conway’s law [43]. The transparent nature of GitHub could

encourage technical connections between projects by providing an awareness of

36

activity across projects. An interesting future research question is understand-

ing how and when these technical dependencies and social connections came to

exist. Did the social connections exist first and result in a technical dependency

or did the technical dependency exist first and result in a social connection? If

the social connections existed first, what was the driver behind the creation of

the technical dependency? Perhaps, the awareness of the other project, enabled

through GitHub’s notifications, was enough to spur a technical dependency in-

dicating that GitHub’s transparency is changing the landscape of OSS projects.

These research questions could be investigated in future research.

While the project owners’ social behaviours (following users and starring

projects) aligned with the technical dependencies in our study, we did not wit-

ness such an alignment for all project contributors. The follower network of

project contributors showed that there were no clear central projects and com-

munities were densely connected. This is in contrast to the technical dependency

network. These results align with recent research that found that the reasons

behind following others extends beyond project coordination needs [44]. Fu-

ture work should investigate the usefulness of following others for coordination

purposes.

It is also worth studying in more detail the coordination needs of developers

on OSS projects. Perhaps the mere existence of a technical dependency does

not imply a coordination need, especially given the transparent environment

of GitHub. Our previous work [45] begun this investigation, but coordination

needs at the ecosystem level are also worthy of investigation.

Ecosystem emergence and evolution. The most prominent nodes in Fig-

ure 4 are not always the most popular projects on GitHub when considering

the number of stars each project has. In fact, the two projects with the most

stars, angular/angular.js and twbs/bootstrap, have significantly smaller ecosys-

tem size and lower degree than other projects. Future work can investigate how

and why ecosystems emerge and why some projects become popular without

growing a large ecosystem. Such a study could include a temporal analysis of

37

the composition of the ecosystem and density of connections together with a

temporal analysis of project history information such as number of contribu-

tors, forks, stars, etc. It would also be worth triangulating results with other

information on important project events now commonly available through blogs

and wikis. Such a study on the evolution of ecosystems can be a first step in

understanding when and why projects accumulate an ecosystem.

Previous research has examined ecosystem growth [46], but this analysis was

focused on the size of the code base (measured in lines of code) of all projects

within an organization. We propose for future research to expand this view by

considering the growth of an ecosystem also based on the number of projects

that the ecosystem comprises of when considering technical dependencies.

Ecosystem size and strength of connections and project success. On many

open source projects, volunteers are crucial to project success as they rely on

volunteers to submit new features and fix bugs. As a project accumulates more

projects in its ecosystem, it is also likely to increase its contributions as devel-

opers on dependent projects will be more likely to fix bugs that they encounter

through their dependency. Future research could investigate this relationship to

identify if the size of a project’s ecosystem is a good predictor of various project

health and success metrics like the number of contributions it receives or the

number of forks it has.

Automatic detection of inter-project dependencies. Another avenue for future

research is creating tools to make developers aware of their technical dependen-

cies outside of their own project. It is important for developers to know who

they need to coordinate with across the ecosystem and to understand how their

tasks fit into the big picture. For example, when a developer on GitHub creates

a cross-reference to an issue on another project, it could be useful for the de-

veloper to be made aware of other projects that have also cross-referenced that

same issue. The issue may be causing problems in many projects and those

project could minimize duplicated e↵ort by being more aware of each other.

Such a tool could increase awareness of coordination needs that extend outside

38

Figure 9: A proposed improvement to GitHub’s Flavored Markdown, which would not only

create a link to the pull request, issue or commit referenced in the comment, but would also

allow users to click to see what other comments in other projects have also made the same

reference.

project boundaries. For example, in Figure 9, we show a prototype we developed

that improves GitHub’s Flavored Markdown by allowing users to click on the

“Get Suggestions” button to see a list of other comments from other projects

that have made a reference to the same pull request, issue or commit. If there

is a large number of other projects that share the dependency, natural language

processing techniques could be used to summarize the information. This will

allow developers to easily review the details from the other projects that share

the same dependency. Similarly, other tools, including those used at IBM, could

be modified to help developers become aware of inter-project dependencies that

are automatically detected by our Reference Coupling method. Reference Cou-

pling could be used to automatically create dependency relationships between

issues or work items in a project’s issue tracking system.

Automatic detection of inter-project dependencies can also be used by soft-

ware engineering researchers in future unrelated studies. For example, a study

of the e↵ects of multi-tasking across multiple projects could include an analysis

on the dependencies that exist between the projects to better understand the

39

reasons for multi-tasking or the amount of context switch that occurs. Many

other software engineering research studies can benefit from an easy way to

identify inter-project dependencies.

Automatic detection of ecosystems. With the ability to automatically detect

individual technical dependencies between projects, it would also be useful to

automatically detect and visualize ecosystems. Such a tool could help developers

gain a better view of the ecosystem surrounding their project. It could also help

researchers in future studies on software ecosystems.

6.2. Threats to Validity

One threat stems from our selection of the GHTorrent dataset to obtain

the GitHub data. GHTorrent may not be a full copy of all GitHub data [32].

Nevertheless, it is a best-e↵ort approach that has been widely accepted in the

research community as evidenced by its inclusion as the dataset for the MSR

2013 Mining Challenge [47] and the many recent papers that utilize its data in

their analysis.

The GHTorrent dataset used in our analysis is from 2014. The structure and

dynamics of ecosystems could have changed since this dataset was captured. Fu-

ture studies should reevaluate ecosystems using more recent datasets. Such a

study can also consider ecosystem evolution since our results provide a snap-

shot that can be compared against. In this direction, Zhang et al. have used

our Reference Coupling method to identify the GitHub projects in the Rails

ecosystem [48]. They found that developers tend to make more cross-references

to other projects over time.

Our manual exploration of cross reference comments illustrates a variety

of types of technical dependencies found in GitHub cross-references, but these

results can not be generalized. While we achieved saturation in our results, our

results could be impacted by selection bias. To mitigate this, we ensured an

equal number of comments for each source (commit, issue, pull request) were

included in our sample. However, the types of dependencies identified seem

reasonable for any software project. Future work can continue this investigation

40

by examining the content of cross-reference comments across a wide range of

projects and code hosting environments.

Cross-references could appear more frequently when there are a higher the

number of shared contributors between the two projects. We have not controlled

for this since the presence of shared contributors does not diminish the existence

of the dependencies. However, it could mean that dependencies on projects

where there are no shared contributors are not easily found using this method.

Future work should investigate this.

7. Conclusion

In this paper, we proposed a new method for detecting technical dependen-

cies between projects, called Reference Coupling, which utilizes user-specified

cross-references between projects. We validated this method on datasets from

GitHub and IBM. We found that Reference Coupling identifies many dependen-

cies which appear to be untracked by developers. The most common type of

dependency that is untracked by developers but found with Reference Coupling

is ‘A↵ected by defect’, which indicates that a task is impacted in some way by

another defect. In these cases, especially, it could be useful for other develop-

ers to be aware of other projects that are impacted by the same defect. This

could allow these projects to coordinate their e↵orts in creating workarounds or

negotiating completion of the defect removal. The Reference Coupling method

enables tools to be developed that can help developers become aware of these

types of shared dependencies.

We also used our Reference Coupling method to identify ecosystems in

GitHub-hosted projects by using an existing community detection algorithm

to identify densely connected clusters of projects. Through an analysis of the

identified ecosystems, we find that most ecosystems are centered around a single

project. While small, unpopular ecosystems remain isolated, most ecosystems

are interconnected. The isolated ecosystems tend to contain projects owned

by the same GitHub user or organization. The popular ecosystems are mostly

41

centered around tools that support software development.

Our Reference Coupling method opens the door for future research in soft-

ware ecosystems including studying the socio-technical relationships, evolution,

health and success of ecosystems.

Acknowledgment

This work was partly funded by NSERC Canada. Thanks to Sunny Wang

and Diksha Sharma for their assistance in the manual content analysis.

References

[1] M. F. Lungu, Reverse engineering software ecosystems, Ph.D. thesis, Uni-

versity of Lugano (2009).

[2] D. Cubranic, G. C. Murphy, J. Singer, K. S. Booth, Hipikat: A project

memory for software development, Transactions on Software Engineering

31 (6) (2005) 446–465.

[3] O. Franco-Bedoya, D. Ameller, D. Costal, X. Franch, Open source software

ecosystems: A systematic mapping, Information & Software Technology 91

(2017) 160–185. doi:10.1016/j.infsof.2017.07.007.

URL https://doi.org/10.1016/j.infsof.2017.07.007

[4] J. Ossher, S. Bajracharya, C. Lopes, Automated dependency resolution

for open source software, in: Proceedings of 7th Working Conference on

Mining Software Repositories, IEEE, 2010, pp. 130–140.

[5] M. Lungu, R. Robbes, M. Lanza, Recovering inter-project dependencies in

software ecosystems, in: Proceedings of the International Conference on

Automated Software Engineering, ACM, 2010, pp. 309–312.

[6] J. Businge, A. Serebrenik, M. van den Brand, Survival of Eclipse third-

party plug-ins, in: Proceedings of 28th International Conference on Soft-

ware Maintenance, IEEE, 2012, pp. 368–377.

42

[7] F. W. Santana, C. M. L. Werner, Towards the analysis of software projects

dependencies: An exploratory visual study of software ecosystems., in: Pro-

ceedings of International Workshop on Software Ecosystems, Citeseer, 2013,

pp. 7–18.

[8] J. M. Gonzalez-Barahona, G. Robles, M. Michlmayr, J. J. Amor, D. M.

German, Macro-level software evolution: A case study of a large software

compilation, Empirical Software Engineering 14 (3) (2009) 262–285.

[9] G. Bavota, G. Canfora, M. Di Penta, R. Oliveto, S. Panichella, How the

apache community upgrades dependencies: An evolutionary study, Empir-

ical Software Engineering (2014) 1–43.

[10] D. M. German, J. M. Gonzalez-Barahona, G. Robles, A model to under-

stand the building and running inter-dependencies of software, in: Pro-

ceedings of 14th Working Conference on Reverse Engineering, IEEE, 2007,

pp. 140–149.

[11] S. Raemaekers, A. van Deursen, J. Visser, Measuring software library

stability through historical version analysis, in: Software Maintenance

(ICSM), 2012 28th IEEE International Conference on, IEEE, 2012, pp.

378–387.

[12] A. Mockus, Amassing and indexing a large sample of version control sys-

tems: Towards the census of public source code history, in: Proceedings of

6th Working Conference on Mining Software Repositories, IEEE, 2009, pp.

11–20.

[13] T. Mens, M. Goeminne, Analysing the evolution of social aspects of open

source software ecosystems, in: S. Jansen, J. Bosch, P. R. J. Campbell,

F. Ahmed (Eds.), Proceedings of the Third International Workshop on

Software Ecosystems, Brussels, Belgium, June 7th, 2011, Vol. 746 of

CEUR Workshop Proceedings, CEUR-WS.org, 2011, pp. 1–14.

URL http://ceur-ws.org/Vol-746/IWSECO2011-1-InvitedPaper-MensGoeminne.pdf

43

[14] T. Mens, B. Adams, J. Marsan, Towards an interdisciplinary, socio-

technical analysis of software ecosystems health, in: Proceedings of the

16th edition of the BElgian-NEtherlands software eVOLution symposium,

Antwerp, Belgium, December 4-5, 2017., 2017, pp. 7–9.

URL http://ceur-ws.org/Vol-2047/BENEVOL 2017 paper 2.pdf

[15] V. D. Blondel, J.-L. Guillaume, R. Lambiotte, E. Lefebvre, Fast unfolding

of communities in large networks, Journal of Statistical Mechanics: Theory

and Experiment 2008 (10) (2008) P10008.

[16] K. Blincoe, F. Harrison, D. Damian, Ecosystems in GitHub and a method

for ecosystem identification using reference coupling, in: Proceedings of the

12th Working Conference on Mining Software Repositories, IEEE Press,

2015, pp. 202–211.

[17] J. Bosch, From software product lines to software ecosystems, in: Soft-

ware Product Lines, 13th International Conference, SPLC 2009, San Fran-

cisco, California, USA, August 24-28, 2009, Proceedings, 2009, pp. 111–119.

doi:10.1145/1753235.1753251.

URL http://doi.acm.org/10.1145/1753235.1753251

[18] S. Jansen, A. Finkelstein, S. Brinkkemper, A sense of community: A

research agenda for software ecosystems, in: 31st International Confer-

ence on Software Engineering, ICSE 2009, May 16-24, 2009, Vancou-

ver, Canada, Companion Volume, 2009, pp. 187–190. doi:10.1109/ICSE-

COMPANION.2009.5070978.

URL https://doi.org/10.1109/ICSE-COMPANION.2009.5070978

[19] K. Manikas, Supporting the evolution of research in software ecosystems:

Reviewing the empirical literature, in: Software Business - 7th Interna-

tional Conference, ICSOB 2016, Ljubljana, Slovenia, June 13-14, 2016,

Proceedings, 2016, pp. 63–78. doi:10.1007/978-3-319-40515-5 5.

[20] M. Lungu, Towards reverse engineering software ecosystems, in: 24th

IEEE International Conference on Software Maintenance (ICSM 2008),

44

September 28 - October 4, 2008, Beijing, China, 2008, pp. 428–431.

doi:10.1109/ICSM.2008.4658096.

URL https://doi.org/10.1109/ICSM.2008.4658096

[21] M. Goeminne, T. Mens, A framework for analysing and visualising open

source software ecosystems, in: Proceedings of the Joint ERCIM Workshop

on Software Evolution (EVOL) and International Workshop on Principles of

Software Evolution (IWPSE), Antwerp, Belgium, September 20-21, 2010.,

2010, pp. 42–47. doi:10.1145/1862372.1862384.

[22] D. Dhungana, I. Groher, E. Schludermann, S. Bi✏, Software ecosystems vs.

natural ecosystems: Learning from the ingenious mind of nature, in: Pro-

ceedings of the 4th European Conference on Software Architecture: Com-

panion Volume, ACM, 2010, pp. 96–102.

[23] M. Syeed, K. M. Hansen, I. Hammouda, K. Manikas, Socio-technical con-

gruence in the Ruby ecosystem, in: Proceedings of The International Sym-

posium on Open Collaboration, ACM, 2014, p. 2.

[24] J. Kabbedijk, S. Jansen, Steering insight: An exploration of the Ruby

software ecosystem, in: Software Business, Springer, 2011, pp. 44–55.

[25] S. Jansen, Measuring the health of open source software ecosystems: Be-

yond the scope of project health, Information & Software Technology

56 (11) (2014) 1508–1519. doi:10.1016/j.infsof.2014.04.006.

[26] M. Lungu, J. Malnati, M. Lanza, Visualizing Gnome with the small project

observatory, in: M. W. Godfrey, J. Whitehead (Eds.), Proceedings of

the 6th International Working Conference on Mining Software Reposito-

ries, MSR 2009 (Co-located with ICSE), Vancouver, BC, Canada, May

16-17, 2009, Proceedings, IEEE Computer Society, 2009, pp. 103–106.

doi:10.1109/MSR.2009.5069487.

URL https://doi.org/10.1109/MSR.2009.5069487

45

[27] K. Manikas, K. M. Hansen, Software ecosystems–a systematic literature

review, Journal of Systems and Software 86 (5) (2013) 1294–1306.

[28] S. Syed, S. Jansen, On clusters in open source ecosystems, in: Proceedings

of International Workshop on Software Ecosystems, Citeseer, 2013, pp. 19–

32.

[29] Y. Yu, G. Yin, H. Wang, T. Wang, Exploring the patterns of social behavior

in GitHub, in: Proceedings of the 1st International Workshop on Crowd-

based Software Development Methods and Technologies, ACM, 2014, pp.

31–36.

[30] F. Thung, T. F. Bissyande, D. Lo, L. Jiang, Network structure of social

coding in GitHub, in: Proceedings of 17th European Conference on Soft-

ware Maintenance and Reengineering, IEEE, 2013, pp. 323–326.

[31] J. MacFarlane, Commonmark spec. 2017, URL http://spec. commonmark.

org/0.25.

[32] G. Gousios, D. Spinellis, GHTorrent: GitHub’s data from a firehose, in:

Proceedings of the 9th Working Conference on Mining Software Reposito-

ries, IEEE, 2012, pp. 12–21.

[33] E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German,

D. Damian, The promises and perils of mining GitHub, in: Proceedings

of the 11th Working Conference on Mining Software Repositories, ACM,

2014, pp. 92–101.

[34] Oracle, Java platform standard edition 7

api specification: Package java.util.regex,

https://docs.oracle.com/javase/7/docs/api/java/util/regex/package-summary.html,

accessed 29 November 2016.

[35] K. A. Neuendorf, The Content Analysis Guidebook, Sage, 2016.

46

[36] J. Corbin, A. Strauss, Basics of Qualitative Research: Techniques and Pro-

cedures for Developing Grounded Theory, Sage, 2008.

[37] D. Freelon, Recal oir: Ordinal, interval, and ratio intercoder reliability as

a web service., International Journal of Internet Science 8 (1).

[38] M. E. Newman, Modularity and community structure in networks, Pro-

ceedings of the National Academy of Sciences 103 (23) (2006) 8577–8582.

[39] M. Bastian, S. Heymann, M. Jacomy, et al., Gephi: an open source soft-

ware for exploring and manipulating networks., Proceedings of Interna-

tional AAAI Conference on Web and Social Media 8 (2009) 361–362.

[40] M. Molloy, B. Reed, Critical subgraphs of a random graph, The Electronic

Journal of Combinatorics 6 (R35) (1999) 2.

[41] J. M. Kleinberg, Authoritative sources in a hyperlinked environment, Jour-

nal of the ACM 46 (5) (1999) 604632.

[42] H. Gall, K. Hajek, M. Jazayeri, Detection of logical coupling based on

product release history, in: Proceedings of International Conference on

Software Maintenance, IEEE, 1998, pp. 190–198.

[43] M. E. Conway, How do committees invent, Datamation 14 (4) (1968) 28–31.

[44] K. Blincoe, D. Damian, Implicit coordination supported by GitHub: A

case study of the rails oss project, in: Open Source Systems: Adoption and

Impact, Springer, 2015, pp. 35–44.

[45] K. Blincoe, G. Valetto, D. Damian, Do all task dependencies require co-

ordination? the role of task properties in identifying critical coordination

needs in software projects, in: Proceedings of the 9th Joint Meeting of

the European Software Engineering Conference and the ACM SIGSOFT

Symposium on the Foundations of Software Engineering, ACM, 2013, pp.

213–223.

47

[46] J. W. Paulson, G. Succi, A. Eberlein, An empirical study of open-source

and closed-source software products, IEEE Transactions on Software Engi-

neering 30 (4) (2004) 246–256. doi:10.1109/TSE.2004.1274044.

[47] G. Gousios, The GHTorent dataset and tool suite, in: Proceedings of the

10th Working Conference on Mining Software Repositories, IEEE Press,

2013, pp. 233–236.

[48] Y. Zhang, Y. Yu, H. Wang, B. Vasilescu, V. Filkov, Within-ecosystem issue

linking: a large-scale study of rails, in: Proceedings of the 7th International

Workshop on Software Mining, ACM, 2018, pp. 12–19.

48

reference coupling: an exploration of inter-project

Documents