engineering the new linkedin profile

Engineering the New Profile

Josh Clemm Tech Lead for the New Profile

March 2013 | LinkedIn

David Fleming!Senior Product Manager at Zoomjax!San Francisco Bay Area | Software!!Previous!Education!

Golden Phase, FixDex!Silicon Valley Business Academy!

Let's compare...

Really a brand new product Refreshed Look & Feel Simplified Design

Surfaced New & Interactive Modules

New Data Insights

New Features Improved in-line

editing experience Modules with In-line

searching & pagination

The New Profile - Goals ●  Represent your entire professional identity

o  Not just your resume

o  Activity, Groups, Following, Connections, Insights about you and your network




●  Needs to be more interactive

o  Keep users engaged on the page

o  Inline pagination, editing, searching





o  Keep users engaged on the page

o  Inline pagination, editing, searching

●  Needs to be fluid, flexible, fast o  Progressive rendering

o  Maintain high performance

So how did we achieve all that?

Using a lot of Technologies

And in particular...

Let's look at our High Level Frontend Architecture

Groups Content Service

Connections Content Service

Profile Content Service

Client/Browser CDN

Load Balancer

SCDS

Fizzy

Profile Web App


Profile Web App



Dust/JS/CSS Server

Retrieve Data Models from

Mid-tier

/profile/view?id=32 top_card.tl, background.tl

/profile/topcard /profile/background




Client/Browser CDN

Load Balancer

SCDS

Fizzy

Profile Web App


Profile Web App



Dust/JS/CSS Server

Retrieve Data Models



Our new architecture uses new tech at all layers




Client/Browser CDN

Load Balancer

SCDS

Fizzy

Profile Web App


Profile Web App



Dust/JS/CSS Server




Let's start at the bottom with Mappers

Mappers - JSON endpoints ●  Convert data models from mid-tier services

into JSON

●  Each have an unique endpoint URL ( /profile/positions?id=42 )

Positions Mapper

JSON

"positions": [{ "position": {"id:{}"}, ]}

Biz Profile Model

Profile Model

Rich Media Rest Model

References Model

Profile Flex Model

Mappers - an example public class PictureMapper extends ProfileParametersAwareMapper !{ ! private PictureContentMap pictureContentMap; ! private static final int ZOOMABLE_DIMENSION = 225; ! ! @Override ! public void doService() ! { ! PictureContentModel picCM = getContent(PictureContentModel.class); //declare content needs ! picCM.criteria().setId(getVieweeId()); //supply any input params ! ! assemble(); //invoke framework to retrieve declared content ! ! if (isResolvedWithoutErrors(picCM)) ! { ! //all went well, create new content map to hold output (JavaBean-like objects) ! pictureContentMap = ContentMap.proxyNew(PictureContentMap.class); ! ! Integer pictureWidth = picCM.getPictureWidth(); ! pictureContentMap.setIsZoomable(pictureWidth >= ZOOMABLE_DIMENSION); ! ! if(pictureWidth != null && pictureWidth > 0) ! { ! pictureContentMap.setWidth(pictureWidth); ! pictureContentMap.setHeight(picCM.getPictureHeight()); ! } ! pictureContentMap.setPictureID(picCM.getPictureID()); ! } ! } ! // tell framework to add our map to final output (uses Jackson to process into JSON) ! addOutput(pictureContentMap); ! } !

1 !2 !3 !4 !5 !6 !7 !8 !9 !

10 !11 !12 !13 !14 !15 !16 !17 !18 !19 !20 !21 !22 !23 !24 !25 !26 !27 !28 !29 !30 !31 !32 !

Mappers - an example public class PictureMapper extends ProfileParametersAwareMapper !{ ! private PictureContentMap pictureContentMap; ! private static final int ZOOMABLE_DIMENSION = 225; ! ! @Override ! public void doService() ! { ! PictureContentModel picCM = getContent(PictureContentModel.class); //declare content needs ! picCM.criteria().setId(getVieweeId()); //supply any input params ! ! assemble(); //invoke framework to retrieve declared content ! ! if (isResolvedWithoutErrors(picCM)) ! { ! //all went well, create new content map to hold output (JavaBean-like objects) ! pictureContentMap = ContentMap.proxyNew(PictureContentMap.class); ! ! Integer pictureWidth = picCM.getPictureWidth(); ! pictureContentMap.setIsZoomable(pictureWidth >= ZOOMABLE_DIMENSION); ! ! if(pictureWidth != null && pictureWidth > 0) ! { ! pictureContentMap.setWidth(pictureWidth); ! pictureContentMap.setHeight(picCM.getPictureHeight()); ! } ! pictureContentMap.setPictureID(picCM.getPictureID()); ! } ! } ! // tell framework to add our map to final output (uses Jackson to process into JSON) ! addOutput(pictureContentMap); ! } !

1 !2 !3 !4 !5 !6 !7 !8 !9 !

10 !11 !12 !13 !14 !15 !16 !17 !18 !19 !20 !21 !22 !23 !24 !25 !26 !27 !28 !29 !30 !31 !32 !

Declare the data you need

Set the data you want coming back as JSON

Mappers - features ●  Modular

o  A single mapper can retrieve data for a section §  Positions, Educations, Groups, etc.

●  Reusable & Combinable

o  Mappers can be used more than once §  Positions Mapper is used for Positions section and the

positions part of Top Card

o  You can Aggregate Mappers under a common root element and a new URL endpoint

Profile's Many Mappers

●  Each section on Profile has either a single Mapper or Aggregated Mapper (like Top Card) for its data

Profile's Many Mappers

●  Each section on Profile has either a single Mapper or Aggregated Mapper (like Top Card) for its data

Summary Mapper

Positions Mapper

Educations Mapper

Picture Mapper

Top Card Mapper

"TopCard": { "positions": {}, "educations":{}, "picture":{}}

Profile Web App

Connections Mapper

JSON "Summary": { "summary": "I'm an experienced..."}

JSON

So we have these Mappers that return JSON for each section.

Who calls each one?




Client/Browser CDN

Load Balancer

SCDS

Fizzy

Profile Web App


Profile Web App



Dust/JS/CSS Server




Fizzy - the UI aggregator

Fizzy

●  Fizzy is an UI aggregator in 2 parts: o  Fizzy Server fetches the content you

want o  Fizzy Client renders it when ready

●  Your base page defines its structure and

which UI components it needs (called "embeds")

*Fizzy Server is an Apache Traffic Server Plugin, Fizzy Client is a JS library

Profile's Embeds

Top Card Embed

Activity Embed

Embed

Embed

Embed

Embed

Yet another embed

Profile's Embeds in code <html> <body> ... <div id=“wrapper”> <div id=“profile”> <script type=“embed” fs-id=“topcard” fs-uri=“/profile/topcard”/> <script type=“embed” fs-id=“background” fs-uri=“/profile/background”/> ... <script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/> </div> <div id=“insights”> <script type=“embed” fs-id=“people_you_may_know” fs-uri=“/profile/pymk”/> <script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/> ... <script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/> </div> </div> </body> </html>

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19

Profile's Embeds in code <html> <body> ... <div id=“wrapper”> <div id=“profile”> <script type=“embed” fs-id=“topcard” fs-uri=“/profile/topcard”/> <script type=“embed” fs-id=“background” fs-uri=“/profile/background”/> ... <script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/> </div> <div id=“insights”> <script type=“embed” fs-id=“people_you_may_know” fs-uri=“/profile/pymk”/> <script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/> ... <script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/> </div> </div> </body> </html>

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19

Each embed specifies a Mapper endpoint that Fizzy will fetch

Fizzy fetches data and sends it to the browser.

What's rendering the actual

markup?




Client/Browser CDN

Load Balancer

SCDS

Fizzy

Profile Web App


Profile Web App



Dust/JS/CSS Server




You guessed it... Dust client templates

{Dust} client templates

●  LinkedIn's latest and greatest rendering layer

●  Logic-less client template language

●  Profile page made up of many* templates

●  Templates + JSON = full markup!

*over 400 actually

{Dust} - what it looks like

<div id="top_card"> <h4>{name}</h4> <h5>{headline}</h5> {#info} <h6>{location} | {industry}</h6> {/info} </div>

{ "name": "Frank Stallone", "headline": "Actor and Less Famous Brother", "info":{ "location": "Hollywood", "industry": "Entertainment" } }

{Dust} - why it's cool for profile

●  Cached markup in CDNs and browsers o  Members browse many profiles in a row

●  Very DRY and reusable o  Improved development speed

o  Templates within templates within templates... §  date range, degree badge, formatted summary field

●  Super easy to refresh a section on the page o  Re-render and replace o  Useful in pagination, inline searching, and inline

editing

{Dust} on profile <hgroup> {>"tl/apps/profile/v2/embed/company_logo"/} <h4> <a href="{link_title_pivot}" name='title'>{title_highlight|s}</a> {?selfView} <span class="edit-tools"> <a class="edit-section">{i18n_Edit}</a> </span> {/selfView} </h4> <h5> {?companyName} {>"tl/apps/profile/v2/embed/company_link" track_param="prof-exp"/} {/companyName} </h5> </hgroup> <span class="experience-date-locale"> {>"tl/apps/profile/v2/partial/daterange"/} {! Location !} {@pre.fmt key="fmt_location" type="geo.region" value="{location}" render="false"/} {?fmt_location}<span class="locality">{fmt_location}</span> {:else} {?locationName} <span class="locality">{locationName}</span> {/locationName} {/fmt_location} </span> {>"tl/apps/profile/v2/partial/summary_field" _summary=summary/} {>"tl/apps/profile/v2/partial/associated_content" trkCodePrefix="exp"/}

{Dust} on profile <hgroup> {>"tl/apps/profile/v2/embed/company_logo"/} <h4> <a href="{link_title_pivot}" name='title'>{title_highlight|s}</a> {?selfView} <span class="edit-tools"> <a class="edit-section">{i18n_Edit}</a> </span> {/selfView} </h4> <h5> {?companyName} {>"tl/apps/profile/v2/embed/company_link" track_param="prof-exp"/} {/companyName} </h5> </hgroup> <span class="experience-date-locale"> {>"tl/apps/profile/v2/partial/daterange"/} {! Location !} {@pre.fmt key="fmt_location" type="geo.region" value="{location}" render="false"/} {?fmt_location}<span class="locality">{fmt_location}</span> {:else} {?locationName} <span class="locality">{locationName}</span> {/locationName} {/fmt_location} </span> {>"tl/apps/profile/v2/partial/summary_field" _summary=summary/} {>"tl/apps/profile/v2/partial/associated_content" trkCodePrefix="exp"/}

Partial templates like this are used on almost all the background sections

We have our Mappers, Fizzy, and Dust templates...

Let's bring it all together.

Feature: Inline Editing

●  Dust + Mappers make this easy ●  Dust templates for view and edit

●  Mappers return just the data we need to refresh

"View" template

"Edit" template

Summary Section




"View" template

"Edit" template

Summary Section When edit link is clicked, render "edit" template




"View" template

"Edit" template

Summary Section

On submit, our endpoint sends back JSON, and we re-render

Inline editing - Example

Either the "view" or "edit" template is shown at a time

Inline editing sounds easy...

●  Issue: Profile data is highly coupled with different sections.

o  Adding/editing a position needs to be reflected on Top Card...

o  Deleting a project needs to also be removed from any associated position...

●  Solution: since we have mappers for each section,

we know which to re-fetch upon a save and refresh the respective templates

What about pagination, search?

●  Same idea but easier (not coupled with other sections)

●  Mappers can take URL offset params

Switch out a partial template containing just list of profiles

Let's talk performance

High Performance Page

●  Profile is the most trafficked page on LinkedIn

●  Profile fetches 48+ different types of content on each page load o  This content results in 250+ total

downstream calls

High Performance Page

●  Profile is the most trafficked page on LinkedIn

●  Profile fetches 48+ different types of content on each page load o  This content results in 250+ total

downstream calls

Bottom Line: We need to be fast, but need to consider downstream fanout

High Performance - Parallel Requests

Fizzy

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

In the beginning, Profile had 15 embeds with 15 different endpoints

which is great for speed...


Fizzy

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

That's a lot of downstream calls, often requesting the same data too.

Content Service

Content Service


Fizzy

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

and those calls call more and more... until...


Fizzy

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Profile App

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

Content Service

aaaaand the site's down


●  Tradeoff: Speed vs Scalability

●  15 parallel calls will be fast but with Profile's load will take down site


●  Tradeoff: Speed vs Scalability

●  15 parallel calls will be fast but with Profile's load will take down site

Batched Endpoints to the rescue

Batching Calls to Mappers

Originally, we had separate endpoints (URIs) for each embed

<html> <body> ... <div id=“wrapper”> <div id=“profile”> <script type=“embed” fs-id=“topcard” fs-uri=“/profile/topcard”/> <script type=“embed” fs-id=“background” fs-uri=“/profile/background”/> ... <script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/> </div> <div id=“insights”> <script type=“embed” fs-id=“peeps_you_may_know” fs-uri=“/profile/pymk”/> <script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/> ... <script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/> </div> </div> </body> </html>

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19

Batching Calls to Mappers

We now tell framework to batch these two endpoints (Fizzy knows how to deliver the right data to the right embed)

<html> <body> ... <div id=“wrapper”> <div id=“profile”> <script type=“embed” fs-id=“topcard” fs-uri=“/profile/mappers?a=topcard,background”/> <script type=“embed” fs-id=“background” fs-uri=“/profile/mappers?a=topcard,background”/> ... <script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/> </div> <div id=“insights”> <script type=“embed” fs-id=“peeps_you_may_know” fs-uri=“/profile/pymk”/> <script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/> ... <script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/> </div> </div> </body> </html>

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19

Batching Calls in Profile

●  Today, we use between 3-5 parallel requests to Profile

●  It's a good balance of speed vs. scalability

●  Batching requests that need the same data has the added benefit of less downstream calls

Progressive Rendering With parallel fetch, Profile modules render when ready for improved perceived performance

How else to improve page load times?

Profiles can be long...

Embed

Embed

Embed

Embed

Embed

Embed

Embed

Profile Page

Optimize for Above the Fold

Embed

Embed

Embed

Embed

Embed

Embed

Embed

Profile Page

The fold

No need to render

No need to render or fetch


●  The New Profile renders above the fold as fast as we can o  Deferring everything at the bottom o  Reduces the static content we need to initially

download


●  The New Profile renders above the fold as fast as we can o  Deferring everything at the bottom o  Reduces the static content we need to initially

download

Sorry guys, we’ll get to you later


●  The New Profile defers fetching the modules at the bottom of the page o  Improves server side assembly times o  Lowers payload and improves network time

o  Improves client rendering times


●  The New Profile defers fetching the modules at the bottom of the page o  Improves server side assembly times o  Lowers payload and improves network time

o  Improves client rendering times

Go fetch!

So we covered the product, technologies, and some

features... let's revisit our goals.

Revisiting the New Profile Goals ●  Needs to surface your entire professional identity

o  Easily expose new data endpoints


o  Inline editing, searching

●  Needs to be

o  Fluid o  Flexible

o  Fast

Revisiting the New Profile Goals ●  Needs to surface your entire professional identity

o  Easily expose new data endpoints (Mappers)


o  Inline editing, searching (Dust + Mappers)

●  Needs to be

o  Fluid (Fizzy progressive rendering) o  Flexible (Fizzy deferred rendering)

o  Fast (Fizzy parallel calls + defer fetch, batching Mappers, Dust template caching)

Takeaways ●  Mappers for each module makes sense

o  You can combine and reuse with ease

o  A must if refreshing part of page

●  When structuring your page, start with many

embeds

o  You can control number of requests, when embeds are rendered, and when to fetch data

●  Many partial templates are a good thing

o  Allows you finer control over what to re-render

o  Improves developer speed

Takeaways cont. ●  Make these technologies work for you

o  Our Mappers are structured to guarantee minimal downstream calls

o  Take advantage of decoupling rendering layer

with server side endpoints §  Engineers can build endpoints

§  Web devs can start the templates with mocked JSON data

Questions?

engineering the new linkedin profile

Technology