2015 at&t developer summit

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks

contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or

entity is not an endorsement by AT&T.

Doug Sillars

High Performance: What These Apps Did May Shock You

ARO Technical Lead

@Dougsillars

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other

marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a

specific company or entity is not an endorsement by AT&T.

The Next 40 Minutes Will Change Your App




Developers Make This Mistake Everyday,And it’s Killing Their Customer’s Battery

• Data Connections account for:• 40% of battery use (screen on)

• 70% screen off

http://thenextweb.com/dd/2013/08/05/how-to-minimize-your-mobile-apps-power-consumption/http://commons.wikimedia.org/wiki/File:Free_EV_charge_station_sign_evinfra.svg




• Cellular Networks are High Latency Environments• Not Optimizing Your Content can lead to delays:

• 500ms:

• 26% frustration

• 8% engagement

• Amazon.com: 100ms delay

• 1% revenue

• 4% of mobile users admit to throwing their phones

You Won’t Believe What Else Network Connections Do

http://www.flickr.com/photos/9009139@N08/1263954439http://blog.radware.com/applicationdelivery/applicationaccelerationoptimization/2013/12/mobile-web-stress-the-impact-of-network-speed-on-emotional-engagement-and-brand-perception-reporthttp://www.mobilejoomla.com/mobile-joomla-blog/172-responsive-design-vs-server-side-solutions-infographic.html




Is Performance Holding Your App Back?

Retention Rate

Usage

Lifetime Value Lifetime Value

Active Users

Session Length

Average Revenue per userApp Launch/Load Time

User Acquisition

User Experience/ Happiness




One Developer Tried This One Simple Trick…

Discovery Adoption Trial

48% of customers abandon if dissatisfied with the performance of a mobile app

http://www.tatvic.com/blog/churn-analysis-for-mobile-app/http://offers2.compuware.com/APM_13_WP_Mobile_App_Survey_Report_Registration.html




Understanding Application Performance




Just Because Someone Says “Unsinkable”

98% of apps tested have potential Performance Optimizations

http://commons.wikimedia.org/wiki/File:RMS_Titanic_3.jpg https://www.flickr.com/photos/kb-monkeywrench/3639224089/




Death by a Thousand CutsOr, It Isn’t Always the Iceberg

Minor issues found in testing

Big problems at scale

http://en.wikipedia.org/wiki/Hans_Brinker,_or_The_Silver_Skates#mediaviewer/File:Hans_Brinker_Madurodam.jpg




• 44% of all issues are reported by users

• 20% issues were only found because of complaints in app reviews

Helps to Find Issues EarlySuccessful Applications Find That Testing

http://info.perfectomobile.com/rs/perfectomobile/images/why-apps-fail-infographic.pdf




The Bigger They Are…

The Quicker Small Issues Magnify




Find My Goat*




Find My Goat

• Connecting to the network every 5 seconds• Creating a new connection every time• 18x more connections than this app’s architecture team expected• The triage team discovered that IP/port collisions = loss in sales




Computers Are Great at Repetitive Stuff…

http://commons.wikimedia.org/wiki/File:Assen_Gert_Sennema_-_Sisyphus_1.jpghttp://commons.wikimedia.org/wiki/File:All_work_and_no_play_makes_Jack_a_dull_boy_(The_Shining)_(7957738500).jpg

But Don’t Punish Your Users




Repetitive Download vs. Caching

100 seconds to load




Caching Enabled!!

30 second startup 2nd startup




Major Snap Sales App

http://pixabay.com/en/woman-smartphone-chatting-girl-410320/




World Famous Goat RacingWas $79, Now $39

User has to wait for image to load

Thumbnails or Dumbnails…

Downloading the same image twice, in different sizes, doesn’t make sense.Download it once and use it in multiple ways.

54 kb

198 kb

World Famous Goat Racing Was $79, Now $39




Major Brick and Mortar Retailer

Image: http://pixabay.com/en/woman-shopping-happy-bags-dresses-169286/




A Rolling Stone Gathers No Moss, but Spinning Wheels Kill Sales

http://www.lukew.com/ff/entry.asp?1797

“With the introduction of these progress indicators, we had made people watch the clock. As a result, time went slower and so did our app.”

~ Luke Wroblewski




AT&T DriveMode




AT&T DriveMode for Android

3 min

480 GPS connections/24hr

~20% Battery




AT&T DriveMode for AndroidUse activity APIs to identify when your users are in motion

11 minute trace

DetectedActivity.IN_VEHICLEDetectedActivity.ON_BICYCLE:DetectedActivity.ON_FOOT:

http://commons.wikimedia.org/wiki/File:Baby_goats_jan_2007_crop.jpghttp://commons.wikimedia.org/wiki/File:Goat_in_a_car.jpghttp://commons.wikimedia.org/wiki/File:2009_goat_and_bicycle_in_India_6512003001.jpg




Major Sports Reporting Application

• Regular NFL season Sept 4 – Feb 1

• 50,000,000 installs on just Android

• So what would you expect the behavior of the app to be in June at 2am?

https://www.flickr.com/photos/perspective/33330283/




Major Sports Reporting Application

Pinging the network 3 minutes 24 x 7 x365Consider 70,000 fans in a stadium with this app trying to hit the network




Advertising Could Be Killing Your App

• Consider your advertising partner like you would a business partner• Do they compress their content?• Do they close their connections? Caching?




Most of these issues have already been fixed by the app developers we’ve worked with, but if you’re wondering why the tops apps are staying on top, it’s because they are constantly checking and watching all of these Best Practices.

Constant Improvement




Don’t Over Think Performance Testing

http://www.glendaleinquiry.ca/2014/09/what-were-we-thinking.html




The Excuses

Time

Usage

Why fix something that isn’t broken

Skill sets

Return on Investment

Cost

Everyone else does the same thing

Too much effort

Users haven’t complained




Performance Testing Guidelines

http://smarterappsguidelines.gsma.com/

http://knowmyapp.org/

http://www.appqualityalliance.org/aqua-performance-test-criteria

http://developer.att.com/application-resource-optimizer




• Free Tools:• Application Resource Optimizer

• Network Attenuator

• Trepn

• Android

• Systrace

• Battery Historian

• Etc.

• Two tests (minimum)

• 40 minutes total testing time

Keep it Simple




What You Find Might Surprise You

Active Use Idle Use

What the User Wants

What the App Needs

GPS




AT&T DefectsDuplicate Content

<5% of KB

5-20% of KB

20+% of KB

Periodic Connections

0-20% of Power

20-50% of Power

50+% of Power

GPS/BT

0-25% Active (s)

25-65% Active (s)

65+% Active (s)

Text Compression

100-500KB

500-1000KB

1000+KB

Closing Connections

0-20% of Power

20-50% of Power

50+% of Power

Sev 3

Sev 2

Sev 1




• Encourage testers to move at a speed that emulates real users

• Create flexible (yet reproducible) test cases

• Test cases can even reflect more of a feature checklist

• The Trace is not about the order of the features, but about the content

• Don’t limit testing to 2 traces if additional are needed

• Some streaming apps can have a trace with just streamed content

• Some apps are most inefficient at startup

• Be Curious

• If something seems odd, explore it

• Screen rotation best practice came out of tester noticing the network lighting up each time they turned the device

Trace Collections Hints




http://bit.ly/HighPerfAndroidApps




Q&A

http://smarterappsguidelines.gsma.com/

http://knowmyapp.org/

http://www.appqualityalliance.org/aqua-performance-test-criteria

http://developer.att.com/application-resource-optimizer

http://bit.ly/HighPerfAndroidApps

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks

contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or

entity is not an endorsement by AT&T.

Thank You

2015 at&t developer summit

Technology