Testing Times for E-Commerce

White PaperSilk PerformerSilk Performer CloudBurst

Prioritizing and Perfecting Performance Tests

2

White PaperTesting Times for E-Commerce

With a majority of the online population already abandoning the high street and buying online, future growth in online sales will largely be driven by an increase in online spending per buyer, surpassing $2,000 per year by 2018.1

Downtime Is DisastrousInternet downtime used to be an inconvenience, but today it can be life or death for a business. Arguably, the single most disastrous crisis that can happen to an online retailer is downtime to its website and mobile apps.

However, the impact of downtime is not all about money. A short outage can impact reputation, customer retention, employee satisfaction and overall confidence.

Putting Performance to the TestWhen it comes to load and performance testing, the first and foremost concerns for online retailers are obviously the efficiency of their web-sites, mobile sites and mobile applications.

However, efficiency is just the tip of the iceberg. Today’s successful retail organizations must be able to test various third party web services, such as PayPal payment services. While integration of third party content (TPC) is common practice, with social media, ad plug-ins, paywalls, widgets and analytics becoming the norm, it is a challenge for testers to ensure that TPC does not compromise website or app performance.

Executive SummaryThe major trend towards e-retail shopping is showing no signs of slowing down any time soon with US online sales continuing to thrive. Forrester Research states that online shopping accounted for almost 9% of the $3.2 trillion total retail market in 2013 and will continue to grow at a compound annual growth rate (CAGR) of nearly 10% through to 2018.1

Amazon Suffers Significant Outage

The website of the world’s largest online retailer, Amazon. com, went down one afternoon in August 2013 for around 15 minutes. Amazon processes $163,622 in transactions per minute on average, meaning its 15 minutes of downtime cost the retailer more than $2.4 million!2

2014 World Cup Worries for ITV

In the UK, ITV’s online viewing service was hit by problems during the opening match. Fans complained with a barrage of criticism via Twitter when ITV Player was unavailable for the game between Brazil and Croatia. The broadcaster was also beleaguered with criticism during the 2010 World Cup in South Africa when more than a million viewers missed the opening goal.4

Kicked into Touch and Falling at the First Hurdle

In May 2014, almost 100,000 tickets for the 2014 Commonwealth Games in Glasgow and half a million tickets for the 2015 Rugby World Cup were in the balance due to problems with ticketing websites for both events. The Games site was suspended while England Rugby 2015 put their go-live date back several weeks.3

__________

1 Forrester Research Online Retail Forecast, 2013 To 2018 (US) www.forrester.com/ Forrester+Research+Online+Retail+Forecast+ 2013+To+2018 +US/fulltext/-/E-RES115941?highlightTerm=us%20 online%20 retail%20forecast%202013&isTurnHighlighting=false

2 MSN News http://news.msn.com/science-technology/amazoncom-loses-millions-during-site-outage

3 Borland Blog http://blog.borland.com/performancetestinggames/488/ 4 The Guardian World Cup 2014 www.theguardian.com/media/2014/

jun/13/itv-player-glitch-world-cup-brazil-croatia

3

There are also various parameters within the back end of online retailers’ systems that need to be tested. Enterprise resource planning (ERP) systems, such as Oracle and SAP, and customer relationship manage-ment systems, such as BC Remedy, should all to be accounted for from a testing point of view.

Challenging Times for TestersExpecting the Unexpected With the proliferation of social media, organizations need to test higher-than-anticipated volumes on their sites and apps. Say, for instance, an A-list celebrity tweeted about a company’s product, potentially driving millions of customers from all over the world to the site, or a promotion goes viral very quickly. If the site cannot handle the load, this downtime will cause a loss for the business in terms of lost revenue.

Testing Different Locations and Speeds In the nineties, many referred to the Internet as the ‘world wide wait,’ but those days are long gone. Today’s testers need to ensure their systems can competently handle loads right across the planet and at different connection speeds. The Internet is global so businesses need to ensure visitors and customers have a satisfying experience wherever they are in the world.

Maintaining high performance is essential and while customers were once willing to wait for around 4 or 5 seconds, today’s web consumers are becoming more demanding and impatient to the point that many literally want pages to load in the blink of an eye (400 milliseconds). People will visit a website less often if it is slower than a close competitor by more than 250 milliseconds.5

On a mobile phone, a web page takes a leisurely nine seconds to load, according to Google, which tracks a huge range of sites from the homes of large companies to the legions of one-person bloggers. Download times on personal computers average about six seconds worldwide and about 3.5 seconds on average in the United States.5 Mobile devices usually have higher latency and slower connection speeds than PCs. But the key point is that all locations and speeds need to be tested from the end user perspective to ensure desktop and mobile customers stay on a site without deeming it too slow, in which case they will quickly defect to the competition.

Decreasing Production Cycle Times With more and more new releases needing to fit into ever-decreasing timeframes, Internet retailers are under increasing pressure with their cycle times. This is due to industry demand, promotions, customer ex-pectation and so forth. Due to Agile development, web developers are

iteratively updating their sites and apps with major retailers, working to cycle times in the region of as little as two weeks to refresh the features and functionalities of their online presence. __________

5 NY Times online www.nytimes.com/2012/03/01/technology/impatient-web-users-flee-slow-loading-sites.html?pagewanted=all&_r=0

4

White PaperTesting Times for E-Commerce

Therefore, testing needs to be closely in sync with development to en-sure these ever more frequent changes are all present and correct. Furthermore, IT budgets and resources are dwindling with constant pres-sure to reduce costs, increasing the challenge for testers and QA teams.

Browser and Device DiversityToday’s developers need to ensure that no matter which browsers tar-get audiences are using to view their website or mobile apps, the site will perform effectively. With the proliferation of available browsers and the fact that different browsers have various connection strategies, which may affect browser speed, along with the increasing use of smart phones and tablets to navigate the Internet, browser testing is a vital part of the website development process.

Other factors to consider include rendering times, which can differ be-tween browsers. Plus an increasing number of sites have mobile ver-sions that differ from desktop versions in that they are less complex, and place an altered load on servers so need to be tested separately. It is also essential to distinguish between mobile apps and websites during testing due to the major variances in user behavior.

Further Management ChallengesGenerating Realistic User Scenarios Organizations can only gain confidence in applications by testing real-istic user scenarios prior to launch. It is essential to know exactly what will happen with a site or application when it goes live. This refers to both new and updated sites. For example, many e-retailers substantially change page content for Black Friday with ads and special promotions, but more images and more hits may combine to hinder response times.

So these loads need to be simulated realistically whilst considering a broad range of criteria, including various profiles for people who are searching and people who are buying, for example.

Identifying Problems EarlyOrganizations must discover problems as early as possible in the release cycle to minimize development costs and reduce project cycle time. This reinforces the fact that both front- and back-end systems need to be tested simultaneously. The longer an issue remains undetected, generally the more expensive and complex it will be to rectify.

Unconsolidated ToolingProcuring and managing various non-integrated testing tools to sup-port the wide range of enterprise environments can be a costly busi-ness. For example, if an organization upgrades to a new ERP system, as soon as those changes are made, the new system needs to be tested. Organizations with different tools to test various technologies need to consolidate their testing into one system and one tool. This has the added benefit that the testers will only need to learn the system once, enabling reuse of assets and skill-sets for a more economical testing environment.

Accurately Planning Capacity RequirementsWith dwindling budgets, organizations need to identify precisely how much server infrastructure they will need to purchase to provide a sat-isfying user experience whilst avoiding over-investing in hardware that is not used. Finding the right budgetary balance is critical to e-retailers.

Black Friday’s Dark Side

86% of companies have experienced one or more instances of downtime during this peak shopping time

Average cost of downtime for one minute is $5,600

59% of customers experiencing errors with an e-commerce platform will not return6

China’s Web Outage

In January 2014, one of the biggest outages in history occurred when China essentially lost the Internet for a full business day. For retailers with significant presence in China, such as Gap and H&M, the outage caused major loss of revenues.7__________

6 Dotcom Monitor www.dotcom-monitor.com/blog/index.php/website-uptime/holiday-website-uptime-performance/#more-4282

7 Retail Online Integration www.retailonlineintegration.com/article/china-s-web-outage-a-cautionary-tale-online-retailers/1#

5

The Cloud ApproachIncreasingly, companies are looking to the Cloud as a way to improve IT performance. They focus the power of the world’s largest data centers onto their computing needs—just when they need it most.

The Cloud provides a virtual set of computing resources that can be securely accessed via the Internet. Cloud infrastructure providers are built on the world’s largest data centers. This allows them to marshal gigantic quantities of computing power as-needed with a pay-as-you-go model to maximize cost efficiencies.

Best of Both Worlds: Testing and the CloudClearly, the Cloud offers significant efficiencies to IT organizations. But can it also help to test applications under peak loads? The answer comes from harnessing cloud resources to host virtual users that can simulate large loads. You can direct these cloud-based virtual users against applications hosted at your company or in the Cloud itself.

Cloud-based peak load testing essentially replicates the model of onsite peak load testing whilst taking advantage of the Cloud. Testers can gain a deeper understanding of the geographical differences in Internet latency and how this will impact end users. It is also cost-effective

because rather than purchasing and maintaining hardware, which will sometimes be sitting idle and taking up space, the Cloud can be used as and when, on a pay-as-you-go basis.

SummaryFor development organizations to remain competitive, their developers and testers require an enterprise-class solution that delivers automated software application performance and load testing. One that delivers realistic, dependable performance tests across the latest technologies, including front and back-office systems, by simulating loads and net-work speeds for web, enterprise and mobile applications.

Micro Focus® Silk Performer™ and Silk Performer CloudBurst enable software quality teams to rapidly launch any size peak-load performance tests without the burden of managing complex infrastructures. Now, organizations can test and diagnose applications under immense global peak loads. The latest version of Silk Performer CloudBurst also sup-ports testing of internal applications from the Cloud through built-in VPN tunneling. In addition, Silk WebMeter effectively reports on the performance and reliability of websites.

To find out more visit: www.borland.com/silkperformer

162-000064-001 | B | 03/16 | © 2016 Micro Focus. All rights reserved. Micro Focus, the Micro Focus logo, and Silk Performer, among others, are trademarks or registered trademarks of Micro Focus or its subsidiaries or affiliated companies in the United Kingdom, United States and other countries. All other marks are the property of their respective owners.

Micro FocusUK HeadquartersUnited Kingdom+44 (0) 1635 565200

U.S. HeadquartersRockville, Maryland301 838 5000877 772 4450

Additional contact information and office locations: www.microfocus.comwww.borland.com