Windows Azure PlatformCustomer Solution Case Study

Ticketing Company Scales to Sell 150,000 Tickets in 10 Seconds by Moving to Cloud Computing Solution

OverviewCountry or Region: United StatesIndustry: Media and entertainment industry

Customer ProfileStarted in 1999, Flavorus is a primary ticket seller that offers companies an online solution that makes it easy to create an engaging sales page, connect with customers, and manage ticket sales.

Business SituationIn early January 2011, Flavorus began researching options for handling ticket sales for the largest paid music festival in the United States. It needed a working solution within three months.

SolutionFlavorus opted to host its ticketing application, Jetstream, in the cloud and uses the sharding capabilities of Microsoft SQL Azure to read and write data across multiple databases in parallel.

Benefits Avoids costs Increases scalability Speeds time-to-market Improves data stability Enhances competitiveness

“Sharding gives us infinite flexibility in terms of sending traffic to different databases. It was an eye-opening experience for us in terms of SQL Azure capabilities.”

James Reichardt, CTO and Lead Programmer, Flavorus

Flavorus sells event tickets for venues, artists, and promoters through its website. In January 2011, it decided to host its ticketing application in the cloud, rather than on-premises, to handle traffic for a large music festival. The company built its Jetstream application to work with the Windows Azure platform and adopted sharding, a partitioning technique that spreads ticketing data across multiple database servers. It spent only a few months developing and testing the solution. During testing, Flavorus sold 150,000 tickets in 10 seconds by using 550 Microsoft SQL Azure databases. When tickets went on sale on April 23, Jetstream worked flawlessly. The company uses the application for all of its high-volume events and benefits from dramatically reduced costs and capital expenditures, exponential scalability, high data reliability, fast time-to-market, and a new competitive landscape.

SituationStarted in 1999, Flavorus is a primary ticket seller, selling tickets to individuals on behalf of its customers, which include venues, artists, and promoters. Its main revenue source is service fees on ticket sales. The company is the largest ticket seller of electronic music events in the United States. “We have advanced client services,” says James Reichardt, Chief Technology Officer (CTO) and Lead Programmer at Flavorus. “We can handle festivals, but we’re still growing. We’re not as big as Ticketmaster yet.”

Flavorus is a midsize company with sales, marketing, customer service and support, design, and development departments. On its website, the company hosts a complete ticketing solution that makes it possible for customers to create an engaging and easy-to-use ticket sales page for an event in minutes. Selling tickets on Flavorus is free, and the company provides flexible ticket-payment options and real-time management tools where customers can view a breakdown of sales and projected sales.

Flavorus provides ticket buyers with a variety of services, including low service fees and multiple ticket types. The company supports traditional tickets, print-at-home tickets, and mobile tickets. “As far as ticket buyers are concerned, usability is the biggest challenge,” says Reichardt. “It is one thing to buy a ticket, but it’s another thing to be able to get into the venue easily—especially when it comes to large events. We offer services that aid concert-goers in getting into the event as fast as possible.” Flavorus recently introduced paperless ticketing that features a customizable card

that fits into a wallet like a credit card. It can hold multiple tickets and is scanned at the entrance, just as a paper ticket would be. Flavorus also makes it possible for its customers to connect with ticket buyers by using marketing tools. “We offer added functionality for both our customers and theirs,” says Reichardt. “We provide built-in social networking tools and tie-ins to popular social networks, and we support the posting of videos, photos, and blogs.”

Ticketing is a unique problem on the Internet because demand is only high for a short time. In the past few years, some ticketing companies have had major issues with large-scale events. For example, in 2010, the ticketing site for Comic-Con International experienced crashes. In January 2011, the Burning Man organization, which uses a queuing method (ticket buyers wait in a virtual line), encountered issues when its ticketing vendor’s website could not handle demand after tickets went on sale. “When big events go on sale, ticket sales can hit some of the highest peak bandwidth, even higher than that experienced by Facebook or Google,” says Reichardt. “As many as 300,000 buyers can hit a ticketing website at one time, vying for 80,000 tickets. An event can sell out in a few seconds.”

Flavorus hosts its own on-premises data center in Los Angeles that features the capacity to handle midsize events. It is traditionally a very technology-focused company. “For a company our size, we have very nice servers with cutting-edge firewalls and load balancing,” says Reichardt. “We have had slowdowns in the past, but our servers can handle a pretty large workload.

29

“The way that SQL Azure is architected, you just can’t lose data. That’s the sort of thing that makes you sleep well at night.”

James Reichardt, CTO and Lead Programmer, Flavorus

For example, for a large event at the Hollywood Palladium in Los Angeles, we sold 2,000 tickets in two hours—about 20 orders a minute.”

In early January 2011, the company started looking into how it would handle ticket sales for the largest paid music festival in the United States—expected to attract as many as 150,000 attendees per day—to be held in Las Vegas, Nevada, in June 2011. In 2010, Flavorus sold tickets for the same event as a secondary ticketing company, and in 2011, because of a venue change, it would be acting as the primary agent. It knew that its servers couldn’t handle the load. “We had never sold tickets for an event that came close to the demand of this event,” says Reichardt. “We didn’t know if the event was going to sell out in five seconds or 10 hours, but when we looked at the scalability of our hardware, there was just no way we could handle it.”

In addition, the company would be servicing ongoing ticket sales for hundreds

of other events at the same time—all of which put demands on its data center for compute time and database querying. Flavorus considered handling the event’s large-scale traffic by bolstering its on-premises solution with greater database capabilities, additional web servers, and higher bandwidth. “From a business perspective, it didn’t make sense to buy hardware that we might not need otherwise,” says Reichardt. “On the flip side, this was our biggest customer, and we had to do it right. We were up against a wall on how to approach scaling for this particular event.”

Flavorus started reviewing options for hosting ticketing sales in the cloud with a comprehensive ticketing solution presented as a set of applications and services accessed over the Internet. The solution would need to be easy to maintain and scale to handle ticketing for individual events that faced a large demand on the day tickets go on sale. Flavorus also had to make sure that any cloud-hosted solution was highly stable and would provide seamless customer service. “The way the Internet is now—with social networking and everything else—if one customer has a bad experience, he has a very strong voice,” says Reichardt. “We needed the system to be stable enough that all of our customers would have a good experience buying tickets and checking into the event.”

Additionally, a primary consideration for Flavorus—and for ticketing companies in general—is to make sure that every ticket sold from a database maps to a physical ticket. The company’s relational database structure has to be flawless. Flavorus needed a solution that would enable high-

39

Figure 1. To accommodate a high volume of ticket sales in a short period of time, Flavorus assigned ticketing data to multiple SQL Azure databases in a process called sharding. During testing, it discovered that it could sell out the event by selling 150,000 tickets in 10 seconds on 550 databases.

volume reading and writing to its ticketing database and an application that was optimized for the cloud. It also needed to launch a fully functioning ticketing application within a short time frame to accommodate its marquee customer.

SolutionFlavorus considered hosting its service with a company that offers a cloud solution for websites. However, Flavorus did some basic testing using that model, and its engineers determined that the solution couldn’t handle the scale of traffic required. “The hosting company offers scalable commodity hardware in that they have a server and run virtual machines in the cloud,” says Reichardt. “You can host a traditional website in the cloud, but the application itself doesn’t scale. That’s a problem when you have the amount of traffic we were facing.”

Instead, Flavorus chose to host its ticket application on the Windows Azure platform. Developers can use Windows Azure to host, scale, and manage web applications on the Internet through Microsoft data centers. The platform includes a set of development tools, services, and management systems familiar to Flavorus. The company builds solutions by using the Microsoft .NET Framework, software that provides a comprehensive programming model and set of application programming interfaces for building applications and services. “We’re a .NET shop and I’m a .NET programmer,” says Reichardt. “Microsoft solutions are much more rich and easy to program.”

Sharding the Data

In early February 2011, Flavorus started working closely with product representatives for the Windows Azure platform. At first, the company wrote its ticketing application—which was built by using Microsoft SQL Server 2008 data management software—for a traditional single-instance database in Microsoft ASP.NET, a web application framework for building dynamic websites, web applications, and web services. It evaluated storage options on the Windows Azure platform and found that Microsoft SQL Azure, a cloud-based relational and self-managed database service, was the fastest option for its performance needs. Flavorus used the .NET Framework to rework the ticket-processing application—which the company calls Jetstream—to run on Windows Azure, the Microsoft cloud services development, hosting, and management platform. The Jetstream application is very simple: it collects the customer’s data and reserves a ticket, while a dynamic webpage calls a database and processes the customer data.

It was easy for Flavorus to port its solution to the Windows Azure platform. The company developed and tested its initial application from February 7 to February 20 and then ran the initial high-volume tests on February 21 and February 22.

The first volume tests on the Windows Azure platform failed. One SQL Azure database did not provide enough computational capacity to process large numbers of tickets and high user traffic. The database automatically restricted connections when the load was exceeded. For its ticketing application, Flavorus

49

“Adopting SQL Azure for our Jetstream ticketing application is a game-changer for Flavorus. It means our salespeople can start knocking on some big doors. From a business standpoint, it is almost unbelievable. It is a windfall.”

James Reichardt, CTO and Lead Programmer, Flavorus

needed to scale beyond what a single SQL Azure database could provide.

The company decided to adopt sharding in SQL Azure to take advantage of the massive scale that a SQL Azure cluster provides. Sharding is a horizontal partitioning technique that spreads data across multiple databases. To this end, the company could turn its single-database ticketing service into an application with data that is split across multiple SQL Azure database servers (Figure 1).

The company was able to scale its on-premises, single-database solution into a sharded solution within two weeks. By doing this, Flavorus gained higher availability by engaging many databases. “By using sharding with SQL Azure, we can have tons of customers on the site at once trying to buy tickets,” says Reichardt. “If any one of the database servers goes down, it doesn’t really matter.”

Flavorus realized that the ticketing application was ideal for sharding because the tickets are all the same. “With sharding, you can put a few tickets on each database if you want,” says Reichardt. “It doesn’t matter which database the ticket is on. You just need to distribute the load. It’s a beautiful thing.”

From February 22 to February 24, Flavorus ran initial high-volume tests using sharding. The ticketing application was able to handle a large amount of ticket sales but hit a limit because of how Internet Information Services 7.5, the default web server role on the Windows Azure platform, handles traffic. To get around this

limitation, the company had to rewrite the data-dependent routing logic.

For debugging purposes, Flavorus needed to access logs to troubleshoot issues with the compute nodes and databases. “I was able to retrieve the logs and track down the line of code that was causing the web servers to have difficulty managing connections to the databases,” says Reichardt.

From March 2 to March 24, Flavorus ran tests using the new logic, gradually increasing the number of sharding instances. By March 25, Reichardt was running volume tests with 7,000 general-admission tickets available in each of the 550 SQL Azure databases. During load testing, he successfully sold 150,000 tickets in 10 seconds. “When we reached that speed, we realized we could fulfill ticketing for any festival in the United States,” says Reichardt. “The Jetstream application literally scaled as I added servers. We were ready to go.”A few days before event tickets went on sale, Flavorus learned that the promoters wanted to sell eight ticket types at different prices. To accommodate this request, the company sharded the ticket data into eight groups of servers, with each group serving a specific ticket type. For example, it used 20 databases for the VIP tickets and 400 databases for the most popular tickets—the general-admission, three-day tickets. Flavorus allocated ticket data by using projections based on the previous year’s ticket sales. “Sharding gives us infinite flexibility in terms of sending traffic to different databases,” says Reichardt. “It was an eye-opening experience for us in terms of SQL Azure capabilities.”

59

“Part of the beauty of SQL Azure is that you can tinker with its capabilities…. But for the most part, the Windows Azure platform handles everything for you. It’s a really beautiful model.”

James Reichardt, CTO and Lead Programmer, Flavorus

Launching Jetstream in the CloudTo deploy Jetstream to production on the Windows Azure platform, Reichardt simply ran a command in the Microsoft Visual Studio 2010 Professional development system, and the application—which comprises 1,391 lines of code—launched in the cloud. He then assigned the application and data to the required number of web servers and database servers.

Tickets went on sale for the event on April 23, and Jetstream worked flawlessly. To support the spike in first-day ticket sales, Flavorus used the Windows Azure platform to host 750 web role instances of the Jetstream application and 550 SQL Azure databases. The application wrote chunks of customer data to multiple sharded SQL Azure databases in parallel. Jetstream was live for two days, after which the company returned the event data to its on-premises servers and continued selling tickets. “For an on-sale event, we really need scalability for a small amount of time—about two hours,” says Reichardt. “About half an hour before sales start, traffic ramps way up. Then you get the latecomers who go to the website to find out if tickets are still available.”

The company continues to use its on-premises environment for day-to-day ticketing sales. It offers the Jetstream service only for customers promoting large-scale events. “By using the cloud model, we can isolate the ticket sales of one event,” says Reichardt. “We can run Jetstream on demand and point the ticket-sales traffic to that application. It doesn’t affect any of the other events on our website. Then, we can shut down Jetstream

and pull the data back into our system to process it—with no interruption in service.”

BenefitsBy designing its Jetstream application to scale for large events by using SQL Azure, Flavorus benefits from greatly reduced costs while increasing the ability to handle high ticket-sales traffic exponentially. The company used Windows Azure technologies to ramp up capacity within a short time frame, easily meeting the deadline for its largest customer. It also benefits from highly reliable data stability. Thanks to cloud capabilities, Flavorus finds itself in a new competitive landscape in which it can host ticket sales for the largest events in the country.

“Part of the beauty of SQL Azure is that you can tinker with its capabilities if you have unique needs—like the way we adopted sharding to scale ticketing on the on-sale day,” says Reichardt. “But for the most part, the Windows Azure platform handles everything for you. It’s a really beautiful model.”

Avoids CostsWith its ticketing application in the cloud, Flavorus can sell tickets for big events without investing in infrastructure. The company avoids purchasing bandwidth and a variety of hardware and software that would need to be replaced in a few years anyway. It also bypasses the costs of administering and maintaining a large on-premises data center. Flavorus estimates that it is saving more than U.S.$100,000 annually in hardware, software, and bandwidth.

69

“Sharding with SQL Azure turned out to be the golden ticket for us. It’s almost infinitely scalable.”

James Reichardt, CTO and Lead Programmer, Flavorus

When putting Jetstream into operation on the Windows Azure platform, Flavorus pays only for each hour of use. The cost of running Jetstream on Windows Azure for two days was minimal, and the on-sale ticketing event was profitable. “Using sharding with SQL Azure is so cost-effective, it is mind-blowing,” says Reichardt. “Plus, we can turn Jetstream off when we’re not using it—and there’s no cost to us.”

Increases ScalabilityThanks to the sharding capabilities of SQL Azure, Flavorus has gone from knowing its servers would crash during high-volume on-sale events to being able to sell 900,000 tickets in a minute. The company has more scalability than it needs. “Sharding with SQL Azure turned out to be the golden ticket for us,” says Reichardt. “It’s almost infinitely scalable. You could continue to ramp it up with more databases and get faster and faster times. It is like night and day compared to what we had done before.”

Speeds Time-to-MarketWhen Flavorus first started investigating options for large-scale ticketing in January 2011, it went to its primary bandwidth provider to inquire about increasing bandwidth within a three-month time frame. As of June 2011, the company was still waiting for its bandwidth to be increased. If it had had to procure and set up hardware and software, it would have taken many months to prepare an on-premises solution. In the meantime, Flavorus developed, tested, and deployed its ticketing application in the cloud within a few months. “From start to finish, developing Jetstream to work on the Windows Azure

platform was an amazingly fast process,” says Reichardt. “We wrote the application in a few days, and within a few months of testing and bug-fixing, we had the functionality to sell 150,000 tickets in 10 seconds. We’re extremely happy we went with SQL Azure because we met our deadline.”

Improves Data StabilityFlavorus relies on the redundancy and data stability that’s available with SQL Azure because the customer data in cloud-based ticket sales is processed after the on-sale event takes place. “The ability of SQL Azure to ensure data recoverability was essential for the Jetstream application,” says Reichardt. “That’s because we weren’t processing credit cards or performing other service calls in real time. The way that SQL Azure is architected, you just can’t lose data. That’s the sort of thing that makes you sleep well at night.”

79

Enhances CompetitivenessPromoters are particularly concerned about ticketing capabilities because the ticket-buying experience is the first exposure that customers have to an event. With the ability to scale ticket sales in the cloud, Flavorus gains a competitive advantage in the market because it can now cater to events of any size. The company can offer its customers a reliable and highly scalable ticketing service that has been proven to work. “Adopting SQL Azure for our Jetstream ticketing application is a game-changer for Flavorus,” says Reichardt. “It means our salespeople can start knocking on some big doors. From a business standpoint, it is almost unbelievable. It is a windfall.”

89

Windows Azure PlatformThe Windows Azure platform provides developers the functionality to build applications that span from consumer to enterprise scenarios. The key components of the Windows Azure platform are:

Windows Azure. Windows Azure is the development, service hosting, and service management environment for the Windows Azure platform. It provides developers with on-demand compute, storage, bandwidth, content delivery, middleware, and marketplace capabilities to build, host, and scale web applications through Microsoft data centers.

Microsoft SQL Azure. Microsoft SQL Azure is a self-managed, multitenant relational cloud database service built on Microsoft SQL Server technologies. It provides built-in high availability, fault tolerance, and scale-out database capabilities, as well as cloud-based data synchronization and reporting, to build custom enterprise and web applications and extend the reach of data assets.

To learn more, visit: www.windowsazure.comwww.sqlazure.com

99

For More InformationFor more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers in the United States and Canada who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to:www.microsoft.com

For more information about Flavorus products and services, call (323) 908-0607 or visit the website at: http://fla.vor.us

This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.

Document published September 2011

Software and Services Windows Azure Platform− Microsoft SQL Azure− Windows Azure

Microsoft Visual Studio− Microsoft Visual Studio 2010

Professional

Technologies− Microsoft .NET Framework 4− Microsoft ASP.NET 4