windows azure: lessons from the field

Windows AzureLessons from the Field

National Architect,

Windows Azure

michael.collier@neudesic.com

@MichaelCollier

www.MichaelSCollier.com

Michael Collier

• Table Storage• Access Control Service (ACS)• Windows Azure Diagnostics• Retry Logic• Deployment• Environments• Tools• Sell it!

What We Are Talking About

• Non-relational data storage• Massive scale (100TB per storage account)• Single Key (Partition Key + Row Key)• Range based partitioning• Requires a different way of thinking

– Multiple entity types in a single table

– Group data into logical units – a partition

– More than 1 key data point? Create your own composite key.

Windows Azure Table Storage

Windows Azure Table StoragePartitionKey

RowKey CourseName Comment CommentCount

43040 0:dafce7ed-47ff-474a-a94c-8b7d555394c1

Darby Creek 1

43040 1:dafce7ed-47ff-474a-a94c-8b7d555394c1:2520576021677371563:b3abfc42-4e66-4306-b39b-c3972fde5ac7

Fast greens!

43012 …. Timberview 5

43016 …. Buck Ridge 3

43016 ….. Great sand!!

Record Type

CourseId

Timestamp

CommentId

Windows Azure Table Storage

public IEnumerable<Course> SelectAllCourses() {var tableClient = storageAccount.CreateCloudTableClient();var ctx = tableClient.GetDataServiceContext(); var results = (from c in ctx.CreateQuery<Course>(tableName)                   where c.RowKey.CompareTo("0:") >= 0 && c.RowKey.CompareTo("0;") < 0 select c).AsTableServiceQuery().ToList();return results; }

Select all the Course entities

Plan for retries

Windows Azure Table Storage

public void Add (Course course, Comment comment){

var tableClient = storageAccount.CreateCloudTableClient();    var ctx = tableClient.GetDataServiceContext();

ctx.AddObject(tableName, course);             ctx.AddObject(tableName, comment);             

ctx.SaveChangesWithRetries(SaveChangesOptions.Batch);}Handle entity group transactions too!

• Claims-based authentication service• Leverages Windows Identity Foundation (WIF)• No need to build your own identity management

solution. What’s your value-add?

• Multiple identity providers– Facebook, Windows Live, Google, Yahoo!, ADFSv2

• Most demos and walkthroughs show how easy ACS is to add . . . But there’s more.

Access Control Service (ACS)

• Install WIF runtime via a startup task• DPAPI not supported – use your own certificate• Change request validation

– Use ASP.NET 2 request validation– Custom validator

Access Control Services (ACS)

• WIF relies on the web.config file• URLs related to the site are set in the web.config . . .

can’t change• Problematic for staging deployments – don’t know the

URL until deployed• Add logic to WebRole’s OnStart() to update the WIF

settings in web.config– Read in configuration settings from .cscfg

– Update and save the web.config

– Changing .cscfg settings can cause a role recycle . . . causing web.confg to update

Access Control Service (ACS)

• Need claims not provided by Identity Provider?– Claims vary by Identity Providers– Windows Live ID – limited usefulness

– Claims Enrichment– Custom implementation of ClaimsAuthenticationManager– Retrieve additional info from data store– Return as additional claims

Access Control Service (ACS)

DEMO TIME!!!

• Ability to persist multiple diagnostic sources across roles

– Log Files

– Event Logs

– Performance Counters

– IIS Logs

• Diagnostics data saved in table or blob storage• Different storage account for diagnostic & app data• Use multiple diagnostic storage accounts & rotate

– Easiest way to clean up large amounts of data in tables

Windows Azure Diagnostics

Configuration via code is easy . . .

. . . but potentially problematic

Windows Azure Diagnostics

• Set diagnostic information via configuration• Special file – diagnostics.wadcfg• File automatically saved to blob storage and accessible

from all instances• Don’t get out of sync• Diagnostics config in code overwrites what is in blob

storage• Allows operations team, not dev, to control settings

Windows Azure Diagnostics

Windows Azure Diagnostics

• Configure Remote Desktop early

• Requires an input endpoint• Changing number of endpoints requires a delete and

redeploy– Can’t perform a VIP swap

• Don’t want it on all the time? Change the settings in .cscfg.

Remote Desktop . . . Your Friend

• Transient Fault Handling Application Block• SQL Database, Windows Azure Storage, Service Bus,

and more• Very extensible and flexible

Plan for Failure – Try, And Try Again!

Plan for Failure . . . And Try Again!

• Upload .cspkg & .cscfg files to Windows Azure portal• Use Visual Studio• Use Windows Azure PowerShell cmdlets

– Humans make mistakes . . . Not good at repetitive tasks

– Handle nearly everything via script

– Works great in development and for production!

– Invoke from other deployment tools like Team Build

– Have a .cmd file that will execute the build and then kick off the deployment

• Put deployment files in blob storage for quick access later

Deployments

• Use subscriptions to control access and billing.

• Get billing and subscription administrators set up . . . . Very difficult to change later (especially the Live ID for account owner)

Logical Environments

Development ProductionQA

Staging

Production

Staging

Production

Staging

Production

• Developers create & deploy services in ‘Development’ as needed. Co-admins for the ‘Development’ subscription.

• QA teams have access to QA. They are co-admins for the QA subscription.

• Operations team is co-admins for ‘Production’.

CS CS

CS DB

CS CS

CS DB

CS

DB

CS

DB

CS

CS

DB

Cloud Storage Studio

Diagnostics Manager

Management Cmdlets

http://azurestorageexplorer.codeplex.com/

Get a Storage ToolNeudesic Azure Storage Explorer Cerebrata

• What are your pain points?– Cost pressures

– Slow to deploy

– Scalability

– Security

• Scenarios– Web Modernization

– Gaming

– Mobility

– Big Data

– Enterprise Application Integration

Awesome! Now Sell It!!

Questions?

• WIF: A Potentially Dangerous Request.Form Value Was Detected– http://

social.technet.microsoft.com/wiki/contents/articles/1725.windows-identity-foundation-wif-a-potentially-dangerous-request-form-value-was-detected-from-the-client-wresult-t-requestsecurityto.aspx

• Install WIF Runtime via Startup Task– http://stackoverflow.com/questions/8697596/azure-service-configuration-error

• Edit & Apply New WIF Config Settings w/o Redeploying– http://

blogs.msdn.com/b/vbertocci/archive/2011/05/31/edit-and-apply-new-wif-s-config-settings-in-your-windows-azure-webrole-without-redeploying.aspx

• Publishing a ACS v2 Federated Identity Web Role– http://blogs.msdn.com/b/davidmcg/archive/2011/04/05/publishing-a-acs-v2-federated-identity-web-role.aspx

• Windows Azure Active Directory Solutions For Developers– http://

social.technet.microsoft.com/wiki/contents/articles/3669.windows-azure-active-directory-solutions-for-developers.aspx

• How to get most out of Windows Azure Tables– http://

blogs.msdn.com/b/windowsazurestorage/archive/2010/11/06/how-to-get-most-out-of-windows-azure-tables.aspx

• Collecting Logging Data by Using Windows Azure Diagnostics– http://msdn.microsoft.com/en-us/library/windowsazure/gg433048.aspx

A Few Great Resources

National Architect,

Windows Azure

michael.collier@neudesic.com

@MichaelCollier

www.MichaelSCollier.com

Thank You