reaching for the sky with asterisk and amazon ec2
Post on 21-Dec-2015
222 views
TRANSCRIPT
Reaching for the sky with Asterisk and Amazon EC2
CLOUD COMPUTING CLASS
• This is an advanced class• Pre-requirements for this class are:• Basic knowledge of Asterisk (dCAP preferred) • SysAdmin level Linux skills (RHCE, LPIC, equivalent)• Your own notebook computer (not a must)• Willingness to think differently about IT and economics• A good sense of humor…
Administrative
• Name: Nir Simionovich• Job Title: Who needs titles anyway…• Asterisk Experience: 7 years• Amazon EC2 Experience: 18 months• Largest EC2 Installation: 3500 concurrent channels• Publications:• AsteriskNOW – Published 2008, Packt Publishing• Asterisk AGI – Published 2009, Packt Publishing
About your instructor
• Questions: Just raise your hand, don’t wait…• Private Questions: Wait for the end of the module…• Specific Implementation: Catch me in the eveninig• Just have fun …
Class Policies
An introduction to Amazon EC2Module 1
CLOUD COMPUTING CLASS
• Per Amazon’s website: “Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.”
• Without the buzz words: “Amazon Elastic Compute Cloud (Amazon EC2) is a virtualization environment, allowing you to initialize virtual servers, without the regular BS related to hosting companies.”
What it Amazon EC2
• Amazon EC2 is not to be mistaken with:• Co-Located servers hosting• Dedicated servers hosting• Web V-Hosting
• EC2 provides virtual servers, similar to VPS hosting• EC2 doesn’t provide common control panels• EC2 doesn’t provide customized network setup
Amazon EC2 is not !
• The EC2 service is built of 3 distinct components:• The Amazon Machine Image (AMI) – A raw image of your virtualized host• Amazon S3 – A storage service where you can store your AMI hosts for future usage• Amazon AWS Admin – An administration service, available as a web service, a web panel or a Firefox plug-in (ElasticFox).
• Other Amazon AWS services can be leveraged, such as: CloudFront, SQS, SimpleDB and others
EC2 Service Components
• Registration is performed via the aws.amazon.com website:
Step 1: Registration
Pay Attention tothis radio button
We won’t explore the entire registration wizard
• EC2 uses your credit card information in order to perform billing• Pay attention to the pricing information provided during the registration wizard – it’s very important• For now, just fill in your credit card information and let’s continue…
Step 2: Pricing and CC info
• Once registration is complete, an email will be sent to you for final confirmation• EC2 requires the installation of X.509 certificate• You can supply your own or create a new one using the EC2 service• To create a new certificate, simply log-on to your account and create the certificate from their• You will also be required to create your Access Key IDs for the service
Step 3: X.509 Certificate and Keys
• Once your X.509 keys had been created, download these and store in a safe place on your computer• Your certificate is used for accessing your servers via SSH – without the usage of a root password• Your Access ID’s are used to allow access from your ElasticFox plug-in and web management to your account
Step 3: X.509 Certificate and Keys
Step 3: X.509 Certificate and Keys
Step 4: Logon to the consolehttp://aws.amazon.com/console/
Step 5: Create a Key-Pair
• Your key pairs are later on used for SSH access
Step 6: Create a Security Group
Remember:All Amazon EC2 instances
are NATTED!
Step 7: Install ElasticFox• ElasticFox is the official production level management console for Amazon EC2• We now switch to using ElasticFox, as it provides greater flexibility than the web console• ElasticFox can be found under the “Developer Tools” section of the AWS website• Once ElasticFox is installed, you need to create your online credentials for it – using the keys presented at step 3
Step 7: Install ElasticFox
Step 7: Launch an Asterisk AMI
Step 8: Create an SSH private key
Save your private keyfile, to be used with PuTTy
Step 9: Connect to your instanceClick this button
• You will be asked for your generated private key file, so just select it• If all goes well, an SSH client to your instance should be made available to you
• You should now be able to:• Initiate new EC2 instances • Logon as ‘root’ to your initiated instances
• You should now be able to create your own EC2 based Asterisk server
Module Recap
• Lab Duration: 30 minutes• Tasks: • Register with Amazon AWS services• Create your keys and certificate• Activate an Asterisk AMI instance• Logon to your Asterisk server via SSH
Module 1: Recap Lab
Amazonomics – The Economics Behind usingAmazon EC2
Module 2
CLOUD COMPUTING CLASS
• Amazonomics is the combination between the words “Amazon” and “Economics” • The term dictates that due to Amazon AWS services, a new economy math is required• Traditional “IT Economy” is based upon monthly and yearly calculations, Amazonomics calculations are based upon hours• While traditional math calculate costs top-down, Amazonomics calculate costs bottom-up
What are Amazonomics
Confused? Can be at start…Hang in there…
• An On-Demand Instance• An On-Demand instance can be described as a virtualized host – in it’s most basic form, that once shut down disappears• On-Demand Instances are billed in accordance to their size, time utilized and operating system• Instances vary from single virtual core units, up to 20 virtual cores per instance
Basic Terms – On Demand Instances
• A Reserved Instance• Similar to an On-Demand instance in functionality• Paid in advance for the entire year (or 3 years)• Provides better utilization for your funds (cost wise)• Once purchased, will remain in your inventory as “reserved” computation power
Basic Terms – Reserved Instances
• Data Transfer costs• Amazon instances consume “inbound” and “outbound” bandwidth• Amazon calculates the cost of this bandwidth on each of the streams• Inbound stream is flat-rated at a per GB cost• Outbound stream is step charged at per GB cost, with TB steps
Basic Terms – Data Transfer
• Elastic IP Addresses• By default – all instances carry a dynamic IP address• Elastic IP addresses allows for the assignment of static IP address to your instances• All instances are NATted, thus, Elastic IP addresses are statically NATted to your EC2 instances
Basic Terms – Elastic IPs
• The basic concept behind EC2 economics is:
Amazon EC2 – Power ala-cart
You pay for what you use!
• If you want reserved power or resources, you pay for that as-well• When you are done consuming resources, simply discard them and don’t pay additional charges
• Theoretical example:• Imagine that we would like to “crack” an SSH2, IDEA based, 1024 bit key• In theory, in order to “crack” the key, we’ll either need a few thousand years or a few thousand computers• Installing and managing a few thousand computers is not feasible, nor is waiting a few thousand years• Using EC2 instances, we can initiate thousands of hosts at the same time, do the job, then discard them• Think of it as an on-demand SETI group that is made available to you
Amazon EC2 – Power ala-cart (cont.)
A CFOs view of Amazon EC2 Power
• Purchasing the above hardware can easily cost around 12,000$ in the first fiscal year• Hosting your development system with Amazon EC2 simply makes sense• Just turn off your hosts at night, and save the money
Instance Size
Instance Cost
Work hours per day
Number of hosts required
Cost per day
Work days per month
Cost per month
Cost per year
Medium $0.20 10 4 $8.00 22 $176.00 $2,112.00
Extra Large 0.8 10 2 $16.00 23 $368.00 $4,416.00
Production on EC2?
• Depending on your application, going production on EC2 isn’t always economical – especially if your application requires full power 24x7• The best track to take is to de-couple your application layer and initiate instant power on EC2 at peak time• Get the best of both worlds – instant power with 24x7 co-location costs
Instance Size
Instance Cost
Work hours per day
Number of hosts required Cost per day
Work days per month
Cost per month Cost per year
Medium $0.20 24 3 $14.40 22 $316.80 $3,801.60
Extra Large 0.8 24 2 $38.40 23 $883.20 $10,598.40High-Powered, Short burstapplications, such as dialers
love EC2!
• Amazonomics change the way we look at costs, relating to IT infrastructure• Companies at their start-up stage can easily utilize Amazon EC2 instances to lower their initial OPEX, turning it into CAPEX• The price of development hardware purchase can easily translate to 3 years worth of EC2 instances – letting you focus on your product and development
Module Recap
• Lab Duration: 30 minutes• Tasks (Theoretical Exercise only): • Design a high-powered Asterisk based dialer platform, utilizing Amazon EC2 components only – based upon economic considerations only• Design constraints are:• Dialer must be able to sustain a total of 1000 concurrent channels• Each EC2 instance can carry up to 120 concurrent channels (medium instance)• Control over the Dialer must be centralized
Module 2: Recap Lab
Network Considerations, Elastic IP addressesand de-coupled architecture
Module 3
CLOUD COMPUTING CLASS
• Amazon EC2 is a fully NATted network• The EC2 cloud environment is fully NATted, which means that all hosts utilize a private IP address space• NATted environments pose an interesting issue what utilized with VoIP in general – and SIP in particular
• Amazon EC2 is geographically de-coupled• When an instance is initiated, the physical location of that instance on the network is somewhat unknown• Relying on the network layer for real time data traversal (block connections, database DSN’s, etc) is no longer available
Network Considerations
In other words:A shift in platform design
and development paradigmsis required to utilize EC2
• Amazon operates 2 AWS global datacenters:• US East – USA East Coast• EU West – Western Europe
• While the information seems redundant to most developers, VoIP usage requires a bit more finesse• Imagine that you are located in the UK, it would make more sense for you to host your Asterisk servers at EU-West and not US-East• When initiating instances, make sure these are initiated in the most suitable location
EC2 Network – A Bird’s Eye View
• Not all VoIP carriers are capable of servicing NATted hosts• Make sure you test your carriers with a simple Asterisk instance on EC2, prior to migrating your system to it• While SIP doesn’t really like NAT, IAX2 loves it – USE IT!
EC2 Network – VoIP Considerations
Remember: If using SIP, media will pass
through your Asterisk AMI – on EC2 that means traffic
payment!
• Amazon operates 2 AWS global datacenters:• US East – USA East Coast• EU West – Western Europe
• While the information seems redundant to most developers, VoIP usage requires a bit more finesse• Imagine that you are located in the UK, it would make more sense for you to host your Asterisk servers at EU-West and not US-East• When initiating instances, make sure these are initiated in the most suitable location
EC2 Network - Datacenters
Even these are spread acrossa few other data centers – inother words, no commitmentfor same location proximity
• Elastic IP addresses are statically mapped IP addresses that can be assigned to your instances, once initiated• You can assign your Elastic IP either from the management console, or via the AWS EC2 API• In any case, your internal host’s IP address always changes – if you turn your instance OFF• This can pose an interesting configuration issue when utilizing Asterisk and SIP (externip and externhost)
EC2 Network – Elastic IP Addresses
De-coupled N-Tier Architecture
CLOUD COMPUTING CLASS
Traditional Asterisk Implementation
Traditional Asterisk Implementation• The main issues with Asterisk N-Tier architecture are these:• The system has a distinct bottle neck with the database.• Storage resources are consumed across the entire network• A direct connection from the Asterisk application to the database servers is required, while utilizing normal database practices
Traditional Asterisk Implementation• The N-Tier approach doesn’t cut it with Cloud Computing structures:• No direct connection between the Asterisk servers and the database servers• No direct connection between the storage servers and the Asterisk/Database servers• No commitment to the geographical location of each server
Hmmmm….. Now What????
New Paradigm Guidelines• It is clear we need a new methodology for developing application, one that takes into account the following:• We can’t rely on proximity between the database and the Asterisk servers• We can’t rely on proximity between the storage and the Asterisk servers• We can’t rely on high-powered servers
De-Coupled N-Tier Architecture
EC2 Asterisk Implementation•Guidelines:• Each Asterisk server holds its own application logic, implemented via normal Asterisk development practices• Retrieval of information from the database servers is performed via XML-RPC methods, carried over an HTTPS secured transport• Storage of audio and content is managed via a Storage Manager and distributor• A web based application Logic implements the XML-RPC server side for the platform
• We now understand that while EC2 provides a highly cost effective solution, its constraints introduce new development and deployment issues• Traditional N-Tier design architectures and paradigms no longer cut-it with EC2• We need to practice a new methodology of designing platform, one that no longer relies on physical attributes and takes into account non-linear approaches
Module Recap
• Lab Duration: 30 minutes• Tasks (Theoretical Exercise only): • Design a high-powered Asterisk based dialer platform, utilizing Amazon EC2 components only – this time using De-Coupled N-Tier architecture• Design constraints are:• Dialer must be able to sustain a total of 1000 concurrent channels• Each EC2 instance can carry up to 120 concurrent channels (medium instance)• Control over the Dialer must be centralized
Module 3: Recap Lab
Running Asterisk compatible AMI – Introduction to the Voxilla AMI
Module 4
CLOUD COMPUTING CLASS
• EC2 is focused on providing a scalable environment for rich web applications• As such, its AMI images aren’t optimized for usage with Asterisk• Mainly, the operational kernel available with EC2 images isn’t fully compatible with Asterisk’s requirements
Is EC2 really suited for Asterisk?
STOP !!!We just killed the entire point!
If AMI images aren’t fullycompatible, what can we do?
Introducing: Voxilla Asterisk AMI• Over the past 12 months, much work had been done in the cloud-computing space• Specifically, work had been done to develop tools for deploying Asterisk in a more reliable manner• Notable work had been performed by Ronald Lewis and specifically by Eric Chamberlain of the Voxilla project• Eric’s work had been concentrating on creating a proper AMI image with a proper kernel, that will be able to utilize the DAHDI_DUMMY driver correctly
Available Voxilla AMI images• There are 2 distinct AMI images available:• ami-e8c22581 – The full Asterisk+FreePBX distribution• ami-0bfa1d62 – Vanilla Asterisk without web interfaces
• If you are a PBX user, you would most probably prefer the first• I prefer using the latter for my application development and testing
Using Voxilla Asterisk AMI
CLOUD COMPUTING CLASS
Step 1: Activate the instance
• When activating the new instance, pay attention to the following:• Data center location (availability zone)• KeyPair (what SSH key to use)• Instance type (c1.medium is preferred)• Security Group (a security group compatible with Asterisk)
Step 1: Activate the instance
Instance Size
SSH KeyPair
Data center location
Security Group
Step 2: Recompile Asterisk (optional)
You can’t use Asterisk RPMavailable from Digium. You
can use SRPM to rebuild the package – BEWARE!
Small Comparison – Codec Translation
Ver 1.6EC2
Ver 1.4Dual
XEON
Amazing !!!Asterisk 1.6 on EC2 is abetter g729 transcoder
than 1.4 on a dedicated Dual XEON Server!
• You are now familiar with Voxilla’s pre-made Asterisk EC2 AMI image• You should now be able to initiate your own EC2 based Asterisk server
Module Recap
• Lab Duration: 30 minutes• Tasks: • Initiate the vanilla Amazon EC2 Voxilla Asterisk AMI• Define 2 SIP phones • Make calls from one SIP phone to another• Repeat the same exercise with the FreePBX AMI
Module 4: Recap Lab
Beyond Voxilla – Creating your own EC2 AMIModule 5
CLOUD COMPUTING CLASS
Congratulations… well, sort of…• Congratulations, you had successfully installed and executed your Clouded Asterisk Open Source PBX system• You had also installed and configured your customized software and application, to work with your Clouded Asterisk installation•Be warned! If you terminate your Asterisk host, all your work will go away and you will need to repeat it!
I win … All your hosts are belong to usYES ! I OWN AMAZON !IN 5 DAYS I WILL HAVE A 5 TRILION DOLLARS – ALL FROM YOUR HOSTS
Ok.. Seriously now…• How can EC2 be a useful tool, if you need to re-do all your work for each initiated instance?• Well, the answer is: YOU DON’T• EC2 provides a set of tools for creating your own AMI image, storing it with Amazon S3, and then activating as many instances of it as you need• If you’ve used any of the Voxilla images, the tools required are already available on your instance• If not - http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351
Step 1: upload keys to your instance• If you may recall, you’ve created a PK and CERT file, earlier this day – we’ll use these now• Copy your PK and CERT files to your instance• Example:
scp pk-XXX.pem cert-xxx.pem [email protected]:/mnt/
pk-XXX.pem – The Private key file that was created earliercert-XXX.pem – The Certificate file that was created earlier
Step 2: image your instance• Create an image of your running instance using the following command:
ec2-bundle-vol -k /mnt/pk-xxx.pem -c /mnt/cert-xxx.pem -u [user-id] -d /mnt
pk-XXX.pem – The Private key file that was created earliercert-XXX.pem – The Certificate file that was created earlieruser-id – Your Amazon AWS user ID (numeric value)
Grab a coffee and cake at this point, better yet,
go out and get some – thistakes time…
Step 3a: Create an S3 Storage Bucket• We need to create a storage bucket, to hold our AMI image on the Amazon S3 storage service• Creating a bucket can be performed using the s3fox tool or using the command line• The easiest is to use the s3fox – available at http://www.s3fox.net
Step 3b: Create an S3 Storage Bucket
Step 4: upload your AMI to S3• Once you’ve imaged your instance, you will find multiple image.part.* files on your file system• These comprise your entire running instance• These now need to be uploaded into S3 using the ec2-upload-bundle tool
ec2-upload-bundle -b [your-s3-bucket] -m /mnt/image.manifest.xml -a [aws-access-key-id] -s [aws-secret-access-key-id]
Your-s3-bucket – the bucket created with s3foxAws-access-key-id – your access key credentialAws-secret-access-key-id – your secret access key credentialImage.manifest.xml – the image manifest file describing the image
[rename this file to something meaningful]
Cross your fingers now, this procedure can fail
a few times till you get itup there - and worst,it’s lengthy to re-start!
Step 5: Register your AMI on EC2• AMI registration is performed using the elasticfox firefox plugin:
• When prompted, enter your bucket name and image manifest filename to register your AMI image
Step 6: Instance Activation• Once your AMI is registered, your “Images” tab would look like this (after filtering):
• Simply click the power button to start-up your AMI image and create a new instance
• You are now familiarized with how to create your own AMI image on Amazon EC2• At this point, we suggest that you take a little break before the final lab – it’s a bit long
Module Recap
• Lab Duration: 60-90 minutes• Tasks: • Build a network of 5 Asterisk hosts• All hosts should have the latest Asterisk 1.6 version• 4 hosts should be installed with FreePBX and be utilized as PBX systems• 1 host should be installed as a vanilla Asterisk and be utilized as a routing system between the nodes• Routing plans and numbering plans are entirely up to you, simply go wild
Module 5: Recap Lab