astricon 2007
TRANSCRIPT
Confidential
AstriCon 2007
September 27
Testing Asterisk
Advanced Test Methodologies
for Enterprise Class IP PBXs
Stefano Carlini & Dario Busso, Klarya srl
Confidential
Klarya @ ATS Group
We develop Open Source IP Communication Solutions
...dai voce alla tua rete!
Confidential
...dai voce alla tua rete!
Customers in Italy
Confidential
Enterprise IP CommunicationWe need Enterprise Class reliability for our IP
Communications platforms and solutions
VoIP / IP Telephony Unified Communications: IM, Presence, File Exchange Web Collaboration
Three years ago (2004) we choose Asterisk as VoIP Solutions
...dai voce alla tua rete!
Confidential
Is Asterisk enterprise ready?Asterisk is/“pretends to be” an Enterprise Class IP PBX
==> the companies providing Asterisk solutions need:
Test the Asterisk performances
(more important) certify and guarantee these performances to the customers
...dai voce alla tua rete!
Confidential
Asterisk & SIP, the key factorSIP’s scalability is a key factor to Asterisk VoIP
technology’s continued adoption
The need to test the Asterisk iPBX servers under heavily saturated scenarios is becoming more acute
The Asterisk SIP servers must be able to bear heavy traffic loads
...dai voce alla tua rete!
Confidential
The tutorial focusThe focus today is to delineate issues involved in testing
Asterisk iPBX servers under heavy call saturation.
We show how to test the Asterisk PBX under different test scenarios in order to find out:
the VoIP Call Setup Rate the maximum number of concurrent VoIP calls the quality of Voip calls.
...dai voce alla tua rete!
Confidential
Important Note
...dai voce alla tua rete!
This is an overview about Asterisk Testing Methodologies
This is not a full and complete study
Purpose: give the guidelines to test your Asterisk PBXs
Confidential
Testing SIP, interesting... but...
What I need ?
test tools, call generators & terminators capable of processing the same significant loads.
Identify o the test scenarioso the Hw & Sw PlatformoThe measurable results
Confidential
Test Tools: Spirent Abacus 5000Protocols:
o SIP, SIP-To Skinny, H.323o MGCP, H.248/MEGACO, SIGTRAN
Interfaces:o TCP/IP: Gigabit Etherneto PSTN: T1, E1, T3, E3o Analog: FXS, FXO
Payload Media:o Tones,o PCM Audioo Video (H.261, H.263)o Audio (G.711, G.723, G.726, G.729A/B)
Confidential
Abacus 5000: IGC3 card (1)
1024 calls, RTP with Packet Path Confirmation 4096 calls, no RTP Registration and full SIP call set up and tear
down Best for Proxy, Registration, Authentication, Parsers, or non-RTP
applications Decoding, PSQM, PESQ, MOS, E-Model, R Factor, J-MOS 256 simultaneous endpoints with compression and decompression G.711u-A law, G.726, G.729AB, G.729B, G.723.1, ... 256 simultaneous PSQM channels 128 simultaneous PESQ channels 128 send or receive T.38 Fax Relay over IP
Confidential
o IGC3 Performance:o65K simultaneous registered SIP users per ICG3
subsystemoUp to 8192 channels of Signaling without voice
for SIP only (packet path confirmation)oUp to 4096 channels of Signaling without voice
for all other VoIP protocols (packet path confirmation)oUp to 1024 channels of RTP, (packet path
confirmation)
Abacus 5000: IGC3 card (2)
Confidential
Test Scenario (1)
Confidential
Test Scenario (2)
Confidential
Test Scenario (3) VoIP Call Setup Rate Maximum Concurrent VoIP Calls Converged VoIP and Data Test VoIP test during DoS attacks VoIP Performance with NAT VoIP Performance over IPSEC
Confidential
System(s) Under Test – SUTs (1)Asterisk “low profile”(SOHO installation)
CPU: Via C3 1 Ghz Memory: 1 GB RAM Ethernet: VIA VT6103
PHYT 10/100 Mbps
Asterisk “high profile”
(Enterprise installation)
CPU: Xeon 3.0 GHz Memory: 1 GB RAM Ethernet: NC373i
Multifunction Gigabit Network Adapter
Confidential
System(s) Under Test – SUTs (2)
Operating System: oDebian Gnu\Linux Etch 4.0oKernel 2.6.18-4
PBX: Asterisk 1.2 NO Real Time Support
Dial Plan: FreePBX 2.2.1 Dial Plan Rules
RDMS: MySQL 5.0.32 CDR Storage
Confidential
The Measurable Results: Abacus 5000Call attempts.Call completions.Percentage of call completions.Call attempts per hour.Script attempts (number of tests).Percentage of script attempts.Errors received for active channels.Call duration.Call set up time.Call tear down time.Clipping.Ring durationBit error rate.
PESQ value.PSQM value.MOS value.RTP packet loss rate.RTP packets out of order.RTP packets late arrival.RTP jitter.One way delay.Round trip delay.Packet rate.Resolution rate for pattern search.Packet loss.Latency.Rate increment/decrement resolution.
... and more...
Confidential
The Measured Results. Focus on:Call attempts.Call completions.Percentage of call completions.Call Setup time.PESQ value.
MOS value.RTP jitter.Round trip delay.Packet loss.Latency.
Confidential
Testing Methodology (1) Standard 3-Way Handshake of SIPoBefore calling can commence, the Abacus 5000 must
issue a REGISTER request for each of the SIP endpoints.
oThe Asterisk is expected to return a "200 OK" message for each.
oAfter the registration process is complete, the calling phase can begin. The Asterisk receives an INVITE request with an SDP payload that describes the desired RTP stream variables for each call.
oThe Asterisk is expected to answer each call with a status message of "200 OK".
oThe Signaling Server then establishes an RTP audio stream for each call
Confidential
Testing Methodology (2) All tests consisted of a steady (fixed Call Per
Second) call ramp up function attempting to reach the desired call volume.
Total test time is 10 minutes for every test. After 10 minutes the calls is terminated and the
channels restarted. We repeat the test with more SIP EndPoints When stop? o When the measured Call Completition is < 99,9%
Confidential
Testing Methodology (3)
Media Path Validation: Packet Path Confirmation. A single test subnet is used to reduce testing
variables. Future testing of SIP should include multi-hop and WAN/LAN scenarios.
Confidential
Test #1: Call Setup Rate (1) Standardo IETF RFC 3261 (Session Initiation Protocol)
PurposeoMax VoIP Call Setup Rate that Asterisk can handle
DescriptionoThis test is an end to end VoIP setup with calls
forwarded by an Asterisk in the middle where the rate of the call establishment is measured. Calls can be generated and terminated by the abacus system.
Materials neededoAbacus 5000 with one ICG3 cardoAsterisk PBX
Confidential
Test #1: Call Setup Rate (2) Setup
Confidential
Test #1: Call Setup Rate (3) ProcedureoConfigure the Abacus to emulate many VoIP calls
(originating and terminating).oConfigure Asterisk to forward VoIP traffic.oSet your call length to '0' so that calls terminate
immediately after establishmentoStart generating calls through the Asterisk at highest
rate possible.oMeasure the call setup rate through the AsteriskoUse a graph to map your call setup rates vs. time
entries.
Confidential
Test #1: Call Setup Rate (4) VariablesoNumber of VoIP clients (the SIP Endpoints)o IP addressing IPv4, IPv6oVoIP protocols SIP or H.323oUse of a proxy or gatekeeper
ResultsoCall Setup Rate (calls/s)
Confidential
Test #2: Max Concurrent Calls (1) Standardo IETF RFC 3261 (Session Initiation Protocol)
PurposeoThis test determines the maximum number of VoIP
calls that can be established through an Asterisk. DescriptionoThis test is an end to end VoIP setup with calls
forwarded by an Asterisk in the middle. Calls can be generated and terminated by the abacus system.
Materials neededoAbacus 5000 with one ICG3 cardoAsterisk PBX
Confidential
Test #2: Max Concurrent Calls (2) Setup
Confidential
Test #2: Max Concurrent Calls (3) ProcedureoConfigure the Abacus to emulate many VoIP calls
(originating and terminating).oConfigure Asterisk to forward VoIP traffic.oGenerate calls starting with one call and increasing
your volume until the maximum number of calls is reached in the Asterisk.
oUse a graph to map your results in number of Sip endpoints vs. normalized number of call completed.
Confidential
Test #2: Max Concurrent Calls (4) VariablesoNumber of VoIP clientso IP addressing IPv4, IPv6oVoice codec typeoVoIP protocols SIP or H.323oUse of a proxy or gatekeeper
ResultsoRTP throughput (MB/S)oMaximum number of calls completedoEnd to end delayoPacket Loss and Jitter
Confidential
Results We perform different kind of test
oNoCDR-NoDPR: Just Asterisk, without CDR recording, without complex Dial Plan rules
oCDR-NoDPR: Asterisk with CDR engine, without complex Dial Plan rules
oNoCDR-DPR: Asterisk without CDR engine, with Dial Plan rules from FreePBX.
Purpose: Find the bottleneck We repeated every test 3 times (to have average
values for the results)
Confidential
Results – Call Setup Rate Test (1) To compare results you need a reference We choose the following Reference Chart from Minacom
(source http://www.minacom.com)
Confidential
Results – Call Setup Rate Test (2)
Confidential
Results – Call Setup Rate Test (3)
Confidential
Results – Max Concurrent Calls
Confidential
Results – Max Concurrent Calls
Confidential
Conclusion (1) To deploy Enterprise-Class Asterisk solutions you
need
– Test– Measure performances and perform optimization– Test Again ;)– Certify your performances to your customers
With the simple (but important) examples we can learn a some important thinks.
Confidential
Conclusion (2)Identify your deployment target
Study the competitor's performances in order to have a reference
Spend time to analyse and optimize your dial plan
Study your system (OS & Applications) in order to optimize the configuration parameters
Confidential
Conclusion (3)A simple example: when you grow the number of the
concurrent calls you get the following error in Asterisk:
ERROR[6637] rtp.c: Unable to allocate socket: Too many open files
To solve the trouble you need increase the number of Operating System Open Files. In Gnu\Linux you need add the following command in startup Asterisk script:
ulimit -c unlimited
