CSN Performance Benchmark Result

 

1. Result summary of the system performance.

 

All the benchmarking tests are performed with 2 hour duration on CSN build 49. Results below show the maximum sustain call rate that system can maintain without service degrades or system failure.

 

Test	Servers Configurations	Scenario	CPU Avg.-Peak	Memory Total Size	Max. Call Rate	Call Pass Rate	Comment
1	SunFire 240, 2CPU, 4G – Single with 30K subscriber	Basic call	26%-40%	3156M	40cps	99.9%	DB activities on same server affect performance.
2	SunFire 240, 2CPU, 4G – Single with 30K subscriber	Basic Call with join	26%-50%	3102M	20cps	99.2%	Same as above.
3	N1280 4CPU, 8G –Single with 100K Subscribers	Basic call	53%-73%	5326M	120cps	99.0%	It can stable at 150cps, a lot of re-trans & failed calls
4	N1280 4CPU, 8G –Single with 100K Subscribers	Basic Call with join	55%-75%	5238M	60cps	99.9%	70cps, service degrade when cache update
5	N1280 8CPU, 16G –Single with 500K Subscribers	Basic call	42%-56%	7089M	180cps	99.6%	Average call rate is 175cps
6	N1280 8CPU, 16G –Single with 500K Subscribers	Basic Call with join	48%-62%	7108M	160cps	99.8%	significant diff. with 4CPU
7	N1280 4CPU, 8G –Pair with 100K Subscribers	Basic call	50%-70%	5238M	120cps	99.7%	It can stable at 150cps, a lot of re-trans & failed calls
8	N1280 4CPU, 8G –Pair with 100K Subscribers	Basic Call with join	52%-72%	5294M	60cps	99.9%	70cps, service degrade when cache update

Note: CPU Avg.  -- CPU average usage when no cache update

         CPU Peak – CPU peak usage during cache update. It will drop down once cache update finishes.

         Memory Size – determine by total subscriber number and sip connector pre-allocated memory size.

 

2. Cache update vs. No cache update

 

-          Cache update has impact to system performance when call rate is high.

-          CSE-SIP connection may bounce in short time and cause some call failure when cache update.

-          If call load is too heavy, cache update may cause all RTT connection drop and failed to resume.

-          Cache update will contribute to the network traffic but does not casue system components to degrade.

-          If no cache update, the result may be better. But all evidence show we can not stable at 200cps currently.

-          For 200cps on 8 CPU platforms without cache update, there will be huge amount of re-trans and message time-out. The average call pass rate is about 187cps.

 

3. Cache update time & resource usage

 

2 CPU platforms, 30K subscriber with 90K devices:

-          9 seconds to load into cache.

-          CSE cache thread CPU peak usage during cache update: 14%

-          CSE memory size with 1 cache image: 114M

-          CSE memory size with 2 cache image: 190M

-          No memory grows for later cache update.

 

4 CPU platforms, 120K subscriber with 340K devices:

-          60 seconds to load into cache.

-          CSE cache thread CPU peak usage during cache update: 20%

-          CSE memory size with 1 cache image: 342M

-          CSE memory size with 2 cache image: 582M

-          No memory grows for later cache update.

 

8 CPU platforms, 500K subscribers with 1M devices:

-          165 seconds to load into cache.

-          CSE cache thread CPU peak usage during cache update: 12%

-          CSE memory size with 1 cache image: 930M

-          CSE memory size with 2 cache image: 1710M

-          No memory grows for later cache update.

 

4. Simplex platform vs. duplex platform (4cpu, 8G)

 

-          We got similar results for simplex and duplex platform.

-          Load sharing with two CSE does not degrade the services.

-          SR replication, Cache update on duplex platform does not degrade the services. (Compare with same call rate)

-          Majority resource usage is on primary host, only secondary CSE consumes some resource on other host.

-          CDR billing buffer file will accumulate for remote DB host on both sides.

-          On simplex platform, if DB is on same server, DB activities like OM report, daily maintenance (CDR load switch, DB backup) will have impact on performance.

 

5. Basic Call vs. Join Call

 

-          JOIN call de-rate the service about 50% compare with basic call scenario (120cps), system resource (CPU) may be a bottleneck on 4 CPU platform.

-          On 8 CPU platform, join scenario has significant improvement. it can stable at 160cps, the bottleneck may lies in software or network capacity.

 

6. Call Rate vs. CPU usage

 

-          SIP connector consumes most of the CPU usage during traffic.

-          CPU total consumption increase 4~5% for every additional 10cps. (4CPU platform, basic call)

 

7. More detailed result about various call rate are saved at:

\\badger\public\Test\Release 3.1\SIP\automation\results\traffic\michaelma\Benchmarking

 

 

CSN Performance Benchmark Hardware Configuration.

 

Servers	Server Type	CPUs	Memory	Subscribers
1	SunFire 240	2	4	30,000
1	Netra 1280	4	8	100,000
1	Netra 1280	8	16	500,000
2	Netra 1280	4	8	100,000

 

 

CSN Performance Benchmark Software Parameters

 

Parameter	Value	Comment
CSN build	49
TCAP	No TCAP invocations included
SIP Registration	No SIP Registrations included
Hold Time	3 minutes
Stable Benchmark Duration	2 hours
SIP Timers	T1=500ms T2=4000ms T4-0ms Invite Linger Timer(D)=0ms General Linger Timer-32000ms SIP_Calls_Max=120K SIP_Transanctions=60K For 8 CPU: SIP_Calls_Max=150K SIP_Transanctions=150K	must be at standard/representative settings
Provisioning Cache Refresh	30 minutes

 

Thanks,

--

Michael Ma

E-Mail: michael.ma@newstep.com

Tel: 416-979-5404 x312