Wednesday, October 5, 2011

SIP Peering KPI’s - How to Measure Answer Seize Ratio

Service providers have for many decades measured key performance indicators for their SS7 interconnects with long-distance or international operators or peering partners. Such measurements are defined in ITU-T Recommendation E.411 "International Network Management – Operational Guidance" and E.422 "Quality of Service for Outgoing International Calls" and include Answer Seize Ratio [ASR], Post Dial Delay [PDD] and Network Efficiency Ratio [NER].

Number of calls
Answered calls (percent)
Subscriber busy (percent)
Circuit Group Congestion (percent)
Switching Equipment Congestion (percent)
Call Failure (percent)
Reset Circuit Signal (percent)
Unallocated Number (percent)
Address Incomplete (percent)
Clear Forward (percent)
Line Out of Service (percent)
Response time (average)
Wait time with answer (average)
Wait time with no answer (average)
Call time (average)
Hold time (average)
Total Call time in minutes

Table 1

Answer Seize Ratio [ASR] is used as a measure of network quality although the measurement also includes user behavior. In other words, if the call was not answered, the network could not be faulted, although the ASR measurement would be reduced by the uncompleted call, indicating lower quality. However, because for a given hour within a day, unanswered calls would always represent the same percentage, from day-to-day, this offset would be normalized out and carriers are able to monitor the trend of ASR and treat it as a relative measurement

Network Efficiency Ratio was designed to eliminate user behavior as a factor and better represent pure network performance.
Network Efficiency Ratio [NER] is defined as:

User Answers or Normal call clearing      -       Cause code: 16
+ User Busy                       -       Cause code: 17
+ Ring No Answer               -       Cause code: 18 & 19
+ Terminal Rejects)            -       Cause code: 21
NER = -------------------------------------------------------x 100
     (Total # of Call Attempts i.e. IAM’s)

SIP is a more flexible protocol and has wider uses than simple call control. Therefore, use cases differ widely and systems such as voicemail and call forwarding skew expected behavior for answered calls

Furthermore, SIP is a more flexible protocol and has wider range of response messages which can be used to specifically indicate certain types of failures from either servers, network devices or the movement or absence or other behavior of users. Accordingly, IETF has defined KPI's equivalent to Answer Seize Ratio and Network Efficiency Ratio. These are described under SIP End-to-End Performance Metrics draft ietf-pmol-sip-perf-metrics-04. For example, the equivalent of ASR is Session Establishment Ratio (SER)

Session Establishment Ratio (SER) is defined as follows, to quote the IETF

   “This metric is used to detect the ability of a terminating UA or
   downstream proxy to successfully establish sessions per new session
   INVITE requests.  SER is defined as the number of new session INVITE
   requests resulting in a 200 OK response, to the total number of
   attempted INVITE requests less INVITE requests resulting in a 3XX
   response.  This metric is similar to Answer Seizure Ratio (ASR)”

The SER is calculated using the following formula:

                 # of INVITE Requests w/ associated 200 OK
  SER = --------------------------------------------------------- x 100
     (Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)

Here is the message flow which defines the basic SER. if the session INVITE request results in an interim response, such as a 302 Redirect response, this should be subtracted from the denominator.

                           UA1                 UA2
                            |                   |
                            |INVITE             |
               |            |                180|
               |            |<------------------|
      Session Established   |                   |
               |            |                   |
               |            |                200|
                            |                   |

In SS7, the fate of the call is determined by the RELEASE message which ends the call. The fate of a SIP call is determined by the Response to each Request for each transaction within the SIP call. For example as above, session establishment is determined by a successful outcome from the call session establishment phase

The SIP Peering KPI’s RFC 6076 also defines Session Establishment Effectiveness Ratio (SEER) which is similar to Network Efficiency Ratio [NER] in the SS7 ISUP circuit switched world.

This metric is complimentary to SER, but excludes the effects of the terminating UAS or endpoint i.e. it excludes user behavior from the metric and therefore more closely reflects the performance of the network.  SEER is defined as the number of INVITE requests resulting in a 200 OK response and INVITE requests resulting in a 480, 486 (Busy Here i.e. that endpoint is busy), or 600 (Busy Everywhere; i.e. the interconnecting network is busy or congested or down) to the total number of INVITE attempts less the 3xx, interim responses.

In order to simplify the formula, the following variable ‘a’ is used to summarize multiple SIP responses:

   a = 3XX, 401, 402, and 407

The SEER is calculated using the following formula:

             # of INVITE Requests w/ associated 200 OK, 480, 486, or 600
SEER = -------------------------------------------------------- x 100
            (Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response)

 SIP response codes
  • 2xx—Successful Responses
    • Eg 200 OK
  • 4xx—Client Failure Responses
    • 401 Unauthorized (Used only by registrars or user agents. Proxies should use proxy authorization 407)
    • 402 Payment Required (Reserved for future use)
    • 480 Temporarily Unavailable
    • 486 Busy Here
  • 6xx—Global Failure Responses
    • 600 Busy Everywhere
    • 603 ADD

Another very useful measurement defined by the IETF here is Session Defects Ratio (SDR), also graphed by Palladion by checking a box. ‘503’ SIP Response messages commonly indicate a route to one of your peering partners is failing due to congestion of the Gateway or SoftSwitch.

Session Defects Ratio (SDR) is the percentage of call attempts receiving the following responses, in relation to total call attempts or INVITES:

o    500 Server Internal Error
o    503 Service Unavailable
o    504 Server Timeout
The SDR is calculated using the following formula:


                      # of INVITE Requests w/ associated 500, 503, or 504
     SDR = ----------------------------------------------------- x 100
                             Total # of INVITE Requests

In addition to this, some carriers like to plot Post Dial Delay, [PDD] and to generate an alert when this measurement exceeds a certain time threshold. PDD is defined as the time interval between transmission of the INVITE and reception of the ‘180 Ringing’ response message. The RFC prefers to define Successful Session Setup [SRD], the SIP equivalent of Post-Selection Delay (defined in E.721). This is an early indication of congestion soon to or about to occur on a given route and is a very useful KPI. Palladion can be configured to provide an SNMP trap or other notification to the Network Operations Center [NOC] of this imminent congestion so preemptive action can be taken.

Here is a table listing all the new RFC 6076 KPI’s with their SS7 or circuit switched equivalent:

SIP Peering KPI’s
RFC 6076
SIP Definition
ISUP Equivalent
ISUP Definition
Registration Request Delay (RRD)
Time of final response – time of REG attempt
Note *

Ineffective Registration Attempts
# IRA/total REG attempts

Session Request Delay (SRD)
Time of response – Time of INVITE               

Successful Session Setup [SRD]
Time of response – Time of INVITE                eg  INVITE            to      180
Post-Selection Delay (defined in E.721)
Failed Session Setup  [SRD] and

INVITE to response indicating failure eg 4XX (excluding 401, 402, and 407 non-failure challenge response codes), 5XX, or 6XX message.
IAM to REL with a cause code indicating a failure
Successful Session Disconnect Delay [SDD]
BYE, to 2XX Ack
Time to Clear a good Call or CIC
Failed Session Disconnect Delay [SDD]
BYE, to Timer F Expires 
Missing RLC i.e. expiry of ISUP T1 timer
Session Duration Time [SDT]
Time of BYE or time out – Time of 200 OK
Time of REL - Time of ANS
Average Call Hold Time (ACHT)
Successful Session Duration Time
200 OK response to an INVITE to BYE
See note below *

Failed session duration SDT
200 OK response to an INVITE, and the resulting Timer F expiration.
“awaiting Answer timer”
SS7 ISUP timer T9 (no response to ACM)
Session Establishment [SER ] Ratio
  # of “good call” INVITEs
---------------------------------- X %
Total INVITEs – interim 3xx Responses
Calls which connect
Calls which fail

Answer Seize Ratio (ASR)
Session Establishment Effectiveness Ratio (SEER)
(# of INVITE Requests w/ associated 200 OK, 480, 486, or 600)
------------------------------ X %
(Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response)
NER = Answers (cc 16)
+UserBusy-cc 17
+ Ring No Answer (cc 18, 19 & 21)
+ Terminal Rejects)
Total call attempts
Network Efficiency Ratio (NER)
ITU E.411.
Ineffective Session Attempts (ISA)
Invites resulting in:-
o  408 Request Timeout
o  500 Server Internal Error
o  503 Service Unavailable
o  504 Server Timeout

# of ISA x 100
-----------------------------               Total # of Session Requests

Ineffective Machine Attempts (IMA) in telephony
   applications of SIP, and was adopted from Telcordia GR-512-CORE
Session Completion Ratio (SCR)
(a Session Completion is any SIP dialog that receives a valid response)

# of Successfully Completed Sessions x 100
Total # of Session Requests

Call Completion Ratio (CCR)

·         * REGISTRATIONs are rare across SIP peering points as these interconnects are typically between two trusted environments. In addition SBC’s are used to protect the sanctity of each network. Similarly an interconnect between two SS7 networks is a trusted and secure interface, so accordingly, no equivalent of REGISTRATIONs exists in SS7, although the MTP3 protocol is used to establish routing between Signaling Point Codes and ISUP control messages to set up voice trunks
·         ** Monitoring for short duration Successful calls in a circuit switched network has not been as important and for a VoIP network as speech quality is not so much of an issue in circuit switched networks. In a VoIP network, short calls are important to monitor because if speech quality is so bad, the callers hang up immediately and call back, ….hopefully. 


·         Basic Telephony SIP End-to-End Performance Metrics Request for Comments: 6076
·         ITU-T Recommendation E.411 : International network management - Operational guidance
·         ITU-T Recommendation E.422 : Observations on international outgoing telephone calls for quality of service
·         Telcordia GR-512-CORE  - LSSGR: Reliability, Section 12

      No comments:

      Post a Comment