Service providers have for many decades measured key performance indicators for their SS7 interconnects with long-distance or international operators or peering partners. Such measurements are defined in ITU-T Recommendation E.411 "International Network Management – Operational Guidance" and E.422 "Quality of Service for Outgoing International Calls" and include Answer Seize Ratio [ASR], Post Dial Delay [PDD] and Network Efficiency Ratio [NER].
Name | Description |
Counted | Number of calls |
ASR | Answered calls (percent) |
SSB | Subscriber busy (percent) |
CGC | Circuit Group Congestion (percent) |
SEC | Switching Equipment Congestion (percent) |
CFL | Call Failure (percent) |
RSC | Reset Circuit Signal (percent) |
UNN | Unallocated Number (percent) |
ADI | Address Incomplete (percent) |
CLF | Clear Forward (percent) |
LOS | Line Out of Service (percent) |
rt | Response time (average) |
wt | Wait time with answer (average) |
wna | Wait time with no answer (average) |
ct | Call time (average) |
ht | Hold time (average) |
Minutes | Total Call time in minutes |
Table 1
Answer Seize Ratio [ASR] is used as a measure of network quality although the measurement also includes user behavior. In other words, if the call was not answered, the network could not be faulted, although the ASR measurement would be reduced by the uncompleted call, indicating lower quality. However, because for a given hour within a day, unanswered calls would always represent the same percentage, from day-to-day, this offset would be normalized out and carriers are able to monitor the trend of ASR and treat it as a relative measurement
Network Efficiency Ratio was designed to eliminate user behavior as a factor and better represent pure network performance.
Network Efficiency Ratio [NER] is defined as:
User Answers or Normal call clearing - Cause code: 16
+ User Busy - Cause code: 17
+ Ring No Answer - Cause code: 18 & 19
+ Terminal Rejects) - Cause code: 21
NER = -------------------------------------------------------x 100
(Total # of Call Attempts i.e. IAM’s)
SIP is a more flexible protocol and has wider uses than simple call control. Therefore, use cases differ widely and systems such as voicemail and call forwarding skew expected behavior for answered calls
Furthermore, SIP is a more flexible protocol and has wider range of response messages which can be used to specifically indicate certain types of failures from either servers, network devices or the movement or absence or other behavior of users. Accordingly, IETF has defined KPI's equivalent to Answer Seize Ratio and Network Efficiency Ratio. These are described under SIP End-to-End Performance Metrics draft ietf-pmol-sip-perf-metrics-04. For example, the equivalent of ASR is Session Establishment Ratio (SER)
Session Establishment Ratio (SER) is defined as follows, to quote the IETF
“This metric is used to detect the ability of a terminating UA or
downstream proxy to successfully establish sessions per new session
INVITE requests. SER is defined as the number of new session INVITE
requests resulting in a 200 OK response, to the total number of
attempted INVITE requests less INVITE requests resulting in a 3XX
response. This metric is similar to Answer Seizure Ratio (ASR)”
The SER is calculated using the following formula:
# of INVITE Requests w/ associated 200 OK
SER = --------------------------------------------------------- x 100
(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)
Here is the message flow which defines the basic SER. if the session INVITE request results in an interim response, such as a 302 Redirect response, this should be subtracted from the denominator.
UA1 UA2
| |
|INVITE |
+----------->|------------------>|
| | 180|
| |<------------------|
Session Established | |
| | |
| | 200|
+----------->|<------------------|
| |
In SS7, the fate of the call is determined by the RELEASE message which ends the call. The fate of a SIP call is determined by the Response to each Request for each transaction within the SIP call. For example as above, session establishment is determined by a successful outcome from the call session establishment phase
The SIP Peering KPI’s RFC 6076 also defines Session Establishment Effectiveness Ratio (SEER) which is similar to Network Efficiency Ratio [NER] in the SS7 ISUP circuit switched world.
This metric is complimentary to SER, but excludes the effects of the terminating UAS or endpoint i.e. it excludes user behavior from the metric and therefore more closely reflects the performance of the network. SEER is defined as the number of INVITE requests resulting in a 200 OK response and INVITE requests resulting in a 480, 486 (Busy Here i.e. that endpoint is busy), or 600 (Busy Everywhere; i.e. the interconnecting network is busy or congested or down) to the total number of INVITE attempts less the 3xx, interim responses.
In order to simplify the formula, the following variable ‘a’ is used to summarize multiple SIP responses:
a = 3XX, 401, 402, and 407
The SEER is calculated using the following formula:
The SEER is calculated using the following formula:
# of INVITE Requests w/ associated 200 OK, 480, 486, or 600
SEER = -------------------------------------------------------- x 100
(Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response)
- 2xx—Successful Responses
- Eg 200 OK
- 4xx—Client Failure Responses
- 401 Unauthorized (Used only by registrars or user agents. Proxies should use proxy authorization 407)
- 402 Payment Required (Reserved for future use)
- 480 Temporarily Unavailable
- 486 Busy Here
- 6xx—Global Failure Responses
- 600 Busy Everywhere
- 603 ADD
Another very useful measurement defined by the IETF here is Session Defects Ratio (SDR), also graphed by Palladion by checking a box. ‘503’ SIP Response messages commonly indicate a route to one of your peering partners is failing due to congestion of the Gateway or SoftSwitch.
Session Defects Ratio (SDR) is the percentage of call attempts receiving the following responses, in relation to total call attempts or INVITES:
o 500 Server Internal Error
o 503 Service Unavailable
o 504 Server Timeout
The SDR is calculated using the following formula:
# of INVITE Requests w/ associated 500, 503, or 504
SDR = ----------------------------------------------------- x 100
Total # of INVITE Requests
In addition to this, some carriers like to plot Post Dial Delay, [PDD] and to generate an alert when this measurement exceeds a certain time threshold. PDD is defined as the time interval between transmission of the INVITE and reception of the ‘180 Ringing’ response message. The RFC prefers to define Successful Session Setup [SRD], the SIP equivalent of Post-Selection Delay (defined in E.721). This is an early indication of congestion soon to or about to occur on a given route and is a very useful KPI. Palladion can be configured to provide an SNMP trap or other notification to the Network Operations Center [NOC] of this imminent congestion so preemptive action can be taken.
Here is a table listing all the new RFC 6076 KPI’s with their SS7 or circuit switched equivalent:
SIP Peering KPI’s RFC 6076 | SIP Definition | ISUP Equivalent | ISUP Definition |
Registration Request Delay (RRD) | Time of final response – time of REG attempt | N/A Note * | |
Ineffective Registration Attempts | # IRA/total REG attempts | N/A | |
Session Request Delay (SRD) | Time of response – Time of INVITE | ||
Successful Session Setup [SRD] | Time of response – Time of INVITE eg INVITE to 180 | Post-Selection Delay (defined in E.721) | IAM to ACM (or ALERTING) |
Failed Session Setup [SRD] and | INVITE to response indicating failure eg 4XX (excluding 401, 402, and 407 non-failure challenge response codes), 5XX, or 6XX message. | N/A | IAM to REL with a cause code indicating a failure |
Successful Session Disconnect Delay [SDD] | BYE, to 2XX Ack | Time to Clear a good Call or CIC | REL to RLC |
Failed Session Disconnect Delay [SDD] | BYE, to Timer F Expires | essage | Missing RLC i.e. expiry of ISUP T1 timer |
Session Duration Time [SDT] | Time of BYE or time out – Time of 200 OK | Time of REL - Time of ANS | Average Call Hold Time (ACHT) |
Successful Session Duration Time | 200 OK response to an INVITE to BYE | See note below * | |
Failed session duration SDT | 200 OK response to an INVITE, and the resulting Timer F expiration. | “awaiting Answer timer” | SS7 ISUP timer T9 (no response to ACM) |
Session Establishment [SER ] Ratio | # of “good call” INVITEs ---------------------------------- X % Total INVITEs – interim 3xx Responses | Calls which connect Calls which fail | Answer Seize Ratio (ASR) |
Session Establishment Effectiveness Ratio (SEER) | (# of INVITE Requests w/ associated 200 OK, 480, 486, or 600) ------------------------------ X % (Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response) | NER = Answers (cc 16) +UserBusy-cc 17 + Ring No Answer (cc 18, 19 & 21) + Terminal Rejects) _____________ Total call attempts | Network Efficiency Ratio (NER) ITU E.411. |
Ineffective Session Attempts (ISA) Invites resulting in:-o 408 Request Timeouto 500 Server Internal Error o 503 Service Unavailable o 504 Server Timeout | # of ISA x 100 ----------------------------- Total # of Session Requests | Ineffective Machine Attempts (IMA) in telephony applications of SIP, and was adopted from Telcordia GR-512-CORE [GR-512]. | |
Session Completion Ratio (SCR) (a Session Completion is any SIP dialog that receives a valid response) | # of Successfully Completed Sessions x 100 ------------------------------------------ Total # of Session Requests | Call Completion Ratio (CCR) | |
Notes:
· * REGISTRATIONs are rare across SIP peering points as these interconnects are typically between two trusted environments. In addition SBC’s are used to protect the sanctity of each network. Similarly an interconnect between two SS7 networks is a trusted and secure interface, so accordingly, no equivalent of REGISTRATIONs exists in SS7, although the MTP3 protocol is used to establish routing between Signaling Point Codes and ISUP control messages to set up voice trunks
· ** Monitoring for short duration Successful calls in a circuit switched network has not been as important and for a VoIP network as speech quality is not so much of an issue in circuit switched networks. In a VoIP network, short calls are important to monitor because if speech quality is so bad, the callers hang up immediately and call back, ….hopefully.
· Basic Telephony SIP End-to-End Performance Metrics Request for Comments: 6076
· ITU-T Recommendation E.411 : International network management - Operational guidance
· ITU-T Recommendation E.422 : Observations on international outgoing telephone calls for quality of service
· Telcordia GR-512-CORE - LSSGR: Reliability, Section 12
