Performance Analysis and Comparison of PESQ and 3SQM in Live 3G Mobile Networks |
The ITU-T’s Perceptual Evaluation of Speech Quality (PESQ) is an intrusive objective assessment tool that has been widely used in telecommunications and IP networks. 3SQM is an ITU-T standard for single-sided non-intrusive quality measurement. The purpose of this paper is to investigate the accuracy of PESQ and 3SQM in evaluating voice quality over live 3G networks. A testbed was setup based on Asterisk to measure voice quality over 3G mobile networks. 192 voice samples from the ITU-T database were recorded via mobile phones during different times of the weekdays and the results of the objective measurements (in terms of PESQ and 3SQM) were analyzed. A further 30 samples were selected (from 192 recorded ones) and carried out informal subjective tests (with 33 subjects). The correlation of the objective and subjective results was analyzed using a 3rd order polynomial regression method. The results showed that overall PESQ (including PESQ-LQO) measures have a high correlation with subjective assessments whereas 3SQM measurements had a fair correlation. This suggests that PESQ is preferred to use for objective speech quality testing in live 3G networks when compared with 3SQM. It was also found that two individual cases in which 3SQM provided better prediction results than PESQ. It can also be noticed that the quality degradation in these cases is mainly due to loss position. It indicates that PESQ still needs to improve its performance in cases such as different loss locations. 3SQM also showed useful in identifying quality problems in Individual tests. Therefore, it is recommended that a co-existence of both measures when investigating speech quality problems in 3G mobile networks.
Goudarzi M, Sun L