Interpreting MOS scores, when can users see a difference? Understanding user experience differences for photo quality

Stamos Katsigiannis, James Scovell, Naeem Ramzan, Lucjan Janowski, Philip Corriveau, Michele Saad, Glenn Van Wallendael

Research output: Contribution to journalArticle

Abstract

The use of no-reference image quality evaluation tools that produce MOS scores, like the VIQET tool which was released by the Video Quality Expert Group, raises the question of whether the produced MOS differences between images correspond to noticeable differences in quality by the consumers. In this work, we attempted to approximate the minimum MOS difference that is required in order for people to be able to distinguish between a higher and a lower quality image under realistic conditions that are commonly encountered in the current consumer space. 91 people participated in a subjective just-noticeable-differences study across three countries that used non-simulated image stimuli, produced and evaluated through crowd sourcing for the validation of the VIQET no-reference image quality tool. The image dataset consisted of 15 different scenes belonging to three different scene types, with a total of 210 different image pairs being used. After evaluating the quality of the collected data, a logistic regression analysis approach was employed in order to estimate the minimum MOS difference required between two images in order for a given percentage of people to be able to detect the higher quality image.
Original languageEnglish
JournalQuality and User Experience
Volume3
Issue number1
Early online date2 May 2018
DOIs
Publication statusE-pub ahead of print - 2 May 2018

Fingerprint

Image quality
Regression analysis
Logistics

Cite this

Katsigiannis, Stamos ; Scovell, James ; Ramzan, Naeem ; Janowski, Lucjan ; Corriveau, Philip ; Saad, Michele ; Van Wallendael, Glenn. / Interpreting MOS scores, when can users see a difference? Understanding user experience differences for photo quality. In: Quality and User Experience. 2018 ; Vol. 3, No. 1.
@article{da5a84a6a96b4b7e8db4208e6f8b1028,
title = "Interpreting MOS scores, when can users see a difference?: Understanding user experience differences for photo quality",
abstract = "The use of no-reference image quality evaluation tools that produce MOS scores, like the VIQET tool which was released by the Video Quality Expert Group, raises the question of whether the produced MOS differences between images correspond to noticeable differences in quality by the consumers. In this work, we attempted to approximate the minimum MOS difference that is required in order for people to be able to distinguish between a higher and a lower quality image under realistic conditions that are commonly encountered in the current consumer space. 91 people participated in a subjective just-noticeable-differences study across three countries that used non-simulated image stimuli, produced and evaluated through crowd sourcing for the validation of the VIQET no-reference image quality tool. The image dataset consisted of 15 different scenes belonging to three different scene types, with a total of 210 different image pairs being used. After evaluating the quality of the collected data, a logistic regression analysis approach was employed in order to estimate the minimum MOS difference required between two images in order for a given percentage of people to be able to detect the higher quality image.",
author = "Stamos Katsigiannis and James Scovell and Naeem Ramzan and Lucjan Janowski and Philip Corriveau and Michele Saad and {Van Wallendael}, Glenn",
year = "2018",
month = "5",
day = "2",
doi = "10.1007/s41233-018-0019-8",
language = "English",
volume = "3",
journal = "Quality and User Experience",
issn = "2366-0139",
publisher = "Springer International Publishing AG",
number = "1",

}

Interpreting MOS scores, when can users see a difference? Understanding user experience differences for photo quality. / Katsigiannis, Stamos; Scovell, James; Ramzan, Naeem; Janowski, Lucjan; Corriveau, Philip; Saad, Michele; Van Wallendael, Glenn.

In: Quality and User Experience, Vol. 3, No. 1, 02.05.2018.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Interpreting MOS scores, when can users see a difference?

T2 - Understanding user experience differences for photo quality

AU - Katsigiannis, Stamos

AU - Scovell, James

AU - Ramzan, Naeem

AU - Janowski, Lucjan

AU - Corriveau, Philip

AU - Saad, Michele

AU - Van Wallendael, Glenn

PY - 2018/5/2

Y1 - 2018/5/2

N2 - The use of no-reference image quality evaluation tools that produce MOS scores, like the VIQET tool which was released by the Video Quality Expert Group, raises the question of whether the produced MOS differences between images correspond to noticeable differences in quality by the consumers. In this work, we attempted to approximate the minimum MOS difference that is required in order for people to be able to distinguish between a higher and a lower quality image under realistic conditions that are commonly encountered in the current consumer space. 91 people participated in a subjective just-noticeable-differences study across three countries that used non-simulated image stimuli, produced and evaluated through crowd sourcing for the validation of the VIQET no-reference image quality tool. The image dataset consisted of 15 different scenes belonging to three different scene types, with a total of 210 different image pairs being used. After evaluating the quality of the collected data, a logistic regression analysis approach was employed in order to estimate the minimum MOS difference required between two images in order for a given percentage of people to be able to detect the higher quality image.

AB - The use of no-reference image quality evaluation tools that produce MOS scores, like the VIQET tool which was released by the Video Quality Expert Group, raises the question of whether the produced MOS differences between images correspond to noticeable differences in quality by the consumers. In this work, we attempted to approximate the minimum MOS difference that is required in order for people to be able to distinguish between a higher and a lower quality image under realistic conditions that are commonly encountered in the current consumer space. 91 people participated in a subjective just-noticeable-differences study across three countries that used non-simulated image stimuli, produced and evaluated through crowd sourcing for the validation of the VIQET no-reference image quality tool. The image dataset consisted of 15 different scenes belonging to three different scene types, with a total of 210 different image pairs being used. After evaluating the quality of the collected data, a logistic regression analysis approach was employed in order to estimate the minimum MOS difference required between two images in order for a given percentage of people to be able to detect the higher quality image.

U2 - 10.1007/s41233-018-0019-8

DO - 10.1007/s41233-018-0019-8

M3 - Article

VL - 3

JO - Quality and User Experience

JF - Quality and User Experience

SN - 2366-0139

IS - 1

ER -