Quality of experience using deep convolutional neural networks and future trends

The development of immersive display technology enables to represent the details of contents more naturally by providing a more realistic viewing environment while increasing immersion. In parallel, quality of experience (QoE) has been dealt with and discussed from both academy and industry to grade consumer products from the quality perspective. However, for quantification of QoE, it is very challengeable to analyze the human perception more accurately, even if it has been studied in many decades. Currently, there is no solid methodology to verify human perception as a closed-form objectively due to the limitation of human perception analysis. Recently, the deep convolutional neural network (CNN) has emerged as a core technology while breaking most performance records in the area of artificial intelligence via intensive training in accordance with the massive dataset. The main motivation of this paper lies in finding new insight into human perception analysis for QoE evaluation through visualization of intermediate node values. This new QoE assessment approach enables us to figure out the human visual sensitivity without using any prior knowledge. Toward the end, we provide a novel clue of how to obtain visual sensitivity, which is expected to be essentially applied for future QoE applications. In addition, we discuss future applications in QoE assessment with respect to the display types.