Knowledge distillation (KD) is a well-known method to reduce inference latency by compressing a cumbersome teacher model to a small student model. Despite the success of KD in the classification task, applying KD to recommender models is challenging due to the sparsity of positive feedback, the ambiguity of missing feedback, and the ranking problem associated with the top-N recommendation. To address the issues, we propose a new KD model for the collaborative filtering approach, namely collaborative distillation (CD). Specifically, (1) we reformulate a loss function to deal with the ambiguity of missing feedback. (2) We exploit probabilistic rank-aware sampling for the top-N recommendation. (3) To train the proposed model effectively, we develop two training strategies for the student model, called the teacher-and the student-guided training methods, selecting the most useful feedback from the teacher model. Via experimental results, we demonstrate that the proposed model outperforms the state-of-the-art method by 5.5-29.7% and 4.8-27.8% in hit rate (HR) and normalized discounted cumulative gain (NDCG), respectively. Moreover, the proposed model achieves the performance comparable to the teacher model.
|Title of host publication||Proceedings - 19th IEEE International Conference on Data Mining, ICDM 2019|
|Editors||Jianyong Wang, Kyuseok Shim, Xindong Wu|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||10|
|Publication status||Published - 2019 Nov|
|Event||19th IEEE International Conference on Data Mining, ICDM 2019 - Beijing, China|
Duration: 2019 Nov 8 → 2019 Nov 11
|Name||Proceedings - IEEE International Conference on Data Mining, ICDM|
|Conference||19th IEEE International Conference on Data Mining, ICDM 2019|
|Period||19/11/8 → 19/11/11|
Bibliographical noteFunding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant (No. NRF-2018R1A2B6009135 and NRF-2019R1A2C2006123) and the Institute of Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government (MSIT) (No.2019-0-00421, AI Graduate School Support Program).
© 2019 IEEE.
All Science Journal Classification (ASJC) codes