Zero-shot learning with self-supervision by shuffling semantic embeddings

Hoseong Kim; Jewook Lee; Hyeran Byun

doi:10.1016/j.neucom.2021.01.037

Zero-shot learning with self-supervision by shuffling semantic embeddings

Hoseong Kim, Jewook Lee, Hyeran Byun

College of Computing

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

Abstract

Zero-shot learning and self-supervised learning have been widely studied due to the advantage of performing representation learning in a data shortage situation efficiently. However, few studies consider zero-shot learning using semantic embeddings (e.g., CNN features or attributes) and self-supervision simultaneously. The reason is that most zero-shot learning works employ vector-level semantic embeddings. However, most self-supervision studies only consider image-level domains, so a novel self-supervision method for vector-level CNN features is needed. We propose a simple way to shuffle semantic embeddings. Furthermore, we propose a method to enrich feature representation and improve zero-shot learning performance effectively. We show that our model outperforms current state-of-the-art methods on the large-scale ImageNet 21K and the small-scale CUB and SUN datasets.

Original language	English
Pages (from-to)	1-8
Number of pages	8
Journal	Neurocomputing
Volume	437
DOIs	https://doi.org/10.1016/j.neucom.2021.01.037
Publication status	Published - 2021 May 21

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (No. 2019R1A2C2003760) and in part by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01361, Artificial Intelligence Graduate School Program (YONSEI UNIVERSITY)).

Publisher Copyright:
© 2021 Elsevier B.V.

All Science Journal Classification (ASJC) codes

Computer Science Applications
Cognitive Neuroscience
Artificial Intelligence

Access to Document

10.1016/j.neucom.2021.01.037

Cite this

@article{07136e846ef34cd198a4918023cef348,

title = "Zero-shot learning with self-supervision by shuffling semantic embeddings",

abstract = "Zero-shot learning and self-supervised learning have been widely studied due to the advantage of performing representation learning in a data shortage situation efficiently. However, few studies consider zero-shot learning using semantic embeddings (e.g., CNN features or attributes) and self-supervision simultaneously. The reason is that most zero-shot learning works employ vector-level semantic embeddings. However, most self-supervision studies only consider image-level domains, so a novel self-supervision method for vector-level CNN features is needed. We propose a simple way to shuffle semantic embeddings. Furthermore, we propose a method to enrich feature representation and improve zero-shot learning performance effectively. We show that our model outperforms current state-of-the-art methods on the large-scale ImageNet 21K and the small-scale CUB and SUN datasets.",

author = "Hoseong Kim and Jewook Lee and Hyeran Byun",

note = "Funding Information: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (No. 2019R1A2C2003760) and in part by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01361, Artificial Intelligence Graduate School Program (YONSEI UNIVERSITY)). Publisher Copyright: {\textcopyright} 2021 Elsevier B.V.",

year = "2021",

month = may,

day = "21",

doi = "10.1016/j.neucom.2021.01.037",

language = "English",

volume = "437",

pages = "1--8",

journal = "Review of Economic Dynamics",

issn = "1094-2025",

publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Zero-shot learning with self-supervision by shuffling semantic embeddings

AU - Kim, Hoseong

AU - Lee, Jewook

AU - Byun, Hyeran

N1 - Funding Information: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (No. 2019R1A2C2003760) and in part by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01361, Artificial Intelligence Graduate School Program (YONSEI UNIVERSITY)). Publisher Copyright: © 2021 Elsevier B.V.

PY - 2021/5/21

Y1 - 2021/5/21

N2 - Zero-shot learning and self-supervised learning have been widely studied due to the advantage of performing representation learning in a data shortage situation efficiently. However, few studies consider zero-shot learning using semantic embeddings (e.g., CNN features or attributes) and self-supervision simultaneously. The reason is that most zero-shot learning works employ vector-level semantic embeddings. However, most self-supervision studies only consider image-level domains, so a novel self-supervision method for vector-level CNN features is needed. We propose a simple way to shuffle semantic embeddings. Furthermore, we propose a method to enrich feature representation and improve zero-shot learning performance effectively. We show that our model outperforms current state-of-the-art methods on the large-scale ImageNet 21K and the small-scale CUB and SUN datasets.

AB - Zero-shot learning and self-supervised learning have been widely studied due to the advantage of performing representation learning in a data shortage situation efficiently. However, few studies consider zero-shot learning using semantic embeddings (e.g., CNN features or attributes) and self-supervision simultaneously. The reason is that most zero-shot learning works employ vector-level semantic embeddings. However, most self-supervision studies only consider image-level domains, so a novel self-supervision method for vector-level CNN features is needed. We propose a simple way to shuffle semantic embeddings. Furthermore, we propose a method to enrich feature representation and improve zero-shot learning performance effectively. We show that our model outperforms current state-of-the-art methods on the large-scale ImageNet 21K and the small-scale CUB and SUN datasets.

UR - http://www.scopus.com/inward/record.url?scp=85100381396&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85100381396&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2021.01.037

DO - 10.1016/j.neucom.2021.01.037

M3 - Article

AN - SCOPUS:85100381396

SN - 1094-2025

VL - 437

SP - 1

EP - 8

JO - Review of Economic Dynamics

JF - Review of Economic Dynamics

ER -

Zero-shot learning with self-supervision by shuffling semantic embeddings

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this