Transfer learning is effective for improving the performance of tasks that are related, and Multi-task learning (MTL) and Cross-lingual learning (CLL) are important instances. This paper argues that hard-parameter sharing, of hard-coding layers shared across different tasks or languages, cannot generalize well, when sharing with a loosely related task. Such case, which we call sparse transfer, might actually hurt performance, a phenomenon known as negative transfer. Our contribution is using adversarial training across tasks, to “soft-code” shared and private spaces, to avoid the shared space gets too sparse. In CLL, our proposed architecture considers another challenge of dealing with low-quality input.
|Title of host publication||ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference|
|Publisher||Association for Computational Linguistics (ACL)|
|Number of pages||9|
|Publication status||Published - 2020|
|Event||57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 - Florence, Italy|
Duration: 2019 Jul 28 → 2019 Aug 2
|Name||ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference|
|Conference||57th Annual Meeting of the Association for Computational Linguistics, ACL 2019|
|Period||19/7/28 → 19/8/2|
Bibliographical noteFunding Information:
This work is supported by Microsoft Research Asia and IITP grant funded by the Korean government (MSIT, 2017-0-01779, XAI). Hwang is a corresponding author.
All Science Journal Classification (ASJC) codes
- Language and Linguistics
- Computer Science(all)
- Linguistics and Language