We tackle the problem of self-training networks for NLU in low-resource environment - few labeled data and lots of unlabeled data. The effectiveness of self-training is a result of increasing the amount of training data while training. Yet it becomes less effective in lowresource settings due to unreliable labels predicted by the teacher model on unlabeled data. Rules of grammar, which describe the grammatical structure of data, have been used in NLU for better explainability. We propose to use rules of grammar in self-training as a more reliable pseudo-labeling mechanism, especially when there are few labeled data. We design an effective algorithm that constructs and expands rules of grammar without human involvement. Then we integrate the constructed rules as a pseudo-labeling mechanism into self-training. There are two possible scenarios regarding data distribution: it is unknown or known in prior to training. We empirically demonstrate that our approach substantially outperforms the state-of-the-art methods in three benchmark datasets for both scenarios.
|Title of host publication||Findings of the Association for Computational Linguistics, Findings of ACL|
|Subtitle of host publication||EMNLP 2021|
|Editors||Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-Tau Yih|
|Publisher||Association for Computational Linguistics (ACL)|
|Number of pages||6|
|Publication status||Published - 2021|
|Event||2021 Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021 - Punta Cana, Dominican Republic|
Duration: 2021 Nov 7 → 2021 Nov 11
|Name||Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021|
|Conference||2021 Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021|
|Period||21/11/7 → 21/11/11|
Bibliographical noteFunding Information:
This research was supported by the Hyundai Motor AIRS company, the NRF grant funded by the Korea government (MSIT) (NRF-2020R1A4A3079947), and the AI Graduate School Program (2020-0-01361). Han is a corresponding author.
© 2021 Association for Computational Linguistics.
All Science Journal Classification (ASJC) codes
- Language and Linguistics
- Linguistics and Language