Chen et al. (2018) proposed ADAN, an architecture based on a feed-forward neural network with three main components, namely: a feature extractor, a sentiment classifier, and a language discriminator. The latter had the purpose of supporting the adversarial training setup, thus covering the scenario where the model was unable to detect whether the input language was from the source dataset or the target one. A similar cross-lingual approach was adopted by Zhang et al. (2020), who developed a system to classify entries from the target language, while only labels from the source language were provided.
Keung et al. (2019) employed the usage of multilingual BERT (Pires et al., 2019) and argued that a language-adversarial task can improve the performance of zero-resource cross-lingual transfers. Moreover, training under an adversarial technique helps the Transformer model align the representations of the English inputs.
Under a Named Entity Recognition training scenario, Kim et al. (2017) used features on two levels (i.e., word and characters), together with Recurrent Neural Networks and a language discriminator used for the domain-adversarial setup. Similarly, Huang et al. (2019) used target language discriminators during the process of training models for low-resource name tagging.
Reference
- Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, and Kilian Weinberger. 2018. Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics, 6:557–570.
- Dejiao Zhang, Ramesh Nallapati, Henghui Zhu, Feng Nan, Cicero dos Santos, Kathleen McKeown, and Bing Xiang. 2020. Unsupervised domain adaptation for cross-lingual text labeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 3527–3536.
- Phillip Keung, Vikas Bhardwaj, et al. 2019. Adversarial learning with contextual embeddings for zeroresource cross-lingual classification and ner. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1355–1360.
- Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual bert? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001.
- Joo-Kyung Kim, Young-Bum Kim, Ruhi Sarikaya, and Eric Fosler-Lussier. 2017. Cross-lingual transfer learning for pos tagging without cross-lingual resources. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 2832–2838.
- Lifu Huang, Heng Ji, and Jonathan May. 2019. Crosslingual multi-level adversarial transfer to enhance low-resource name tagging. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3823–3833.