Call for Papers

MetaNLP workshop invites submissions that investigate the theoretical and experimental nature of meta learning methodologies and their applications to NLP. Relevant research directions include, but not limited to:

• New meta-learning approaches
• Application of meta-learning models to NLP tasks
• Generalizability of meta-learned models across domains, tasks, or languages
• Understanding of why do meta-learning methods work for NLP, for example:
• What does the model learn in meta-learning tasks?
• Are there some meta-learning approaches that are suitable for some NLP applications but not others?
• Comparative study on meta-learning approaches.

Popular meta-learning topics include, but not limited to:

• Learning optimizer
• Learning model initialization
• Learning metrics or distance measurement
• Learning training algorithm
• Few shot learning
• Network architecture search

We welcome three categories of papers: regular workshop papers, cross-submissions, and extended abstracts. Only the regular workshop paper will be included in the proceedings. The extended Abstracts and cross-submissions will simply be hosted on our websites. Submissions portal will be announced later.

• Regular Workshop Papers The submissions should be in ACL 2021 style between 4 and 8 pages, excluding the references. Authors can add supplementary material in addition to the 8 pages, but reviewers are not required to review the extra material. The papers should present novel research. The review will be double blind and thus all submissions should be anonymized. Double submission is allowed, but the paper accepted by another conference should be moved to cross-submissions.
• Extended Abstracts Preliminary but interesting ideas that have not been published before can be submitted as extended abstracts. Ideas and works that would benefit from additional exposure and discussion but are not ready for publication are welcome to be submitted. The submissions should be up to 2 pages long including the references. The review will be double blind and thus all submissions should be anonymized.
• Cross-Submissions We also invite works on relevant topics that have appeared in or submitted to alternative venues. Accepted cross-submissions will be presented as posters, with an indication of the original venue. Selection of cross-submissions will be determined solely by the organizing committee.

Program

TBD

Reading

Meta learning is one of the fastest growing research areas in the deep learning scope. However there is no standard definition for meta learning. Usually the main goal is to design models that can learn new tasks rapidly with few in domain training examples, by having models to pre-learn from many, relevant or not, training tasks in a way that the models ar e easy to be generalized to new tasks. For better understanding the scope of meta learning, we provide several online courses and papers describing the works falling into the area. These works are just for showcasing, and we definitely encourage people with research not covered here but sharing the same goal mentioned above to submit.

Online Courses

• CS 330: Deep Multi-Task and Meta Learning
• Hung-Yi Lee’s Lecture (in Mandarin)

Papers

Meta Learning Technology

• Learning to Initialize:
  • Chelsea Finn, Pieter Abbeel, and Sergey Levine, “Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks”, ICML, 2017
  • Sebastian Flennerhag, Pablo G. Moreno, Neil D. Lawrence, Andreas Damianou, Transferring Knowledge across Learning Processes, ICLR, 2019
• Learning to optimize:
  • Sachin Ravi, Hugo Larochelle, Optimization as a model for few-shot learning, ICLR, 2017
  • Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, Nando de Freitas, Learning to learn by gradient descent by gradient descent, NIPS, 2016
• Learning to compare
  • Jake Snell, Kevin Swersky, Richard S. Zemel, Prototypical Networks for Few-shot Learning, NIPS, 2017
  • Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra, Matching Networks for One Shot Learning, NIPS, 2016
  • Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, Timothy M. Hospedales, Learning to Compare: Relation Network for Few-Shot Learning, CVPR, 2018
• Learning the whole learning algorithm
  • Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap, Meta-Learning with Memory-Augmented Neural Networks, ICML, 2016
  • Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel, A Simple Neural Attentive Meta-Learner, ICLR, 2018
• Network architecture search:
  • RL based
    • Barret Zoph, Quoc V. Le, Neural Architecture Search with Reinforcement Learning, ICLR 2017
    • Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le, Learning Transferable Architectures for Scalable Image Recognition, CVPR, 2018
    • Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, Jeff Dean, Efficient Neural Architecture Search via Parameter Sharing, ICML, 2018
  • Evolution based
    • Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc Le, Alex Kurakin, Large-Scale Evolution of Image Classifiers, ICML 2017
    • Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V Le, Regularized Evolution for Image Classifier Architecture Search, AAAI, 2019
    • Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu, Hierarchical Representations for Efficient Architecture Search, ICLR, 2018
  • Supernetwork based
    • Hanxiao Liu, Karen Simonyan, Yiming Yang, DARTS: Differentiable Architecture Search, ICLR, 2019

Applications to Natural language understanding:

• [Bansal, et al., arXiv’19] Trapit Bansal, Rishikesh Jha, Andrew McCallum, Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks, arXiv, 2019
• [Bose, et al., arXiv’19] Avishek Joey Bose, Ankit Jain, Piero Molino, William L. Hamilton, Meta-Graph: Few shot Link Prediction via Meta Learning, arXiv, 2019
• [Campagna, et al., ACL’20] Giovanni Campagna, Agata Foryciarz, Mehrad Moradshahi, Monica Lam, Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking, ACL 2020
• [Chen, et al., EMNLP’19] Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, Huajun Chen, Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs, EMNLP 2019
• [Chen, et al., INTERSPEECH'20] Yi-Chen Chen, Jui-Yang Hsu, Cheng-Kuang Lee, Hung-yi Lee, DARTS-ASR: Differentiable architecture search for multilingual speech recognition and adaptation, INTERSPEECH 2020
• [Chen, et al., ACL’20] Zhiyu Chen, Harini Eavani, Wenhu Chen, Yinyin Liu, William Yang Wang, Few-Shot NLG with Pre-Trained Language Model, ACL 2020
• [Chien, et al., INTERSPEECH’19] Jen-Tzung Chien, Wei Xiang Lieow, Meta Learning for Hyperparameter Optimization in Dialogue System, INTERSPEECH, 2019
• [Coope, et al., ACL’20] Samuel Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, Matthew Henderson, Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations, ACL 2020
• [Dou, et al., EMNLP’19] Zi-Yi Dou, Keyi Yu, Antonios Anastasopoulos, Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks, EMNLP 2019
• [Geng, et al., EMNLP’19] Ruiying Geng, Binhua Li, Yongbin Li, Xiaodan Zhu, Ping Jian, Jian Sun, Induction Networks for Few-Shot Text Classification, EMNLP, 2019
• [Geng, et al., ACL’20] Ruiying Geng, Binhua Li, Yongbin Li, Jian Sun, Xiaodan Zhu, Dynamic Memory Induction Networks for Few-Shot Text Classification, ACL 2020
• [Gu, et al., EMNLP’18] Jiatao Gu, Yong Wang, Yun Chen, Kyunghyun Cho, Victor O.K. Li, Meta-Learning for Low-Resource Neural Machine Translation, EMNLP, 2018
• [Guo, et al., ACL’19] Daya Guo, Duyu Tang, Nan Duan, Ming Zhou, Jian Yin, Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing, ACL, 2019
• [Hou, et al., ACL’20] Yutai Hou, Wanxiang Che, Yongkui Lai, Zhihan Zhou, Yijia Liu, Han Liu, Ting Liu, Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network, ACL 2020
• [Hu, et al., ACL’19] Ziniu Hu, Ting Chen, Kai-Wei Chang, Yizhou Sun, Few-Shot Representation Learning for Out-Of-Vocabulary Words, ACL 2019
• [Huang, et al., NAACL’18] Po-Sen Huang, Chenglong Wang, Rishabh Singh, Wen-tau Yih, Xiaodong He, Natural Language to Structured Query Generation via Meta-Learning, NAACL 2018
• [Huang, et al., ACL’20] Yi Huang, Junlan Feng, Min Hu, Xiaoting Wu, Xiaoyu Du, Shuo Ma, Meta-Reinforced Multi-Domain State Generator for Dialogue Systems, ACL 2020
• [Kim, et al., ACL’20] Hwichan Kim, Tosho Hirasawa, Mamoru Komachi, Zero-shot North Korean to English Neural Machine Translation by Character Tokenization and Phoneme Decomposition, ACL 2020
• [Luo, et al., INTERSPEECH'20] Hongyin Luo, Shang-Wen Li, James Glass, Prototypical q networks for automatic conversational diagnosis and few-shot new disease adaption, INTERSPEECH 2020
• [Lv, et al., EMNLP’19] Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations, EMNLP 2019
• [Madotto, et al., ACL’19] Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, Pascale Fung, Personalizing Dialogue Agents via Meta-Learning, ACL 2019
• [Mohtarami, et al., EMNLP’19] Mitra Mohtarami, James Glass, Preslav Nakov, Contrastive language adaptation for cross-lingual stance detection, EMNLP 2019
• [Mu, et al., ACL’20] Jesse Mu, Percy Liang, Noah Goodman, Shaping Visual Representations with Language for Few-shot Classification, ACL 2020
• [Obamuyide, et al., ACL’19] Abiola Obamuyide, Andreas Vlachos, Model-Agnostic Meta-Learning for Relation Classification with Limited Supervision, ACL 2019
• [Qian, et al., ACL’19] Kun Qian, Zhou Yu, Domain Adaptive Dialog Generation via Meta Learning, ACL 2019
• [Shah, et al., ACL’19] Darsh Shah, Raghav Gupta, Amir Fayazi, Dilek Hakkani-Tur, Robust zero-shot cross-domain slot filling with example values, ACL 2019
• [Song, et al., ACL’20] Yiping Song, Zequn Liu, Wei Bi, Rui Yan, Ming Zhang, Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks, ACL 2020
• [Sun, et al., EMNLP’18] Jingyuan Sun, Shaonan Wang, Chengqing Zong, Memory, Show the Way: Memory Based Few Shot Word Representation Learning, EMNLP 2018
• [Sun, et al., EMNLP’19] Shengli Sun, Qingfeng Sun, Kevin Zhou, Tengchao Lv, Hierarchical Attention Prototypical Networks for Few-Shot Text Classification, EMNLP 2019
• [Surís, et al., arXiv’19] Dídac Surís, Dave Epstein, Heng Ji, Shih-Fu Chang, Carl Vondrick, Learning to Learn Words from Narrated Video, arXiv, 2019
• [Tan, et al., EMNLP’19] Ming Tan, Yang Yu, Haoyu Wang, Dakuo Wang, Saloni Potdar, Shiyu Chang, Mo Yu, Out-of-Domain Detection for Low-Resource Text Classification Tasks, EMNLP 2019
• [Wang, et al., EMNLP’19] Zihao Wang, Kwun Ping Lai, Piji Li, Lidong Bing, Wai Lam, Tackling Long-Tailed Relations and Uncommon Entities in Knowledge Graph Completion, EMNLP 2019
• [Winata, et al., ACL’20] Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, Peng Xu, Pascale Fung, Meta-Transfer Learning for Code-Switched Speech Recognition, ACL 2020
• [Wu, et al., EMNLP’19] Jiawei Wu, Wenhan Xiong, William Yang Wang, Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification, EMNLP 2019
• [Wu, et al., AAAI’20] Qianhui Wu, Zijia Lin, Guoxin Wang, Hui Chen, Börje F. Karlsson, Biqing Huang, Chin-Yew Lin, Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources, AAAI 2020
• [Xiong, et al., EMNLP’18] Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, William Yang Wang, One-Shot Relational Learning for Knowledge Graphs, EMNLP 2018
• [Yan, et al., ACL’20] Guangfeng Yan, Lu Fan, Qimai Li, Han Liu, Xiaotong Zhang, Xiao-Ming Wu, Albert Y.S. Lam, Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification, ACL 2020
• [Yan, et al., ACL’20] Ming Yan, Hao Zhang, Di Jin, Joey Tianyi Zhou, Multi-source meta transfer for low resource multiple-choice question answering, ACL 2020
• [Ye, et al., ACL’20] Zhiquan Ye, Yuxia Geng, Jiaoyan Chen, Jingmin Chen, Xiaoxiao Xu, SuHang Zheng, Feng Wang, Jun Zhang, Huajun Chen, Zero-shot Text Classification via Reinforced Self-training, ACL 2020
• [Ye, et al., ACL’19] Zhi-Xiu Ye, Zhen-Hua Ling, Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification, ACL 2019
• [Yu, et al., ACL’20] Changlong Yu, Jialong Han, Haisong Zhang, Wilfred Ng, Hypernymy Detection for Low-Resource Languages via Meta Learning, ACL 2020
• [Yu, et al., ACL’18] Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, Bowen Zhou, Diverse Few-Shot Text Classification with Multiple Metrics, ACL 2018
• [Zhang, et al., ACL’20] Biao Zhang, Philip Williams, Ivan Titov, Rico Sennrich, Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation, ACL 2020
• [Zhao, et al. EMNLP’19] Zhenjie Zhao, Xiaojuan Ma, Text Emotion Distribution Learning from Small Sample: A Meta-Learning Approach, EMNLP 2019