publications

2025

1. Look Both Ways and No Sink: Converting LLMs into Text Encoders without Training

   Ziyong Lin, Haoyi Wu, Shu Wang, and 3 more authors

   In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025
2. Reflectevo: Improving meta introspection of small llms by learning self-reflection

   Jiaqi Li, Xinyi Dong, Yang Liu, and 6 more authors

   arXiv preprint arXiv:2505.16475, 2025
3. From hours to minutes: Lossless acceleration of ultra long sequence generation up to 100k tokens

   Tong Wu, Junzhe Shen, Zixia Jia, and 2 more authors

   arXiv preprint arXiv:2502.18890, 2025

2024

1. Varying sentence representations via condition-specified routers

   Ziyong Lin, Quansen Wang, Zixia Jia, and 1 more author

   In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024
2. LangSuit\textperiodcentered E: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments

   Zixia Jia, Mengmeng Wang, Baichen Tong, and 2 more authors

   In Findings of the Association for Computational Linguistics ACL 2024, 2024
3. Combining Supervised Learning and Reinforcement Learning for Multi-Label Classification Tasks with Partial Labels

   Zixia Jia, Junpeng Li, Shichuan Zhang, and 2 more authors

   In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024

2023

1. Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models

   Junpeng Li, Zixia Jia, and Zilong Zheng

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023
2. ACL2023

   Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field

   Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Jul 2023

   Abs HTML

   Prior works on joint Information Extraction (IE) typically model instance (e.g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding. We find that the previous models generally consider binary type dependency scoring of a pair of instances, and leverage local search such as beam search to approximate global solutions. To better integrate cross-instance interactions, in this work, we introduce a joint IE framework (CRFIE) that formulates joint IE as a high-order Conditional Random Field. Specifically, we design binary factors and ternary factors to directly model interactions between not only a pair of instances but also triplets. Then, these factors are utilized to jointly predict labels of all instances. To address the intractability problem of exact high-order inference, we incorporate a high-order neural decoder that is unfolded from a mean-field variational inference method, which achieves consistent learning and inference. The experimental results show that our approach achieves consistent improvements on three IE tasks compared with our baseline and prior work.

3. SemEval2023

   DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition

   Zeqi Tan*, Shen Huang*, Zixia Jia*, and 7 more authors

   In Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), Jul 2023

   Abs

   The MultiCoNER II shared task aims to tackle multilingual named entity recognition (NER) in fine-grained and noisy scenarios, and it inherits the semantic ambiguity and low-context setting of the MultiCoNER I task. To cope with these problems, the previous top systems in the MultiCoNER I either incorporate the knowledge bases or gazetteers. However, they still suffer from insufficient knowledge, limited context length, single retrieval strategy. In this paper, our team DAMO-NLP proposes a unified retrieval-augmented system (U-RaNER) for fine-grained multilingual NER. We perform error analysis on the previous top systems and reveal that their performance bottleneck lies in insufficient knowledge. Also, we discover that the limited context length causes the retrieval knowledge to be invisible to the model. To enhance the retrieval context, we incorporate the entity-centric Wikidata knowledge base, while utilizing the infusion approach to broaden the contextual scope of the model. Also, we explore various search strategies and refine the quality of retrieval knowledge. Our system wins 9 out of 13 tracks in the MultiCoNER II shared task. Additionally, we compared our system with ChatGPT, one of the large language models which have unlocked strong capabilities on many tasks. The results show that there is still much room for improvement for ChatGPT on the extraction task.

2022

1. AAAI2022

   Span-based semantic role labeling with argument pruning and second-order inference

   Zixia* Jia, Zhaohui* Yan, Haoyi Wu, and 1 more author

   Proceedings of the AAAI Conference on Artificial Intelligence, Jul 2022

   Abs HTML

   Prior works on joint Information Extraction (IE) typically model instance (e.g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding. We find that the previous models generally consider binary type dependency scoring of a pair of instances, and leverage local search such as beam search to approximate global solutions. To better integrate cross-instance interactions, in this work, we introduce a joint IE framework (CRFIE) that formulates joint IE as a high-order Conditional Random Field. Specifically, we design binary factors and ternary factors to directly model interactions between not only a pair of instances but also triplets. Then, these factors are utilized to jointly predict labels of all instances. To address the intractability problem of exact high-order inference, we incorporate a high-order neural decoder that is unfolded from a mean-field variational inference method, which achieves consistent learning and inference. The experimental results show that our approach achieves consistent improvements on three IE tasks compared with our baseline and prior work.

2. NAACL2022

   SHARP: Search-Based Adversarial Attack for Structured Prediction

   Liwen Zhang*, Zixia Jia*, Wenjuan Han*, and 2 more authors

   In Findings of the Association for Computational Linguistics: NAACL 2022, Jul 2022
3. NAACL2022

   ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition

   Xinyu Wang, Min Gui, Yong Jiang, and 5 more authors

   In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jul 2022
4. AACL2022

   An Empirical Study of Pipeline vs. Joint Approaches to Entity and Relation Extraction

   Zhaohui Yan*, Zixia Jia*, and Kewei Tu

   In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, Jul 2022

2021

1. ACL2021

   Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor

   Xinyu Wang*, Yong Jiang*, Zhaohui Yan*, and 6 more authors

   Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, Jul 2021

   Abs HTML

   Knowledge distillation is a critical technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). The objective function of knowledge distillation is typically the cross-entropy between the teacher and the student’s output distributions. However, for structured prediction problems, the output space is exponential in size; therefore, the cross-entropy objective becomes intractable to compute and optimize directly. In this paper, we derive a factorized form of the knowledge distillation objective for structured prediction, which is tractable for many typical choices of the teacher and student models. In particular, we show the tractability and empirical effectiveness of structural knowledge distillation between sequence labeling and dependency parsing models under four different scenarios: 1) the teacher and student share the same factorization form of the output structure scoring function; 2) the student factorization produces more fine-grained substructures than the teacher factorization; 3) the teacher factorization produces more fine-grained substructures than the student factorization; 4) the factorization forms from the teacher and the student are incompatible.

2. IWPT2021

   Enhanced Universal Dependency Parsing with Automated Concatenation of Embeddings

   Xinyu Wang*, Zixia Jia*, Yong Jiang, and 1 more author

   In Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021), Jul 2021

2020

1. ACL2020

   Semi-supervised semantic dependency parsing using CRF autoencoders

   Zixia Jia, Youmi Ma, Jiong Cai, and 1 more author

   Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020

   Abs HTML

   Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations. We propose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework. Our encoder is a discriminative neural semantic dependency parser that predicts the latent parse graph of the input sentence. Our decoder is a generative neural model that reconstructs the input sentence conditioned on the latent parse graph. Our model is arc-factored and therefore parsing and learning are both tractable. Experiments show our model achieves significant and consistent improvement over the supervised baseline.

2019

1. CoNLL2019

   ShanghaiTech at MRP 2019: Sequence-to-Graph Transduction with Second-Order Edge Inference for Cross-Framework Meaning Representation Parsing

   Xinyu Wang, Yixian Liu, Zixia Jia, and 2 more authors

   In Proceedings of the Shared Task on Cross-Framework Meaning Representation Parsing at the 2019 Conference on Natural Language Learning, Jul 2019