About Me

Prospective students: I am always looking for strong and motivated Postdoc/PhD/Master/RA students to join my group. Please drop me an email with your CV if you are interested. 详见招生信息

Dr. Jing Li is currently a Professor at Harbin Institute of Technology (Shenzhen) . Prior to that, he was a Research Scientist at the Inception Institute of Artificial Intelligence (IIAI), Abu Dhabi, United Arab Emirates. He was a Research Fellow in the School of Computer Science and Engineering, Nanyang Technological University (NTU), Singapore. He obtained his PhD degree in Computer Science, at NTU, Singapore in 2018. He received his B.E. and M.E. both from University of Electronic Science and Technology of China (UESTC), China. His research aims to build up semantic Web systems to support information needs of web users via deep text understanding, information extraction, machine intelligent question answering, knowledge representation, as well as social media analysis.

Research Interests

Natural Language Processing / Information Retrieval

Large Langauge Models and Generative AI (PEFT, safety, etc.)
Information Extraction and Knowledge Acquisition (NER, QA, etc.)
Knowledgeable NLP (Knowledge Graph, Reasoning, etc.)
Trustworthy NLP (Robust/adversarial, Low-resource, etc.)
AI for Software Engineering (Documentation Mining, Programming, etc.)

Machine Learning

Transfer Learning
Meta-learning
Adversarial Learning

Selected Publications [Full List] [Google Scholar]

Books

Authors: Xiaoyan Zhu, Jing Li, Yu Hao, Han Xiao and Minlie Huang
Publisher: Publishing House of Electronics Industry
ISBN: 9787121389924
Publish time: June 1st, 2020
Links: JD.com, Tmall.com

Rethinking Document-Level Relation Extraction: A Reality Check
Jing Li, Yequan Wang, Shuai Zhang and Min Zhang
ACL-23- Findings of The 61st Annual Meeting of the Association for Computational Linguistics, 2023.
PDF Abstract BibTex

Recently, numerous efforts have continued to push up performance boundaries of document-level relation extraction (DocRE) and have claimed significant progress in DocRE. In this paper, we do not aim at proposing a novel model for DocRE. Instead, we take a closer look at the field to see if these performance gains are actually true. By taking a comprehensive literature review and a thorough examination of popular DocRE datasets, we find that these performance gains are achieved upon a strong or even untenable assumption in common: all named entities are perfectly localized, normalized, and typed in advance. Next, we construct four types of entity mention attacks to examine the robustness of typical DocRE models by behavioral probing. We also have a close check on model usability in a more realistic setting. Our findings reveal that most of current DocRE models are vulnerable to entity mention attacks and difficult to be deployed in real-world end-user NLP applications. Our study calls more attentions for future research to stop simplifying problem setups, and to model DocRE in the wild rather than in an unrealistic Utopian world.

@inproceedings{li2023rethinking, title={Rethinking Document-Level Relation Extraction: A Reality Check}, author={Li, Jing and Wang, Yequan and Zhang, Shuai and Zhang, Min}, booktitle = {Findings of The 61st Annual Meeting of the Association for Computational Linguistics (ACL)}, year={2023} }

Few-Shot Relation Extraction With Dual Graph Neural Network Interaction
Jing Li, Shanshan Feng and Billy Chiu
IEEE TNNLS-23- IEEE Transactions on Neural Networks and Learning Systems, 2023.
PDF Abstract BibTex

Recent advances in relation extraction with deep neural architectures have achieved excellent performance. However, current models still suffer from two main drawbacks: 1) they require enormous volumes of training data to avoid model overfitting and 2) there is a sharp decrease in performance when the data distribution during training and testing shift from one domain to the other. It is thus vital to reduce the data requirement in training and explicitly model the distribution difference when transferring knowledge from one domain to another. In this work, we concentrate on few-shot relation extraction under domain adaptation settings. Specifically, we propose, a novel graph neural network (GNN) based approach for few-shot relation extraction. leverages an edge-labeling dual graph (i.e. an instance graph and a distribution graph) to explicitly model the intraclass similarity and interclass dissimilarity in each individual graph, as well as the instance-level and distribution-level relations across graphs. A dual graph interaction mechanism is proposed to adequately fuse the information between the two graphs in a cyclic flow manner. We extensively evaluate on FewRel1.0 and FewRel2.0 benchmarks under four few-shot configurations. The experimental results demonstrate that can match or outperform previously published approaches. We also perform experiments to further investigate the parameter settings and architectural choices, and we offer a qualitative analysis.

@article{li2023few, title={Few-Shot Relation Extraction With Dual Graph Neural Network Interaction}, author={Li, Jing and Feng, Shanshan and Chiu, Billy}, journal={IEEE Transactions on Neural Networks and Learning Systems}, year={2023}, publisher={IEEE} }

A Survey on Deep Learning for Named Entity Recognition
Jing Li, Aixin Sun, Jianglei Han and Chenliang Li
IEEE TKDE-22- IEEE Transactions on Knowledge and Data Engineering, 34(1): 50-70, 2022.
PDF Abstract BibTex

Named entity recognition (NER) is the task to identify text spans that mention named entities, and to classify them into predefined categories such as person, location, organization etc. NER serves as the basis for a variety of natural language applications such as question answering, text summarization, and machine translation. Although early NER systems are successful in producing decent recognition accuracy, they often require much human effort in carefully designing rules or features. In recent years, deep learning, empowered by continuous real-valued vector representations and semantic composition through nonlinear processing, has been employed in NER systems, yielding state-of-the-art performance. In this paper, we provide a comprehensive review on existing deep learning techniques for NER. We first introduce NER resources, including tagged NER corpora and off-the-shelf NER tools. Then, we systematically categorize existing works based on a taxonomy along three axes: distributed representations for input, context encoder, and tag decoder. Next, we survey the most representative methods for recent applied techniques of deep learning in new NER problem settings and applications. Finally, we present readers with the challenges faced by NER systems and outline future directions in this area.

@article{jing22nersurvey, author = {Jing Li and Aixin Sun and Jianglei Han and Chenliang Li}, title = {A Survey on Deep Learning for Named Entity Recognition}, journal = {IEEE Transactions on Knowledge and Data Engineering (TKDE)}, volume = {34}, number = {1}, pages = {50--70}, year = {2022}, url = {https://doi.org/10.1109/TKDE.2020.2981314}, doi = {10.1109/TKDE.2020.2981314}, }

Neural Text Segmentation and Its Application to Sentiment Analysis
Jing Li, Billy Chiu, Shuo Shang and Ling Shao
IEEE TKDE-22- IEEE Transactions on Knowledge and Data Engineering, 34(2): 828-842, 2022.
PDF Abstract BibTex Demo

Text segmentation is a fundamental task in natural language processing. Depending on the levels of granularity, the task can be defined as segmenting a document into topical segments, or segmenting a sentence into elementary discourse units (EDUs). Traditional solutions to the two tasks heavily rely on carefully designed features. The recently proposed neural models do not need manual feature engineering, but they either suffer from sparse boundary tags or cannot efficiently handle the issue of variable size output vocabulary. In light of such limitations, we propose a generic end-to-end segmentation model, namely SEGBOT, which first uses a bidirectional recurrent neural network to encode an input text sequence. SEGBOT then uses another recurrent neural networks, together with a pointer network, to select text boundaries in the input sequence. In this way, SEGBOT does not require any hand-crafted features. More importantly, SEGBOT inherently handles the issue of variable size output vocabulary and the issue of sparse boundary tags. In our experiments, SEGBOT outperforms state-of-the-art models on two tasks: document-level topic segmentation and sentence-level EDU segmentation. As a downstream application, we further propose a hierarchical attention model for sentence-level sentiment analysis based on the outcomes of SEGBOT. The hierarchical model can make full use of both word-level and EDU-level information simultaneously for sentence-level sentiment analysis. In particular, it can effectively exploit EDU-level information, such as the inner properties of EDUs, which cannot be fully encoded in word-level features. Experimental results show that our hierarchical model achieves new state-of-the-art results on the Movie Review and Stanford Sentiment Treebank benchmarks.

@article{li22segsenti, author = {Jing Li and Billy Chiu and Shuo Shang and Ling Shao}, title = {Neural Text Segmentation and Its Application to Sentiment Analysis}, journal = {IEEE Transactions on Knowledge and Data Engineering (TKDE)}, volume = {34}, number = {2}, pages = {828--842}, year = {2022}, url = {https://doi.org/10.1109/TKDE.2020.2983360}, doi = {10.1109/TKDE.2020.2983360}, }

Neural Named Entity Boundary Detection
Jing Li, Aixin Sun and Yukun Ma
IEEE TKDE-22- IEEE Transactions on Knowledge and Data Engineering, 33(4): 1790-1795, 2021.
PDF Abstract BibTex Demo

In this paper, we focus on named entity boundary detection , which is to detect the start and end boundaries of an entity mention in text, without predicting its type. The detected entities are input to entity linking or fine-grained typing systems for semantic enrichment. We propose BdryBot , a recurrent neural network encoder-decoder framework with a pointer network to detect entity boundaries from a given sentence. The encoder considers both character-level representations and word-level embeddings to represent the input words. In this way, BdryBot does not require any hand-crafted features. Because of the pointer network, BdryBot overcomes the problem of variable size output vocabulary and the issue of sparse boundary tags. We conduct two sets of experiments, in-domain detection and cross-domain detection, on six datasets. Our results show that BdryBot achieves state-of-the-art performance against five baselines. In addition, our proposed approach can be further enhanced when incorporating contextualized language embeddings into token representations.

@article{li21bdrybot, author = {Jing Li and Aixin Sun and Yukun Ma}, title = {Neural Named Entity Boundary Detection}, journal = {IEEE Transactions on Knowledge and Data Engineering (TKDE)}, volume = {33}, number = {4}, pages = {1790--1795}, year = {2021}, url = {https://doi.org/10.1109/TKDE.2020.2981329}, doi = {10.1109/TKDE.2020.2981329}, }

Sequence Labeling with Meta-Learning
Jing Li, Peng Han, Xiangnan Ren, Jilin Hu, Lisi Chen and Shuo Shang
IEEE TKDE-21- IEEE Transactions on Knowledge and Data Engineering, 2021, in Press.
PDF Abstract BibTex

Recent neural architectures in sequence labeling have yielded state-of-the-art performance on single domain data such as newswires. However, they still suffer from (i) requiring massive amounts of training data to avoid overfitting; (ii) huge performance degradation when there is a domain shift in the data distribution between training and testing. In this paper, we investigate the problem of domain adaptation for sequence labeling under homogeneous and heterogeneous settings. We propose MetaSeq, a novel meta-learning approach for domain adaptation in sequence labeling. Specifically, MetaSeq incorporates meta-learning and adversarial training strategies to encourage robust, general and transferable representations for sequence labeling. The key advantage of MetaSeq is that it is capable of adapting to new unseen domains with a small amount of annotated data from those domains. We extensively evaluate MetaSeq on named entity recognition, part-of-speech tagging and slot filling tasks under homogeneous and heterogeneous settings. The experimental results show that MetaSeq achieves state-of-the-art performance against eight baselines. Impressively, MetaSeq surpasses the in-domain performance using only 16.17% and 7% of target domain data on average for homogeneous settings, and 34.76%, 24%, 22.5% of target domain data on average for heterogeneous settings.

@article{jing22nersurvey, author = {Jing Li and Peng Han and Xiangnan Ren and Jilin Hu and Lisi Chen and Shuo Shang}, title = {Sequence Labeling with Meta-Learning}, journal = {IEEE Transactions on Knowledge and Data Engineering (TKDE)}, volume = {34}, number = {1}, pages = {50--70}, year = {2021}, url = {https://doi.org/10.1109/TKDE.2021.3118469}, doi = {10.1109/TKDE.2021.3118469}, }

Domain Generalization for Named Entity Boundary Detection via Meta-Learning
Jing Li, Shuo Shang and Lisi Chen
IEEE TNNLS-21- IEEE Transactions on Neural Networks and Learning Systems, 32(9): 3819-3830, 2021.
PDF Abstract BibTex

Named entity recognition (NER) aims to recognize mentions of rigid designators from text belonging to predefined semantic types, such as person, location, and organization. In this article, we focus on a fundamental subtask of NER, named entity boundary detection, which aims at detecting the start and end boundaries of an entity mention in the text, without predicting its semantic type. The entity boundary detection is essentially a sequence labeling problem. Existing sequence labeling methods either suffer from sparse boundary tags (i.e., entities are rare and nonentities are common) or they cannot well handle the issue of variable size output vocabulary (i.e., need to retrain models with respect to different vocabularies). To address these two issues, we propose a novel entity boundary labeling model that leverages pointer networks to effectively infer boundaries depending on the input sequence. On the other hand, training models on source domains that generalize to new target domains at the test time are a challenging problem because of the performance degradation. To alleviate this issue, we propose METABDRY, a novel domain generalization approach for entity boundary detection without requiring any access to target domain information. Especially, adversarial learning is adopted to encourage domain-invariant representations. Meanwhile, metalearning is used to explicitly simulate a domain shift during training so that metaknowledge from multiple resource domains can be effectively aggregated. As such, METABDRY explicitly optimizes the capability of ``learning to generalize,'' resulting in a more general and robust model to reduce the domain discrepancy. We first conduct experiments to demonstrate the effectiveness of our novel boundary labeling model. We then extensively evaluate METABDRY on eight data sets under domain generalization settings. The experimental results show that METABDRY achieves state-of-the-art results against the recent seven baselines.

@article{li21domaingen, author = {Jing Li and Shuo Shang and Lisi Chen}, title = {Domain Generalization for Named Entity Boundary Detection via Metalearning}, journal = {IEEE Transactions on Neural Networks and Learning Systems (TNNLS)}, volume = {32}, number = {9}, pages = {3819--3830}, year = {2021}, url = {https://doi.org/10.1109/TNNLS.2020.3015912}, doi = {10.1109/TNNLS.2020.3015912}, }

Few-Shot Named Entity Recognition via Meta-Learning
Jing Li, Billy Chiu, Shanshan Feng and Hao Wang
IEEE TKDE-20- IEEE Transactions on Knowledge and Data Engineering, 2020, in Press.
PDF Abstract BibTex

Few-shot learning under the N-way K-shot setting (i.e., K annotated samples for each of N classes) has been widely studied in relation extraction (e.g., FewRel) and image classification (e.g., Mini-ImageNet). Named entity recognition (NER) is typically framed as a sequence labeling problem where the entity classes are inherently entangled together because the entity number and classes in a sentence are not known in advance, leaving the N-way K-shot NER problem so far unexplored. In this paper, we first formally define a more suitable N-way K-shot setting for NER. Then we propose FewNER, a novel meta-learning approach for few-shot NER. FewNER separates the entire network into a task-independent part and a task-specific part. During training in FewNER, the task-independent part is meta-learned across multiple tasks and a task-specific part is learned for each single task in a low-dimensional space. At test time, FewNER keeps the task-independent part fixed and adapts to a new task via gradient descent by updating only the task-specific part, resulting in it being less prone to overfitting and more computationally efficient. The results demonstrate that FewNER achieves state-of-the-art performance against nine baseline methods by significant margins on three adaptation experiments.

@article{li20fewshot, author = {Jing Li and Billy Chiu and Shanshan Feng and Hao Wang}, title = {Few-Shot Named Entity Recognition via Meta-Learning}, journal = {IEEE Transactions on Knowledge and Data Engineering (TKDE)}, volume = {Early Access}, year = {2021}, url = {https://doi.org/10.1109/TKDE.2020.3038670}, doi = {10.1109/TKDE.2020.3038670}, }

MetaNER: Named Entity Recognition with Meta-Learning
Jing Li, Shuo Shang and Ling Shao
WWW-20- The Web Conference, 2020. Acceptance rate: 217/1129 (19.2%).
PDF Abstract BibTex

Recent neural architectures in named entity recognition (NER) have yielded state-of-the-art performance on single domain data such as newswires. However, they still suffer from (i) requiring massive amounts of training data to avoid overfitting; (ii) huge performance degradation when there is a domain shift in the data distribution between training and testing. In this paper, we investigate the problem of domain adaptation for NER under homogeneous and heterogeneous settings. We propose MetaNER, a novel meta-learning approach for domain adaptation in NER. Specifically, MetaNER incorporates meta-learning and adversarial training strategies to encourage robust, general and transferable representations for sequence labeling. The key advantage of MetaNER is that it is capable of adapting to new unseen domains with a small amount of annotated data from those domains. We extensively evaluate MetaNER on multiple datasets under homogeneous and heterogeneous settings. The experimental results show that MetaNER achieves state-of-the-art performance against eight baselines. Impressively, MetaNER surpasses the in-domain performance using only 16.17% and 34.76% of target domain data on average for homogeneous and heterogeneous settings, respectively.

@inproceedings{li20metaner, author = {Jing Li and Shuo Shang and Ling Shao}, title = {MetaNER: Named Entity Recognition with Meta-Learning}, booktitle = {The Web Conference 2020 (WWW)}, pages = {429--440}, year = {2020}, url = {https://doi.org/10.1145/3366423.3380127}, }

Adversarial Transfer for Named Entity Boundary Detection with Pointer Networks
Jing Li, Deheng Ye and Shuo Shang
IJCAI-19- The 28th International Joint Conference on Artificial Intelligence, Pages 5053-5069, 2019. Acceptance rate: 850/4752 (17.9%).
PDF Abstract BibTex

In this paper, we focus on named entity boundary detection, which aims to detect the start and end boundaries of an entity mention in text, without predicting its type. A more accurate and robust detection approach is desired to alleviate error propagation in downstream applications, such as entity linking and fine-grained typing systems. Here, we first develop a novel entity boundary labeling approach with pointer networks, where the output dictionary size depends on the input, which is variable. Furthermore, we propose AT-Bdry, which incorporates adversarial transfer learning into an end-to-end sequence labeling model to encourage domain-invariant representations. More importantly, AT-Bdry can reduce domain difference in data distributions between the source and target domains, via an unsupervised transfer learning approach (i.e., no annotated target-domain data is necessary). We conduct Formal Text to Formal Text, Formal Text to Informal Text and ablation evaluations on five benchmark datasets. Experimental results show that AT-Bdry achieves state-of-the-art transferring performance against recent baselines.

@inproceedings{li19advt, author = {Jing Li and Deheng Ye andd Shuo Shang}, title = {Adversarial Transfer for Named Entity Boundary Detection with Pointer Networks}, booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI)}, pages = {5053--5059}, year = {2019}, url = {https://doi.org/10.24963/ijcai.2019/702}, }

SegBot: A Generic Neural Text Segmentation Model with Pointer Network
Jing Li, Aixin Sun and Shafiq Joty
IJCAI-18-The 27th International Joint Conference on Artificial Intelligence and the 23rd European Conference on Artificial Intelligence. Pages 4166-4172, 2018. Acceptance rate: 710/3470 (20.5%).
PDF Abstract BibTex Demo

Text segmentation is a fundamental task in natural language processing that comes in two levels of granularity: (i) segmenting a document into a sequence of topical segments (topic segmentation), and (ii) segmenting a sentence into a sequence of elementary discourse units (EDU segmentation). Traditional solutions to the two tasks heavily rely on carefully designed features. The recently proposed neural models do not need manual feature engineering, but they either suffer from sparse boundary tags or they cannot well handle the issue of variable size output vocabulary. We propose a generic end-to-end segmentation model called SegBot. SegBot uses a bidirectional recurrent neural network to encode input text sequence. The model then uses another recurrent neural network together with a pointer network to select text boundaries in the input sequence. In this way, SegBot does not require hand-crafted features. More importantly, our model inherently handles the issue of variable size output vocabulary and the issue of sparse boundary tags. In our experiments, SegBot outperforms state-of-the-art models on both topic and EDU segmentation tasks.

@inproceedings{LiSJ18segbot, author = {Jing Li and Aixin Sun and Shafiq R. Joty}, title = {SegBot: {A} Generic Neural Text Segmentation Model with Pointer Network}, booktitle = {Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI)}, pages = {4166--4172}, year = {2018}, url = {https://doi.org/10.24963/ijcai.2018/579}, doi = {10.24963/ijcai.2018/579}, }

Learning to Answer Programming Questions with Software Documentation through Social Context Embedding
Jing Li, Aixin Sun and Zhenchang Xing
INS-18- Information Sciences. Volumes 448–449, Pages 36-52, June 2018, Elsevier.
PDF Abstract BibTex

Official software documentation provides a comprehensive overview of software usages, but not on specific programming tasks or use cases. Often there is a mismatch between the documentation and a question on a specific programming task because of different wordings. We observe from Stack Overflow that the best answers to programmers’ questions often contain links to formal documentation. In this paper, we propose a novel deep-learning-to-answer framework, named QDLinker, for answering programming questions with software documentation. QDLinker learns from the large volume of discussions in community-based question answering site to bridge the semantic gap between programmers’ questions and software documentation. Specifically, QDLinker learns question-documentation semantic representation from these question answering discussions with a four-layer neural network, and incorporates semantic and content features into a learning-to-rank schema. Our approach does not require manual feature engineering or external resources to infer the degree of relevance between a question and documentation. Through extensive experiments, results show that QDLinker effectively answers programming questions with direct links to software documentation. QDLinker significantly outperforms the baselines based on traditional retrieval models and Web search services dedicated for software documentation retrieval. The user study shows that QDLinker effectively bridges the semantic gap between the intent of a programming question and the content of software documentation.

@article{L2ALiSX18, author = {Jing Li and Aixin Sun and Zhenchang Xing}, title = {Learning to answer programming questions with software documentation through social context embedding}, journal = {Information Sciences}, volume = {448-449}, pages = {36--52}, year = {2018}, url = {https://doi.org/10.1016/j.ins.2018.03.014}, doi = {10.1016/j.ins.2018.03.014}, }