XU-DISSERTATION-2020.pdf (3.54 MB)

Lifelong Representation Learning for NLP Applications

Download (3.54 MB)
thesis
posted on 01.05.2020, 00:00 by Hu Xu
Representation learning lives at the heart of deep learning for natural language processing (NLP). Traditional representation learning (such as softmax-based classification, pre-trained word embeddings, and language models, graph representations) focuses on learning general or static representations with the hope to help any end task. As the world keeps evolving, emerging knowledge (such as new tasks, domains, entities or relations) typically come with a small amount of data with shifted distributions that challenge the existing representations to be effective. As a result, how to effectively learn representations for new knowledge becomes crucial. Lifelong learning is a machine learning paradigm that aims to build an AI agent that keeps learning from the evolving world, like humans' learning from the world. This dissertation focuses on improving representations on different types of new knowledge (classification, word-level, contextual-level, and knowledge graph) for a myriad of NLP end tasks, ranging from text classification, sentiment analysis, entity recognition, question answering to the more complex dialog system. With the help of lifelong representation learning, models' performance on tasks is greatly improved beyond existing general representation learning.

History

Advisor

Yu, Philip S.Liu, Bing

Chair

Yu, Philip S.

Department

Computer Science

Degree Grantor

University of Illinois at Chicago

Degree Level

Doctoral

Degree name

PhD, Doctor of Philosophy

Committee Member

Gmytrasiewicz, Piotr Parde, Natalie Xie, Sihong

Submitted date

May 2020

Thesis type

application/pdf

Language

en

Usage metrics

Categories

Exports