Senior Capstone Experience by Michelle Ly ’21
Submitted to the Department of Computer Science
Advised by Dr. Kyle Wilson
Description: This paper presents an overview of XLNet’s state-of-the-art system in the field of Natural Language Processing and specifically text generation. I look at previous language models Transformer-XL and BERT in order to understand the foundations that XLNet is built upon. Context is vital for the advancement of Natural Language Processing, not only in terms of text generation but understanding how these machine learning systems can improve. Overall, XLNet generates text that sounds human-passable, and it’s also able to build around a new prompt based on its pretrained knowledge. This collaboration between machines and humans on improving text generation is what natural language processing focuses on. XLNet, while on the cutting edge of machine learning, is already being surpassed by bigger models. However, it still deserves recognition for what it does bring to the field of natural language processing.
Washington College Review 