The NLP Lab at the University of Waterloo, led by Professor Olga Vechtomova, conducts research in Natural Language Processing (NLP) with a focus on creativity, generative models, and multimodal systems. Our work explores how AI can interact with and enhance creative processes, with applications ranging from text generation to music and art.
In this area of research, we explore the potential of AI to enhance artistic expression and create interactive, dynamic experiences. Our systems are designed to inspire and support human creativity, contributing to the exploration of new artistic processes.
LyricJam: This project is centered around a system that generates lyrics in real-time during live instrumental music performances. By analyzing the music, the system creates lyrics that complement the performance. LyricJam has received considerable media coverage and has been used in live settings, demonstrating the potential of AI in artistic expression.
LyricJam Sonic: Building on the original LyricJam, LyricJam Sonic focuses on generating evolving soundscapes that respond in real-time to live music. This project has been presented at international conferences, showing its capability to create interactive and dynamic artistic experiences.
We have worked on text style transfer, particularly in developing models that can separate and manipulate different aspects of text, such as syntax and meaning. This research is useful for applications like personalized content generation and creative writing.
Disentangled Representation Learning for Non-Parallel Text Style Transfer: We developed a method that effectively separates style and content in language models using a combination of multi-task and adversarial objectives. This approach enables high-performance style transfer on non-parallel text data, ensuring accurate style transfer, content preservation, and fluent language generation, outperforming several existing methods.
Stochastic Wasserstein Autoencoder: We developed a model that improves sentence generation by learning latent representations of text. This model supports the creation of diverse and high-quality text outputs.
Polarized-VAE: We developed a model that disentangles the semantics and syntax of sentences by using proximity measures to reflect the similarity between data points. This approach, called Polarized-VAE, supports the creation of diverse and high-quality text by learning distinct attributes in the latent space, outperforming traditional VAE models and offering a flexible framework for various disentanglement tasks.
Variational Attention Models: We’ve explored how Variational models can better focus on relevant parts of input data in sequence-to-sequence tasks, which is useful for applications like translation, summarization, and creative writing.
The lab has made contributions to text summarization and sentence simplification, developing models that can create concise summaries or simplify complex text without losing important information. These models are particularly valuable for educational and assistive tools.
Iterative Edit-Based Unsupervised Sentence Simplification: We introduced a novel approach to sentence simplification that iteratively edits sentences at the word and phrase level. Guided by a scoring function that balances fluency, simplicity, and meaning preservation, our model operates without the need for parallel training data. This method is more controllable and interpretable than previous approaches and achieves performance nearly on par with state-of-the-art supervised models, as demonstrated on the Newsela and WikiLarge datasets.
Unsupervised Sentence Summarization via Discrete Optimization We developed an unsupervised method for sentence summarization that balances language fluency and information retention using a custom objective function based on language modeling and semantic similarity metrics. By employing discrete optimization, our approach achieves new state-of-the-art results in unsupervised sentence summarization, as measured by ROUGE scores. Additionally, we highlight that the ROUGE F1 metric is sensitive to summary length, a factor often overlooked, and recommend that future evaluations should account for output length when comparing summarization systems.