UE Neural Language Processing

CONTENU

• Machine Learning and Deep Learning Refresher

  This session reviews the foundations needed for neural language processing, including learning paradigms, evaluation, optimisation, generalisation and neural-network training. It also introduces the essential stages of data preparation: exploratory data analysis, missing values, outliers, feature scaling, class imbalance and information leakage. Students will learn how to build reliable, reproducible pipelines that connect raw data to trainable neural models.

• Exploratory Data Analysis

  This practical session introduces exploratory data analysis and preprocessing for machine-learning workflows. You will inspect data structure and distributions, identify missing values and outliers, examine class imbalance, and apply appropriate cleaning and scaling strategies. The exercises illustrate how data quality and preprocessing choices affect model training, evaluation, and generalisation.

• Text Preprocessing

  In this practical session, you will explore the main techniques used to preprocess textual data for NLP. Through hands-on exercises on English movie synopses, you will compare different tokenisation, stemming, lemmatisation, and stop-word removal approaches and analyse how they transform raw text before modelling.

• Text Encoding and Representation

  This course explores how textual data can be converted into numerical representations for neural language processing. It covers sparse document representations such as bag-of-words, n-grams, and TF–IDF, as well as distributional approaches based on co-occurrence matrices, SVD, and Word2Vec. The course concludes with trainable embeddings, contextual representations, and pretraining, highlighting the strengths and limitations of each approach.

• Text Data Encoding and Embeddings

  In this practical session, you will explore different approaches to representing text numerically for NLP. Starting from a complete preprocessing pipeline, you will experiment with binary, count, and TF–IDF representations, measure document similarity, apply SVD for dimensionality reduction, and train Word2Vec embeddings to investigate how words and documents can be represented in vector spaces.

• LSTM, GRU & RNN

  This lecture introduces recurrent neural architectures for modelling sequential data and language. We will examine how RNNs process sequences, why they struggle with long-term dependencies, and how LSTMs and GRUs use gating mechanisms to improve memory. The lecture also covers bidirectional and stacked architectures, sequence preparation, encoder–decoder models, and the transition from recurrence to attention and Transformers.

• Text Classification with RNNs, LSTMs and GRUs

  In this practical session, you will explore recurrent neural networks for text classification using the 20 Newsgroups dataset. You will first establish a TF–IDF baseline, then implement and compare SimpleRNN, bidirectional LSTM, and GRU architectures. Finally, you will evaluate their performance and analyse the strengths and limitations of each recurrent approach.

• Transformers with Hugging Face

  In this two-hour practical session, students will use the Hugging Face ecosystem to apply and fine-tune Transformer models for sentiment classification. The activities cover pipelines, tokenization, direct model inference, IMDb dataset preparation, fine-tuning with the Trainer API, evaluation and error analysis. Students will progressively complete code exercises and examine the strengths and limitations of their model.

• Text classification with Transformers

  In this two-hour lab, students will use Hugging Face Transformers to classify French news articles. They will explore tokenization and attention masks, compare frozen Transformer embeddings with LoRA fine-tuning, and evaluate the models using accuracy, macro-F1, a confusion matrix, and error analysis.

• LLMs
  Aucune description
• LLMs with Ollama
  Aucune description
• Individual Project Work Session

  This session is dedicated to the development of your individual NLP project. You will work independently on your implementation, experiments, and evaluation, with opportunities to ask questions and receive guidance. The project will form part of the final course assessment.

• Individual Project Work Session -2
  Aucune description