Скачать или смотреть CSCI 3151 - M48 - Sequence modelling & vanilla RNNs

This module introduces sequence modelling as a new setting where inputs and outputs are ordered in time and can be variable length. We compare fixed-size models you’ve seen earlier in the course (for tabular data and images) with tasks like language modelling, sentiment over a sentence, and forecasting in time series, where the order of observations matters just as much as their values. This motivates recurrent neural networks (RNNs) as a way to process one step at a time while carrying forward a hidden state that summarizes the past.

We then develop the vanilla RNN mathematically and visually: unrolling the recurrence over time, defining hidden states h_t, and distinguishing common setups such as many-to-one (sequence classification) and many-to-many (next-step prediction). Using small PyTorch examples on text or simple signals, you’ll see how RNNs are trained with backpropagation through time, how they relate to the gradient-flow ideas from earlier modules, and where they begin to struggle with long-range dependencies—setting the stage for gated architectures like LSTMs and GRUs in the next module.

Course module page:
https://web.cs.dal.ca/~rudzicz/Teaching/CS...