Modul CS4575-KP04

Sequence Learning (SEQL)

Dauer:

1 Semester

Angebotsturnus:

Jedes Sommersemester

Leistungspunkte:

Studiengang, Fachgebiet und Fachsemester:

Master Medizinische Informatik 2019 (Wahlpflicht), Medical Data Science / Künstliche Intelligenz, 1. oder 2. Fachsemester
Master Psychologie 2016 (Wahlpflicht), Wahlpflicht, Beliebiges Fachsemester
Master Biophysik 2023 (Wahlpflicht), Wahlpflicht, Beliebiges Fachsemester
Master Medieninformatik 2020 (Wahlpflicht), Wahlpflicht, Beliebiges Fachsemester
Master Medizinische Ingenieurwissenschaft 2020 (Wahlpflicht), Wahlpflicht, Beliebiges Fachsemester
Master Entrepreneurship in digitalen Technologien 2020 (Wahlpflicht), Wahlpflicht, Beliebiges Fachsemester
Master Informatik 2019 (Wahlpflicht), Wahlpflicht, Beliebiges Fachsemester

Lehrveranstaltungen:

CS4575-V: Sequence Learning (Vorlesung, 2 SWS)
CS4575-Ü: Sequence Learning (Übung, 1 SWS)

Workload:

75 Stunden Selbststudium
45 Stunden Präsenzstudium

Lehrinhalte:

Introduction to Sequence Learning (Formalisms, Metrics, Recapitulation of Relevant Machine Learning Techniques)
Recurrent Neural Networks (Simple RNN Models, Backpropagation Through Time)
Gated Recurrent Networks (Vanishing Gradient Problem in RNNs, Long Short-Term Memories, Gated Recurrent Units, Stacked RNNs)
Important Techniques for RNNs (Teacher Forcing, Scheduled Sampling, h-Detach)
Bidirectional RNNs and related concepts
Hierarchical RNNs and Learning on Multiple Time Scales
Online Learning and Learning without BPTT (Real-Time Recurrent Learning, e-Prop, Forward Propagation Through Time)
Reservoir Computing (Echo State Networks, Deep ESNs)
Spiking Neural Networks (Spiking Neuron Models, Learning in SNNs, Neuromorphic Computing, Recurrent SNNs)
Temporal Convolution Networks (Causal Convolution, Temporal Dilation, TCN-ResNets)
Introduction to Transformers (Sequence-to-Sequence Learning, Basics on Attention, Self-Attention and the Query-Key-Value Principle, Large Language Models)
State Space Models (Structured State Space Sequence Models, Mamba)

Qualifikationsziele/Kompetenzen:

Students get a comprehensive understanding of most relevant sequence learning approaches
Students learn to analyze the challenges in sequence learning tasks and to identify well-suited approaches to solve them
Students will understand the pros and cons of various sequence learning models
Students can implement common and custom sequence learning models for time series analysis, classification, and forecasting
Students know how to analyze the models and results, to improve the model parameters, and to interpret the model predictions and their relevance

Vergabe von Leistungspunkten und Benotung durch:

Klausur oder mündliche Prüfung nach Maßgabe des Dozenten

Modulverantwortlicher:

Prof. Dr. Sebastian Otte

Lehrende:

Institut für Robotik und Kognitive Systeme

MitarbeiterInnen des Instituts
Prof. Dr. Sebastian Otte

Literatur:

Goodfellow, I., Bengio, Y., & Courville, A. (2016): Deep Learning - MIT Press. ISBN 978-0262035613
Prince, S. J. D. (2023): Understanding Deep Learning - The MIT Press. ISBN 978-0262048644
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020): Mathematics for Machine Learning - Cambridge University Press, 2020. ISBN 978-1108470049
Nakajima, K., & Fischer, I. (2021): Reservoir Computing: Theory, Physical Implementations, and Applications - Cambridge University Press, 2020. ISBN 978-1108470049
Sun, R., & Giles, C. (2001): Sequence Learning: Paradigms, Algorithms, and Applications - Springer Berlin Heidelberg. ISBN 978-3540415978
Bishop, C. M. (2006): Pattern Recognition and Machine Learning - Springer. ISBN 978-0387310732
Recent publications on the related topics:

Sprache:

Wird nur auf Englisch angeboten

Bemerkungen:

Admission requirements for taking the module:
- None, but it is recommended to complete the course Deep Learning (CS4295-KP04) first

Admission requirements for participation in module examination(s):
- Successful completion of exercise assignments as specified at the beginning of the semester

Module Exam(s):
- CS4575-L1: Sequence Learning, exam, 90 min

Laut Beschluss des Prüfungsausschusses Informatik vom 19.8.2024 kann dieses Modul von Studierenden Master Informatik SGO ab 2019 im Bereich 5. Wahlpflichtfach gewählt werden.