Introduction

In this class we are going to look at the basic concepts behind modern day Data Science techniques. We will always try to not only discuss the theory but also use Python to illustrate the content programmatically.

The main reference for these notes is the excellent Brunton and Kutz (2022), see citations throughout the notes. If python code is adapted form code blocks provided by Brunton and Kutz (2022), see github this is indicated and for these blocks you can also find the MATLAB equivalent in the book or on github.

These notes are intended for engineering students and therefore the mathematical concepts will rarely include rigorous proofs.

We start of by recalling the main concepts of 1  Linear Algebra and some statistics on 2  Data sets to make sure everybody as an the same page for notation and problematically in Python.

We continue with matrix decomposition, namely 3  Eigendecomposition and 4  Singular Value Decomposition. These basic decomposition are used to illustrate certain concepts we need in later chapters but also to show how the change of basis is influencing problems, their solution and computational properties. Furthermore, we dive into the first concepts used in machine learning where matrix computations build the foundation. As illustration we use applications for engineering and image processing.

The SVD allows us to neatly transition to 5  Linear Regression, generalize to 6  Non-linear Regression and discuss optimization and learning properties with the help of LASSO and Model Selection/Identification and over-/underfitting in 7  Optimizers. We mainly use toy examples but where appropriate we look at the world population or unemployment data to illustrate a concept.

In the next part we look at aspects of signal processing often found in engineering and especially mechatronics and therefore we discuss 8  Fourier Transform, 9  Laplace Transform as well as 10  Wavelet Transform transform and how they are extended to 11  Two-Dimensional Transform. Examples range from toy examples to solving electric circuit problems or image processing.

The fifth part is based on Brunton and Kutz (2022) Chapter 3 where we look at aspects of 12  Sparsity and Compression as well as the rather new topic of 13  Compressed Sensing.

To round the content of these notes we look at statistics with some basis of 14  Bayesian Statistics and the engineering application via 15  Kalman Filter.

Brunton, Steven L., and J. Nathan Kutz. 2022. Data-Driven Science and Engineering - Machine Learning, Dynamical Systems, and Control. 2nd ed. Cambridge: Cambridge University Press.