R Tutorial: Parameters vs hyperparameters

Want to learn more? Take the full course at https://learn.datacamp.com/courses/hy... at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work.

---
Welcome to this course on hyperparameter tuning in R.
In this course you will learn:
what hyperparameters are and what makes them different from regular parameters
why hyperparameter tuning is an important step towards optimizing your machine learning models
and how you can apply hyperparameter tuning with the packages caret, mlr and h2o.

My name is Shirin and I started out as a traditional biologist. I spent a lot of time in the lab. But eventually, it became clear that what I really enjoyed above all else was working with data. That's why I spent two years as a bioinformatics Postdoc at the University of Münster in Germany before I started working as a Data Scientist for codecentric.

I also write a Data Science blog where I play around with different datasets, analyses, and visualization techniques.

So, why do we use the strange word "hyper-parameter"? And how are hyperparameters different from model parameters?
In this chapter, we will work with a dataset about breast cancer patient samples. 10 features describe the diagnosis of benign or malignant tissue masses. Here, we use them to build a classification model.

Let's have a look at a simple linear model.

A linear model models the relationship between variables by fitting a linear function. Here, we will pick two features at random: perimeter_worst & fractal_dimension_mean and look at their linear relationship. We could, of course, make our linear model much more complex by adding additional features and more complex interactions, but for this purpose, we will keep it simple.

The summary function will give us an overview of the fitted linear model and its results,

like residuals,

coefficients, and statistics.

The results of our fitted linear model give the model parameters.

Thus, model *parameters* are the *result* of model fitting.
In machine learning, we use the word training instead of model fitting, so we can say that the model parameters are being fit (or found) during training.

Let's look again at our linear model: here we want to find the coefficients,

which we can think of as the slope and intercept of our model.

Slope and intercept are best understood when visualized;

they describe the best line through our data points.

slope describes the steepness of this line, while
intercept describes the point where our line crosses the y-axis.

Okay, you now know what model parameters are. But what about hyperparameters?

Hyperparameters are defined before training,

they specify HOW the training is supposed to happen, this means they define options in function calls:

We can find out which options to define by looking at the arguments or formals of a function or by going to its help page.
In our linear model, `method` is a hyperparameter.

So, to recap:

Model parameters were found during training, like coefficients. In machine learning these could be the weights of a neural network.

Hyperparameters were defined before training; these could be the learning rate in a neural net or the number of trees in a random forest.

But why would we want to tune hyperparameters?
Imagine we are creating a fantasy football team: we want to find the best combination of players to maximize our chances of winning.

In machine learning, we have hyperparameters - which we could think of like fantasy football players; each hyperparameter can take a range of values - just as players can be assigned different positions on the field.

Just like with our fantasy football team, we want to find the best combination of hyperparameters, so that our model performs as well as possible.

Time to put this into practice.

#R #RTutorial #DataCamp #Hyperparameter #Tuning #Parameters #hyperparameters