Uncovering How Machine Learning Systems Learn

The whole goal of using machine learning is to learn the rules that can be used to automate a given task. Without the data and ability to learn the patterns hidden in the data(inexplicitly), machine learning would not be the headlines of the internet nowadays.

In order for a machine to learn such patterns, we need three things:

1. Input data
2. Actual output
3. Loss and optimization functions

Let's talk about each requirement.

1. Input data

Data is the primary input for any machine learning algorithm.

Data can be structured, or unstructured. Structured data are usually in spreadsheet(tabular) format. Unstructured data are images, video, sound, and text.

The problem we are solving is what tells the type of data we need. If you want to predict if this tweet is positive or negative, you will collect many positive and negative tweets. In this case, tweets are text data, so unstructured data.

Below is a summary of structured and unstructured data

2. Actual output

The actual output or what we also call a label is like the description of the input data. For the example of classifying tweets, the labels are positive and negative. Another example. If the task is image classification(say a car and truck), the labels are car and truck.

For many machine learning blocks, you will see labels also referred to answers or results.

3. Loss and Optimization Functions

Loss and optimizer are the foundation of learning in machine learning (a term coined by Arthur Samuel). Without the ability of the system to measure the errors and correct them, learning would not be possible.

In order to measure if a machine learning algorithm is doing well on mapping input data and output, we have to measure the difference/distance between the predicted output and actual output.

During the model training, such difference is measured by a loss function and will be minimized by an optimization function.

Ideally, the loss will decrease gradually, or step by step, to the point where it is minimum. Or when it converges. This is what makes a learning curve.

Take a look at the graph below, but pay attention to the decreasing variable or loss.

In many (if not most) cases, the appropriate loss and optimization function depends on the task. There are regression losses, classification losses, and various optimizers. Choosing an appropriate loss is not a trivial task, but there are clear specifics.

Choosing the right optimizer for a given problem can be hard. Stick to Adam. It is safe... and forgiving(CC: Andrej).

---

To summarize, machine learning systems are made of 3 main things: Input data, output label, and functions that measure and minimize loss(loss and optimizer).

Below is your visual takeaway.

---

Thank you for reading.

Every week, I write one article about machine learning techniques, ideas, or best practices. You can support me by sharing the article with friends or tech communities that you are a part of.

And connect with me on Twitter!