What Is Machine Learning?

Artificial intelligence (AI) is a field of technology that aims to enable computers to perform intellectual tasks typically carried out by humans.

Machine learning, a subfield of AI, learns the rules needed for prediction and decision-making by fitting mathematical models to observed data.

A deep neural network, also known as a deep network, is one type of model used in machine learning.

Deep learning is an approach that uses deep neural networks to learn rules and features from data.

Deep learning is used in a wide range of applications, including machine translation, which converts text from one language into another; computer vision, which detects specific objects in images; and speech recognition used in digital assistants.

Machine learning methods can be broadly divided into three categories: supervised learning, unsupervised learning, and reinforcement learning.

Supervised Learning

In supervised learning, a model represents the relationship used to obtain a predicted output from input data.

A model can be expressed as a function, and the process of providing an input to the model and calculating an output is called inference.

A model contains not only inputs but also parameters.

When the parameter values change, the model’s output also changes, even when the same input is provided.

Training or learning a model means adjusting its parameters so that it can appropriately represent the relationship between inputs and outputs.

Training uses training data consisting of input data paired with the corresponding correct outputs.

The learning algorithm repeatedly adjusts the parameters so that the model’s predictions become closer to the correct outputs.

If the model successfully learns the relationship between inputs and outputs from the training data, it is expected to make reasonable predictions for new inputs whose correct outputs are unknown.

Thus, supervised learning is characterized by learning the relationship between inputs and outputs from data for which the correct outputs are provided.

Unsupervised Learning

Unsupervised learning is the process of building a model from input data that does not have corresponding correct answers or output labels.

Rather than learning a mapping from inputs to correct outputs, as in supervised learning, its purpose is to describe and understand the structures and features contained in the data.

In unsupervised learning, training is performed using a set of unlabeled observations $\{x_i\}$.

Unsupervised learning has a wide range of objectives, including generating new samples similar to the training data, representing or transforming data, removing noise, completing missing information, and compressing data.

Reinforcement Learning

Reinforcement learning (RL) is based on the concept of an agent that acts within an environment.

At each time step, the agent selects an action according to the state of the environment.

The action changes the state of the environment and may result in the agent receiving a reward.

In a video game, for example, the game can be regarded as the environment, the character being controlled as the agent, the character’s movements as actions, and the score earned as the reward.

In financial trading, the financial market can be regarded as the environment, a virtual trader as the agent, buying and selling assets as actions, and the resulting profit as the reward.

The next state resulting from an action is not always uniquely determined. Even when the same action is selected in the same state, the environment may transition to different states.

The objective of reinforcement learning is to learn how the agent should select actions in order to obtain high rewards over the long term.

However, a reward is not always received immediately after an action and may occur only after a certain amount of time has passed.

This gives rise to the temporal credit assignment problem: determining how much each action contributed to a reward received later.

In addition to selecting actions that have already proved effective through previous learning, the agent must also try unfamiliar actions in order to discover potentially better ones.

Thus, another characteristic of reinforcement learning is the need to balance exploitation, which uses knowledge already acquired, with exploration, which involves trying new actions.

About This Article

References
・Understanding Deep Learning I: Supervised Learning and the Foundations of Models, Simon J. D. Prince, translated by Toshiaki Kurokawa
・Understanding Deep Learning II: Unsupervised Learning, Reinforcement Learning, and Ethics, Simon J. D. Prince, translated by Toshiaki Kurokawa


Note: This article was organized based on the author’s understanding, with reference to the publications listed above.

Chapter 1: Foundations of Machine Learning and Deep Learning