A function is a rule that maps elements of one set to elements of another set.

A function $f$ from a set $\mathcal{X}$ to a set $\mathcal{Y}$ is written as $f:\mathcal{X}\rightarrow\mathcal{Y}$.
In this case, each element of the set $\mathcal{X}$ is associated with exactly one element of the set $\mathcal{Y}$.

A function in which distinct inputs always map to distinct outputs is called an injective function.
For an injective function, the set $\mathcal{Y}$ may contain elements that are not associated with any input.

A function in which every element of the set $\mathcal{Y}$ is associated with at least one element of the set $\mathcal{X}$ is called a surjective function.
For a surjective function, multiple inputs may map to the same output.

A function that is both injective and surjective is called a bijective function or bijection.
For a bijective function, the elements of the two sets correspond one-to-one, with neither duplication nor omission.

A diffeomorphism is a bijection for which both the forward mapping and the inverse mapping are smooth—that is, differentiable as many times as required.

Lipschitz Constant

A function $\boldsymbol{f}[\boldsymbol{z}]$ is said to be Lipschitz continuous if there exists a constant $\beta$ such that the following inequality holds for any $\boldsymbol{z}_1$ and $\boldsymbol{z}_2$.

Here, $\beta$ is the Lipschitz constant.
Equation (1-1) describes how much the distance between the outputs can change in relation to a change in the distance between two inputs.

A function with a small Lipschitz constant changes gradually in response to changes in its input, whereas a function with a large Lipschitz constant may exhibit rapid changes in its output.

A function with a Lipschitz constant of less than 1 is called a contraction mapping.

In a space satisfying the appropriate conditions, the Banach fixed-point theorem shows that repeatedly applying a contraction mapping converges to a unique fixed point, regardless of the initial value.

A fixed point is a point whose value remains unchanged when the function is applied.

If two functions with Lipschitz constants $\beta_1$ and $\beta_2$ are added, their sum is also Lipschitz continuous, with a Lipschitz constant no greater than $\beta_1+\beta_2$.

For the affine transformation $\boldsymbol{f}[\boldsymbol{z}]=\boldsymbol{A}\boldsymbol{z}+\boldsymbol{b}$, the Lipschitz constant with respect to the Euclidean distance is given by the spectral norm of the matrix $\boldsymbol{A}$, which is its largest singular value.

Convexity

A function is called a convex function if, for any two points on its graph, the line segment connecting them always lies on or above the graph of the function.
Conversely, a function is called a concave function if the line segment connecting any two points on its graph always lies on or below the graph.

For a convex function, every local minimum is also a global minimum.
However, a convex function with a flat region may have more than one minimum point.

By contrast, for a strictly convex function, in which the line segment lies strictly above the graph except at its endpoints, there is at most one minimum point.
The corresponding relationships are reversed for concave functions, so every local maximum is also a global maximum.

A strictly concave function has at most one maximum point.

A region in $\mathbb{R}^{D}$ is called a convex set if, for any two points selected from the region, the entire line segment connecting those points is contained within the region.

Convexity is important when considering the loss functions used to train models. Because every local minimum of a convex function is also a global minimum, the properties of its minima are easier to understand than those of a non-convex function.

Under appropriate conditions, methods such as gradient descent can be used to find the global minimum of a convex function.

Function Notation

In this article, arguments are enclosed in square brackets following the name of a function.
For example, $\log[x]$ denotes a function that returns the logarithm of the variable $x$.

A function that returns a vector is written in lowercase bold type.
For example, $\boldsymbol{y}=\boldsymbol{\mathrm{mlp}}[\boldsymbol{x},\boldsymbol{\phi}]$ represents a function that takes the vector $\boldsymbol{x}$ and the parameter vector $\boldsymbol{\phi}$ as arguments and returns the vector $\boldsymbol{y}$.

A function that returns a matrix or tensor is written in uppercase bold type.
For example, $\boldsymbol{Y}=\boldsymbol{\mathrm{Sa}}[\boldsymbol{X},\boldsymbol{\phi}]$ represents a function that takes the matrix $\boldsymbol{X}$ and the parameter vector $\boldsymbol{\phi}$ as arguments and returns the matrix $\boldsymbol{Y}$.

When the specific argument of a function is not stated, a bullet symbol is used, as in $f[\bullet]$.

Special Functions

Deep learning uses various special functions, including the exponential function, logarithm, and gamma function.

Exponential Function

The exponential function $y=\exp[x]=e^x$ maps a real number $x\in\mathbb{R}$ to a positive real number $y\in\mathbb{R}_{>0}$.
Regardless of the real value of $x$, the output of the exponential function is always positive.

Logarithm

The logarithm $x=\log[y]$ is the inverse of the exponential function.
It takes a positive real number $y\in\mathbb{R}_{>0}$ as its input and returns a real number $x\in\mathbb{R}$.

Therefore, if $y=e^x$, then $x=\log[y]$.

Gamma Function

The gamma function $\Gamma[x]$ is defined for $x>0$ by the following equation.

The gamma function extends the factorial to positive real numbers. For positive integers $x\in\{1,2,\ldots\}$, the relation $\Gamma[x]=(x-1)!$ holds.

Dirac Delta Function

The Dirac delta function $\delta[\boldsymbol{z}]$ is zero everywhere except at $\boldsymbol{z}=\boldsymbol{0}$ and is treated so that its integral over the entire domain equals 1.
Intuitively, all of its area or volume can be regarded as being concentrated at $\boldsymbol{z}=\boldsymbol{0}$.

Strictly speaking, it is not an ordinary function but is treated as a distribution or generalized function. Nevertheless, the term Dirac delta function is widely used in physics, engineering, and applied mathematics.

A dataset consisting of N data points can be represented as a probability distribution composed of N delta functions, each centered at a data point $\boldsymbol{x}_i$ and assigned a weight of 1/N.

A delta function is generally represented by an arrow on a graph.

The Dirac delta function has the property of extracting the value of a function at a specific point.
This property is expressed by the following equation.

Equation (1-3) shows that multiplying the function $f[\boldsymbol{x}]$ by a delta function centered at $\boldsymbol{x}_0$ and integrating over the entire domain yields $f[\boldsymbol{x}_0]$.

Functions for Expressing Minimum and Maximum Values

$\min$ and $\operatorname{arg\,min}$ are both related to minimizing a function, but they return different values.
$\underset{x}{\min}\,f[x]$ returns the minimum value of the function $f[x]$ over the range of possible values of $x$.

By contrast, $\underset{x}{\operatorname{arg\,min}}\,f[x]$ represents the value of $x$ that minimizes the function $f[x]$.
If multiple values of $x$ minimize the function, they are treated as a set.

If the minimizing value of $x$ is unique and is denoted by $y$, the following relationship holds.

Similarly, $\underset{x}{\max}\,f[x]$ returns the maximum value of the function, whereas $\underset{x}{\operatorname{arg\,max}}\,f[x]$ returns the value of $x$ that maximizes the function, or the set of such values.

About This Article

References
・Understanding Deep Learning I: Supervised Learning and the Foundations of Models, Simon J. D. Prince, translated by Toshiaki Kurokawa
・Understanding Deep Learning II: Unsupervised Learning, Reinforcement Learning, and Ethics, Simon J. D. Prince, translated by Toshiaki Kurokawa
・“Convex Optimization—Convex Sets,” Stephen Boyd, Lieven Vandenberghe, Parth Nobel, Stanford University
・“Convex Optimization—Convex Functions,” Stephen Boyd, Lieven Vandenberghe, Parth Nobel, Stanford University
・“DLMF §5.2 Definitions—Gamma Function,” National Institute of Standards and Technology (NIST)
・“DLMF §1.17 Integral and Series Representations of the Dirac Delta,” National Institute of Standards and Technology (NIST)

(Last accessed: September 21, 2026)


Note: This article was organized and restructured based on the author’s understanding, with reference to the publications listed above.

深層学習に関する数学・確率の基礎