1. Classification losses

每次输入一个样本，对样本进行类别预测，根据预测类别和真实标签得到对应的分类损失。

2. Pairwise losses

每次输入两个样本，数据集包含了这两个样本是否相似的信息。计算损失时根据模型在这两个样本上的输出和相似信息进行计算。

3. Triplet losses

每次输入三个样本，\(x, x_-, x_+\)。其中\(x_-和x_+\)分别为\(x\)的负样本（不相似）和正样本（相似）。根据这三个数据在模型的输出以及对应的相似信息得到损失。

Quadruplet losses

每次输入四个数据，这四个数据都是不同的，其中包含了一对相似数据和一对不相似数据。根据四个数据在模型的输出和对应的相似信息得到损失。

疑问

• 不同losses之间的优缺点以及适用场景。

参考

• Ustinova E, Lempitsky V. Learning Deep Embeddings with Histogram Loss[J]. 2016.

Classification losses. It has been observed in [8] and confirmed later in multiple works (e.g. [15])
that deep networks trained for classification can be used for deep embedding. In particular, it is
sufficient to consider an intermediate representation arising in one of the last layers of the deep
network. The normalization is added post-hoc. Many of the works mentioned below pre-train their
embeddings as a part of the classification networks.
Pairwise losses. Methods that use pairwise losses sample pairs of training points and score them
independently. The pioneering work on deep embeddings [3] penalizes the deviation from the unit
cosine similarity for positive pairs and the deviation from -1 or -0:9 for negative pairs. Perhaps,
the most popular of pairwise losses is the contrastive loss [5, 20], which minimizes the distances in
the positive pairs and tries to maximize the distances in the negative pairs as long as these distances
are smaller than some margin M. Several works pointed to the fact that attempting to collapse all
positive pairs may lead to excessive overfitting and therefore suggested losses that mitigate this
effect, e.g. a double-margin contrastive loss [12], which drops to zero for positive pairs as long as
their distances fall beyond the second (smaller) margin. Finally, several works use non-hinge based
pairwise losses such as log-sum-exp and cross-entropy on the similarity values that softly encourage
the similarity to be high for positive values and low for negative values (e.g. [24, 27]). The main
problem with pairwise losses is that the margin parameters might be hard to tune, especially since
the distributions of distances or similarities can be changing dramatically as the learning progresses.
While most works “skip” the burn-in period by initializing the embedding to a network pre-trained for classification [24], [22] further demonstrated the benefit of admixing the classification loss during
the fine-tuning stage (which brings in another parameter).
Triplet losses. While pairwise losses care about the absolute values of distances of positive and
negative pairs, the quality of embeddings ultimately depends on the relative ordering between positive
and negative distances (or similarities). Indeed, the embedding meets the needs of most practical
applications as long as the similarities of positive pairs are greater than similarities of negative pairs
[19, 26]. The most popular class of losses for metric learning therefore consider triplets of points
x0; x+; x-, where x0; x+ form a positive pair and x0; x- form a negative pair and measure the
difference in their distances or similarities. Triplet-based loss can then e.g. be aggregated over all
triplets using a hinge function of these differences. Triplet-based losses are popular for large-scale
embedding learning [4] and in particular for deep embeddings [13, 14, 17, 21, 28]. Setting the margin
in the triplet hinge-loss still represents the challenge, as well as sampling “correct” triplets, since the
majority of them quickly become associated with zero loss. On the other hand, focusing sampling on
the hardest triplets can prevent efficient learning [17]. Triplet-based losses generally make learning
less constrained than pairwise losses. This is because for a low-loss embedding, the characteristic
distance separating positive and negative pairs can vary across the embedding space (depending on
the location of x0), which is not possible for pairwise losses. In some situations, such added flexibility
can increase overfitting.
Quadruplet losses. Quadruplet-based losses are similar to triplet-based losses as they are computed
by looking at the differences in distances/similarities of positive pairs and negative pairs. In the case
of quadruplet-based losses, the compared positive and negative pairs do not share a common point
(as they do for triplet-based losses). Quadruplet-based losses do not allow the flexibility of tripletbased losses discussed above (as they includes comparisons of positive and negative pairs located in
different parts of the embedding space). At the same time, they are not as rigid as pairwise losses, as
they only penalize the relative ordering for negative pairs and positive pairs. Nevertheless, despite
these appealing properties, quadruplet-based losses remain rarely-used and confined to “shallow”
embeddings [9, 30]. We are unaware of deep embedding approaches using quadruplet losses. A
potential problem with quadruplet-based losses in the large-scale setting is that the number of all
quadruplets is even larger than the number of triplets. Among all groups of losses, our approach
is most related to quadruplet-based ones, and can be seen as a way to organize learning of deep
embeddings with a quarduplet-based loss in an efficient and (almost) parameter-free manner.