Reuction operations

Reduction operations

A reduction operations on a tensor is an operation that reduces the number of elements contained within the tensor.

Tensors give us the ability to manage out data. Reduction operations allow us to perform operations on elements within a single tensor.

Suppose we have the following 3$\times$3 rank-2 tensor.

> t = torch.tensor([
    [0, 1, 0],
    [2, 0, 2],
    [0, 3, 0]
], dtype=torch.float32)

Common tensor reduction operations

> t.sum()
tensor(8.)

> t.numel()
9

> t.prod()
tensor(0.)

> t.mean()
tensor(.8889)

> t.std()
tensor(1.1667)

Reducing tensors by axes

Suppose we have the following tensor:

> t = torch.tensor([
    [1,1,1,1],
    [2,2,2,2],
    [3,3,3,3]
], dtype=torch.float32)

This time , we will specify a dimension to reduce.

> t.sum(dim=0)
tensor([6., 6., 6., 6.])

> t.sum(dim=1)
tensor([4., 8., 12.])

Argmax tensor reduction operation

Argmax returns the index location of the maximum value inside a tensor.

t = torch.tensor([
    [1,0,0,2],
    [0,3,3,0],
    [4,0,0,5]
], dtype=torch.float32)

If we don't specific an axis to the argmax() method, it returns the index location of the max value from the flattened tensor, which in the case is indeed 11.

> t.max()
tensor(5.)

> t.argmax()
tensor(11)

> t.flatten()
tensor([1., 0., 0., 2., 0., 3., 3., 0., 4., 0., 0., 5.])

Work with specific axis now:

> t.max(dim=0)
(tensor([4., 3., 3., 5.]), tensor([2, 1, 1, 2]))

> t.argmax(dim=0)
tensor([2, 1, 1, 2])

> t.max(dim=1)
(tensor([2., 3., 5.]), tensor([3, 1, 3]))

> t.argmax(dim=1)
tensor([3, 1, 3])

In practice, we often use the argmax() function on a network's output prediction tensor, to determine which category has the highest prediction value.