broadcasting Theano vs. Numpy
broadcast mechanism allows a scalar may be added to a matrix, a vector to a matrix or a scalar to a vecotor.
Examples
T and F stands for True and False respectively, denoting which dimension can be broadcasted.
Diference
- numpy broadcast dynamically;
- theano needs to knows, for any operations which supports broadcasting, which dimensions will need to be broadcasted.
Numpy Broadcasting
broadcasting describe how numpy treats arrays with difference shapes during arithmetic operations:
the smaller array is broadcast across the larger array so that they have compatible shapes
Simple Case
in this case, the two arrays must have exactly the same shape:
a=np.array([1.0,2.0,3.0])
b=np.array([2.0,2.0,2.0])
print a*b
>>> array([2., 4., 6.])
numpy broadcast mechanism relaxes this constraint when the arrays' shape meet certain constraints:
- they are equal, or
- one of them is 1
a=np.array([1.0,2.0,3.0])
b=2
print a*b
>>> array([2., 4., 6.])
rules
Image (3d array): 256 x 256 x 3
Scale (1d array): 3
Result (3d array): 256 x 256 x 3
A (4d array): 8 x 1 x 6 x 1
B (3d array): 7 x 1 x 5
Result (4d array): 8 x 7 x 6 x 5
more examples
A (2d array): 5 x 4
B (1d array): 1
Result (2d array): 5 x 4
A (2d array): 5 x 4
B (1d array): 4
Result (2d array): 5 x 4
A (3d array): 15 x 3 x 5
B (3d array): 15 x 1 x 5
Result (3d array): 15 x 3 x 5
A (3d array): 15 x 3 x 5
B (2d array): 3 x 5
Result (3d array): 15 x 3 x 5
A (3d array): 15 x 3 x 5
B (2d array): 3 x 1
Result (3d array): 15 x 3 x 5