利用Matrix Multiplication实现Convolutions
以feature map : channel=1,stride=1,padding=0为例,解决:1、如何将feature map和 filter kernel进行unfold ;2、如何进行卷积运算的问题。
import numpy as np
from numpy import zeros
#将feature map unfold
def unfold_matrix(X, k):
n, m = X.shape[0:2]
xx = zeros(((n - k + 1) * (m - k + 1), k**2))
row_num = 0
def make_row(x):
return x.flatten()
for i in range(n- k+ 1):
for j in range(m - k + 1):
#collect block of m*m elements and convert to row
xx[row_num,:] = make_row(X[i:i+k, j:j+k])
row_num = row_num + 1
return xx
w = np.array([[1, 2, 3], [4, 5, 6], [-1, -2, -3]], np.float32)
#x = np.random.randn(5,5)
x = np.array([[-0.21556299, -0.11002319, -0.3499612, 1.49290769, -0.50435978],
[ 0.06348409, 0.66873375, 0.14251138, -1.6414004 , -0.91561852],
[-2.52451962, -1.97544675, -0.24609529, -1.11489934, -1.44793437],
[ 1.26260575, -0.62047366, 0.12274525, 0.25200227, -0.83925847],
[-1.54336488, -0.05100702, 0.36608208, 0.51712927, -0.97133877],
[-1.54336488, -0.05100702, 0.36608208, 0.51712927, -0.97133877]])
n, m = x.shape[0:2]
k = w.shape[0]
x_unfolded = unfold_matrix(x, k)
w_flat = w.flatten()
yy = np.matmul(x_unfolded, w_flat)
yy = yy.reshape((n-k+1, m-k+1))
print(yy)
卷积运算方式发展史
CuDNN v1.0 released in Aug 2014使用矩阵乘法实现卷积运算,因为大量优化矩阵运算的库,使得这种方式已经能高效的实现卷积的运算。但这种方式不是最高效的,更高效的实现方式有Fast Fourier Transform (FFT) and the Winograd transformation等。一般来说,Fast Fourier Transform (FFT)适合于大尺度的filter sizes,而Winograd更适合与小尺度的filter sizes(3×3 or 5×5)。