09_图像卷积
一.图像的矩阵表示方法
- 对于单通道的图像,i和j分别表示行和列的索引,输入图像为src,则i从1~src.rows-1,j从1~src.cols-1时,src可以表示成如下形式:
src=⎣⎡I(i−1,j−1)I(i,j−1)I(i+1,j−1)I(i−1,j)I(i,j)I(i+1,j)I(i−1,j+1)I(i,j+1)I(i+1,j+1)⎦⎤
- 对于多通道的图像,以三通道为例,i和j分别表示行和列的索引,输入图像为src,则i从1~src.rows-1,j从1~src.cols-1时,src可以表示成如下形式:
src=⎣⎡I1(i−1,j−1)I1(i,j−1)I1(i+1,j−1)I2(i−1,j−1)I2(i,j−1)I2(i+1,j−1)I3(i−1,j−1)I3(i,j−1)I3(i+1,j−1)I1(i−1,j)I1(i,j)I1(i+1,j)I2(i−1,j)I2(i,j)I2(i+1,j)I3(i−1,j)I3(i,j)I3(i+1,j)I1(i−1,j+1)I1(i,j+1)I1(i+1,j+1)I2(i−1,j+1)I2(i,j+1)I2(i+1,j+1)I3(i−1,j+1)I3(i,j+1)I3(i+1,j+1)⎦⎤
二.图像的卷积运算
- 图像的卷积运算是指,依次从图像取出与卷积核相同size的矩阵I与卷积核绕中心位置旋转
180。(先沿X轴翻转,再沿y轴翻转)后的结果对应位置处的值相乘相加,所得结果再放入矩阵I的中心位置后,得到新的矩阵,即为卷积后的结果
- 假设有如下卷积核:
kernel=⎣⎡10−120−210−1⎦⎤
则kernel绕中心位置旋转
180。后得到:
kernel‘=⎣⎡−101−202−101⎦⎤
对于单通道图像,与kernel卷积的计算公式如下:
I(i,j)=−1∗I(i−1,j−1)+−2∗I(i−1,j)+−1∗I(i−1,j+1)+0∗I(i,j−1)+0∗I(i,j)+0∗I(i,j+1)+1∗I(i+1,j−1)+2∗I(i+1,j)+1∗I(i+1,j+1)
对于多通道图像,以三通道图像为例,与kernel卷积的计算公式如下:
I1(i,j)=−1∗I1(i−1,j−1)+−2∗I1(i−1,j)+−1∗I1(i−1,j+1)+0∗I1(i,j−1)+0∗I1(i,j)+0∗I1(i,j+1)+1∗I1(i+1,j−1)+2∗I1(i+1,j)+1∗I1(i+1,j+1)
I2(i,j)=−1∗I2(i−1,j−1)+−2∗I2(i−1,j)+−1∗I2(i−1,j+1)+0∗I2(i,j−1)+0∗I2(i,j)+0∗I2(i,j+1)+1∗I2(i+1,j−1)+2∗I2(i+1,j)+1∗I2(i+1,j+1)
I3(i,j)=−1∗I3(i−1,j−1)+−2∗I3(i−1,j)+−1∗I3(i−1,j+1)+0∗I3(i,j−1)+0∗I3(i,j)+0∗I3(i,j+1)+1∗I3(i+1,j−1)+2∗I3(i+1,j)+1∗I3(i+1,j+1)
其中:
i∈[1,src.rows()−1)j∈[1,src.cols()−1)
三.一次性读取Mat对象中全部的像素信息
int channels = src.channels();
int rows = src.rows();
int cols = src.cols();
byte[] buffer = new byte[cols*rows*channels];
src.get(0, 0, buffer);
四.从图像中获取与卷积核同样大小的像素信息
- 假设单通道图像的像素信息和卷积核如下:
src=⎣⎢⎢⎢⎢⎡0000000000000000000000000⎦⎥⎥⎥⎥⎤
kernel=⎣⎡000000000⎦⎤
将图像的像素信息全部读取到缓冲数组后有:
buffer=[0000000000000000000000000]
可以得到src中第i行j列(i,j从0开始)的像素值对应buffer中的第i*cols+j个元素
同理,kernel的顶点对应buffer中的第i*cols+j个元素,并且i,j的取值范围如下:
i∈[0,src.rows()−kernel.rows()+1)j∈[0,src.cols()−kernel.cols()+1)
所以,对于单通道图像,可以通过如下方式得到src中,从第i行j列开始的与kernel同样大小的像素信息:
int startPos = i*rows+j
pixel = buffer[startPos + l + cols * k]
其中:
i∈[0,src.rows()−kernel.rows()+1)j∈[0,src.cols()−kernel.cols()+1)k∈[0,kernel.rows())l∈[0,kernel.cols())
而对于多通道图像,可以通过如下方式得到src中,从第i行j列开始的与kernel同样大小的像素信息:
int startPos = i*rows+j
pixel = buffer[(startPos + l + cols * k)*src.channels + channel]
其中:
i∈[0,src.rows()−kernel.rows()+1)j∈[0,src.cols()−kernel.cols()+1)k∈[0,kernel.rows())l∈[0,kernel.cols())channel∈[0,src.channels())
获取到与卷积核同样大小的像素信息后就可以进行卷积运算了,并且要把运算结果重新放回与卷积核同样大小的图像的中心位置,其中心位置的像素与buffer中元素对于关系如下:
int offsetX = (kernel.rows()-1)/2;
int offsetY = (kernel.cols()-1)/2;
int startPos = i*src.cols() + j;
int index = (startPos + offsetX + src.cols() * offsetY) * src.channels() + channel
其中:
i∈[0,src.rows()−kernel.rows()+1)j∈[0,src.cols()−kernel.cols()+1)channel∈[0,src.channels())
index即为与卷积核同样大小的图像的中心位置的像素,在buffer中与之对应的元素的下标
五.通过遍历Mat对象实现图像卷积
Mat srcMat = new Mat();
Utils.bitmapToMat(src, srcMat);
Imgproc.cvtColor(srcMat, srcMat, Imgproc.COLOR_BGRA2BGR);
int kRows = kernel.rows();
int kCols = kernel.cols();
if(!((kCols%2==1) && (kRows%2==1))) {
throw new RuntimeException("The size of kernal is not suitable!");
}
int offsetX = (kCols-1)/2;
int offsetY = (kRows-1)/2;
Core.flip(kernel, kernel, 0);
Core.flip(kernel, kernel, 1);
double[] kernels = new double[kRows*kCols];
kernel.get(0,0,kernels);
int channels = srcMat.channels();
int rows = srcMat.rows();
int cols = srcMat.cols();
byte[] srcBuffer = new byte[rows*cols*channels];
srcMat.get(0, 0, srcBuffer);
byte[] dstBuffer = new byte[rows*cols*channels];
for(int i=0; i<(rows - kRows + 1); i++) {
for(int j=0; j<(cols - kCols + 1); j++) {
int startPos = i*cols + j;
double[] bgra = new double[channels];
for (int channel = 0; channel < channels; channel++) {
for (int k = 0; k < kRows; k++) {
for (int l = 0; l < kCols; l++) {
bgra[channel] += (srcBuffer[(startPos + l + cols * k) * channels + channel] & 0xff) * kernels[k * kCols + l];
}
}
dstBuffer[(startPos + offsetX + cols * offsetY) * channels + channel] = (byte) saturateCast(bgra[channel]);
}
}
}
Mat dst = new Mat(srcMat.size(), srcMat.type());
dst.put(0,0,dstBuffer);
Utils.matToBitmap(dst, src);
六.通过遍历Bitmap对象实现图像卷积
Mat srcMat = new Mat();
Utils.bitmapToMat(src, srcMat);
Imgproc.cvtColor(srcMat, srcMat, Imgproc.COLOR_BGRA2BGR);
int kRows = kernel.rows();
int kCols = kernel.cols();
if(!((kCols%2==1) && (kRows%2==1))) {
throw new RuntimeException("The size of kernal is not suitable!");
}
int offsetX = (kCols-1)/2;
int offsetY = (kRows-1)/2;
Core.flip(kernel, kernel, 0);
Core.flip(kernel, kernel, 1);
double[] kernels = new double[kRows*kCols];
kernel.get(0,0,kernels);
int width = src.getWidth();
int height = src.getHeight();
int[] srcPixels = new int[width * height];
int[] dstPixels = new int[width * height];
src.getPixels(srcPixels, 0, width, 0, 0, width, height);
for(int i=0; i<(height - kRows + 1); i++) {
for(int j=0; j<(width - kCols + 1); j++) {
int startPos = i*width + j;
double a = 0, r = 0, g = 0, b = 0;
for (int k = 0; k < kRows; k++) {
for (int l = 0; l < kCols; l++) {
a = 255;
r += ((srcPixels[startPos + l + width * k] >> 16) & 0xff) * kernels[k * kCols + l];
g += ((srcPixels[startPos + l + width * k] >> 8) & 0xff) * kernels[k * kCols + l];
b += ((srcPixels[startPos + l + width * k]) & 0xff) * kernels[k * kCols + l];
}
}
a = saturateCast(a);
r = saturateCast(r);
g = saturateCast(g);
b = saturateCast(b);
int pixel = (((int)a & 0xff) << 24) | (((int)r & 0xff) << 16) | (((int)g & 0xff) << 8) | ((int)b & 0xff);
dstPixels[startPos + offsetX + width * offsetY] = pixel;
}
}
src.setPixels(dstPixels, 0, width, 0, 0, width, height);
七.通过OpenCv接口(Imgproc.filter2D)实现图像卷积
Mat srcMat = new Mat();
Utils.bitmapToMat(src, srcMat);
Imgproc.cvtColor(srcMat, srcMat, Imgproc.COLOR_BGRA2BGR);
int kRows = kernel.rows();
int kCols = kernel.cols();
if(!((kCols%2==1) && (kRows%2==1))) {
throw new RuntimeException("The size of kernal is not suitable!");
}
int offsetX = (kCols-1)/2;
int offsetY = (kRows-1)/2;
Core.flip(kernel, kernel, 0);
Core.flip(kernel, kernel, 1);
int[] kernels = new int[kRows*kCols];
kernel.get(0,0,kernels);
Imgproc.filter2D(srcMat, srcMat, srcMat.depth(), kernel);
Utils.matToBitmap(srcMat, src);
八.运行结果
其中:
kernel=⎣⎡10−120−210−1⎦⎤