opencv task2----停车场车位识别
前言
这个项目虽然识别技术很老,相比深度学习来识别来说也相对复杂、落后,但是是一个很好的练习opencv的项目。
github
https://github.com/yanjingke/opencv_park
权重
链接:https://pan.baidu.com/s/1k8JrkUGu98FKQGr9MxhJjg
提取码:0z1p
预备知识
1. inRange()函数
cv2.inRange函数设阈值,去除背景部分
cv2.inRange(hsv, lower_red, upper_red)
hsv指的是原图
lower_red指的是图像中低于这个lower_red的值,图像值变为0
upper_red指的是图像中高于这个upper_red的值,图像值变为0
而在lower_red~upper_red之间的值变成255
2. Canny算子
下面这幅图对边缘检测做出了很好的对比,但是我们常用Sobel算子和Canny算子做为实际开发。
cv2.Canny(image, threshold1, threshold2[, edges[, apertureSize[, L2gradient ]]])
image:源图像
threshold1:阈值1
threshold2:阈值2
apertureSize:可选参数,Sobel算子的大小()
其中,较大的阈值2用于检测图像中明显的边缘,但一般情况下检测的效果不会那么完美,边缘检测出来是断断续续的。所以这时候用较小的第一个阈值用于将这些间断的边缘连接起来。
函数返回的是二值图,包含检测出的边缘。
3. cvFillPoly
cvFillPoly用于一个单独被多边形轮廓所限定的区域内进行填充。函数可以填充复杂的区域
cv2.fillConvexPoly(image, triangle,color)
image:指的是原图
triangle:指需要需要提供多边形的顶点
color:指需要填充的颜色
img = np.zeros((1080, 1920, 3), np.uint8)
triangle = np.array([[0, 0], [1500, 800], [500, 400]])
cv2.fillConvexPoly(img, triangle, (255, 255, 255))
plt.imshow(img)
plt.show()
4.HoughLinesP
HoughLinesP函数是统计概率霍夫线变换函数,该函数能输出检测到的直线的端点 (x_{0}, y_{0}, x_{1}, y_{1}),其函数原型为HoughLines,后面加个P是HoughLine的升级版本,据说检测速度更快。
cv2.HoughLinesP(image, rho, theta, threshold[, lines[, minLineLength[, maxLineGap]]])
image参数表示边缘检测的输出图像,该图像为单通道8位二进制图像。
rho参数表示参数极径 r 以像素值为单位的分辨率,这里一般使用 1 像素。
theta参数表示参数极角 \theta 以弧度为单位的分辨率,这里一般使用 1度。
threshold参数表示检测一条直线所需最少的曲线交点。
lines参数表示储存着检测到的直线的参数对 (x_{start}, y_{start}, x_{end}, y_{end}) 的容器,也就是线段两个端点的坐标。
minLineLength参数表示能组成一条直线的最少点的数量,点数量不足的直线将被抛弃。
maxLineGap参数表示能被认为在一条直线上的亮点的最大距离。
停车场车位识别
需求分析
在本次中需要识别车位被占用情况,及识别出空车位和别占用用的车位。空车位并用mask画出来标记上,并且统计出总共车位和剩余车位的数量。
具体步骤
1.对图片(视频)做出预处理,并产生车位线的坐标 。ps:很多小伙伴最近问我,视频怎么识别,其实视频转换图片就几行代码,只是转换成了帧图片(具体百度吧)。
2.对处理后的图片产生车位线的坐标,存入字典序列中
3.车位线坐标,裁剪车位,利用深度学习分类网络,训练裁剪的车位有没有汽车。ps:这里采用vgg,不过vgg个人感觉这个网络太大了,在这种背景不太复杂的情况可以采用mobilenet。
4.利用训练好的对车位2分类识别,识别有没有车位。
对图片预处理
代码:
def img_process(test_images,park):
white_yellow_images = list(map(park.select_rgb_white_yellow, test_images))
park.show_images(white_yellow_images)
gray_images = list(map(park.convert_gray_scale, white_yellow_images))
park.show_images(gray_images)
edge_images = list(map(lambda image: park.detect_edges(image), gray_images))
park.show_images(edge_images)
roi_images = list(map(park.select_region, edge_images))
park.show_images(roi_images)
list_of_lines = list(map(park.hough_lines, roi_images))
line_images = []
for image, lines in zip(test_images, list_of_lines):
line_images.append(park.draw_lines(image, lines))
park.show_images(line_images)
rect_images = []
rect_coords = []
for image, lines in zip(test_images, list_of_lines):
new_image, rects = park.identify_blocks(image, lines)
rect_images.append(new_image)
rect_coords.append(rects)
park.show_images(rect_images)
delineated = []
spot_pos = []
for image, rects in zip(test_images, rect_coords):
new_image, spot_dict = park.draw_parking(image, rects)
delineated.append(new_image)
spot_pos.append(spot_dict)
park.show_images(delineated)
final_spot_dict = spot_pos[1]
print(len(final_spot_dict))
代码分解:
利用inRange 读图片进行处理,过滤掉一些复杂的背景。
white_yellow_images = list(map(park.select_rgb_white_yellow, test_images))
def select_rgb_white_yellow(self,image):
#过滤掉背景
lower = np.uint8([120, 120, 120])
upper = np.uint8([255, 255, 255])
# lower_red和高于upper_red的部分分别变成0,lower_red~upper_red之间的值变成255,相当于过滤背景
white_mask = cv2.inRange(image, lower, upper)
self.cv_show('white_mask',white_mask)
# 与操作,和原图做出位运算
#dst = cv.bitwise_and(src1, src2[, dst[, mask]]
#src1:图1 src2:图2 mask:图1和图2’与’操作的掩码输出图像
masked = cv2.bitwise_and(image, image, mask = white_mask)
self.cv_show('masked',masked)
return masked
转换为灰度图
gray_images = list(map(park.convert_gray_scale, white_yellow_images))
def convert_gray_scale(self,image):
return cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
利用canny进行边缘检测
edge_images = list(map(lambda image: park.detect_edges(image), gray_images))
def detect_edges(self,image, low_threshold=50, high_threshold=200):
return cv2.Canny(image, low_threshold, high_threshold)
为了简便我们需要手动找到我们需要识别的区域,红色区域是我们需要识别的区域。利用红色的点裁剪出我们需要识别的区域。
roi_images = list(map(park.select_region, edge_images))
def select_region(self,image):
"""
手动选择区域
"""
# first, define the polygon by vertices
rows, cols = image.shape[:2]
pt_1 = [cols*0.05, rows*0.90]
pt_2 = [cols*0.05, rows*0.70]
pt_3 = [cols*0.30, rows*0.55]
pt_4 = [cols*0.6, rows*0.15]
pt_5 = [cols*0.90, rows*0.15]
pt_6 = [cols*0.90, rows*0.90]
vertices = np.array([[pt_1, pt_2, pt_3, pt_4, pt_5, pt_6]], dtype=np.int32)
point_img = image.copy()
point_img = cv2.cvtColor(point_img, cv2.COLOR_GRAY2RGB)
for point in vertices[0]:
cv2.circle(point_img, (point[0],point[1]), 10, (0,0,255), 4)
self.cv_show('point_img',point_img)
return self.filter_region(image, vertices)
cv2.fillPoly(mask, vertices, 255) # 在mask上画多边形,由这vertices的点组成的,填充为白色
cv2.bitwise_and #过滤掉其他部分我们不需要的部分
def filter_region(self,image, vertices):
"""
剔除掉不需要的地方
"""
mask = np.zeros_like(image)
if len(mask.shape)==2:
cv2.fillPoly(mask, vertices, 255)
self.cv_show('mask', mask)
return cv2.bitwise_and(image, mask)
利用 hough_lines粗略检测直线的端点
list_of_lines = list(map(park.hough_lines, roi_images))
def hough_lines(self,image):
#输入的图像需要是边缘检测后的结果
#minLineLengh(线的最短长度,比这个短的都被忽略)和MaxLineCap(两条直线之间的最大间隔,小于此值,认为是一条直线)
#rho距离精度,theta角度精度,threshod超过设定阈值才被检测出线段
return cv2.HoughLinesP(image, rho=0.1, theta=np.pi/10, threshold=15, minLineLength=9, maxLineGap=4)
过滤到一部分不是车位线的线
line_images = []
for image, lines in zip(test_images, list_of_lines):
line_images.append(park.draw_lines(image, lines))
park.show_images(line_images)
def draw_lines(self,image, lines, color=[255, 0, 0], thickness=2, make_copy=True):
# 过滤霍夫变换检测到直线
if make_copy:
image = np.copy(image)
cleaned = []
for line in lines:
for x1,y1,x2,y2 in line:
if abs(y2-y1) <=1 and abs(x2-x1) >=25 and abs(x2-x1) <= 55:
cleaned.append((x1,y1,x2,y2))
cv2.line(image, (x1, y1), (x2, y2), color, thickness)
print(" No lines detected: ", len(cleaned))
return image
对每一列车道线坐标进行排序,分出一个一个区域,并且画出矩形框。
rect_images = []
rect_coords = []
for image, lines in zip(test_images, list_of_lines):
new_image, rects = park.identify_blocks(image, lines)
rect_images.append(new_image)
rect_coords.append(rects)
park.show_images(rect_images)
Step1部分和上面代码重复代码,可以不用管。(为了方便大家清楚流程)
def identify_blocks(self,image, lines, make_copy=True):
if make_copy:
new_image = np.copy(image)
#Step 1: 过滤部分直线
cleaned = []
for line in lines:
for x1,y1,x2,y2 in line:
if abs(y2-y1) <=1 and abs(x2-x1) >=25 and abs(x2-x1) <= 55:
cleaned.append((x1,y1,x2,y2))
#Step 2: 对直线按照x1进行排序
import operator
list1 = sorted(cleaned, key=operator.itemgetter(0, 1))
#Step 3: 找到多个列,相当于每列是一排车
clusters = {}
dIndex = 0
clus_dist = 10
for i in range(len(list1) - 1):
print(list1[i+1][0])
print(list1[i ][0])
distance = abs(list1[i+1][0] - list1[i][0])
if distance <= clus_dist:
if not dIndex in clusters.keys(): clusters[dIndex] = []
clusters[dIndex].append(list1[i])
clusters[dIndex].append(list1[i + 1])
else:
dIndex += 1
#Step 4: 得到坐标
rects = {}
i = 0
for key in clusters:
all_list = clusters[key]
cleaned = list(set(all_list))
if len(cleaned) > 5:
cleaned = sorted(cleaned, key=lambda tup: tup[1])
avg_y1 = cleaned[0][1]
avg_y2 = cleaned[-1][1]
avg_x1 = 0
avg_x2 = 0
for tup in cleaned:
avg_x1 += tup[0]
avg_x2 += tup[2]
avg_x1 = avg_x1/len(cleaned)
avg_x2 = avg_x2/len(cleaned)
rects[i] = (avg_x1, avg_y1, avg_x2, avg_y2)
i += 1
print("Num Parking Lanes: ", len(rects))
#Step 5: 把列矩形画出来
buff = 7
for key in rects:
tup_topLeft = (int(rects[key][0] - buff), int(rects[key][1]))
tup_botRight = (int(rects[key][2] + buff), int(rects[key][3]))
cv2.rectangle(new_image, tup_topLeft,tup_botRight,(0,255,0),3)
return new_image, rects
对车位线进行微调,并根据车位线的间隔分割出车位线,并建立索引。
delineated = []
spot_pos = []
for image, rects in zip(test_images, rect_coords):
new_image, spot_dict = park.draw_parking(image, rects)
delineated.append(new_image)
spot_pos.append(spot_dict)
def draw_parking(self,image, rects, make_copy = True, color=[255, 0, 0], thickness=2, save = True):
if make_copy:
new_image = np.copy(image)
gap = 15.5
spot_dict = {} # 字典:一个车位对应一个位置
tot_spots = 0
#微调
adj_y1 = {0: 20, 1:-10, 2:0, 3:-11, 4:28, 5:5, 6:-15, 7:-15, 8:-10, 9:-30, 10:9, 11:-32}
adj_y2 = {0: 30, 1: 50, 2:15, 3:10, 4:-15, 5:15, 6:15, 7:-20, 8:15, 9:15, 10:0, 11:30}
adj_x1 = {0: -8, 1:-15, 2:-15, 3:-15, 4:-15, 5:-15, 6:-15, 7:-15, 8:-10, 9:-10, 10:-10, 11:0}
adj_x2 = {0: 0, 1: 15, 2:15, 3:15, 4:15, 5:15, 6:15, 7:15, 8:10, 9:10, 10:10, 11:0}
for key in rects:
tup = rects[key]
x1 = int(tup[0]+ adj_x1[key])
x2 = int(tup[2]+ adj_x2[key])
y1 = int(tup[1] + adj_y1[key])
y2 = int(tup[3] + adj_y2[key])
cv2.rectangle(new_image, (x1, y1),(x2,y2),(0,255,0),2)
num_splits = int(abs(y2-y1)//gap)
for i in range(0, num_splits+1):
y = int(y1 + i*gap)
cv2.line(new_image, (x1, y), (x2, y), color, thickness)
# self.cv_show("AA",new_image)
if key > 0 and key < len(rects) -1 :
#竖直线
x = int((x1 + x2)/2)
cv2.line(new_image, (x, y1), (x, y2), color, thickness)
# 计算数量
if key == 0 or key == (len(rects) -1):
tot_spots += num_splits +1
else:
tot_spots += 2*(num_splits +1)
# 字典对应好
if key == 0 or key == (len(rects) -1):
for i in range(0, num_splits+1):
cur_len = len(spot_dict)
y = int(y1 + i*gap)
spot_dict[(x1, y, x2, y+gap)] = cur_len +1
else:
for i in range(0, num_splits+1):
cur_len = len(spot_dict)
y = int(y1 + i*gap)
x = int((x1 + x2)/2)
spot_dict[(x1, y, x, y+gap)] = cur_len +1
spot_dict[(x, y, x2, y+gap)] = cur_len +2
print("total parking spaces: ", tot_spots, cur_len)
if save:
filename = 'with_parking.jpg'
cv2.imwrite(filename, new_image)
return new_image, spot_dict
将建立好的字典序列存入文件中方便下次使用
with open('spot_dict.pickle', 'wb') as handle:
pickle.dump(final_spot_dict, handle, protocol=pickle.HIGHEST_PROTOCOL)
park.save_images_for_cnn(test_images[0],final_spot_dict)
利用vgg进行预测
深度学习模块我就不做过多讲解。我附上代码有兴趣的可以去看看。
总结
多刊,多读,多写,代码和算法能力就起来了。