Tracking multiple moving objects in moving camera images

Master thesis in electrical engineering (communication-system)

Abstract

Tracking multiple moving objects in moving camera images

The subject of tracking one target One of the most important issues in the science of machine vision. The problem of tracking multiple targets in images obtained from a moving camera has been evaluated in this thesis. To solve this problem, two general methods are presented in this thesis. The first method is mainly based on block matching algorithms. In this method, by using motion compensation and categorizing different motions in the image, the target and background areas will be determined. Then the tracking operation has been done using the Kalman filter. According to the obtained results, this method has the ability to track targets with high accuracy in most situations. The second proposed method is basically based on the well-known point matching algorithm called SIFT. In this method, by making a match between the key points of the target area and the corresponding points in the next frame, an attempt is made to find the target area in successive frames. The problem of tracking the target areas in the frame sequence is done by Kalman filter. Due to the fact that the target is checked locally in this method, this method has great ability in different tracking situations such as fading. According to the graphs and tables, it can be concluded that this method has a much stronger performance than most existing methods in terms of accuracy in many tracking cases. At the same time, with the introduction of technology into the personal life of society, the presence of devices and tools that play the role of the interface between man and machine is felt more and more day by day. An example of these devices are cameras[1]. The widespread use of these tools in today's societies, especially in more industrialized countries, is undeniable. Therefore, today, improving the quality and facilities of cameras is considered as an important factor in increasing their efficiency. One of the most important branches of science that investigates these issues is called machine vision[2].

One ??of the main goals of machine vision is to make cameras intelligent in order to use them in surveillance systems[3], commercial, military and other applications. For this purpose, extensive studies have been conducted in order to create new methods of intelligentization and also to improve existing methods. Most of these studies are focused on the detection [4] and tracking [5] of targets [6]. The general purpose of conducting such studies is to reduce the volume of calculations and increase the accuracy in the stages of detection and tracking. In general, target detection means detecting a region of the image that can be considered as a candidate [7] for the target region. For example: specifying the areas of the image that are related to the license plate, or also revealing the areas of the image that can be considered as areas related to the human face. Also, the purpose of tracking is to specify the desired area in the set of consecutive frames. In this way, the general direction of the target will be determined in a sequence of time during successive frames.

In the following, we will briefly introduce different tracking systems and their components as well as their operation.

1-1-1-The structure of tracking systems

Different tracking systems are divided into different categories based on their applications. Cameras and targets are the main components of such systems. Therefore, just as these components play a decisive role in the type of tracking systems, they are also very important in determining the type of methods used in these systems. The systems have significant differences based on the number, type, and other conditions of the cameras and objectives. Likewise, these differences can be seen in the tracking methods used in them.

1-1-1-1-camera

The camera, as the main component of the tracking system, has the task of creating a sequence of frames over time. The type of cameras used, the number and the way they are placed will play a big role in determining the appearance of the frames. This effect is sometimes to the extent that it creates methods with a different foundation.

As an example, the tracking methods in systems with visible cameras [8] are completely different from the methods used in systems with infrared cameras [9]. This problem comes from the fact that infrared cameras have a kind of prior information [10] about the targets. This means that in the images obtained from these cameras, the targets have stronger color intensity [11] than their surroundings. As a result, they have more detection capabilities. Although some of the presented algorithms can be applied to images obtained from both visible and infrared cameras, the efficiency of these algorithms in these images is significantly different.

In addition, the number of cameras used is also one of the most important factors in determining the method used in tracking. The difference in the viewing angle of the cameras creates different images from different angles of a certain scene. In this situation, it is necessary to find the corresponding points in the frames obtained from all cameras and also to calibrate [12] the cameras. It can be seen that these methods are generally different from tracking methods based on a single camera.

In addition to the above, camera motion[13] must also be considered in some cases. This means that sometimes, in addition to the targets, the camera also has movement. In these cases, the components in the consecutive frames move relative to each other. This is despite the fact that some of these movements are due to the movement of the camera and some are also due to the presence of movement in objects. Therefore, the ultimate goal is to distinguish between the movements that are caused by the camera and the movements that are real. The need to perform this action is one of the main things that is not considered in tracking targets in fixed camera systems. rtl;">

Visual object tracking is perceived as one of the most important subjects in Computer Vision. The problem of the multi-object tracking in image sequences captured from a moving camera is evaluated in this thesis. To address the problem, two distinct methods are proposed. The first method is mainly based on Block Matching Algorithm (BMA) in which the target region and background are distinguished using classified motion vectors obtained from Motion Compensation. At the end, tracking is done by using Kalman Filter. According to the results, this approach can obtain a high level of accuracy for object detection in many situations. The second proposed method is particularly based on a well-known points matching algorithm which is called SIFT. In this approach, finding target regions in consecutive frames is accomplished by applying match between the key-points of target region and target candidate region in the next frame. The Kalman Filter plays a significant role in determining the target candidate region in the next frame. Since the target is evaluated locally, this approach can effectively handle a wide variety of situations such as occlusion. Figures and tables illustrate that the mentioned method can outperform existing tracking techniques in terms of accuracy.