Automatic control of model helicopter flight by machine vision

Computer Engineering Master Thesis (M.Sc)

Abstract:

Today, automatic control is widely used in transportation systems. There are several reasons for using this method, the most important of which are cost reduction, the ability to make devices lighter, more safety for the passengers or to keep them away from danger. The last two cases are more visible in systems that have the ability to fly. In this thesis, a helicopter flight control system is designed, which has the capability of remote control based on image processing. The model helicopter is tracked using a camera connected to a computer, using image processing algorithms. After tracking the helicopter, the commands are sent from the computer to the hardware at a transmission rate of 9600 frames per second. The hardware made in this project has a hand-made board to digitize the analog parameters of the radio control as well as the spectrum for receiving commands from the computer. The helicopter is detected using image processing and color tracking algorithms, then the position of the X,Y direction of the helicopter is determined on the screen and displayed. In order to perform high-speed tracking, the methods used for object detection as well as thresholding and segmentation have been used. The detection of the helicopter in the frame was done by the computer webcam and the Microsoft Kinect camera. The implementation of the required programs and algorithms has been done in C++ language in Visual Studio 2013 environment. The image processing was done by the cvInRenge color tracking function available in the OpenCV image processing library. The computer detects the flying helicopter by means of the webcam connected to it, then the computer sends the speed and left and right commands to the radio control through the serial port so that the helicopter is placed in the center of the frame.

Key words: machine vision, helicopter detection, color recognition, radio control, image processing library

Chapter 1

  General Research

Research overview

The importance of computer vision can be seen through many applications in life. Computer vision has applications such as cancer diagnosis, security programs, biometrics and other things. Image processing today is mostly referred to as digital image processing, which is a branch of computer science that deals with processing digital signals that represent images taken with a digital camera or images scanned by a scanner. Image processing has two main branches: image enhancement and machine vision. Image enhancement includes methods such as using faders and contrast enhancement to improve the visual quality of images and ensure they are displayed correctly in the destination environment (such as a printer or computer monitor), while machine vision deals with methods that can be used to understand the meaning and content of images to be used in tasks such as robotics. The sequence of video images has a specific purpose that must be done with the desired accuracy. Although the history of creating the phenomenon of object tracking goes back to military issues, today, due to the very wide applications of object tracking in various fields, for example, traffic control and detection of unusual movements, this category and its various aspects have received special attention. Among the issues that have always made the performance of tracking algorithms difficult are their interaction with target recognition methods, variable appearance of targets, and simultaneous tracking of multiple targets. In this research, a method with higher productivity and more accurate than the previous methods for tracking objects using webcam imaging is introduced. In this system, the video images taken from the flight of the model helicopter are processed, identified and extracted. The proposed algorithm is divided into several. First, it detects the static background and removes the noise from it. This background is used to subtract moving objects. After that, during a stage of image filtering, the shadows and noises of the filmed image are removed, and finally, moving objects or moving objects are separated, identified and tracked using the bubble routing method. The testing of this system was done on images of a person in the yard and city roads.. The experimental results show that the proposed model for detecting moving objects and tracking them works well and can improve the estimation of movement and movement path of objects in terms of speed and accuracy. Object tracking is used in autonomous aerial vehicles (or drones). Another method used to track aerial vehicles is color recognition, which is used using an image processing algorithm to identify color.

Definition of the problem and research objectives

The purpose of this research project is to create a system to control an unmanned helicopter model to place the helicopter in the center of the image received from the webcam connected to the computer. be This helicopter is tracked using a webcam connected to the computer and sends commands to the helicopter in the shortest possible time. Commands include speed and direction. Therefore, it is necessary to use a suitable tracking method to find the helicopter. The final result of this project will be an automatic control helicopter, and perform some pre-defined movements.

Objectives

The main objectives of this project are:

Connecting the radio control board of the model helicopter to the computer to digitize the analog radio control.

Identifying and tracking the helicopter by image processing to determine the position and orientation.  

Sending automatic commands from the computer to the helicopter and placing the helicopter in the center of the frame.

Helicopter landing at the point determined by clicking the mouse on the image frame.

Organization of the thesis

In chapter 2, some of the background related to the project is introduced and presented, and the review of previous work done in this field is discussed. In this section, with the aim of some insights, we discuss methods such as subtraction of two images and color identification algorithms that are used for this project. The tools and design used for this project as well as the required software, such as libraries and algorithms, the hardware required to carry out the project are explained in chapter 3. The implementation of the project is mentioned in chapter 4, which shows the technical explanation of the methods used to carry out this project. The system test and the obtained results are given in chapter 5 and each test is described along with its result. The results of the analysis and the capabilities and limitations of the system are discussed, and finally, the conclusion is given in chapter 5, which summarizes the results and implications of this system and future work in chapter 5. In the helicopter class, it pays to be there. The main focus of this project is on image processing and color tracking of the helicopter.

Autonomous vehicles

Helicopter control is more difficult than other vehicles due to its complexity [1]. Helicopters are very useful vehicles, however, many real-world applications such as autonomous helicopters have many advantages and uses. They can be used for safety, including assisting pilots in the event of loss of control or providing airspace surveillance. They can also be used to monitor environmental factors, such as volcanoes or weather conditions. Another common use is in dangerous situations such as going into mines or destroying them or moving in contaminated spaces. [1]. At the same time, it can be used in systems used on the surface of seas and oceans. Intelligence and control of autonomous vehicles are generally divided into three categories: internal intelligence, external intelligence, and a combination of internal intelligence and external intelligence. Internal intelligence is when all tracking and navigation is done inside the vehicle. The big challenge in them is to create an autonomous vehicle that can find and move its way on a road in a certain amount of time. For example, the vehicle named Stanley has a GPS system, camera, antenna and connected sensors, and all its intelligence is done inside the system [21]. As another example, the intelligence within the system of an autonomous helicopter model was developed as part of a doctoral thesis [3].