Improving the construction and composition of fuzzy rules using the colonial competition algorithm

Master's Thesis in Computer Engineering (Artificial Intelligence)

Abstract

Extracting general [1] and comprehensible classifications from data plays an important role in many areas and problems. So far, several methods for classification [2] and pattern recognition [3] have been introduced. One of the successful and unique methods in the field of classification and pattern recognition of input data is the use of fuzzy techniques for the soft division of the feature space and of course the use of an effective architecture in connecting these subspaces for fuzzy decision making and classification.  Being able to extract the best and most efficient fuzzy rules from the data is still a very important field for researchers.

In this study, a new method for weighting the fuzzy rules using the evolutionary algorithm of colonial competition is presented so that more important rules can be considered using more optimized weights. In this thesis, the operators of the colonial competition algorithm are redefined to make suitable fuzzy rules.

In fact, the Ishibuchi technique is presented for the first phase, i.e. the generation of rules, and the colonial competition technique is presented for the second phase, i.e. weighting them. In the next step, the generation and evolution of fuzzy rules with the colonial competition algorithm is proposed. This method increases the efficiency of the classifier for the classification rate. Finally, the goal is to build a compact rule set with a small number of rules, which have a short length and therefore high interpretability.

The proposed algorithm is compared and evaluated with non-fuzzy basic classifiers such as SVM, C4.5, 1NN and Naive Bayes and the fuzzy classifier algorithms that will be explained.

Keywords: classification, pattern recognition, colonial competition algorithm, classifier Fuzzy classifiers, non-fuzzy classifiers, rule weighting. Chapter 1. Introduction (images are available in the main file) 1. Introduction

In this chapter, it provides a general description of the motivation for choosing the topic, fuzzy classifiers, as well as a description of the problem, applications, and challenges. At the end of the chapter, the objectives of the thesis are mentioned in summary.

1-1. Introduction

Until now, data mining scientists have made many efforts to correctly separate similar samples. Extracting common classifications [1] and understandable from data is an important role in many areas and problems. So far, several methods for classification [2] and pattern recognition [3] have been introduced. One of the successful and unique methods in the field of classification and pattern recognition of input data is the use of fuzzy techniques for the soft division of the feature space and of course the use of an effective architecture in connecting these subspaces for decision making and fuzzy classification. Fuzzy classification is the process of grouping elements within fuzzy sets with a membership function[4][1]. In fact, first the search space is divided into parts so that the whole space is covered and then a fuzzy set is placed on each of these subspaces. A community of fuzzy sets called fuzzy space defines fuzzy linguistic values ??or fuzzy classes to which an object can belong. After that, if and then fuzzy rules [5] are generated according to the allocation method. The modeling of fuzzy systems is displayed as a set of these rules.

1-2. Motivation

Fuzzy classifiers have the unique property of interpretability and are able to represent the knowledge of how to recognize patterns for an expert as an instruction. Fuzzy classifiers pursue four basic goals. Maximize the accuracy of the classifier, create a classifier with the most interpretability, maximize the stability of the classifier and reduce the sensitivity to noise. So far, different methods have been presented for creating rules, how to allocate subspaces, how to infer each rule, and finally how to merge rules. It is obvious that natural language[6] is the focus of fuzzy rule structure despite knowledge extraction, the problem of mathematical proof of classifier efficiency including providing an upper bound[7] for training error[8] and testing error[9] is.In other words, increasing the generalization [10] of these classifiers mathematically like the group reinforcement classifier [11] is a very difficult task. Therefore, the methods of revelations [12] and super revelations [13] are often used in the form of trial and error in formulating rules and their integration, because they search the subspace to obtain the best combination of rules [2]-[4]. Ishiboshi[14][5] presented a method for space allocation in the form of regular and repetitive division, which can be called as one of the most effective fuzzy classifier methods, which became the basis of many subsequent researches in this field.

1-3. Problem description

The learning process of a fuzzy classification system must solve various problems in order to create a language classification system with a correct behavior. including being able to create sets of fuzzy rules that have a necessary level of cooperation between these fuzzy rules. 2- Selecting an inference function that selects a method to combine information obtained from fuzzy rules in classifying samples. 3- In problems with high dimensions, fuzzy rules suffer from exponential growth in their size. The first two problems are related to the process of knowledge extraction, which can be solved by different learning processes based on repetitive algorithms such as artificial neural networks [5-6] or genetic algorithm [2-4]. The third option can be managed in two ways: by compressing and reducing the set of rules, removing unnecessary rules with the aim of creating a classification system with higher efficiency. And the second solution is done with feature selection process.

In general, the goal of the problem is to provide a general framework for the evolution of fuzzy rules. Many solutions have been proposed in this field, but all of them differ in at least one of the following, the number of rules that are coded in each member of the population, the type of expression of the coded rules in each member, and the type and purpose of the evolutionary process. [7-8] These algorithms include genetic algorithms [15], particle group optimization [16], simulated fusion [17] and so on.

Since the evolutionary algorithms [18] multifactorially [19] perform the search in the feature space, their circulation is as random as possible. These properties have turned evolutionary algorithms into powerful tools for all kinds of optimization problems. [2], [4] Among the problems raised in the field of optimization, optimization of the structure and parameters of classifiers. Obviously, the more parameters a classifier has, the optimal setting of these parameters manually is very difficult, and in some cases impossible. Therefore, evolutionary algorithms have been widely used to learn parameters and determine the structure of different classifiers. Among these researches, we can mention the improvement of neural network structure by genetic algorithm [9], which genetic algorithm tries to prune the connection between neurons and layer them in a way to improve the efficiency of classification.

The advantage of combining fuzzy rules and evolutionary algorithms is that the set of created rules are more interpretable and can deal with uncertainty [20] and ambiguity, and they can also search the feature space in an exploratory manner. For example, evolutionary algorithms have been used in the input section on how to allocate spaces and determine the parameters of membership functions (such as slope and variance) [10].

1-4. Challenges

According to the fact that most of the major and well-known methods of evolutionary calculations are computer simulations of natural and biological processes, in this paper, a hybrid method for improving fuzzy classifiers is presented, which is adapted from the evolutionary algorithm of colonial competition [11] to improve the learning of its parameters. This thesis presents the imperial competition algorithm [21] for the purpose of extracting common and understandable classifiers from data in the form of a rule system. In this research, we are trying to present a new structure on the fuzzy platform, in which the structure, distribution of rules is adapted from the colonial competition algorithm[22], but the spirit of the rules is fuzzy. In addition, due to the creation of appropriate harmony in the optimization of the structure of laws and also the integration of laws, it is suggested to use the optimization algorithm of colonial competition.

In this algorithm, some examples that have a high degree of fitness [23] (imperialist [24]) and are the center of empires, try to pull the rest of the examples (colony) [25] towards them.