chapter 3: evolutionary fuzzy...

Chapter 3: Evolutionary Fuzzy Modeling

73

A Genetic-Fuzzy Approach to Measure Multiple Intelligence


Genetic- Fuzzy Systems(GFS)

Machine Learning Models

Advantages

Limitations

Applications

Need of Research Framework


74


3.1 Chapter Overview

This chapter presents significant design views of genetic learning. In order to achieve

hybridization of Fuzzy Logic (FL) with Genetic Algorithms (GA), several approaches

have been considered. The chapter presents a detailed discussion on available models

for Genetic-Fuzzy hybridization and applications developed based on existing

approaches. The extensive review on the genetic rule learning is presented as a part of

evolutionary fuzzy modeling. The evolutionary fuzzy modeling focuses on four major

approaches of Genetic- Fuzzy model, i.e. the Michigan, the Pittsburg, the Iterative

Rule Learning (IRL) and Genetic Cooperative-Competitive Learning (GCCL). The

characteristics of all these approaches have been discussed. The chapter

accommodates a discussion on the sub components of major hybrid approaches along

with their limitations and comparisons among one another. In the area of intelligent

decision support system, major application domains are taking advantages of machine

learning methods, especially Genetic-Fuzzy hybridization. A survey during research

in the area of various designed applications is documented. This investigation covers

several important applications domains where the machine intelligence is required to

be built. The real life applications of varied domains such as classification, medicine,

control systems, robotics, travel industry, stock and share, networking, etc employ

hybrid structures of GFS in order to achieve optimized rule learning. The chapter

finally draws conclusions from the current state of the art in the application of rule

learning from Genetic-Fuzzy hybridization. Subject to the conclusion made, the

chapter justifies the scope of the research carried out and reported in this thesis. As a

result of extensive research review on Genetic-Fuzzy hybridization, it has been

observed that no generalized framework using evolutionary fuzzy approach has been

developed in the field of education to solve the problems which lack mathematical

formulation.

3.2 Importance of Soft Computing

Soft Computing (SC) is not merely a clearly defined field but also a discipline that

deals with hybrid intelligent systems [174, p.239]. SC techniques are integrated

techniques to find solutions for the problems which are highly complex, ill defined

and difficult to model. It has been found that real world problems carry rapidly

changing information as well as they are somehow imprecise and uncertain in nature.


75


To provide solutions for such kind of tasks using hard computing or traditional

computing is considered to be highly expensive and improper. It has been seen that

even after utilization of hard computing paradigm; real life applications suffer from

infeasible, non traceable and costly solutions to achieve high precision in their

outcome. This becomes the prime reason behind the transition from hard computing

to soft computing. Soft computing techniques are meant to operate in an environment

that is subjected to uncertainty and imprecision. According to Zadeh [126, p.1], the

guiding principle of soft computing is:

“Exploit the tolerance for imprecision, uncertainty, partial truth, and approximation to

achieve tractability, robustness, low solution cost and better rapport with reality.”

The family of soft computing is constructed using four prime techniques: namely

Fuzzy Logic (FL), Evolutionary Computation (EC), Neural Networks (NN) and

Probabilistic Reasoning (PR) as stated earlier in Chapter 1 (refer Figure 1.1 of

Chapter 1). Due to the characteristics of above mentioned techniques; soft computing

is distinguished from hard computing. These constituents are more flexible and robust

in providing solutions for real life problems. All four methodologies EC, FL, NN and

PR constitute the empire of soft computing which have been conceptualized and

developed before past fifty years. Each method is capable of providing distinguished

as well as sharable advantages and obviously carries certain weaknesses also. They

are considered complementary rather than competitive as desirable features lacking in

one approach are present in another. Initially, they had been applied for complex tasks

in isolation [166, p. 80]. As stated in earlier chapters, Evolutionary Computation (EC)

refers to the computer-based problem solving systems that use computational models

of evolutionary process. In order to implement evolutionary processes, EC includes

four methods namely Genetic Algorithms (GA), Evolutionary Strategies (ES),

Evolutionary Programming (EP) and Genetic Programming (GP). GA is one of the

prime optimization techniques from the tree of evolutionary search and optimization

that performs parallel, stochastic but direct search method to evolve the best solution.

Table 3.1 presents advantages and limitations of the principle constituents of Soft

computing that does provide a significant reason for inclusion of the constituents

[143, p.261 ; 63, p.390].


76


Table 3.1: Advantages and Limitations of Principle Constituents of SC

Constituents of

SC Advantages Limitations

GA Natural evolution and optimization Inability of storing and

handling imprecision

FL Approximate reasoning,

imprecision Inability of learning

NN Learning and implicit knowledge

representation

Inability for

optimization

PR Uncertainty Inability of learning

GA provides a means to encode and to evolve rule antecedent aggregation operators,

different rule semantics, rule- based aggregation operators and de-fuzzification

methods. Therefore, GAs remain today as one of the fewest knowledge acquisition

schemes available to design and in some sense, optimize Fuzzy Rule Based Systems

(FRBSs) with respect to the design decisions, allowing decision makers to decide

what components are fixed and which ones evolve according to the performance

measures [54]. Such characteristics lead to the design of an intelligent system with

GA fuzzy hybridization which is a promising research field of modern computational

intelligence concerned with the development of the next generation of intelligent

systems.

3.2.1 Hybridization in Soft Computing

Actually, soft computing constituents are derived from varied domains as logic,

biology, physiology and statistics. In the initial stage of development of soft

computing, independent methods were utilized among community of practitioners.

Their success progressively attracted researchers in the other fields also. Indeed, they

model in different extents natural processes such as evolution, learning, or reasoning

[27, p.31].

The integration of different learning and adaptation techniques is required to

overcome limitation of individual methods so that synergetic effects through

hybridization or fusion of these techniques can be made achievable. Due to the lack of


77


common framework; most of these approaches need to follow an adhoc and domain

specific design methodology. However, they are observed to be very successful but

still it becomes difficult to compare such various types of hybrid systems and

evaluation of their performance [31].

3.2.2 Integration of Evolutionary Algorithms and Fuzzy Logic

Recent years have contributed to large number of new hybrid evolutionary systems.

There are several ways to hybridize a conventional evolutionary algorithm for solving

optimization problems.

Evolutionary computing is based on Evolutionary Algorithms (EA). Genetic

Algorithms being one of the prominent types of EA were not specifically designed as

machine learning techniques like other approaches such as neural networks but have

been successfully applied to many search, combinatorial and optimization problems.

However, it is well known that a learning task can be modeled as an optimization

problem, and thus can be solved through evolution which is efficiently offered by EA

[55]. However, one of the significant limitations of GA; shown in the Table 3.1 is an

inability to deal with imprecision. This limitation can be solved using integration of

GA with FL. The principles and operations of Genetic Algorithm and fuzzy

computations have been described in Chapter 1 and Chapter 2 of the thesis.

Fuzzy Logic provides a mathematical way to represent and deals with vagueness of

everyday life. FL is presented not as a control methodology, but as a way of

processing data by allowing partial set membership rather than a crisp set membership

or non-membership [22].With the help of Fuzzy Logic based system; the knowledge

representation has been possible in a human understandable way using linguistic rules

to explain decision processes; but at the same time, fuzzy systems are suffering from

inability of self learning [166, p.89] as well as requires documentation of knowledge

which needs further continuous maintenance. At the other end; Genetic Algorithms

provide robust search capabilities both global and local in complex spaces whereas

Fuzzy Systems present flexible inference methods in order to deal with imprecision

and uncertainty [117]. The linguistic representation of knowledge allows a human to

interact with an FS in an intuitive, seamless manner. Hence, hybridization of FL with

GA becomes essential to achieve advantages of both the aforementioned approaches.


78


In this aspect of hybridization of GA and FL, two main hybrid approaches are

analyzed [166, pp. 86-88; 45, p.180] which are narrated as under:

1. Fuzzy Logic Assisted Evolutionary Algorithm

This approach is also known as Genetic Algorithms controlled by Fuzzy Logic. In this

approach GA utilizes FL in order to adjust its parameters or operators to improve its

performance. Inherent parameters of GA such as fitness function and stopping

criterion are fuzzified. Hence, computational resources can be optimized from over

usages. This approach is further classified into two popular and successful subtypes

which are narrated as under:

Adaptive GA that adapt control parameters

In this approach, FLCs are used to dynamically compute optimal values for the GA

parameters. The objective is to adapt GA optimization process [166, p.87]. The

example of this approach is explained as under:

The control parameters of GA such as mutation and crossover rates, population size,

etc. can be dynamically computed by Fuzzy Logic controller (FLC). Eg. H. Y. Xu and

G. Yukovich, have categorized the size of populations as small, medium and large

[90].

GA with fuzzified genetic operators

In this approach, different fuzzified versions of genetic operators have been proposed

such as fuzzy connective crossover for real-coded Genetic Algorithms [56], [60], [57]

and soft operators [85].

2. Genetic Fuzzy Systems

In this type of approach, GA is used to improve the performance of FLC. As GA is

computationally expensive, GA based tuning is generally carried out off-line but if

integration of FLC is obtained with optimization, it can be controlled online [45,

p.180]. In order to hybrid GA with FS, it is required to incorporate fuzzy knowledge

in GA. The research design focuses on genetic rule learning for fuzzified knowledge

representation. In order to achieve the same, available models of Genetic Fuzzy

systems are discussed in detail in the upcoming sections.


79


3.2.3 Genetic Fuzzy Systems

A GFS is basically a fuzzy system augmented by a learning process based on

evolutionary computation, which includes any method of EC family such as Genetic

Algorithms, genetic programming, and evolutionary strategies [5].

The most extensive GFS type is the Genetic Fuzzy Rule Based System (GFRBS),

where GA is employed to learn or tune (optimizing parameter) different components

of a Fuzzy Rule Based System (FRBS). In the design of GFS, a GA is used to look up

the performance of Fuzzy Logic controller (FLC) but the performance of FLC

depends on its Knowledgebase (KB) consisting Database (DB) and Rulebase (RB)

[45, p.170]. In order to achive design of FRBS, tasks such as designing inference

mechanism as well as generation of fuzzy rule set (KB or FRB) are required to be

satisfied [166, p.89].

As referred in Table 3.1 FRBSs are not able to learn themselves, but require the KB to

be derived from expert knowledge. In order to remove such limitation, evolutionary

learning process becomes essential to employ to automate FRBS design. By utilizing

this type of learning process FRBS can be defined automatically. The stated type of

design can be considered as an optimization or search problem. In order to solve

optimization problems, GAs are selected due to major capabilities such as:

Being global search method, GAs can explore large a search space;

Able to find near optimal solutions in complex search spaces; and

Able to provide generic code structure and independent performance.

Due to the above mentioned capabilities, it is possible to incorporate a priori

knowledge in GA which may be in form of linguistic variable, fuzzy membership

function parameters, fuzzy rules, etc. Finally, Genetic- Fuzzy system can be designed

as shown in Figure 3.1.

Figure 3.1 illustrates the general structure of Genetic Fuzzy Rule Based System. The

design of GFRBS is constituted using three layers which are explained as under.

1. Interface Layer

The bottom layer of the Genetic-Fuzzy Rule Based System is named as interface

layer. It is composed of three major components: Environment, FRBS and Output

Interface. This layer is basically structured using input interface, comparison with


80


fuzzy system and processed output of FRBS. Here, the input interface defines the

members of an application domain and interact with FL layer which is responsible for

designing and implementing the fuzzy system. The FL layer is explained as under.

2. FL Layer

The middle layer of the Genetic Fuzzy Rule Based System is named as FL layer. It

interacts with the interface layer to have input variables. This layer consists of various

components such as Fuzzification Interface, Inference Mechanism and

Defuzzification Interface. This layer is responsible to design the processes which are

related to fuzzy system implementation. The input interface decides the members or

the variables of the application domain in order to generate fuzzification. The

inference mechanism is designed using the members which are selected in the input

interface and will be involved in the process of generating the FRBS. The output

interface is responsible for defuzzification of input variables. It provides the results

produced by Fuzzy System (FS).

Figure 3.1: General Structure of Genetic Fuzzy Rule Based System

Environment Environment Comparison with Fuzzy Rule

Based Systems

Design Process

Genetic Based Learning Process

Knowledge Base= Rule Base + Data Base

Inte

rfac

e L

ayer

F

L L

a y e

r

Rep

osi

tory

Lay

er

o

Members or

Variables of

Application Domain

Input

Interface

Components

of Fuzzy

System

FRBS Output of

Processed FRBS

Output

Interface


81


3. Repository Layer

The top most layer of the Genetic Fuzzy Rule Based System is named as repository

layer. This layer is responsible for designing of GFRBS. In order to design, GFRBS,

evolutionary techniques are required to be incorporated in order to achieve automatic

generation or modification of entire part of the Knowledge Base (KB). KB is a

combination of Data Base (DB) and Rule Base (RB). The parameters of

knowledgebase include fuzzy rules and membership functions. Both the constituents

interact with inference mechanism of the middle layer. In order to achieve

optimization, it is required to find an appropriate KB.

3.2.3.1 Genetic Learning Process

In the aspect represented by the Figure 3.1, designing GFS requires to incorporate

genetic learning process for designing or optimizing the knowledgebase. GFRBS is a

design method for FRBSs which incorporates evolutionary techniques to achieve the

automatic generation or modification of the entire or part of the knowledgebase. In

order to evolve rules, GA needs to search the genotype space as well as it also

requires some mechanism generating new variants from the currently existing

candidate solutions. The objective of the search process is to maximize or minimize a

fitness function that describes the desired effectiveness of the system.

In summary, the genetic process is the result of the interaction between the evaluation,

selection and creation of genetically encoded candidate solutions, which represents

the contents of the KB of an FRBS.

Approaches for Rule Base Learning with Genetic Algorithm

Generating system which employs learning approaches is capable enough of changing

their underlying structure with the objective of improving their performance or quality

of their knowledge according to certain criteria. With reference to genetic rule

learning using GA in order to optimize FLC, major approaches are identified as

follows:

1. Approach A: GA based optimization for automatic constructed FLC

The approach “A” is mainly suitable for solving complex tasks. For a complex task, it

is very difficult to design KB manually because in such cases performance of GFS

cannot be achieved as per expectations. Automatic design of FLC through GA


82


becomes solution for complex tasks. GA utilizes information of DB, RB and

consequent part of each rule of the FLC. The predefined training set is utilized by GA

strings to optimize FLC. As a result, optimal FLC can be evolved through iterations.

To design system based on such approach is a critical task.

2. Approach B: GA based optimization of manually constructed FLC

The working characteristic of the approach “B” is based on designer‟s knowledge of

the process to be controlled. In order to process linguistic knowledge representation,

ranges of variables are decided and different linguistic terms are designed as per

requirements of the problem. The number of input combinations for the design of

FLC depends on the number of input variables and their linguistic terms. In order to

propose the DB of the FLC manually, it is needed to have distribution of membership

functions and variables by designer and appropriate design of RB is made possible,

later. This type of design of FLC may be flexible but it is not optimal as well [45,

p.174]. In this approach, a GA is used to tune the DB and/or RB of the FLC with the

help of training scenarios. After the GA-based tuning is over, the FLC will be able to

determine the output for a set of inputs within a reasonably accuracy limit.

Figure 3.2 represents parallel approaches namely “A” and “B” along with their

characteristics for developing Genetic- Fuzzy systems.

Figure 3.2: Approaches for GA based Optimization for FLC

Approach A: GA Based Optimization for Automatically

Constructed FLC

• Handles complex tasks

• Utlizes knowledge of experts of GA

• More complex

• Must generate an optimal FLC

• More time consuming process from execution of GA perspective

Approach B: GA Based Optimization for Manually Constructed FLC

• Handles easy task

• Utilzes knowledge of designer

• Less complex

• May not generate an optimal FLC

• Less time consuming process from execution of GA perspective


83


3.3 Techniques of Hybridization for GFS

Although, GA was not specifically designed for learning, they are used as global

search algorithms. Apart from the searching task, they do offer a set of advantages for

machine learning. Many methodologies for machine learning are based on the search

of a good model and they are very flexible because the same GA can be used with

different representations [54, p.33]. In a rule based system, to achieve task of learning

rules, there are two major styles available to encode rules within populations of

individuals of GA as shown in Figure 3.3.

Style 1: The ‘‘Chromosome = Rule’’ Approach

In the style 1, each individual codifies a single rule, and the whole rule set is provided

by combining several individuals in a population known as rule cooperation or via

different evolutionary runs known as rule competition [101]. For example, the

Michigan approach and the Iterative Rule Learning (IRL) approach are representative

approaches of Style 1.

Figure 3.3: Popular Styles of Rule Encoding in Genetic-Fuzzy Hybrid Systems

Style 2: The „„Chromosome = Set of Rules‟‟ Approach

In style 2, each individual represents a rule set so it is popularly known as the

“Chromosome = Set of rules”. In this case, a chromosome evolves a complete RB and

they compete among them along with the evolutionary process. E.g. the Pittsburgh

approach is the representative approach of style 2 [183].

Style 1

Chromosome="Single Rule"

Ex. The Michingan Approach

Ex. The IRL Appraoch

Style 2

Chromosome= "Set of Rules"

Ex. The Pittusburg Approach


84


In order to provide rule base learning process with different levels of complexities,

evolutionary algorithm provides three major approaches for evolving such rule

systems namely the Michigan Approach, the Pittsburg Approach, the Iterative Rule

Learning (IRL) approach as shown in Figure 3.4.

Figure 3.4: Major Approaches for Genetic Fuzzy System

3.3.1 The Michigan Approach - Classifier System

Learning Classifier System (LCS) is a machine learning technique which combines

reinforcement learning, evolutionary computing and other heuristics to produce

adaptive systems. This approach represents “Chromosome=rule” approach as shown

in Figure 3.3. In order to generate populations of rules automatically, Classifier

Systems (CSs) or production rule systems are designed [99, pp. 263-296]. Learning

Classifier Systems are a kind of rule-based system with general mechanisms for

processing the rules in parallel, for adaptive generation of new rules, and for testing

the effectiveness of existing rules [100]. These mechanisms make possible

performance and learning without the “fragile” characteristics of the most expert

systems in AI [102, p. 3].

CSs are especially parallel, message passing, rule based systems that learn through

credit assignment and rule discovery [166, p.130]. CSs typically operate in

environments that exhibit one or more of the following characteristics:

The Michigan Approach

The Pittusburg Approach

The Itreative Rule Learning (IRL)

Approach

Genetic Fuzzy System


85


Large amount of irrelevant or noisy data can be incorporated;

Uninterrupted real time required for action; and

Implicit or vague goals.

Figure 3.5 shows comparison of traditional expert system and classifier system along

with characteristics of each of them. These characteristics are narrated as under.

CSs are different than traditional expert systems by providing several advantages such

as rules are needed to design by knowledge engineer in expert system while in

Classifier System (CS), classifier rules are generated by a GA. In the expert system,

rules are stored and processed in a simple way as well as no additional parameter for

rule system are available to calculate rule strength as well as evolution of it cannot be

achieved. Each classifier has to pass through performance evaluation system and have

a strength parameter; a number is assigned to each classifier, after that. Figure 3.5

distinguishes characteristics of expert system and classifier system.

Figure 3.5: Comparison of Traditional Expert System and Classifier System

The prototype organization of Classifier Systems (CSs) is composed of three sub

systems [194, p.49] which are narrated as shown below:

Traditional Expert System

(ES)

• Rules are normally designed by knowledge engineer

• Rules have condition and action parts

• Decision making is possible

• No evolution is achieved

Classfier System

(CS)

• Classfiers are generated through GA

• Message list is associated with each of the classifiers

• Message list contains current state and internal message

• Each classifer has strength associated in form of number


86


1. A production system with a rule base which processes incoming messages from

the environment and sends output messages to the environment.

2. A design of credit system which receives pay-off from the environment and

determines which rules had been responsible for the feedback.

3. A Genetic Algorithm which recombines existing rules and introduces new rules.

Figure 3.6 illustrated above mentioned subsystems along with major task of each one.

Figure 3.6: Subsystems of Classifier Systems (CS)

After implementing the above sub-systems, the working of classifier system is

summarized as below:

The working of classifiers is strengthened through the Credit Assignment (CA)

system. Basically, a GA selects high fitness classifiers as parents forming offspring by

recombining components from the parent classifiers. Here, traditional fitness function

of GA is not used; instead the fitness of a classifier is determined by strength

calculated with the CA system. In typical CS implementations, from the set of

classifiers, high strength classifiers constitute the GA population. The strategy of GA

is to replace the worst set of classifiers by newly created strong classifiers. Due to this

strategy, the high performance of the classifier can be achieved [166, pp.132-136].

The following approaches are the fundamental approaches based on evolutionary

fuzzy modeling using the Michigan approach.

Evolutionary fuzzy modeling has since been applied to an ever-growing number of

domains, branching into areas as diverse as chemistry, medicine, telecommunications,

biology, and geophysics. Some applications are briefly narrated as follows:

The Production System

(Rule Base)

The Credit Assignment System

(Assignment of Strength to Classifier )

The Classifier Discovery System

(Learning of Classifiers through

GA)


87


A Genetic Algorithm based learning process has been proposed to determine the

optimal data base of a Fuzzy Logic controller keeping its rule base which is

previously defined [34].

In 1999, H.Ishibushi,et al.[79] have presented fuzzy classifier systems for high

dimensional pattern classification problems and in 2000, Juang et al. have presented

genetic learning for fuzzy controller design [30]. In 1999, Y.Shi et al. discussed GA

based method to evolve fuzzy expert system for large complex problems. This system

can evolve the rule set along with optimal number of rules inside it, tune the

membership functions and evolve the membership function [203].

M. Valenzuela- Rendon , has proposed linguistic RB learning using Michigan

approach [148]. This proposal presents the first GFS based on the Michigan approach

for learning RBs with DNF fuzzy rules. This approach incorporates a reward

distribution scheme that requires knowledge of the correct action, and thus, must be

considered as a supervised learning algorithm [149].

In GFS, the fuzzy inference system is represented by the entire population having

several rules participating. In order to propose the best action, these rules are in

constant competition and cooperate to form an efficient fuzzy system. Identification

of the specific rules responsible for good system behavior is a very difficult task. This

task requires design of fitness function capable of measuring goodness of a single

fuzzy rule as well as the quality of its cooperation with other fuzzy rules in the

population to give the best action as an output [27, p.33]. The design of a fitness

function of this kind is not an easy task [4]. In 1978, the learning classifier systems

based on Michigan approach are designed by J. H. Holland & J. S. Reitman [101]. In

1988, K.A. De Jong has analyzed two different approaches for learning the rule set of

production system [110]. In 1991, the significant research work on the classifier

systems has been proposed by M.Valenzuela- Rendon [148].

Optimum design of Fuzzy Logic controllers using Genetic Algorithms is presented by

[48]. Here, the GA is used to modify the fuzzy relational matrix of a one-input, one-

output fuzzy model using fuzzy relation matrix rather than the decision table. The

major limitations of The Michigan approach are presented as follows:


88


Limitations of The Michigan Approach

Michigan approach is mainly concerned with continuous (on-line) learning in

non inductive problems. One of the major limitations is the risk involved due

to random modification done by GA for applications especially designed to

achieve security [194, pp.54-55]. The reason behind this limitation is

explained as under:

Online rule learning process is achieved in which every single rule is

manipulated and hence requires an intense analysis of the performance of

every rule. Such a system can adapt to varying environmental circumstances

automatically. In this case, the rule is modified every time.

The other limitation observed is: the Michigan approach based system may

fail if working with complex environment; i.e. there is only a low probability

that important state sequences are observed repeatedly.

Further, this approach represents the knowledge of a single entity that learns

through interaction with the environment and later being adapted to it rather

than to evolution of possible solutions in form of rules [166, p.142].

3.3.2 The Pittsburgh Approach

It represents “Chromosome is Rule set” approach as shown in Figure 3.3. The central

idea is to impart intelligence through evolution. In this approach, an evolution can be

generated through competition among the individuals and adaptation to the

environment [166, p.142].

This approach is particularly tailored for training in both inductive and non inductive

problems. In the Pittsburgh approach, each individual represents a complete entity of

knowledge and due to this type of structure; different individuals do not require

interacting with one another for the evaluation of the knowledge. Hence, the need of

credit assignment, and, thus the definition of complex algorithms with that purpose

are not required. The individuals responsible in generating the evaluation are assigned

the performance measure. Figure 3.7 represents the block diagram of subsystems of

Pittsburg rule learning approach.


89


Figure 3.7: The Subsystems of the Pittsburg Rule Learning Approach

The major components of Figure 3.7 are described as follows:

Rule base and the Population of RBs

In order to generate the learning process, it is required to generate a population of

potential solutions to the problem. The population of potential solutions is designed

from RBs through a common processing structure to solve a specific problem. Each

RB in the population is evaluated by applying it to solve the problem. Feedback is

generated from the environment. Each RB is evaluated independently and no

interaction between individuals of the population occurs during the evaluation. To

start the learning process, an initial population of RB is required. In some cases, it is

obtained from available knowledge, while in other cases; it is randomly generated

[166, pp.143-144].

The Evaluation System

As stated in the Pittsburg system, each RB is to be evaluated. This evaluation of RB is

based on the effect of the interaction of the rule based system, applying the

corresponding RB, with the environment. As a result of this interaction, the

environment generates a feedback that is used by the evaluation system to generate

the evaluation of the RB. The evaluation is quite different depending on the

application and the environment [166, p.144]. The evaluation system often becomes

the most time consuming element of the process.

Comparatively, this approach offers more simplicity compared to the Michigan

approach by producing independent evaluation of population. Here, larger

Rulebase System

Population of RB

RBs Discovery

System

Evaluation System

EnvironmentFeedback


90


computational efforts for independent evaluations are required; hence this approach is

free from conflicts, generated as a result of interaction.

The Rule Base Discovery System

Once the process of evaluation of population of RB is completed, new RBs are to be

searched, and, RB discovery system is to be introduced. This system generates a new

population along with set of genetic operator to the previous generation [166, p.144].

The rule base discovery system for the Pittsburg approach is different from the

Michigan approach by the following characteristics:

The Level of Replacement

In the Michigan approach, the number of replaced individuals at each generation has

to be low enough to preserve the system performance, as it is the result of the

interaction between the individuals whereas in Pittsburgh approach the performance

of the best individual is achieved and consequently due to that performance, there can

remain steady as long as the best individual is maintained.

The Timing of Evaluation and Discovery

In the Michigan approach, discovery is applied with a lower frequency than credit

assignment. In this approach, continuous learning is made possible to reach a steady

state situation before creating a new generation while the Pittsburg approach

implements predefined training cycle which has to be applied for each individual in

the population. Consequently, the discovery phase takes place after a complete

training cycle for each individual [166, pp.144-145].

The Pittsburg approach facilitates to include additional optimization criteria in the

fitness function, thus affords the implementation of multi-objective optimization. In

1991, Thrift [172] has presented a pioneer work on the Pittsburgh approach for

learning Rule Base. This method works by using a complete decision table that

represents a special case of crisp relation defined over the collections of fuzzy sets

corresponding to the input and output variables. Finally, the GA employs an integer

coding.

Summarizing the above mentioned characteristics of the Michigan and the Pittsburg

approaches, Figure 3.8 is constructed which shows distinguishable characteristics of

the Michigan approach and the Pittsburg approach:


91


Figure 3.8: Comparison of the Michigan and the Pittsburg Approaches

Different kinds of Rule based systems have been designed and evolved using this

approach among them significant examples are presented as follows:

Concept learning can be evolved by GABIL system which consists of RBs

with boolean rules [111]. This system continuously learns and refines concept

classification rules form its interaction with environment.

Genetic Based Inductive Learning (GIL) systems designed by C.Z.Janikow,

utilize coding of RBs using logical representation for learning concept

descriptions [38].

Classification with real valued representation for continuous variables that

utilizes RBs using Genetic Algorithm is achieved [15].

Evolutionary learning of fuzzy rules (ELF) method is proposed and applied to

the problem of guiding an autonomous robot by A. Bonarini in 1993. The key

The Michigan Approach

It operates on single rule in an

online process or a simulated

environment

It concerns with online learning in

non inductive problems

Candidate solution is embeded in

performance system

The whole population consistutes

RB and single entity is evaluated

through performance system

Complexity of fitness evaluation

is high

The PittusBurg Approach

It operates on multiple rules constitued in FRBS

It concerns with training of

inductive and non inductive

problems

Candidate solutions exist in a

seperate entity

Entire population of RB is

evaluated once at a time &

feedback is assigned to RB under

evaluation

Complexity of fitness evaluation

is low


92


issue of ELF is to find a small rule base which only contains important rules

[4].

Limitations of The Pittsburg Approach

The Pittsburg approach evaluates the population of the entire fuzzy system for every

generation. It evaluates fitness for each RB of the population and repeats the same

process for the entire RB, thus demanding more computational resources. Hence, the

computational cost of this approach is high, which is the major limitation of the

Pittsburg approach [27, p.34].

3.3.3 The Iterative Rule Learning Approach (IRL)

The major drawback of the Michigan approach and the Pittsburg approach is the

consumption of huge amount of computer memory for searching numerous fuzzy

rules. To overcome the above stated problem, an iterative rule learning approach

(IRL) is designed. The IRL (Iterative Rule Learning) approach is based on the

approach of “Chromosome is rule”. The prime reason to develop this approach is to

integrate the best features of the Michigan and the Pittsburg approaches. In this

approach, a new rule is added to the rule set, in an iterative fashion, for every run of

GA. The Itreative Rule Learning (IRL) approach was first proposed by Supervised

Inductive Algorithm (SIA) designed by G.Venturini [68] in 1993 and it has been

widely developed in the field of GFRBs [8, 13,162,165].

This approach works by combining the styles of the Michigan and the Pittsburg

approaches. Similar to the Michigan approach, each chromosome in the population

represents a single rule, but similar to the Pittsburg approach, only the best individual

are considered to form part of the final solution. As a result of this type of

computation, the generated RB ultimately discards the remaining chromosomes in the

population. Therefore, in the iterative model, the GA provides a partial solution to the

problem of learning, and, it is repeated multiple times to obtain the complete set of

rules. Hence, the chances of generating best solution are possible [166, pp.148-149].

3.3.3.1 Multistage Processes of Iterative Rule Learning

The learning process utilizing Iterative Rule Learning (IRL) is composed of two

major stages. This type of GFRBSs based on IRL approach is popularly known as


93


multi-stage GFRBSs [8]. Figure 3.9 represents two pioneer stages required to develop

IRL.

Figure 3.9: Different Stages of IRL

In order to generate an RB, which constitutes a true solution to the learning problem,

the GA is embedded into an iterative scheme similar to the following [194, pp.43-44]:

Step 1: Use a GA to obtain a rule for the system.

Step 2: Incorporate this rule into the final set of rules.

Step 3: Penalize this rule.

Step 4: If the set of rules generated till now is adequate to solve the problem,

end up returning it as the solution. Otherwise, return to step 1.

An evolutionary algorithm is used to find a single rule, thus providing a partial

solution. The evolutionary algorithm is then used iteratively for the discovery of new

rules, until an appropriate rule base is built. A penalization process is applied each

time a new rule is added. As an effect of this process, redundant rules are avoided [27,

p.34]. Chromosomes compete with every GA run, choosing the best rule per run. The

global solution is formed by these best rules. Learning algorithms that use the IRL

approach do not imagine any relationship between them in the process of obtaining

rules. Therefore, the final set of rules usually needs an a posteriori process which will

modify and/or fit the said set [45, pp. 33-34].

In order to implement a learning algorithm based on GAs using the IRL approach,

following sub components are required [166, p.150].

A Generation Process

• Representation of knowledge in

data set through chromosome

• Execution of Iterative covering

method

• Execution of Fuzzy rule

generating method frequently to

find best rules

A Post Processing Process

• Simplification of the rule set

• Selection and refinement of

rules

• Refinement of previous rule set

to remove redundent rules

• Demonstration of best

performance of final rule set


94


Figure 3.10 represents sub components of IRL which are discussed as follows:

Figure 3.10: Sub Components of IRL

1. A criterion for selecting the best rule in each iteration

This component is used to determine the good rules. The selection criteria about the

rule strength are also presented which includes number of examples covered, criteria

of consistency of the rule or criteria of simplicity.

2. A Penalty Criterion

This component of criterion is often associated, although it is not necessary, with the

elimination of the examples covered by the previous rules.

3. A Determination Criterion

This component is used to determine the assurance about the completeness of the set

of rules. It starts working when enough rules are available to represent the examples

in the training set. The main functionality of this component is to check whether all

the examples in the training set are sufficiently covered or not.

Advantages offered by IRL Approach

The significant advantage of IRL is that it reduces the search space, because in

each sequence of iterations, the learning method only searches for a single best

rule instead of the whole RB.

This approach combines the speed of the Michigan approach with the

simplicity of the fitness evaluation of the Pittsburgh approach.

A Criterion for

Selecting the Best

Rule

A Penalty Criterion

Determination

Criterion for

Training Set


95


The Michigan approach provides online learning for non-inductive learning

problems whereas IRL approach facilitates off line inductive learning

problems.

There are different examples of IRL approach for genetic rule learning in the

literature; the major among them are presented below:

Supervised Inductive Algorithm (SIA) uses a single GA that goes on detecting the

rules and eliminating the examples covered by the latter. SIA can only work with

crisp data [68]. MOGUL (Methodology to obtain GFRBSs under the IRL approach)

has been proposed for designing different types of FRBS for variety of the problem

domains i.e fuzzy modeling, fuzzy control and fuzzy classification [61,158, 164]. In

order to work with MOGUL, a user has to define evolutionary process in each of the

GFRBS learning stages. MOGUL works with different fuzzy models i.e. descriptive

Mamdani type [158, 163] and approximate Mamdani type [61, 157] as well as TSK

FRBSs [159].

SLAVE (Structural Learning Algorithm in Vague Environment) is a genetic learning

process based on IRL approach to design DNF Mamdani type FRBSs that was

proposed by A.Gonza'lez in 1993 [13] and later refined in various research papers of

A.Gonza'lez, and R.Perez [9,10,11,12,128]. SLAVE launches a new GA to find a new

rule after having eliminated the examples covered by the last rule obtained. SLAVE

was designed to work with or without linguistic information.

The genetic generation process runs a GA for obtaining the best rule according to

different features, assigns a relative covering value to every example, and removes the

examples with a covering value greater than a constant [58,59].

3.3.4 Other Approaches of Genetic Rule Learning

Another major approach is identified as Genetic Cooperative-Competitive Learning

(GCCL) to provide genetic rule learning. In this approach, the rule base can be

encoded with the help of a complete population or a subset of population [54, p.34].

In this model the chromosomes compete and cooperate simultaneously with one

another.


96


COGIN (Coverage-based Genetic Induction) proposed by Greene and Smith in

1993[47], REGAL [7] and LOGENPRO [142, p.71] are the successful examples of

GCCL kind of learning approach.

3.4 Work done so far in the Area of GFS Hybridization

It has been observed that there are many real world applications successfully designed

to support the need of intelligent decision support and to achieve optimization using

Genetic-Fuzzy hybridization. The major applications obtained as a result of the above

mentioned approaches and explained as follows:

Pattern Recognition constitutes an important application of FRBS as FRBSs provide a

suitable mean to classify incomplete and imprecise data [129, pp.15-36; 204, pp.152-

178].

As mentioned earlier, SLAVE has been applied to solve classification problem of

myocardial infarction [14]. The set of the linguistic rules generated by SLAVE is

more understandable and more useful to the clinical experts. SLAVE has also been

applied to diagnose malignancy. In order to diagnose breast cancer in females,

Wisconsin Breast Cancer Diagnosis (WBCD) database is designed [33] using the nine

characteristics with 683 instances representing the classification of malignant

attributes. The diagnostic accurate classifier system using evolutionary fuzzy rules has

been developed by C. A. Pena-Reyes and M. Sipper [28].

C.H.Wang,et al.proposed several GA-based knowledge integration strategies to

automatically integrate multiple rule sets in a distributed-knowledge environment.

Also, a self-integrating knowledge-based brain tumor diagnostic system based on

these strategies was successfully developed. Here, Genetic Algorithm generates an

optimal or nearly optimal rule set from these initial knowledge inputs. Furthermore, a

rule-refinement scheme is proposed to refine inference rules via interaction with the

environment [32].

The Genetic-Fuzzy rule base system (GFRBS) using the Pittsburg approach has also

been designed for classification in breast cancer diagnosis [166, pp.377-378]. Here,

KB consists of membership function and its parameters. Later, it identifies the best

single rule for classification.


97


GFS modeling has been used to estimate human dental age from tooth eruption status

and patient chronological ages. Here, an FRBS is compared with the expert

knowledge and is automatically generated from a set of data collected by the

researcher. GFRBS based on the Pittsburg Approach has been designed [136,137].

GFRBS for linguistic and fuzzy modeling has been achieved in different applications

for variety of domains, which are briefly narrated as follows:

There are two different genetic learning processes consist of two stages are designed

for linguistic FRBS based on first generating RB. These two RB learning processes

considered for the first stage are: WM adhoc data driven method and Thirft‟s GFRBS

[16,165,158]. An adaptation of the genetic learning process is designed [78] which

generates multiple RBs from different fuzzy partitions and joining them in a global

RB and applying a genetic selection process to obtain the best sub set of rules [167].

In 1994, the subjective qualification of rice tastes using sensory test is designed by H.

Ishibushi et al.. This test has been performed to identify quality of rice using genetic

fuzzy classification [77, 86].

In the field of behavior based robotics, GA based system has been designed for

robotic system in order to navigate, plan and operate in the real world with intelligent

behavior. Automatic generation of design of robotics system is possible through

Evolutionary approaches. In order to achieve automatic generation of design, the

objective function is designed such a way that it is capable to observe the robotic

behavior evolved by the means of the EA [166, pp.414-416].

A learning classifier system based on Michigan-style Learning Fuzzy-Classifier

System (LCS) has been designed for the supervised learning tasks. Here, the problem

of interpretability in LCSs, and design of Fuzzy-UCS - an online accuracy-based

LFCS architecture has been addressed [17].

In order to optimize social regulation policies, the decision support system is designed

by A. Abraham et al., which is popularly known as EvoPOL. EvoPOL is a fuzzy

inference based decision support system that uses an evolutionary algorithm (EA) to

optimize the if-then rules and its parameters for social regulation policies[2, pp.814-

826].

A computational intelligence framework has been designed to cope with the problem


98


of controlling the dynamical model for a disease in order to provide some

understanding related to the medical treatment. A genetic fuzzy system approach to

control a nonlinear dynamic model of the HIV infection is presented by [139].

The web based decision support system popularly known as WBISC-DSS is an

admission process which is designed using evolutionary fuzzy computing for

university of Berkeley, California. This system has been developed to allow automatic

adjusting of the user‟s preferences. These preferences can be seen as the parameters of

the Fuzzy Logic model in the form of degrees of importance of the used variables

[182].

M. A. Lee & H. Takagi have proposed the design method for fuzzy system which uses

Genetic Algorithm. Here, GA integrates three design stages; membership functions,

the number of fuzzy rules, and the rule consequent parameters to obtain optimal fuzzy

system [137].

A decision model for trading system has been designed that combines Fuzzy Logic

and technical analysis to find patterns and trends in the financial indices [198]. The

fuzzy model is optimized by utilizing a Genetic Algorithm and the historical data.

The classification of rules in dermatology data sets and breast cancer data set for

medicine has been discovered by GA. In this work; skin diseases like psoriasis,

seboreic dermatitis, lichen planus, pityriasis rosea, chronic dermatitis and pityriasis

rubra pilaris are discovered by implementing several if-then rules using GA. This

system has also been implanted in order to determine the patients suffering from

breast cancer. The goal of this research work is to find out the patients to whom the

cancer may re-occur [147].

The bilingual question classification through Genetic Algorithm with machine

learning methods has been designed with an Integrated Genetic Algorithm (GA) and

Machine Learning (ML) approach for the question classification in cross-language

question answering [150].

An evolutionary system is designed for discovering fuzzy classification rules. In order

to perform classification task, three methods were used to extract fuzzy classification

rules using EA. These three methods are genetic selection of fuzzy rules from a large

number of fuzzy candidate rules, genetic reduction of genetic space, and genetic

learning of fuzzy classification rules [141].


99


In order to predict, IPO under pricing, a prediction system has been designed using

the Genetic Algorithm. The Michigan approach has been utilized for generating the

Evolutionary Rule-Based System for IPO under-pricing [49].

A Genetic- Fuzzy hybridization approach has been implemented in travel choice

behavior models to provide insight into the travel choice behavior that is required in

design and planning of transport systems of the urban public transport networks [184].

An approach to a classical network optimization problem or the transportation

problem has been designed to optimize the distribution network for identifying

shortest route for the distribution of vehicles. In order to achieve optimization,

Genetic Algorithm is designed while fuzzy sets have been used to represents the

imprecise information related to provisional information such as costs, demands and

other variables [52].

3.5 Conclusion

The chapter presents the significance of hybridization of soft computing methods in

order to design solution for real life applications with desired characteristics. Being

two prime constituents, GA and FS provide advantages for machine learning, search

and optimization. Here, the need of hybridization of GA with FS is explained. In

order to achieve such integration, two major GA-Fuzzy hybrid approaches have been

analyzed: Fuzzy Logic assisted evolutionary algorithm and Genetic- Fuzzy Systems.

The chapter explains GFS in broad ranges of approaches for genetic rule learning. The

general structure of GFS has been presented along with two major approaches for

optimized rule learning using GA for automatic generated FLC as well as manually

designed FLC. The techniques of Genetic-Fuzzy hybridization are structured using

two popular styles of encoding:”Chromosome is rule “approach and “Chromosome is

set of rule “approach. Genetic-Fuzzy hybridization approaches are designed using

either of the two above stated styles. GFS hybridization is classified into three major

approaches: the Michigan, the Pittsburg and The IRL. The chapter explains each of

these approaches along with subcomponents, working style, advantages and

limitations. It also compares characteristics of traditional expert system with

Classifier System for Genetic- Fuzzy hybridization and hence presents the importance

of designing GFS. Comparative evaluation of each of the hybrid method leads to the

design of IRL. The chapter also narrates various advantages of the Iterative Rule


100


Learning approach. Extensive literature survey of GFS on the varied domains such as

classification, medicine, machine learning, management, etc. has been presented.

In summary, it can be stated that hybridization of Genetic-Fuzzy approaches is

efficient in order to achieve rule learning in an optimized way and hence can be one

of the most suitable methods for the machine learning. In order to design a system,

based on genetic rule learning from fuzzified input, Genetic-Fuzzy System is

required. The approach similar to IRL is required to achieve the genetic rule learning

using FLC. It has been observed from the literature review on GFS, that real life

problems which are based on mathematical formula can be easily designed and

implemented using GA with FLC. The significant observation also says that there is

not any generic framework yet developed for problem lacking mathematical

formulation using genetic fuzzy hybrid approach order to achieve optimization for

rule learning The research work is a step towards the same.

chapter 3: evolutionary fuzzy...

Documents