Metaheuristic Clustering / Edition 1

Table of Contents

1 Metaheuristic Pattern Clustering-An Overview 1

1.1 Introduction 1

1.2 The Clustering Problem 6

1.2.1 Basic Definitions 6

1.2.2 Proximity Measures 8

1.2.3 Clustering Validity Indices 9

1.2.3.1 The Davis-Bouldin (DB) Index 9

1.2.3.2 The Dunn and Dunn Like Indices 10

1.2.3.3 S_Dbw Validity Index 10

1.2.3.4 Partition Coefficient 11

1.2.3.5 Classification Entropy 12

1.2.3.6 Xie-Beni Index 12

1.2.3.7 The PS Measure 12

1.2.3.8 The PBMF Index 13

1.2.3.9 The CS Measure 13

1.3 The Classical Clustering Algorithms 14

1.3.1 Hierarchical Clustering Algorithms 14

1.3.2 Partitional Clustering Algorithms 16

1.3.2.1 The k-Means Algorithm 18

1.3.2.2 The k-Medoids Algorithm 19

1.3.2.3 The Fuzzy c-Means Algorithm 19

1.3.2.4 The Expectation-Maximization Algorithm 20

1.3.2.5 The k-Harmonic Means Algorithm 21

1.3.3 Density-Based Clustering Algorithms 22

1.3.4 Grid-Based Clustering Algorithms 23

1.3.5 A Comparative View of the Traditional Clustering Algorithms 23

1.4 Population Based Optimization Techniques 26

1.4.1 Optimization Algorithms 26

1.4.2 The Evolutionary Computing (EC) Family 28

1.4.3 The Evolutionary Algorithms 29

1.4.3.1 Evolutionary Strategies (ESs) 30

1.4.3.2 Evolutionary Programming (EP) 30

1.4.3.3 Genetic Algorithms (GAs) 31

1.4.3.4 Genetic Programming (GPs) 33

1.4.4 Swarm Intelligence Algorithms 33

1.4.4.1 The Particle Swarm Optimization (PSO) 34

1.4.4.2 The Ant Colony Optimization (ACO) 35

1.4.5 Evolutionary Computing (EC) Techniques in Pattern Clustering 36

1.5 Clustering Methods Based on Evolutionary Algorithms 36

1.5.1 The GA-Based Partitional Clustering Algorithms-Earlier Approaches 37

1.5.2 ClusteringAlgorithms Based on ES, EP, and GP 38

1.6 Clustering Using Swarm Intelligence Algorithms 39

1.6.1 The Ant Colony Based Clustering Algorithms 39

1.6.2 The PSO-Based Clustering Algorithms 40

1.7 Automatic Clustering: Evolutionary Vs. Classical Approaches 42

1.7.2 Genetic Clustering with Unknown Number of Clusters K (GCUK) Algorithm 43

1.7.3 The FVGA Algorithm 44

1.7.4 The Dynamic Clustering with Particle Swarm Optimization Algorithm 45

1.8 Clustering with Evolutionary Multi-objective Optimization 45

1.8.1 Multi-objective Optimization Problem (MOP) 45

1.8.2 Evolutionary Multi-objective Optimization (EMO) 46

1.8.3 Clustering Using EMO Algorithms (EMOAs) 48

1.9 Innovation and Research: Main Contributions of This Volume 49

1.10 Conclusions 53

References 53

2 Differential Evolution Algorithm: Foundations and Perspectives 63

2.1 Introduction 63

2.2 Differential Evolution: A First Glance 64

2.2.1 Initialization of the Parameter Vectors 64

2.2.2 Mutation with Differential Operators 66

2.2.3 Crossover 68

2.2.4 Selection 72

2.2.5 Summary of DE Iteration 73

2.3 The Complete Differential Evolution Algorithm Family of Storn and Price 77

2.4 Control Parameters of the Differential Evolution 79

2.5 Important Variants of the Differential Evolution Algorithm 81

2.5.1 Differential Evolution Using Trigonometric Mutation 81

2.5.2 Differential Evolution Using Arithmetic Recombination 82

2.5.3 Self Adaptive Differential Evolution 84

2.5.4 The DE/rand/1/Either-Or Algorithm 86

2.5.5 The Opposition-Based Differential Evolution 86

2.5.6 The Binary Differential Evolution Algorithm 89

2.5.7 Differential Evolution with Adaptive Local Search 90

2.5.8 Self-adaptive Differential Evolution (SaDE) with Strategy Adaptation 92

2.5.9 DE with Neighborhood-Based Mutation 93

2.5.9.1 The DE/target-to-best/1 - A Few Drawbacks 93

2.5.9.2 Motivations for the Neighborhood-Based Mutation 94

2.5.9.3 The Local and Global Neighborhood-Based Mutations in DE 95

2.5.9.4 Control Parameters in DEGL 97

2.5.9.5 Runtime Complexity of DEGL - A Discussion 99

2.5.9.6 Comparative Performance of DEGL 102

2.6 Hybridization of Differential Evolution with Other Stochastic Search Techniques 104

2.7 Conclusions 106

References 107

3 Modeling and Analysis of the Population-Dynamics of Differential Evolution Algorithm 111

3.1 Introduction 111

3.2 The Mathematical Model of the Population-Dynamics in DE 112

3.2.1 Assumptions 113

3.2.2 Modeling Different Steps of DE 114

3.3 A State Space Formulation of the DE Population 122

3.4 Lyapunov Stability Analysis of the DE Population 124

3.5 Computer Simulation Results 129

3.6 Conclusions 131

Appendix 132

References 133

4 Automatic Hard Clustering Using Improved Differential Evolution Algorithm 137

4.1 Introduction 137

4.2 The DE-Based Automatic Clustering Algorithm 138

4.2.1 Vector Representation 138

4.2.2 Designing the Fitness Function 140

4.2.3 Avoiding Erroneous Vectors 146

4.2.4 Modification of the Classical DE 147

4.2.5 Pseudo-code of the ACDE Algorithm 148

4.3 Experiments and Results for Real Life Datasets 148

4.3.1 The Datasets Used 149

4.3.2 Population Initialization 149

4.3.3 Parameter Setup for the Algorithms Compared 150

4.3.4 Simulation Strategy 150

4.3.5 Empirical Results 151

4.3.6 Discussion on the Results (for Real Life Datasets) 161

4.4 Application to Image Segmentation 162

4.4.1 Image Segmentation as a Clustering Problem 162

4.4.2 Experimental Details and Results 162

4.4.3 Discussion on Image Segmentation Results 165

4.5 Conclusions 172

Appendix: Statistical Tests Used 172

References 173

5 Fuzzy Clustering in the Kernel-Induced Feature Space Using Differential Evolution Algorithm 175

5.1 Introduction 175

5.2 The Kernel-Induced Clustering 177

5.3 The Kernel-Induced Clustering Technique with DEGL 181

5.3.1 Kernelization of the Xie-Beni Index 181

5.3.2 Summary of the Integrated Clustering Approach 183

5.4 Experimental Results 184

5.4.1 General Comparison with Other Clustering Algorithms 184

5.4.2 Scalability of the DEGL-Based Clustering Algorithm 194

5.5 Application to Image Pixel Clustering 197

5.5.1 Parametric Setup for the Contestant Algorithms 197

5.5.2 The Test-Suite for Comparison 198

5.5.3 Quantitative Validation of Clustering Results 198

5.5.4 The Simulation Strategy 199

5.5.5 Experimental Results 200

5.5.6 Discussion on the Results 202

5.6 Conclusions 208

References 208

6 Clustering Using Multi-objective Differential Evolution Algorithms 213

6.1 Introduction 213

6.2 Multi-objective Optimization Using Differential Evolution Algorithm 215

6.2.1 The Pareto Differential Evolution (PDE) 215

6.2.2 The Multi-Objective Differential Evolution (MODE) 216

6.2.3 Differential Evolution for Multi-objective Optimization (DEMO) 216

6.2.4 Non-dominated Sorting DE (NSDE) 218

6.3 The Multi-objective Clustering Scheme 218

6.3.1 Search-Variable Representation 218

6.3.2 Selecting the Objective Functions 219

6.3.3 Selecting the Best Solutions from Pareto-front 221

6.3.4 Evaluating the Clustering Quality 222

6.4 Experiments and Results 223

6.4.1 Datasets Used 223

6.4.2 Parameters for the Algorithms 223

6.4.3 Presentation of Results 224

6.4.4 Significance and Validation of Microarray Data Clustering Results 228

6.5 Conclusions 236

References 237

7 Conclusions and Future Research 239

7.1 Cluster Analysis Using Metaheuristics: A Roadmap of This Volume 239

7.2 Potential Application Areas for Clustering Schemes 241

7.3 Future Research Directions 242

References 245

Index 249

Read More