Optimizing Models Using Continuous Ant Algorithms. Oleg Kovarik, Pavel Kordik

Abstract. While constructing inductive models of a given system, we need to optimize parameters of units the system is composed of. These parameters are often real-valued variables and we can use a large scale of continuous optimization methods to locate their optimum. Each of these methods can give different results for problems of various nature or complexity. In our experiments, the usually best performing gradient based Quasi-Newton method was unable to optimize parameters for a well known problem of two intertwined spirals; its classification accuracy was close to 50%. Therefore, we compared several continuous optimization algorithms performance on this particular problem. Our results show that two probabilistic algorithms inspired by ant behaviour are able to optimize parameters of model units for this problem with the classification accuracy of 70%.

Keywords. AntAlgorithms, ContinuousOptimization, Inductive Modelling.

References.
1. Pavel Kordik. Fully Automated Knowledge Extraction using Group of Adaptive Models Evolution. PhD thesis, Czech Technical University in Prague, FEE, Dep. of Comp. Sci. and Computers, FEE, CTU Prague, Czech Republic, September 2006.

2. Salane and Tewarson. A unified derivation of symmetric quasi-newton update formulas. Applied Math, 25:29–36, 1980.

3. A.G. Ivakhnenko, E.A. Savchenko, and G.A. Ivakhnenko. Gmdh algorithm for optimal model choice by the external error criterion with the extension of definition by model bias and its applications to the committees and neural networks. Pattern Recognition and Image Analysis, 12(4):347353, 2002.

4. Min Kong and Peng Tian. A direct application of ant colony optimization to function optimization problem in continuous domain. In Ant Colony Optimization and Swarm Intelligence, 5th International Workshop, ANTS 2006, Brussels, Belgium, September 4-7, 2006. Proceedings, Lecture Notes in Computer Science, pages 324–331. Springer, 2006.

5. Krzysztof Socha. ACO for continuous and mixed-variable optimization. In Marco Dorigo, Mauro Birattari, Christian Blum, Luca Maria Gambardella, Francesco Mondada, and Thomas Stutzle, editors, Ant Colony Optimization and Swarm Intelligence, 4th International Workshop, ANTS 2004, Brussels, Belgium, September 5 - 8, 2004, Proceedings, volume 3172 of Lecture Notes in Computer Science, pages 25–36. Springer, 2004.

6. Marco Dorigo, Gianni Di Caro, and Luca Maria Gambardella. Ant algorithms for discrete optimization. Artificial Life, 5(2):137–172, 1999.