You are currently browsing the tag archive for the ‘Combinatorial Optimization’ tag.

Figure – Neural circuit controller of the virtual ant (page 3, fig. 2). [URL: http://arxiv.org/abs/1507.08467 ]

Intelligence and decision in foraging ants. Individual or Collective? Internal or External? What is the right balance between the two. Can one have internal intelligence without external intelligence? Can one take examples from nature to build in silico artificial lives that present us with interesting patterns? We explore a model of foraging ants in this paper that will be presented in early September in Exeter, UK, at UKCI 2015. (available on arXiv [PDF] and ResearchGate)

Cristian Jimenez-Romero, David Sousa-Rodrigues, Jeffrey H. Johnson, Vitorino Ramos; “** A Model for Foraging Ants, Controlled by Spiking Neural Networks and Double Pheromones**“, UKCI 2015 Computational Intelligence – University of Exeter, UK, September 2015.

**Abstract**: A model of an Ant System where ants are controlled by a spiking neural circuit and a second order pheromone mechanism in a foraging task is presented. A neural circuit is trained for individual ants and subsequently the ants are exposed to a virtual environment where a swarm of ants performed a resource foraging task. The model comprises an associative and unsupervised learning strategy for the neural circuit of the ant. The neural circuit adapts to the environment by means of classical conditioning. The initially unknown environment includes different types of stimuli representing food (rewarding) and obstacles (harmful) which, when they come in direct contact with the ant, elicit a reflex response in the motor neural system of the ant: moving towards or away from the source of the stimulus. The spiking neural circuits of the ant is trained to identify food and obstacles and move towards the former and avoid the latter. The ants are released on a landscape with multiple food sources where one ant alone would have difficulty harvesting the landscape to maximum efficiency. In this case the introduction of a double pheromone mechanism (positive and negative reinforcement feedback) yields better results than traditional ant colony optimization strategies. Traditional ant systems include mainly a positive reinforcement pheromone. This approach uses a second pheromone that acts as a marker for forbidden paths (negative feedback). This blockade is not permanent and is controlled by the evaporation rate of the pheromones. The combined action of both pheromones acts as a collective stigmergic memory of the swarm, which reduces the search space of the problem. This paper explores how the adaptation and learning abilities observed in biologically inspired cognitive architectures is synergistically enhanced by swarm optimization strategies. The model portraits two forms of artificial intelligent behaviour: at the individual level the spiking neural network is the main controller and at the collective level the pheromone distribution is a map towards the solution emerged by the colony. The presented model is an important pedagogical tool as it is also an easy to use library that allows access to the spiking neural network paradigm from inside a Netlogo—a language used mostly in agent based modelling and experimentation with complex systems.

**References**:

[**1**] C. G. Langton, “Studying artificial life with cellular automata,” Physica D: Nonlinear Phenomena, vol. 22, no. 1–3, pp. 120 – 149, 1986, proceedings of the Fifth Annual International Conference. [Online]. Available: http://www.sciencedirect.com/ science/article/pii/016727898690237X

[**2**] A. Abraham and V. Ramos, “Web usage mining using artificial ant colony clustering and linear genetic programming,” in Proceedings of the Congress on Evolutionary Computation. Australia: IEEE Press, 2003, pp. 1384–1391.

[**3**] V. Ramos, F. Muge, and P. Pina, “Self-organized data and image retrieval as a consequence of inter-dynamic synergistic relationships in artificial ant colonies,” Hybrid Intelligent Systems, vol. 87, 2002.

[**4**] V. Ramos and J. J. Merelo, “Self-organized stigmergic document maps: Environment as a mechanism for context learning,” in Proceddings of the AEB, Merida, Spain, February 2002. ´

[**5**] D. Sousa-Rodrigues and V. Ramos, “Traversing news with ant colony optimisation and negative pheromones,” in European Conference in Complex Systems, Lucca, Italy, Sep 2014.

[**6**] E. Bonabeau, G. Theraulaz, and M. Dorigo, Swarm Intelligence: From Natural to Artificial Systems, 1st ed., ser. Santa Fe Insitute Studies In The Sciences of Complexity. 198 Madison Avenue, New York: Oxford University Press, USA, Sep. 1999.

[**7**] M. Dorigo and L. M. Gambardella, “Ant colony system: A cooperative learning approach to the traveling salesman problem,” Universite Libre de Bruxelles, Tech. Rep. TR/IRIDIA/1996-5, ´ 1996.

[**8**] M. Dorigo, G. Di Caro, and L. M. Gambardella, “Ant algorithms for discrete optimization,” Artif. Life, vol. 5, no. 2, pp. 137– 172, Apr. 1999. [Online]. Available: http://dx.doi.org/10.1162/ 106454699568728

[**9**] L. M. Gambardella and M. Dorigo, “Ant-q: A reinforcement learning approach to the travelling salesman problem,” in Proceedings of the ML-95, Twelfth Intern. Conf. on Machine Learning, M. Kaufman, Ed., 1995, pp. 252–260.

[**10**] A. Gupta, V. Nagarajan, and R. Ravi, “Approximation algorithms for optimal decision trees and adaptive tsp problems,” in Proceedings of the 37th international colloquium conference on Automata, languages and programming, ser. ICALP’10. Berlin, Heidelberg: Springer-Verlag, 2010, pp. 690–701. [Online]. Available: http://dl.acm.org/citation.cfm?id=1880918.1880993

[**11**] V. Ramos, D. Sousa-Rodrigues, and J. Louçã, “Second order ˜ swarm intelligence,” in HAIS’13. 8th International Conference on Hybrid Artificial Intelligence Systems, ser. Lecture Notes in Computer Science, J.-S. Pan, M. Polycarpou, M. Wozniak, A. Carvalho, ´ H. Quintian, and E. Corchado, Eds. Salamanca, Spain: Springer ´ Berlin Heidelberg, Sep 2013, vol. 8073, pp. 411–420.

[**12**] W. Maass and C. M. Bishop, Pulsed Neural Networks. Cambridge, Massachusetts: MIT Press, 1998.

[**13**] E. M. Izhikevich and E. M. Izhikevich, “Simple model of spiking neurons.” IEEE transactions on neural networks / a publication of the IEEE Neural Networks Council, vol. 14, no. 6, pp. 1569–72, 2003. [Online]. Available: http://www.ncbi.nlm.nih. gov/pubmed/18244602

[**14**] C. Liu and J. Shapiro, “Implementing classical conditioning with spiking neurons,” in Artificial Neural Networks ICANN 2007, ser. Lecture Notes in Computer Science, J. de S, L. Alexandre, W. Duch, and D. Mandic, Eds. Springer Berlin Heidelberg, 2007, vol. 4668, pp. 400–410. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-74690-4 41

[**15**] J. Haenicke, E. Pamir, and M. P. Nawrot, “A spiking neuronal network model of fast associative learning in the honeybee,” Frontiers in Computational Neuroscience, no. 149, 2012. [Online]. Available: http://www.frontiersin.org/computational neuroscience/10.3389/conf.fncom.2012.55.00149/full

[**16**] L. I. Helgadottir, J. Haenicke, T. Landgraf, R. Rojas, and M. P. Nawrot, “Conditioned behavior in a robot controlled by a spiking neural network,” in International IEEE/EMBS Conference on Neural Engineering, NER, 2013, pp. 891–894.

[**17**] A. Cyr and M. Boukadoum, “Classical conditioning in different temporal constraints: an STDP learning rule for robots controlled by spiking neural networks,” pp. 257–272, 2012.

[**18**] X. Wang, Z. G. Hou, F. Lv, M. Tan, and Y. Wang, “Mobile robots’ modular navigation controller using spiking neural networks,” Neurocomputing, vol. 134, pp. 230–238, 2014.

[**19**] C. Hausler, M. P. Nawrot, and M. Schmuker, “A spiking neuron classifier network with a deep architecture inspired by the olfactory system of the honeybee,” in 2011 5th International IEEE/EMBS Conference on Neural Engineering, NER 2011, 2011, pp. 198–202.

[**20**] U. Wilensky, “Netlogo,” Evanston IL, USA, 1999. [Online]. Available: http://ccl.northwestern.edu/netlogo/

[**21**] C. Jimenez-Romero and J. Johnson, “Accepted abstract: Simulation of agents and robots controlled by spiking neural networks using netlogo,” in International Conference on Brain Engineering and Neuro-computing, Mykonos, Greece, Oct 2015.

[**22**] W. Gerstner and W. M. Kistler, Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge: Cambridge University Press, 2002.

[**23**] J. v. H. W Gerstner, R Kempter and H. Wagner, “A neuronal learning rule for sub-millisecond temporal coding,” Nature, vol. 386, pp. 76–78, 1996.

[**24**] I. P. Pavlov, “Conditioned reflexes: An investigation of the activity of the cerebral cortex,” New York, 1927.

[**25**] E. J. H. Robinson, D. E. Jackson, M. Holcombe, and F. L. W. Ratnieks, “Insect communication: ‘no entry’ signal in ant foraging,” Nature, vol. 438, no. 7067, pp. 442–442, 11 2005. [Online]. Available: http://dx.doi.org/10.1038/438442a

[**26**] E. J. Robinson, D. Jackson, M. Holcombe, and F. L. Ratnieks, “No entry signal in ant foraging (hymenoptera: Formicidae): new insights from an agent-based model,” Myrmecological News, vol. 10, no. 120, 2007.

[**27**] D. Sousa-Rodrigues, J. Louçã, and V. Ramos, “From standard ˜ to second-order swarm intelligence phase-space maps,” in 8th European Conference on Complex Systems, S. Thurner, Ed., Vienna, Austria, Sep 2011.

[**28**] V. Ramos, D. Sousa-Rodrigues, and J. Louçã, “Spatio-temporal ˜ dynamics on co-evolved stigmergy,” in 8th European Conference on Complex Systems, S. Thurner, Ed., Vienna, Austria, 9 2011.

[**29**] S. Tisue and U. Wilensky, “Netlogo: A simple environment for modeling complexity,” in International conference on complex systems. Boston, MA, 2004, pp. 16–21.

Figure – ** Attractor basins** (fig.2 pp.6 on Mária Ercsey-Ravasz and Zoltán Toroczkai, “

*Optimization hardness as transient chaos in an analog approach to constraint satisfaction*“, Nature Physics, vol. 7, p. 966-970, 2011.)

*Mária Ercsey-Ravasz* and *Zoltán Toroczkai* have proposed a way of mapping satisfiability problems to differential equations and a deterministic algorithm that solves them in polynomial continuous time at the expense of exponential energy functions (so the discrete approximation of the algorithm does not run in polynomial time, and an analogue system would need exponential resources).

The map assigns a phase space to a problem; the algorithm chooses random initial conditions from within that phase space. In the graphs above and below, they pick a 2-d subspace of the phase space and for each initial point in that space they illustrate 1) the particular solution the algorithm finds, 2) the corresponding “solution cluster”, an equivalence class of solutions that identifies two solutions if they differ in exactly one variable assignment, and 3) the time it takes to solve the problem. Each row adds another clause to satisfy.

The especially interesting part of the paper is the notion of an ** escape rate**, the proportion of the trajectories still searching for a solution after a time

*t*. In a companion paper, they show that the escape rate for

*Sudoku*combinatorial instances (

*The Chaos Within Sudoku*,

*Nature*, August 2012) correlates strongly with human judgements of hardness. This

*escape rate*is similar to the

*Kolmogorov complexity*in that it gives a notion of hardness to individual problem instances rather than to classes of problems. Full paper could be retrieved from arXiv:

*Mária Ercsey-Ravasz*and

*Zoltán Toroczkai*, “

**Optimization hardness as transient chaos in an analog approach to constraint satisfaction**“, Nature Physics, vol. 7, p. 966-970, 2011. (at arXiv on August 2012).

Figure – ** Attractor basins for 3-XORSAT** (fig.8 pp.18 on Mária Ercsey-Ravasz and Zoltán Toroczkai, “

*Optimization hardness as transient chaos in an analog approach to constraint satisfaction*“, Nature Physics, vol. 7, p. 966-970, 2011.)

Figure (click to enlarge) – ** Orders of common functions** (via

*Wikipedia*): A list of classes of functions that are commonly encountered when analyzing the running time of an algorithm. In each case,

*c*is a constant and

*n*increases without bound. The slower-growing functions are generally listed first. For each case, several examples are given.

Fig.1 – (click to enlarge) The optimal shortest path among *N*=1265 points depicting a Portuguese *Navalheira* crab as a result of one of our latest Swarm-Intelligence based algorithms. The problem of finding the shortest path among *N* different points in space is *NP-hard*, known as the *Travelling Salesmen Problem* (*TSP*), being one of the major and hardest benchmarks in Combinatorial Optimization (link) and Artificial Intelligence. (*V. Ramos*, *D. Rodrigues*, 2012)

This summer my kids just grab a tiny Portuguese *Navalheira *crab on the shore. After a small photo-session and some baby-sitting with a lettuce leaf, it was time to release it again into the ocean. He not only survived my kids, as he is now entitled into a new World Wide Web on-line life. After the *Shortest path Sardine* (link) with 1084 points, here is the *Crab* with 1265 points. The algorithm just run as little as 110 iterations.

Fig. 2 – (click to enlarge) Our 1265 initial points depicting a TSP Portuguese *Navalheira* crab. Could you already envision a minimal tour between all these points?

As usual in *Travelling Salesmen problems* (*TSP*) we start it with a set of points, in our case 1084 points or cities (fig. 2). Given a list of cities and their pairwise distances, the task is now to find the *shortest possible tour* that visits each city exactly once. The problem was first formulated as a mathematical problem in 1930 and is one of the most intensively studied problems in optimization. It is used as a benchmark for many optimization methods.

Fig. 3 – (click to enlarge) Again the shortest path *Navalheira* crab, where the optimal contour path (in black: first fig. above) with 1265 points (or cities) was filled in dark orange.

TSP has several applications even in its purest formulation, such as planning, logistics, and the manufacture of microchips. Slightly modified, it appears as a sub-problem in many areas, such as DNA sequencing. In these applications, the concept city represents, for example, customers, soldering points, or DNA fragments, and the concept distance represents travelling times or cost, or a similarity measure between DNA fragments. In many applications, additional constraints such as limited resources or time windows make the problem considerably harder.

What follows (fig. 4) is the original crab photo after image segmentation and just before adding *Gaussian* noise in order to retrieve several data points for the initial TSP problem. The algorithm was then embedded with the extracted *x*,*y* coordinates of these data points (fig. 2) in order for him to discover the minimal path, in just 110 iterations. For extra details, pay a visit onto the *Shortest path Sardine* (link) done earlier.

Fig. 4 – (click to enlarge) The original crab photo after some image processing as well as segmentation and just before adding *Gaussian* noise in order to retrieve several data points for the initial TSP problem.

Fig.1 – (click to enlarge) The optimal shortest path among *N*=1084 points depicting a Portuguese sardine as a result of one of our latest Swarm-Intelligence based algorithms. The problem of finding the shortest path among *N* different points in space is *NP-hard*, known as the *Travelling Salesmen Problem* (*TSP*), being one of the major and hardest benchmarks in Combinatorial Optimization (link) and Artificial Intelligence. (*D. Rodrigues*, *V. Ramos*, 2011)

Almost summer time in Portugal, great weather as usual, and the perfect moment to eat sardines along with friends in open air esplanades; in fact, a lot of grilled sardines. We usually eat grilled sardines with a tomato-onion salad along with barbecued cherry peppers in salt and olive oil. That’s tasty, believe me. But not tasty enough however for me and one of my colleagues, *David Rodrigues* (blog link/twitter link). We decided to take this experience a little further on, creating the first ** shortest path sardine**.

Fig. 2 – (click to enlarge) Our 1084 initial points depicting a TSP Portuguese sardine. Could you already envision a minimal tour between all these points?

As usual in *Travelling Salesmen problems* (*TSP*) we start it with a set of points, in our case 1084 points or cities (fig. 2). Given a list of cities and their pairwise distances, the task is now to find the *shortest possible tour* that visits each city exactly once. The problem was first formulated as a mathematical problem in 1930 and is one of the most intensively studied problems in optimization. It is used as a benchmark for many optimization methods. TSP has several applications even in its purest formulation, such as planning, logistics, and the manufacture of microchips. Slightly modified, it appears as a sub-problem in many areas, such as DNA sequencing. In these applications, the concept city represents, for example, customers, soldering points, or DNA fragments, and the concept distance represents travelling times or cost, or a similarity measure between DNA fragments. In many applications, additional constraints such as limited resources or time windows make the problem considerably harder. (link)

Fig. 3 – (click to enlarge) A well done and quite grilled shortest path sardine, where the optimal contour path (in blue: first fig. above) with 1084 points was filled in black colour. Nice T-shirt!

Even for toy-problems like the present 1084 *TSP sardine*, the amount of possible paths are incredible huge. And only one of those possible paths is the optimal (minimal) one. Consider for example a TSP with *N*=4 cities, *A*, *B*, *C*, and *D*. Starting in city *A*, the number of possible paths is 6: that is 1) A to B, B to C, C to D, and D to A, 2) A-B, B-D, D-C, C-A, 3) A-C, C-B, B-D and D-A, 4) A-C, C-D, D-B, and B-A, 5) A-D, D-C, C-B, and B-A, and finally 6) A-D, D-B, B-C, and C-A. I.e. there are (*N*–*1*)! [i.e., *N*–*1 factorial*] possible *paths*. For *N*=3 cities, 2×1=2 possible paths, for *N*=4 cities, 3x2x1=6 possible paths, for *N*=5 cities, 4x3x2x1=24 possible paths, … for *N*=20 cities, 121.645.100.408.832.000 possible paths, and so on.

The most direct solution would be to try all permutations (ordered combinations) and see which one is cheapest (using computational brute force search). The running time for this approach however, lies within a polynomial factor of *O*(*n*!), the factorial of the number of cities, so this solution becomes impractical even for only 20 cities. One of the earliest and oldest applications of dynamic programming is the *Held–Karp algorithm* which only solves the problem in time *O*(*n*^{2}2^{n}).

In our present case (*N*=1084) we have had to deal with 1083 factorial possible paths, leading to the astronomical number of 1.19×10^{2818 }possible solutions. That’s roughly 1 followed by 2818 zeroes! – better now to check this *Wikipedia* entry on very* large numbers*. Our new *Swarm-Intelligent* based algorithm, running on a normal PC was however, able to formulate a minimal solution (fig.1) within just several minutes. We will soon post more about our novel self-organized stigmergic-based algorithmic approach, but meanwhile, if you enjoyed these drawings, do not hesitate in asking us for a grilled cherry pepper as well. We will be pleased to deliver you one by email.

p.s. – This is a joint twin post with *David Rodrigues*.

Fig. 4 – (click to enlarge) Zoom at the end sardine tail optimal contour path (in blue: first fig. above) filled in black, from a total set of 1084 initial points.

Figure – A comic strip by *Randall Munroe* (at xkcd.com – a webcomic of romance, sarcasm, math, and language) about *Computational Complexity*, the *Travelling Salesman* (TSP) problem, and – *last, but not least* – about crowd-sourcing the whole thing into *ebay*! … LOL

… hey wait, just before you ROTFL yourself on the floor, just check this out. For some problem cases it might just work. Here is one of those recent cases: Interactive, online games are being used to crack complex scientific conundrums, says a report in *Nature*. And the wisdom of the ‘multiplayer’ crowd is delivering a new set of search strategies for the prediction of protein structures. The problem at hands is nothing less than *protein folding*, not an easy one. You can check *Nature*‘s journal video here. While the online game in question is known as ** Foldit** (link – image below).

Figure – Solving puzzle #48 with FOLDIT, the online game.

There are a lot of consequences with this approach. Here from an article of the Spanish *El Pais* newspaper (in, *Malen Ruiz de Elvira*, “*Los humanos ganan a los ordenadores – Un juego en red para resolver un problema biológico obtiene mejores resultados que un programa informático*“, *El Pais*, Madrid – August 8, 2010):

[…] Miles de jugadores en red, la mayoría no especializados, han demostrado resolver mejor la forma que adoptan las proteínas que los programas informáticos más avanzados, han hallado científicos de la *Universidad de Washington* (en Seattle). Averiguar cómo se pliegan las largas cadenas de aminoácidos de las proteínas en la naturaleza -su estructura en tres dimensiones- es uno de los grandes problemas de la biología actual, al que numerosos equipos dedican enormes recursos informáticos. […] Sin embargo la predicción por ordenador de la estructura de una proteína representa un desafío muy grande porque hay que analizar un gran número de posibilidades hasta alcanzar la solución, que se corresponde con un estado óptimo de energía. Es un proceso de optimización. […] Para comprobar su pericia, los científicos plantearon a los jugadores 10 problemas concretos de estructuras de proteínas que conocían pero que no se habían hecho públicas. Encontraron que en algunos de estos casos, concretamente cinco, el resultado alcanzado por los mejores jugadores fue más exacto que el de Rosetta. En otros tres casos las cosas quedaron en tablas y en dos casos ganó la máquina. […] Además, las colaboraciones establecidas entre algunos de los jugadores dieron lugar a todo un nuevo surtido de estrategias y algoritmos, algunos de los cuales se han incorporado ya al programa informático original. “Tan interesantes como las predicciones de Foldit son la complejidad, la variedad y la creatividad que muestra el proceso humano de búsqueda”, escriben los autores del trabajo, entre los que figuran, algo insólito en un artículo científico, “los jugadores de Foldit”. […] “Estamos en el inicio de una nueva era, en la que se mezcla la computación de los humanos y las máquinas”, dice *Michael Kearns*, un experto en el llamado pensamiento distribuido. […]

Video – *Matthew Todd* lecture at Google Tech Talk April, 2010 – Open Science: how can we crowdsource chemistry to solve important problems?

The idea of course, is not new. All these distributed human learning systems, started with the *SETI at home* project (link), originally launched in 1999, by the *Berkeley University* in California. But I would like to drawn your attention, instead, to some other works on it. First is a video by *Matthew Todd* (School of Chemistry, *University of Sydney*). His question his apparently simple: *how can we crowdsource chemistry to solve important problems? *(above). Second, a well known introductory paper on *Crowd-Sourcing* by *Daren C. Brabham* (2008), with several worldwide examples:

[…] **Abstract:** Crowdsourcing is an online, distributed problem-solving and production model that has emerged in recent years. Notable examples of the model include Threadless, iStockphoto, Inno-Centive, the Goldcorp Challenge, and user-generated advertising contests. This article provides an introduction to crowdsourcing, both its theoretical grounding and exemplar cases, taking care to distinguish crowdsourcing from open source production. This article also explores the possibilities for the model, its potential to exploit a crowd of innovators, and its potential for use beyond for-profit sectors. Finally, this article proposes an agenda for research into crowdsourcing. […] in Daren C. Brabham, “*Crowdsourcing as a Model for Problem Solving – An Introduction and Cases*“, *Convergence*: The International Journal of Research into New Media Technologies, London, Los Angeles, New Delhi and Singapore Vol 14(1): 75-90, 2008.

## Recent Comments