logo EDITE Jean-Baptiste MOURET
Identité
Jean-Baptiste MOURET
État académique
Thèse soutenue le 2008-12-05
Sujet: PRESSIONS SELECTIVES MULTIPLES POUR L'EVOLUTION DE RESEAUX DE NEURONES DESTINES A LA ROBOTIQUE
Laboratoire: personnel permanent
Encadrement de thèses (depuis 2007)
0
Voisinage
Ellipse bleue: doctorant, ellipse jaune: docteur, rectangle vert: permanent, rectangle jaune: HDR. Trait vert: encadrant de thèse, trait bleu: directeur de thèse, pointillé: jury d'évaluation à mi-parcours ou jury de thèse.
Productions scientifiques
oai:hal.upmc.fr:hal-00633930
Online Discovery of Locomotion Modes for Wheel-Legged Hybrid Robots: a Transferability-based Approach
Wheel-legged hybrid robots promise to combine the e ciency of wheeled robots with the versatility of legged robots: they are able to roll on simple terrains, to dynamically adapt their posture and even to walk on uneven grounds. Al- though di erent locomotion modes of such robots have been studied, a pivotal question remains: how to automatically adapt the locomotion mode when the environment changes? We here propose that the robot autonomously discov- ers its locomotion mode using optimization-based learning. To that aim, we introduce a new algorithm that relies on a forward model and a stochastic multi-objective optimization. Three objectives are optimized: (1) the average displacement speed, (2) the expended energy and (3) the transferability score, which re ects how well the behavior of the robot is in agreement with the pre- dictions of the forward model. This transferability function is approximated by conducting 20 experiments of one second on the real robot during the op- timization. In the three investigated situations ( at ground, grass-like terrain, tunnel-like environment), our method found e cient controllers for forward locomotion in 1 to 2 minutes: the robot used its wheels on the at ground, it walked on the grass-like terrain and moved with a lowered body in the tunnel- like environment.
CLAWAR'11: Proceedings of the 14th International Conference on Climbing and Walking Robots 14th International Conference on Climbing and Walking Robotsproceeding, seminar, workshop without peer review 2011-09-06
oai:hal.upmc.fr:hal-00473132
Using Behavioral Exploration Objectives to Solve Deceptive Problems in Neuro-evolution
Encouraging exploration, typically by preserving the diversity within the population, is one of the most common method to improve the behavior of evolutionary algorithms with deceptive fitness functions. Most of the published approaches to stimulate exploration rely on a distance between genotypes or phenotypes; however, such distances are difficult to compute when evolving neural networks due to (1) the algorithmic complexity of graph similarity measures, (2) the competing conventions problem and (3) the complexity of most neural-network encodings. In this paper, we introduce and compare two conceptually simple, yet efficient methods to improve exploration and avoid premature convergence when evolving both the topology and the parameters of neural networks. The two proposed methods, respectively called behavioral novelty and behavioral diversity are built on multiobjective evolutionary algorithms and on a user-defined distance between behaviors. They can be employed with any genotype. We benchmarked them on the evolution of a neural network to compute a Boolean function with a deceptive fitness. The results obtained with the two proposed methods are statistically similar to those of NEAT and substantially better than those of the control experiment and of a phenotype-based diversity mechanism.
Proceedings of the 11th Annual conference on Genetic and evolutionary computation The 11th Annual conference on Genetic and evolutionary computation (GECCO'09)proceeding with peer review 2009
oai:hal.upmc.fr:hal-00633928
How to Promote Generalisation in Evolutionary Robotics: the ProGAb Approach
In Evolutionary Robotics (ER), controllers are assessed in a single or a few environments. As a consequence, good performances in new di erent contexts are not guaranteed. While a lot of ER works deal with robustness, i.e. the ability to perform well on new contexts close to the ones used for evaluation, no current approach is able to promote broader generalisation abilities without any assumption on the new contexts. In this paper, we introduce the ProGAb approach, which is based on the standard three data sets methodology of supervised machine learning, and compare it to state-of- the-art ER methods on two simulated robotic tasks: a navi- gation task in a T-maze and a more complex ball-collecting task in an arena. In both applications, the ProGAb ap- proach: (1) produced controllers with better generalisation abilities than the other methods; (2) needed two to three times fewer evaluations to discover such solutions.
GECCO'11: Proceedings of the 13th annual conference on Genetic and evolutionary computation Conference on Genetic and Evolutionary Computationproceeding with peer review 2011-07
oai:hal.upmc.fr:hal-00633927
Crossing the Reality Gap in Evolutionary Robotics by Promoting Transferable Controllers
The reality gap, that often makes controllers evolved in simulation inefficient once transferred onto the real system, remains a critical issue in Evolutionary Robotics (ER); it prevents ER application to real-world problems. We hypothesize that this gap mainly stems from a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: best solutions in simulation often rely on bad simulated phenomena (e.g. the most dynamic ones). This hypothesis leads to a multi-objective formulation of ER in which two main objectives are optimized via a Pareto-based Multi-Objective Evolutionary Algorithm: (1) the fitness and (2) the transferability. To evaluate this second objective, a simulation-to-reality disparity value is approximated for each controller. The proposed method is applied to the evolution of walking controllers for a real 8-DOF quadrupedal robot. It successfully finds effi- cient and well-transferable controllers with only a few experiments in reality.
GECCO'10: Proceedings of the 12th annual conference on Genetic and evolutionary computation Conference on Genetic and Evolutionary Computationproceeding with peer review 2010-07
oai:hal.upmc.fr:hal-00473153
Automatic system identification based on coevolution of models and tests
In evolutionary robotics, controllers are often designed in simulation, then transferred onto the real system. Nevertheless, when no accurate model is available, controller transfer from simulation to reality means potential performance loss. It is the reality gap problem. Unmanned aerial vehicles are typical systems where it may arise. Their locomotion dynamics may be hard to model because of a limited knowledge about the underlying physics. Moreover, a batch identification approach is difficult to use due to costly and time consuming experiments. An automatic identification method is then needed that builds a relevant local model of the system concerning a target issue. This paper deals with such an approach that is based on coevolution of models and tests. It aims at improving both modeling and control of a given system with a limited number of manipulations carried out on it. Experiments conducted with a simulated quadrotor helicopter show promising initial results about test learning and control improvement.
CEC'09: Proceedings of the Eleventh Congress on Evolutionary Computation Eleventh Congress on Evolutionary Computation (CEC'09)proceeding with peer review 2009
oai:hal.upmc.fr:hal-00473147
Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity
The bootstrap problem is often recognized as one of the main challenges of evolutionary robotics: if all individuals from the first randomly generated population perform equally poorly, the evolutionary process won't generate any interesting solution. To overcome this lack of fitness gradient, we propose to efficiently explore behaviors until the evolutionary process finds an individual with a non-minimal fitness. To that aim, we introduce an original diversity-preservation mechanism, called behavioral diversity, that relies on a distance between behaviors (instead of genotypes or phenotypes) and multi-objective evolutionary optimization. This approach has been successfully tested and compared to a recently published incremental evolution method (multi-subgoal evolution) on the evolution of a neuro-controller for a light-seeking mobile robot. Results obtained with these two approaches are qualitatively similar although the introduced one is less directed than multi-subgoal evolution.
CEC'09: Proceedings of the Eleventh conference on Congress on Evolutionary Computation Eleventh conference on Congress on Evolutionary Computation (CEC'09)proceeding with peer review 2009
oai:hal.upmc.fr:hal-00473135
Evolving modular neural-networks through exaptation
Despite their success as optimization methods, evolutionary algorithms face many difficulties to design artifacts with complex structures. According to paleontologists, living organisms evolved by opportunistically co-opting characters adapted to a function to solve new problems, a phenomenon called \emph{exaptation}. In this paper, we draw the hypotheses (1) that exaptation requires the presence of multiple selection pressures, (2) that Pareto-based multi-objective evolutionary algorithms (MOEA) can create such pressures and (3) that the modularity of the genotype is a key to enable exaptation. To explore these hypotheses, we designed an evolutionary process to find the structure and the parameters of neural networks to compute a Boolean function with a modular structure. We then analyzed the role of each component using a Shapley value analysis. Our results show that: (1) the proposed method is efficient to evolve neural networks to solve this task; (2) genotypic modules and multiple selections gradients needed to be aligned to converge faster than the control experiments. This prominent role of multiple selection pressures contradicts the basic assumption that underlies most published modular methods for the evolution of neural networks, in which only the modularity of the genotype is considered.
CEC'09: Proceedings of the Eleventh conference on Congress on Evolutionary Computation Eleventh conference on Congress on Evolutionary Computation (CEC'09)proceeding with peer review 2009
oai:hal.upmc.fr:hal-00687617
The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics
The reality gap, that often makes controllers evolved in simulation inefficient once transferred onto the physical robot, remains a critical issue in Evolutionary Robotics (ER). We hypothesize that this gap highlights a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: the most efficient solutions in simulation often exploit badly modeled phenomena to achieve high fitness values with unrealistic behaviors. This hypothesis leads to the Transferability approach, a multi-objective formulation of ER in which two main objectives are optimized via a Pareto-based Multi-Objective Evolutionary Algorithm: (1) the fitness and (2) the transferability, estimated by a simulation-to-reality (STR) disparity measure. To evaluate this second objective, a surrogate model of the exact STR disparity is built during the optimization. This Transferability approach has been compared to two reality-based optimization methods, a noise-based approach inspired from Jakobis minimal simulation methodology and a local search approach. It has been validated on two robotic applications: 1) a navigation task with an e-puck robot; 2) a walking task with an 8-DOF quadrupedal robot. For both experimental set-ups, our approach successfully finds efficient and well-transferable controllers only with about ten experiments on the physical robot.
IEEE Transactions on Evolutionary Computationpeer-reviewed article 2012
oai:hal.upmc.fr:hal-00687621
On the Relationships between Synaptic Plasticity and Generative Systems.
The present paper analyzes the mutual relationships between generative and developmental systems (GDS) and synaptic plasticity when evolving plastic artificial neural networks (ANNs) in reward-based scenarios. We first introduce the concept of synaptic Transitive Learning Abilities (sTLA), which reflects how well an evolved plastic ANN can cope with learning scenarios not encountered during the evolution process. We subsequently report results of a set of experiments designed to check that (1) synaptic plasticity can help a GDS to fine-tune synaptic weights and (2) that with the investigated generative encoding (EvoNeuro), only a few learning scenarios are necessary to evolve a general learning system, which can adapt itself to reward-based scenarios not tested during the fitness evaluation.
GECCO'11: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation GECCO'11proceeding with peer review 2011
oai:hal.upmc.fr:hal-00687630
Stochastic optimization of a chain sliding mode controller for the mobile robot maneuvering
In this study we present a chain sliding mode controller for the control of a four wheeled autonomous mobile robot performing aggressive turning maneuver to 90 degrees on a slippery surface. The controller consists of a set of local sliding mode controllers and the hyperplanes of switching between them. The parameters of the sliding mode controllers and the hyperplanes are obtained using methods of multiobjective stochastic optimization applied to a model of the robot. The obtained controller is used to drive the mobile robot. The results show that the controller allowed the robot to execute the aggressive maneuver. Moreover, the turn radius obtained with the controller was twice less than the minimal turn radius admitted by the robot's geometry and the steering system.
International Conference on Intelligent Robots and Systems (IROS) International Conference on Intelligent Robots and Systems (IROS)proceeding with peer review 2011
oai:hal.upmc.fr:hal-00687639
Importing the Computational Neuroscience Toolbox into Neuro-Evolution---Application to Basal Ganglia
Neuro-evolution and computational neuroscience are two scientific domains that produce surprisingly different artificial neural networks. Inspired by the "toolbox" used by neuroscientists to create their models, this paper argues two main points: (1) neural maps (spatially-organized identical neurons) should be the building blocks to evolve neural networks able to perform cognitive functions and (2) well-identified modules of the brain for which there exists computational neuroscience models provide well-defined benchmarks for neuro-evolution. To support these claims, a method to evolve networks of neural maps is introduced then applied to evolve neural networks with a similar functionality to basal ganglia in animals (i.e. action selection). Results show that: (1) the map-based encoding easily achieves this task while a direct encoding never solves it; (2) this encoding is independent of the size of maps and can therefore be used to evolve large and brain-like neural networks; (3) the failure of direct encoding to solve the task validates the relevance of action selection as a benchmark for neuro-evolution.
GECCO'10: Proceedings of the 12th annual conference on Genetic and evolutionary computation GECCO'10proceeding with peer review 2010
oai:hal.upmc.fr:hal-00687633
Sferes_v2: Evolvin' in the Multi-Core World
This paper introduces and benchmarks Sferesv2, a C++ framework designed to help researchers in evolutionary computation to make their code run as fast as possible on a multi-core computer. It is based on three main concepts: (1) including multi-core optimizations from the start of the design process; (2) providing state-of-the art implementations of well-selected current evolutionary algorithms (EA), and especially multiobjective EAs; (3) being based on modern (template-based) C++ techniques to be both abstract and efficient. Benchmark results show that when a single core is used, running time of classic EAs included in Sferesv2 (NSGA-2 and CMA-ES) are of the same order of magnitude than specialized C code. When n cores are used, typical speed-ups range from 0.75n to 0.9n; however, parallelization efficiency critically depends on the time to evaluate the fitness function.
Evolutionary Computation (CEC), 2010 IEEE Congress on CEC 2010proceeding with peer review 2010
oai:hal.upmc.fr:hal-00687646
MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars
Recent work in the evolutionary computation field suggests that the implementation of the principles of modularity (functional localization of functions), repetition (multiple use of the same sub-structure) and hierarchy (recursive composition of sub-structures) could improve the evolvability of complex systems. The generation of neural networks through evolutionary algorithms should in particular benefit from an adapted use of these notions. We have consequently developed modular encoding for neural networks based on attribute grammars (MENNAG), a new encoding designed to generate the structure of neural networks and parameters with evolutionary algorithms, while explicitly enabling these three above-mentioned principles. We expressed this encoding in the formalism of attribute grammars in order to facilitate understanding and future modifications. It has been tested on two preliminary benchmark problems: cart-pole control and robotic arm control, the latter being specifically designed to evaluate the repetition capabilities of an encoding. We compared MENNAG to a direct encoding, ModNet, NEAT, a multi-layer perceptron with a fixed structure and to reference controllers. Results show that MENNAG performs better than comparable encodings on both problems, suggesting a promising potential for future applications.
Evolutionary Intelligencepeer-reviewed article 2008
Soutenance
Thèse: PRESSIONS SELECTIVES MULTIPLES POUR L'EVOLUTION DE RESEAUX DE NEURONES DESTINES A LA ROBOTIQUE
Soutenance: 2008-12-05