publications

2024

1. End-to-End and Highly-Efficient Differentiable Simulation for Robotics

   Quentin Le Lidec, Louis Montaut, De Mont-Marin Yann, and 1 more author

   In preprint , May 2024

   Abs arXiv Bib PDF Code Website

   Over the past few years, robotics simulators have largely improved in efficiency and scalability, enabling them to generate years of simulated data in a few hours. Yet, efficiently and accurately computing the simulation derivatives remains an open challenge, with potentially high gains on the convergence speed of reinforcement learning and trajectory optimization algorithms, especially for problems involving physical contact interactions. This paper contributes to this objective by introducing a unified and efficient algorithmic solution for computing the analytical derivatives of robotic simulators. The approach considers both the collision and frictional stages, accounting for their intrinsic nonsmoothness and also exploiting the sparsity induced by the underlying multibody systems. These derivatives have been implemented in C++, and the code will be open-sourced in the Simple simulator. They depict state-of-the-art timings ranging from 5 microseconds for a 7-dof manipulator up to 95 microseconds for 36-dof humanoid, outperforming alternative solutions by a factor of at least 100.

   @inproceedings{simple2024gradients,
     title = {End-to-End and Highly-Efficient Differentiable Simulation for Robotics},
     author = {Le Lidec, Quentin and Montaut, Louis and Yann, De Mont-Marin and Carpentier, Justin},
     url = {https://simple-robotics.github.io/publications/simple-contact-solver/},
     booktitle = {preprint},
     year = {2024},
     address = {},
     month = may
   }

2. From Compliant to Rigid Contact Simulation: a Unified and Efficient Approach

   Justin Carpentier, Quentin Le Lidec, and Louis Montaut

   In Proceedings of Robotics: Science and Systems , Jul 2024

   Abs arXiv Bib PDF Code Website

   Whether rigid or compliant, contact interactions are inherent to robot motions, enabling them to move or manipulate things. Contact interactions result from complex physical phenomena, that can be mathematically cast as Nonlinear Complementarity Problems (NCPs) in the context of rigid or compliant point contact interactions. Such a class of complementarity problems is, in general, difficult to solve both from an optimization and numerical perspective. Over the past decades, dedicated and specialized contact solvers, implemented in modern robotics simulators (e.g., Bullet, Drake, MuJoCo, DART, Raisim) have emerged. Yet, most of these solvers tend either to solve a relaxed formulation of the original contact problems (at the price of physical inconsistencies) or to scale poorly with the problem dimension or its numerical conditioning (e.g., a robotic hand manipulating a paper sheet). In this paper, we introduce a unified and efficient approach to solving NCPs in the context of contact simulation. It relies on a sound combination of the Alternating Direction Method of Multipliers (ADMM) and proximal algorithms to account for both compliant and rigid contact interfaces in a unified way. To handle ill-conditioned problems and accelerate the convergence rate, we also propose an efficient update strategy to adapt the ADMM hyperparameters automatically. By leveraging proximal methods, we also propose new algorithmic solutions to efficiently evaluate the inverse dynamics involving rigid and compliant contact interactions, extending the approach developed in MuJoCo. We validate the efficiency and robustness of our contact solver against several alternative contact methods of the literature and benchmark them on various robotics and granular mechanics scenarios. Our code is made open-source.

   @inproceedings{simple2024contacts,
     title = {From Compliant to Rigid Contact Simulation: a Unified and Efficient Approach},
     author = {Carpentier, Justin and Le Lidec, Quentin and Montaut, Louis},
     url = {https://simple-robotics.github.io/publications/simple-contact-solver/},
     booktitle = {Proceedings of Robotics: Science and Systems},
     year = {2024},
     address = {Delft, Netherlands},
     month = jul
   }

3. GJK++: Leveraging Acceleration Methods for Faster Collision Detection

   Louis Montaut, Quentin Le Lidec, Vladimir Petrik, and 2 more authors

   In IEEE Transactions on Robotics , Mar 2024

   Abs Bib PDF

   Collision detection is a fundamental computational problem in various domains, such as robotics, computational physics, and computer graphics. In general, collision detection is tackled as a computational geometry problem, with the so-called Gilbert, Johnson, and Keerthi (GJK) algorithm being the most adopted solution nowadays. While introduced in 1988, GJK remains the most effective solution to compute the distance or the collision between two 3D convex geometries. Over the years, it was shown to be efficient, scalable, and generic, operating on a broad class of convex shapes, ranging from simple primitives (sphere, ellipsoid, box, cone, capsule, etc.) to complex meshes involving thousands of vertices. In this article, we introduce several contributions to accelerate collision detection and distance computation between convex geometries by leveraging the fact that these two problems are fundamentally optimization problems. Notably, we establish that the GJK algorithm is a specific sub-case of the well-established Frank-Wolfe (FW) algorithm in convex optimization. By adapting recent works linking Polyak and Nesterov accelerations to Frank-Wolfe methods, we also propose two accelerated extensions of the classic GJK algorithm. Through an extensive benchmark over millions of collision pairs involving objects of daily life, we show that these two accelerated GJK extensions significantly reduce the overall computational burden of collision detection, leading to up to two times faster computation timings. Finally, we hope this work will significantly reduce the computational cost of modern robotic simulators, allowing the speed-up of modern robotic applications that heavily rely on simulation, such as reinforcement learning or trajectory optimization.

   @inproceedings{montaut2024gjk,
     title = {GJK++: Leveraging Acceleration Methods for Faster Collision Detection},
     author = {Montaut, Louis and Le Lidec, Quentin and Petrik, Vladimir and Sivic, Josef and Carpentier, Justin},
     url = {https://hal.science/hal-04070039},
     booktitle = {IEEE Transactions on Robotics},
     year = {2024},
     address = {},
     month = mar,
     doi = {}
   }

4. Reconciling RaiSim with the Maximum Dissipation Principle

   Quentin Le Lidec, and Justin Carpentier

   In IEEE Transactions on Robotics , Jan 2024

   Abs Bib PDF

   Recent progress in reinforcement learning (RL) in robotics has been obtained by training control policy directly in simulation. Particularly in the context of quadrupedal locomotion, astonishing locomotion policies depicting high robustness against environmental perturbations have been trained by leveraging RaiSim simulator. While being more realistic than its counterparts, it has been shown recently that RaiSim does not obey the maximum dissipation principle, a fundamental principle when simulating rigid contact interactions. In this note, we detail these relaxations and propose an algorithmic correction of the RaiSim contact algorithm to handle the maximum dissipation principle adequately. Our experiments empirically demonstrate our approach leads to more physically-consistent simulation.

   @inproceedings{lelidec2024raisim,
     author = {Le Lidec, Quentin and Carpentier, Justin},
     title = {Reconciling RaiSim with the Maximum Dissipation Principle},
     url = {https://hal.science/hal-04067291},
     booktitle = {IEEE Transactions on Robotics},
     year = {2024},
     address = {},
     month = jan,
     doi = {}
   }

2023

1. Contact Models in Robotics: a Comparative Analysis

   Quentin Le Lidec, Wilson Jallet, Louis Montaut, and 3 more authors

   In IEEE Transactions on Robotics , Apr 2023

   Abs Bib PDF Code

   Physics simulation is ubiquitous in robotics. Whether in model-based approaches (e.g., trajectory optimization), or model-free algorithms (e.g., reinforcement learning), physics simulators are a central component of modern control pipelines in robotics. Over the past decades, several robotic simulators have been developed, each with dedicated contact modeling assumptions and algorithmic solutions. In this article, we survey the main contact models and the associated numerical methods commonly used in robotics for simulating advanced robot motions involving contact interactions. In particular, we recall the physical laws underlying contacts and friction (i.e., Signorini condition, Coulomb’s law, and the maximum dissipation principle), and how they are transcribed in current simulators. For each physics engine, we expose their inherent physical relaxations along with their limitations due to the numerical techniques employed. Based on our study, we propose theoretically grounded quantitative criteria on which we build benchmarks assessing both the physical and computational aspects of simulation. We support our work with an open-source and efficient C++ implementation of the existing algorithmic variations. Our results demonstrate that some approximations or algorithms commonly used in robotics can severely widen the reality gap and impact target applications. We hope this work will help motivate the development of new contact models, contact solvers, and robotic simulators in general, at the root of recent progress in motion generation in robotics.

   @inproceedings{lelidec2023contacts,
     author = {Le Lidec, Quentin and Jallet, Wilson and Montaut, Louis and Laptev, Ivan and Schmid, Cordelia and Carpentier, Justin},
     title = {Contact Models in Robotics: a Comparative Analysis},
     url = {https://hal.science/hal-04067291},
     booktitle = {IEEE Transactions on Robotics},
     year = {2023},
     address = {},
     month = apr,
     doi = {}
   }

2. Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control

   Quentin Le Lidec, Wilson Jallet, Ivan Laptev, and 2 more authors

   In 2023 - IEEE International Conference on Robotics and Automation (ICRA) , May 2023

   Abs Bib PDF

   Reinforcement learning (RL) and trajectory optimization (TO) present strong complementary advantages. On one hand, RL approaches are able to learn global control policies directly from data, but generally require large sample sizes to properly converge towards feasible policies. On the other hand, TO methods are able to exploit gradient-based information extracted from simulators to quickly converge towards a locally optimal control trajectory which is only valid within the vicinity of the solution. Over the past decade, several approaches have aimed to adequately combine the two classes of methods in order to obtain the best of both worlds. Following on from this line of research, we propose several improvements on top of these approaches to learn global control policies quicker, notably by leveraging sensitivity information stemming from TO methods via Sobolev learning, and augmented Lagrangian techniques to enforce the consensus between TO and policy learning. We evaluate the benefits of these improvements on various classical tasks in robotics through comparison with existing approaches in the literature.

   @inproceedings{lelidec2022policy,
     author = {Le Lidec, Quentin and Jallet, Wilson and Laptev, Ivan and Schmid, Cordelia and Carpentier, Justin},
     title = {Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control},
     url = {https://hal.archives-ouvertes.fr/hal-03780392v1},
     booktitle = {2023 - IEEE International Conference on Robotics and Automation (ICRA) },
     year = {2023},
     address = {London},
     month = may,
     doi = {}
   }

3. Differentiable Collision Detection: a Randomized Smoothing Approach

   Louis Montaut, Quentin Le Lidec, Antoine Bambade, and 3 more authors

   In 2023 - IEEE International Conference on Robotics and Automation (ICRA) , May 2023

   Abs Bib PDF

   Collision detection appears as a canonical operation in a large range of robotics applications from robot control to simulation, including motion planning and estimation. While the seminal works on the topic date back to the 80s, it is only recently that the question of properly differentiating collision detection has emerged as a central issue, thanks notably to the ongoing and various efforts made by the scientific community around the topic of differentiable physics. Yet, very few solutions have been suggested so far, and only with a strong assumption on the nature of the shapes involved. In this work, we introduce a generic and efficient approach to compute the derivatives of collision detection for any pair of convex shapes, by notably leveraging randomized smoothing techniques which have shown to be particularly adapted to capture the derivatives of non-smooth problems. This approach is implemented in the HPP-FCL and Pinocchio ecosystems, and evaluated on classic datasets and problems of the robotics literature, demonstrating few micro-second timings to compute informative derivatives directly exploitable by many real robotic applications including differentiable simulation.

   @inproceedings{montaut2022differentiable,
     title = {Differentiable Collision Detection: a Randomized Smoothing Approach},
     author = {Montaut, Louis and Le Lidec, Quentin and Bambade, Antoine and Petrik, Vladimir and Sivic, Josef and Carpentier, Justin},
     url = {https://arxiv.org/abs/2209.09012},
     booktitle = {2023 - IEEE International Conference on Robotics and Automation (ICRA) },
     year = {2023},
     address = {London},
     month = may,
     doi = {}
   }

2022

1. Augmenting differentiable physics with randomized smoothing

   Quentin Le Lidec, Louis Montaut, Cordelia Schmid, and 2 more authors

   In Robotics Science and Systems, Workshop on Differentiable Simulation For Robotics , Jun 2022

   Abs Bib PDF

   In the past few years, following the differentiable programming paradigm, there has been a growing interest in computing the gradient information of physical processes (e.g., physical simulation, image rendering). However, such processes may be non-differentiable or yield uninformative gradients (i.d., null almost everywhere). When faced with the former pitfalls, gradients estimated via analytical expression or numerical techniques such as automatic differentiation and finite differences, make classical optimization schemes converge towards poor quality solutions. Thus, relying only on the local information provided by these gradients is often not sufficient to solve advanced optimization problems involving such physical processes, notably when they are subject to non-smoothness and non-convexity issues. In this work, inspired by the field of zero-th order optimization, we leverage randomized smoothing to augment differentiable physics by estimating gradients in a neighborhood. Our experiments suggest that integrating this approach inside optimization algorithms may be fruitful for tasks as varied as mesh reconstruction from images or optimal control of robotic systems subject to contact and friction issues.

   @inproceedings{lelidec2022diffphys,
     author = {Le Lidec, Quentin and Montaut, Louis and Schmid, Cordelia and Laptev, Ivan and Carpentier, Justin},
     title = {{Augmenting differentiable physics with randomized smoothing}},
     url = {https://arxiv.org/abs/2206.11884},
     booktitle = {Robotics Science and Systems, Workshop on Differentiable Simulation For Robotics},
     year = {2022},
     address = {New York City, NY, USA},
     month = jun,
     doi = {}
   }

2. Collision Detection Accelerated: An Optimization Perspective

   Louis Montaut, Quentin Le Lidec, Vladimir Petrík, and 2 more authors

   In Proceedings of Robotics: Science and Systems , Jun 2022

   Abs Bib PDF Code

   Collision detection between two convex shapes is an essential feature of any physics engine or robot motion planner. It has been often tackled as a computational geometry problem, with the Gilbert, Johnson and Keerthi (GJK) algorithm being the most common approach today. In this work we show that collision detection is fundamentally a convex optimization problem. In particular, we establish that the GJK algorithm is a specific sub-case of the well-established Frank-Wolfe (FW) algorithm in convex optimization. We introduce a new collision detection algorithm by adapting recent works linking Nesterov acceleration and Frank-Wolfe methods. We benchmark the proposed accelerated collision detection method on two datasets composed of strictly convex and non-strictly convex shapes. Our results show that our approach significantly reduces the number of iterations to solve collision detection problems compared to the state-of-the-art GJK algorithm, leading to up to two times faster computation times.

   @inproceedings{montaut2022collacc,
     author = {Montaut, Louis and Le Lidec, Quentin and Petrík, Vladimir and Sivic, Josef and Carpentier, Justin},
     title = {{Collision Detection Accelerated: An Optimization Perspective}},
     url = {http://www.roboticsproceedings.org/rss18/p039.html},
     booktitle = {Proceedings of Robotics: Science and Systems},
     year = {2022},
     address = {New York City, NY, USA},
     month = jun,
     doi = {10.15607/RSS.2022.XVIII.039}
   }

2021

1. Leveraging randomized smoothing for optimal control of nonsmooth dynamical systems

   Quentin Le Lidec, Fabian Schramm, Louis Montaut, and 3 more authors

   Nonlinear Analysis: Hybrid Systems, Dec 2021

   Abs Bib PDF

   Optimal Control (OC) algorithms such as Differential Dynamic Programming (DDP) take advantage of the derivatives of the dynamics to efficiently control physical systems. Yet, in the presence of nonsmooth dynamical systems, such class of algorithms are likely to fail due, for instance, to the presence of discontinuities in the dynamics derivatives or because of non-informative gradient during the solving. On the contrary, Reinforcement Learning (RL) algorithms have shown better empirical results in scenarios exhibiting nonsmooth effects (contacts, frictions, etc). Our approach leverages recent works on Randomized Smoothing (RS) to tackle nonsmoothness issues commonly encountered in Optimal Control, and provides key insights on the interplay between RL and OC through the prism of RS methods. This naturally leads us to introduce the Randomized Differential Dynamic Programming (R-DDP) algorithm accounting for deterministic but non-smooth dynamics in a very sample-efficient way. The experiments demonstrate that our method is able to solve classic robotic problems with dry friction and frictional contacts, where classical OC algorithms are likely to fail and RL algorithms require in practice a prohibitive number of samples to find an optimal solution.

   @article{lelidec2021randomized,
     title = {{Leveraging randomized smoothing for optimal control of nonsmooth dynamical systems}},
     author = {Le Lidec, Quentin and Schramm, Fabian and Montaut, Louis and Schmid, Cordelia and Laptev, Ivan and Carpentier, Justin},
     journal = {Nonlinear Analysis: Hybrid Systems},
     volume = {52},
     pages = {101468},
     year = {2021},
     month = dec,
     url = {https://hal.archives-ouvertes.fr/hal-03480419},
     publisher = {Elsevier}
   }

2. Differentiable rendering with perturbed optimizers

   Quentin Le Lidec, Ivan Laptev, Cordelia Schmid, and 1 more author

   Advances in Neural Information Processing Systems, Dec 2021

   Abs Bib PDF Code

   Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision. Solutions to this inverse and ill-posed problem typically involve a search for models that best explain observed image data. Notably, images depend both on the properties of observed scenes and on the process of image formation. Hence, if optimization techniques should be used to explain images, it is crucial to design differentiable functions for the projection of 3D scenes into images, also known as differentiable rendering. Previous approaches to differentiable rendering typically replace non-differentiable operations by smooth approximations, impacting the subsequent 3D estimation. In this paper, we take a more general approach and study differentiable renderers through the prism of randomized optimization and the related notion of perturbed optimizers. In particular, our work highlights the link between some well-known differentiable renderer formulations and randomly smoothed optimizers, and introduces differentiable perturbed renderers. We also propose a variance reduction mechanism to alleviate the computational burden inherent to perturbed optimizers and introduce an adaptive scheme to automatically adjust the smoothing parameters of the rendering process. We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction. By providing informative gradients that can be used as a strong supervisory signal, we demonstrate the benefits of perturbed renderers to obtain more accurate solutions when compared to the state-of-the-art alternatives using smooth gradient approximations.

   @article{lelidec2021differentiable,
     title = {Differentiable rendering with perturbed optimizers},
     author = {Le Lidec, Quentin and Laptev, Ivan and Schmid, Cordelia and Carpentier, Justin},
     journal = {Advances in Neural Information Processing Systems},
     volume = {34},
     year = {2021},
   }

3. Differentiable simulation for physical system identification

   Quentin Le Lidec, Igor Kalevatykh, Ivan Laptev, and 2 more authors

   IEEE Robotics and Automation Letters, Dec 2021

   Abs Bib PDF Code

   Simulating frictional contacts remains a challenging research topic in robotics. Recently, differentiable physics emerged and has proven to be a key element in modelbased Reinforcement Learning (RL) and optimal control fields. However, most of the current formulations deploy coarse approximations of the underlying physical principles. Indeed, the classic simulators lose precision by casting the Nonlinear Complementarity Problem (NCP) of frictional contact into a Linear Complementarity Problem (LCP) to simplify computations. Moreover, such methods deploy non-smooth operations and cannot be automatically differentiated. In this paper, we propose (i) an extension of the staggered projections algorithm for more accurate solutions of the problem of contacts with friction. Based on this formulation, we introduce (ii) a differentiable simulator and an efficient way to compute the analytical derivatives of the involved optimization problems. Finally, (iii) we validate the proposed framework with a set of experiments to present a possible application of our differentiable simulator. In particular, using our approach we demonstrate accurate estimation of friction coefficients and object masses both in synthetic and real experiments.

   @article{lelidec2021diffsim,
     title = {Differentiable simulation for physical system identification},
     author = {Le Lidec, Quentin and Kalevatykh, Igor and Laptev, Ivan and Schmid, Cordelia and Carpentier, Justin},
     journal = {IEEE Robotics and Automation Letters},
     volume = {6},
     number = {2},
     pages = {3413--3420},
     year = {2021},
     publisher = {IEEE},
   }