Quentin Le Lidec

I am a postdoctoral researcher at the Courant Institute of Mathematical Sciences at New York University, working with Yann LeCun on learning world models for robotics.

My research interests are in machine learning, optimization, and computer vision, particularly as applied to robotics. I also work on the open-source Pinocchio and Simple libraries, making differentiable physics tools accessible for the robotics community.

Prior to that, I received my PhD in computer science from ENS Paris where I was working under the supervision of Justin Carpentier, Ivan Laptev, and Cordelia Schmid. Even before that, I received an engineering degree from École Polytechnique and an MSc in Mathematics, Computer Vision, and Machine Learning from École Normale Supérieure Paris-Saclay.

Email  /  Scholar  /  Github  /  LinkedIn

News

• Dec 2024: I defended my PhD on Differentiable optimization for robotics: simulation, learning and control.
• Nov 2024: I organized a workshop on Differentiable Optimization Everywhere: Simulation, Estimation, Learning, and Control at CORL'24 in Munich!
• Jul 2024: The paper From Compliant to Rigid Contact Simulation: a Unified and Efficient Approach was accepted to RSS'24!

Research

I'm interested in machine learning, optimization, and computer vision, particularly as applied to robotics. My work focuses on differentiable physics simulation, contact modeling, and learning-based approaches for robot control and manipulation.

A novel framework combining diffusion policy with trajectory optimization, using gradient-based solvers to generate locally optimal trajectories and leveraging their feedback gains to enrich Sobolev training with first-order information for more efficient policy learning.

Optimization is ubiquitous in modern robotics. This thesis aims to bring fundamental robotics tools to the differentiable programming paradigm, looking at robot simulation through the lens of numerical optimization techniques and proposing approaches for learning controllers.

A unified and efficient algorithmic solution for computing analytical derivatives of robotic simulators, achieving state-of-the-art timings from 5 microseconds for a 7-dof manipulator up to 95 microseconds for 36-dof humanoid.

A unified approach to solving contact interactions using ADMM and proximal algorithms, handling both compliant and rigid contact interfaces with automatic hyperparameter adaptation.

Accelerated collision detection by establishing GJK as a Frank-Wolfe algorithm and introducing Nesterov acceleration, leading to up to two times faster computation.

Algorithmic correction of the RaiSim contact algorithm to properly handle the maximum dissipation principle for more physically-consistent simulation.

Comprehensive survey and benchmark of contact models in robotics simulators, exposing physical relaxations and computational limitations with open-source implementations.

Combining reinforcement learning and trajectory optimization using Sobolev learning and augmented Lagrangian techniques to learn global control policies faster.

Generic approach to compute derivatives of collision detection for convex shapes using randomized smoothing techniques, with microsecond computation times.

Showing collision detection is fundamentally a convex optimization problem and introducing accelerated collision detection using Nesterov acceleration.

Leveraging randomized smoothing to estimate gradients in optimization problems involving non-smooth physical processes for mesh reconstruction and optimal control.

Introducing Randomized Differential Dynamic Programming (R-DDP) to handle nonsmooth dynamics with contacts and friction in a sample-efficient way.

Studying differentiable renderers through randomized optimization and perturbed optimizers, with applications to 6D pose estimation and 3D mesh reconstruction.

Differentiable simulation framework for accurate estimation of friction coefficients and object masses using staggered projections and analytical derivatives.

Code

I contribute to several open-source robotics libraries, making differentiable physics tools accessible for the robotics community.

A highly efficient library for differentiable simulation of multibody systems with contacts. Simple provides analytical derivatives for rigid body dynamics and contact interactions, enabling gradient-based optimization for robotics applications.

A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives. Pinocchio is widely used in robotics for motion planning, control, and simulation of complex robotic systems.

This repository contains C++ implementations of physical formulations of contacts and their associated numerical solvers. It also provides benchmarks between these different contact formulations to evaluate their physical accuracy and computational efficiency.

A simple differentiable QCQP (Quadratically Constrained Quadratic Programming) solver with Python bindings that can be used in PyTorch for end-to-end optimization workflows.

Miscellanea

Academic Service: Reviewer for NeurIPS, ICML, RSS, ICRA, IROS, T-RO, RA-L, L4DC, Humanoids.

Teaching: Teaching assistant for "Convex optimization" course at MSc MVA, ENS Paris-Saclay.

Website template based on Jon Barron's source code.