Tactile and proprioceptive feedback condition a learned behavior prior. Downstream policies act in its latent space to learn in-hand reorientation and coordinated arm–hand manipulation.
Multi-scale specialist policies are distilled into a contact-aware prior, then reused through residual latent actions. We evaluate transfer to new object shapes, commanded-axis rotation, and grasp-to-pose tasks, with sim-to-real demonstrations.
Preprint · 2026
Grounding Generated Video Plans in Simulation Towards Versatile Dexterous Controllers
Generated human–object interactions become 3D motion references. Reinforcement learning grounds those references in simulation to train dexterous controllers for real-world execution.
A metric hand–object reconstruction pipeline connects generated videos to simulation. Reference-tracking policies are trained and distilled across objects and trajectories; real-robot evaluations cover grasping, pushing, pulling, and pose adjustment.
CoRL · 2026
Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction
Simulation supplies contact information missing from human video. Visual-tactile policies learn from these reconstructed interactions and transfer to a real dexterous hand.
A privileged reinforcement-learning teacher tracks reconstructed human–object demonstrations. Distillation produces a policy driven by point clouds, tactile signals, and proprioception, enabling zero-shot transfer to real grasping and tool-use tasks.
CoRL · 2026
FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-rich Manipulation
A generalist tactile policy learns from heterogeneous touch sensors. A shared, morphology-aware token space supports transfer to new sensors and robot configurations.
Sensor-specific encoders map image-, array-, and state-based touch into a shared tactile representation. Pretraining spans approximately 3,000 hours and 21 sensors; downstream finetuning evaluates transfer across five hardware configurations.
IROS · 2026
Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
An RL-trained in-hand copilot assists both human data collection and robot execution. MoDE-VLA brings force and tactile feedback into a vision-language-action model for bimanual tasks.
IMCopilot provides stable grasping and in-hand rotation as callable skills. A mixture-of-experts module incorporates force and tactile signals into a pretrained VLA, evaluated on apple peeling, gear assembly, charger plugging, and tube rearrangement.
IEEE Transactions on Robotics · 2026
EROAM: Event-based Camera Rotational Odometry and Mapping in Real-time
Wanli Xing, Shijie Lin, Linhan Yang, Zeqing Zhang, Yanjun Du, Maolin Lei, Yipeng Pan, Chen Wang, Jia Pan
Events projected onto a sphere support high-rate camera rotation estimation and detailed panoramic mapping, with bounded map density for long sequences.
Event Spherical ICP directly aligns event geometry on the unit sphere. Incremental map management keeps computation efficient for CPU-based rotational tracking at 1 kHz. The system estimates rotation rather than full six-degree-of-freedom motion.
IROS · 2025
EventSync: Joint Recovery of Temporal Offsets and Relative Orientations for Wide-Baseline Event Cameras
Wanli Xing, Shijie Lin, Guangze Zheng, Linhan Yang, Yanjun Du, Jia Pan
Joint recovery of temporal offsets and relative orientations for wide-baseline event cameras.
The related 2023 preprint uses shared object motion and epipolar geometry to synchronize cameras and estimate relative orientation. The link below identifies that earlier version separately from the IROS 2025 publication.
Earlier version: Simultaneous Synchronization and Calibration for Wide-baseline Stereo Event Cameras (2023)
Nature Communications · 2024
Embodied neuromorphic synergy for lighting-robust machine vision to see in extreme bright
Neuromorphic events anticipate changes in scene illumination and guide camera exposure, improving visual perception through extreme lighting transitions.
A peripheral event-based irradiance estimator drives exposure control before conventional frame feedback can react. Evaluations include tunnel-exit driving, mixed-reality hand tracking, and 3D reconstruction.
Figure credit: Lin et al., Nature Communications (2024), Figure 4. Reproduced unchanged under CC BY-NC-ND 4.0. License
IEEE Robotics and Automation Letters · 2023
Target-free Extrinsic Calibration of Event-LiDAR Dyad using Edge Correspondences
Wanli Xing, Shijie Lin, Lei Yang, Jia Pan
Geometric and reflectivity edges connect event-camera measurements with LiDAR point clouds for target-free extrinsic calibration.
A dense LiDAR map and motion-compensated event edges provide correspondences for estimating the rigid sensor transform. Indoor and outdoor experiments evaluate calibration without a specialized target or controlled lighting.
