TactiDex: A Real-World Tactile-Guided Benchmark for Human-Like Dexterous Manipulation

Suting Ni1,2, Hanbing Zhang1,2, Zhenyu Wei1, Guo Chen1, Chixuan Zhang1, Ye Shi1,2, Jingya Wang1,*,
1Shanghaitech University 2InstAdapt
*Corresponding author
Paper arXiv Video
TactiDex Data Collection and Annotation Framework

Overview of the TactiDex Data Collection and Annotation Framework. (Left) Our hardware setup integrates a high-precision motion capture system with a novel dual glove design—an inner capture glove for kinematics and an outer tactile glove for whole-hand pressure mapping, strictly synchronized via an eSync module. Manipulated objects are digitally reconstructed via 3D scanning. (Center) A user performs complex, contact-rich interactions (e.g., bimanual grinding) within the capture volume. (Right) The pipeline yields strictly synchronized multi-modal data streams, including tactile signals, hand kinematics, and hand-object 6D poses, enriched with comprehensive language descriptions and hierarchical task phase annotations.

Abstract

Tactile feedback is fundamental to Hand-Object Interaction (HOI), governing contact formation, force regulation, and stable manipulation, making it essential for achieving true human-like dexterous manipulation. Yet, current human-to-robot dexterous transfer pipelines primarily rely on kinematic trajectories, resulting in motion imitation without physically grounded interaction. To address this, we introduce TactiDex, a real-world tactile-guided benchmark specifically designed to move dexterous manipulation beyond kinematic mimicry toward contact-level human-likeness. TactiDex provides a comprehensive dataset that elegantly aligns whole-hand tactile signals with multi-granularity kinematic and object states, coupled with standardized evaluation metrics. Building upon this data paradigm, we propose a tactile-driven transfer framework that effectively translates human demonstrations into physically plausible robotic execution. We introduce TactiSkill, a framework built upon a novel tri-component tactile reward that innovatively uses tactile signals as structured supervision. This reward unifies guidance, human-like alignment, and contact constraints into a single objective. Through comprehensive experiments on both single and bimanual tasks, we demonstrate that TactiSkill achieves superior performance in manipulation success and physical realism. This work lays a crucial foundation for advancing tactile-aware dexterous manipulation.

Overview

Overview pipeline figure.
Overview of the TactiSkill Pipeline. (Left) Data Collection & Preparation: Raw multi-modal signals from human demonstrations undergo a two-stage tactile-constrained post-optimization and sensor-to-sim mapping, yielding the physically aligned TactiDex dataset. (Center) Tactile-Guided Tri-Component Transfer Framework: We train a residual tracking policy using an asymmetric Actor-Critic architecture, where the Critic receives privileged simulated contact forces. A physics-aware tri-component reward (Tactile Guidance, Human-Like Alignment, and Contact Constraints) enforces force-level human-likeness. (Right) Real-World Deployment: The composite policy enables zero-shot sim-to-real transfer, achieving physically grounded, human-like dexterous manipulation on actual robotic hardware across diverse tasks.

Video

Key Figures from the Paper and Appendix

Demonstrations of the TactiDex Dataset

Dataset visualization figure.

Multi-modal Human Demonstrations from the TactiDex Dataset. Each frame pairs the 3D kinematic hand-object interaction with its synchronized tactile pressure heat map (where purple indicates low pressure and yellow indicates high pressure). These sequences demonstrate how our dataset captures the precise, task-specific force distributions required for complex object manipulation.

Simulation Results

Simulation results figure.

Qualitative Results of TactiSkill. Transfer sequences for single-hand (LH/RH) and bimanual (BiH) tasks. Left: Optimized human HOI reference and whole-hand tactile prior. Right (colored panels): Five-frame execution rollout of the simulated dexterous hand. Guided by our tactile supervision, the policy establishes physically grounded contact, effectively avoiding unnatural hovering or penetrations.

Real-World Results

Real-world experiment figure.

Real-world Deployment. Execution on our bimanual Franka Inspire platform with contact-rich execution snapshots.

Qualitative Comparison

Qualitative comparison figure.

Qualitative Comparison of Contact Quality. Frame-by-frame rollouts of LH (TelephoneHand), RH (PingPang), and BiH (KettleBlack) tasks. (Left) The kinematic-only baseline often results in unnatural hovering or severe mesh penetration, as explicitly highlighted by the red magnified views. (Right) Our TactiSkill policy seamlessly establishes flush, stable contact with diverse object geometries, effectively preventing physical artifacts as shown in the green magnified views.

Dataset Demos

Simulation Demos

Real-World Demos

BibTeX

@misc{ni2026tactidexrealworldtactileguidedbenchmark,
  title         = {TactiDex: A Real-World Tactile-Guided Benchmark for Human-Like Dexterous Manipulation},
  author        = {Suting Ni and Hanbing Zhang and Zhenyu Wei and Guo Chen and Chixuan Zhang and Ye Shi and Jingya Wang},
  year          = {2026},
  eprint        = {2607.09190},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2607.09190},
}