System Overview
My workflow is designed for repeatability: gesture-driven intent → stable Cartesian control → simulation-first validation → real robot deployment → logging + evaluation. This keeps iteration fast, safe, and measurable.
Key idea: The UX is driven by hand gestures. Users select a mode/axis using a menu
(lock/unlock), then issue positive/negative action commands. The same control interface works in
MuJoCo and on the Franka FR3 for Sim2Real validation.
Sim2Real Evidence (3 images)
1) System pipeline: EMG+IMU → feature extraction → SVM gesture classifier → UX menu → command generator →
MuJoCo ↔ Real Franka FR3 (same command structure).
2) Simulation (MuJoCo): The controller is validated in repeatable scenarios (fast rollouts, safe iteration) before hardware trials.
3) Real robot (Franka FR3): Hardware validation of the same gesture-driven workflow in a lab setup (contact-rich task).
Detailed quantitative outcomes (pose error, forces/torques, learning effects, and user questionnaires) are shown on the Results page.
Core Blocks (what each part does)
1) Input + Intent (Gestures → Commands)
- EMG + IMU → gesture prediction (SVM)
- UX menu / state machine to select mode/axis (lock/unlock)
- Action gestures produce + / − motion commands
- Safety: dead-man (rest), bounded steps, rate limiting
EMG/IMUSVMUX MenuSafety
2) Control Layer (Stable Robot Motion)
- Commanded Cartesian deltas mapped to joints (Jacobian / IK)
- Stable motion through smoothing + bounded updates
- Orientation handled explicitly (keep fixed or controlled rotation)
- Optional compliance logic (impedance-style behavior) for contact tasks
Cartesian ControlJacobian IKStability
3) Plant (Simulation + Real Hardware)
- MuJoCo first: fast iteration, repeatable scenarios, contact testing
- Real FR3 second: validate behavior under real dynamics + friction
- Same interface format (commands + feedback) for Sim2Real consistency
MuJoCoFranka FR3Sim2Real
4) Evaluation (Measurable Outcomes)
- Log commands, states, success/failure reasons
- Compute metrics (error, forces/torques, time, stability)
- Use results page to show evidence (plots, tables, videos)
MetricsLoggingReproducibility
Stability & Safety Strategies I Use
- Command bounding: clamp Δx/Δy/Δz and Δrot per step
- Rate limiting + smoothing: limit command frequency and filter spikes
- Lock/unlock UX: only move when an axis/mode is explicitly locked
- Hold behavior: maintain target pose to prevent drift/sagging
- Failure visibility: log “why it failed” (stale, drift, collision, instability)
Note: Detailed quantitative results (pose error / force / torque / workload questionnaires) are shown on the Results page.