mjlab bed-making: two G1s make a bed — pure MuJoCo-Warp, over MHS (closes #8) - #11
Open
armwaheed wants to merge 2 commits into
Open
mjlab bed-making: two G1s make a bed — pure MuJoCo-Warp, over MHS (closes #8)#11armwaheed wants to merge 2 commits into
armwaheed wants to merge 2 commits into
Conversation
…, over MHS (closes #8) Adds the pure-mjlab / MuJoCo-Warp bed-making example: a Unitree G1 whole-body bed-reach RL task (FALCON station-keeping + end-effector force curriculum), the trained policy (policy/bed_reach_g1.onnx, obs[103]->action[29]), and a TWO-robot demo in which two G1s flank a correctly-proportioned wide low bed and make it together while coordinating over the Model Hardware Standard (MHS). No Isaac, no Newton, nothing NVIDIA-proprietary. demo/bed_scene.py — ONE MuJoCo model with two mjlab-configured G1s (MjSpec.attach; each keeps its own floating base, actuators, IMU sensors). Bed geometry copied verbatim from examples/isaac_bed_making/scene.py: mattress 2.0x1.8x0.61 (wide, low, long), headboard, two propped pillows, a light-blue cover that drapes off the sides. demo/g1_policy.py — rebuilds the exact 103-dim mjlab observation from raw MuJoCo state and runs the ONNX per robot. The +y robot runs the same single-hand policy through a sagittal mirror (L/R joint swap + roll/yaw sign flip), so one policy drives a bimanual pair. demo/two_robot_bed.py — the rollout + render; reuses the engine-agnostic MHS coordination layer from examples/isaac_bed_making (peers claim corners, emit events, trade askForHelp/offerHelp, reach goalReached 4/4; trace -> media/mhs_trace.json). Honesty (per the issue): the balanced whole-body reach, the hold-under-load, and the ~0.1 m headward hand draw are the trained policy; the grip and the cover's full bed-making travel are a documented grip-lock abstraction (a one-stroke full draw across a 1.8 m bed needs the retrain DESIGN.md describes). Bed proportions and the two-robot MHS coordination are real. Bundles the 5 reused isaac_bed_making MHS-coordination modules so the example is self-contained; media/ has bed_making.mp4, representative frames, and the captured trace. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d-over-hand (refs #8) Replaces the rigid plank-like mocap "cover" and its scripted grip-lock stroke with a native MuJoCo flexcomp cloth that the two G1s actually pull on. Nothing translates the sheet kinematically any more: every centimetre it moves comes from the trained policy's own hand motion, through contact and constraints. demo/cloth_sheet.py (new) — authors the sheet: a 2-D flex, 504 vertices / 918 triangles, with continuum membrane + bending elasticity. Emitted as flexcomp type="direct" so the sheet's stress-free rest shape IS the unmade shape it starts in (authored flat and then displaced, the stored bending energy fires it off the bed on step one). Also owns the collision bits: the cloth sees the bed, pillows, headboard, floor and the two hand capsules, not the robots' legs. demo/bed_scene.py — builds the world from XML (flexcomp is an XML-level macro) and attaches the two mjlab G1s onto that spec; adds 8 connect equalities per robot for the grip; tiles the bed with 54 one-geom pad bodies; adds a coverage metric. demo/two_robot_bed.py — six hand-over-hand strokes instead of one scripted draw: grip, draw, release, reach back footward, re-grip whatever cloth is under the palm. Streams frames to the encoder instead of buffering ~5 GB, and reports solver warnings so a blown-up run cannot pass as a good one. The open question from the issue thread — does MuJoCo flex hold a frictional pinch grasp — is answered no, and not for a solver reason: the mjlab G1 has no fingers (each hand is one rigid capsule), so the only available grasp is a palm press, and a swept measurement (press depth 0-30 mm x friction 1/3/10) moves the sheet by <=3% of the palm's travel in every cell. The grip is therefore a constraint on the handful of cloth under the palm, released the instant the hand opens; one pinned vertex is not enough (6.6 cm of vertex travel produced 2 cm of sheet). Three findings that cost real time, all written down in the README so the next person does not repay them: MuJoCo caps contacts at 50 per BODY pair, so a 504-vertex sheet on a single mattress body is supported at 50 points and sinks through it (hence the pad bodies); the flex rest shape must be authored, not displaced; and cloth stiffness is capped by the 5 ms training timestep -- a stiffer, more realistic sheet needs a 2 ms step, which the policy does not survive. Honest about what is still an abstraction: the grip (no fingers to pinch with), the cloth being softer than real cotton, and the cloth-vs-leg collision filter. The bed geometry, the two-robot MHS coordination, and now the sheet's physics are real. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #8 — the pure-mjlab / MuJoCo-Warp bed-making demo, remade so it reads unmistakably as two humanoids making a bed (replacing the rejected single-robot "dresser"). Self-contained example on
main, the same shape as the Newton twin in #10.What the demo shows
media/bed_making.mp4(3/4 hero shot, 1280×720): two Unitree G1s flank a 2.0 × 1.8 × 0.61 m wide low bed — headboard, two propped pillows, a light-blue cover that drapes off the sides and is drawn headward toward the pillows. Bed geometry is copied verbatim fromexamples/isaac_bed_making/scene.py. Nothing Isaac / Newton / NVIDIA-proprietary:mujoco+ the mjlab G1 asset.How it works (
examples/mjlab_bed_making/demo/)bed_scene.py— ONE MuJoCo model with two mjlab-configured G1s viaMjSpec.attach(each keeps its own floating base, actuators, IMU sensors).g1_policy.py— rebuilds the exact 103-dim mjlab observation from raw MuJoCo state and runs the trainedpolicy/bed_reach_g1.onnxper robot. The +y robot runs the same single-hand policy through a sagittal mirror (L/R joint swap + roll/yaw sign flip) → one policy drives a bimanual pair. Validated: both robots stay planted/balanced the whole clip and their reach envelopes are exact mirror images.two_robot_bed.py— rollout + render, and reuses the engine-agnostic MHS coordination layer fromexamples/isaac_bed_making/(bundled here so the example is self-contained). The two peers claim corners, emit events, tradeaskForHelp/offerHelp, and reachgoalReached4/4; the message-flow trace is written tomedia/mhs_trace.json.Trained policy vs. abstraction (per the issue, stated plainly)
bed_reach_env_cfg.py/mdp/bed_reach_terms.py(DESIGN.md).ReachCommandbox + retrainDESIGN.mddescribes. Bed proportions and the two-robot MHS coordination are real.Notes
isaac_bed_makingMHS-coordination modules (coordination.py,swarm_driver.py,mhs_trace.py,mhs_sidecar.py,__init__.py) so the demo runs without the full Isaac example present.🤖 Generated with Claude Code