AlbertPro

A 14 cm quadruped that learns to walk in MuJoCo — and walks on an ESP32.

Albert is a small 4-legged robot (14 × 11 × 2 cm body, 8 servos). This repo holds the whole sim2real loop: a PPO policy trained in MuJoCo, exported as a plain C header, and run on an ESP32 that drives the servos over I²C.

MuJoCo + PPO  ──►  policy.h / trajectory.h  ──►  ESP32 + PCA9685  ──►  8 servos
   (RL/)              (auto-generated)              (ESP32/)

It walks

The learned gait, one full cycle — the policy runs at 100 Hz and the body advances at ~0.25 m/s:

It learns

Same camera, same 8-second episode, four checkpoints from the same training run — round 49 flails, round 249 shuffles forward, by round 849 it is trotting:

round 49	round 249	round 849 → 2499
+0.01 m	+0.18 m	+0.53 m per 8 s episode

Full-resolution 1080p60 renders live in RL/movies/: albert_day.mp4 · albert_night.mp4 · albert_learning.mp4

Robot

Body	14 × 11 × 2 cm, ~0.13 m standing height
Legs	4 × (hip + knee) = 8 DOF, 5 cm upper / 5 cm lower
Actuators	8 servos via PCA9685 (I²C), position control
Brain	ESP32, Bluetooth serial for commands
Neutral pose	hip 0.90 rad, knee −1.40 rad

RL/ — training in MuJoCo

PPO + GAE on a ΔΔθ action space: the network outputs changes to a persistent delta buffer, i.e. acceleration-level joint control, which makes the resulting gait smooth by construction instead of by reward shaping.

State  (24): joint pos (8) + joint vel (8) + delta (8)
Action  (8): ΔΔθ  →  delta  →  joint targets
Policy:      24 → ReLU(64) → tanh(8)      2120 floats, 8.3 KB
Control:     100 Hz (10 ms), 5 physics substeps at 2 ms

Notebook	What it does
AllLegs_parallel.ipynb	4 MuJoCo worker processes (~4× faster) + the cinematic renderer
AllLegs_def.ipynb	reference implementation — sequential PPO
AllLegs_curriculum.ipynb	curriculum variant

Reward: forward-velocity tracking (target 0.25 m/s) + alive bonus + yaw penalty, with regularisation on actions, delta magnitude, joint velocity and joint-limit excursion.

cd RL
conda env create -f environment.yml
conda activate albert-rl
jupyter notebook AllLegs_parallel.ipynb

Training writes checkpoints to models/, curves to plots/, videos to movies/ (all but the best model and the showcase renders are gitignored — they are big).

Cinematic renderer

Any checkpoint can be re-rendered as finished footage:

render_cinematic('movies/albert_day.mp4',   weights=best_w, look='day')
render_cinematic('movies/albert_night.mp4', weights=best_w, look='night')
render_cinematic('movies/slowmo.mp4',       weights=best_w, slowmo=0.4)
render_progress_montage('movies/albert_learning.mp4', look='day', n_clips=8)

It generates dog_cinematic_<look>.xml from dog.xml — visuals only, bodies / joints / actuators are byte-identical, so the physics you film is the physics you trained. On top: a 4-shot camera choreography with a damped follow cam, motion blur from control-rate→fps resampling, and a bloom + vignette grade. Two looks — day (blue sky, procedural concrete slabs) and night (dark studio, glossy cyan grid).

ESP32/ — running it on the robot

The trained policy is exported two ways:

• policy.h — the network weights as C arrays plus a self-contained policy_step(state, ctrl, delta), for running the policy on-device at 100 Hz.
• trajectory.h — one gait cycle baked to a fixed-point table (angle_rad = OFFSET[j] + traj[step][j] / 10000.0), for open-loop playback.

Sketch	What it does
Albert_AML_traj/	plays back one exported gait cycle
Albert_AML_meta/	four gaits (forward / back / left / right), switched over Bluetooth

Both use Adafruit_PWMServoDriver (PCA9685) and BluetoothSerial. Send a single character over BT or USB serial: f forward, b back, l left, r right, s stop.

Joint angles map to servo pulses as pulse = CENTER + angle_rad * PULSE_PER_RAD * DIR[ch], with per-channel direction signs so left and right legs mirror correctly.

Flashing

1. Arduino IDE → board ESP32 Dev Module.
2. Install libraries: Adafruit PWM Servo Driver Library, Adafruit BusIO.
3. Copy the freshly exported header into the sketch folder (RL/export/trajectory.h → ESP32/Albert_AML_traj/trajectory.h).
4. Upload, power the servo rail separately from the ESP32, pair over Bluetooth.

Layout

RL/
  AllLegs_parallel.ipynb   parallel PPO + cinematic renderer
  AllLegs_def.ipynb        reference sequential PPO
  AllLegs_curriculum.ipynb curriculum variant
  _parallel_worker.py      subprocess rollout worker
  dog.xml                  MuJoCo model (physics + visuals)
  dog_cinematic_*.xml      generated: same physics, film visuals
  meshes/                  trunk / upper_leg / lower_leg STL
  models/                  best policy (.pth) + policy.h
  trajectories/            exported gait: .npy / .csv / .h
  movies/  plots/          showcase renders and training curves
ESP32/
  Albert_AML_traj/         single-gait playback sketch
  Albert_AML_meta/         four-gait sketch, Bluetooth switching
  trajectory.h             latest exported gait
docs/                      images used by this README

Assembled

simulation/ builds the whole robot -- printed parts and bought ones -- and drives it with the trained gaits, using solid-node. It adds nothing to the design and changes nothing in it: every printed part is one body of hardware/albert_pro.stl, every joint is placed and limited by RL/dog.xml, every gait is one of the tables in ESP32/, and every bought part is dimensioned by the printed part that receives it.

The model is two parallel assemblies of one machine:

printed	the 18 bodies of the print plate: two trunk shells, four legs of four pieces
sourced	8 micro servos, 8 horns, 16 M2 screws, 16 M2 nuts

They mirror each other joint for joint and are driven from one binding, so the hardware moves with the parts that hold it.

pip install "solid-node[viewer]"
solid build                          # build and publish the model
solid test --faceted simulation/albert.py   # the fast loop
solid test simulation/albert.py             # the certified run
solid develop                        # drive it in a browser

Driving it

Four sliders, not eight servo channels:

driver	range	what it does
height	67.4 – 86.9 mm	how tall Albert stands, measured trunk to floor, feet under their hips
travel	−1 … +1	traj_bwd ← stand → traj_fwd
heading	−1 … +1	traj_left ← straight → traj_right
trot	0 … 1	the meta-learned trot of ESP32/trajectory.h

All eight joints are still there underneath as ports, and each gait plays one pass of its own table over the timeline, so the robot walks on its own and stands still when the gait sliders are at zero.

Buttons: Rest, Stand, Crouch, Walk, Back, Turn, Trot.

Looking at it in pieces

Built from the ground up, each of these is a model in its own right:

solid build simulation/trunk.py:Trunk       # the two printed shells
solid build simulation/leg.py:LegBench      # one leg, hip and knee on sliders
solid build                                 # the whole robot

What the model found in the design

Assembling the parts and asking every question at once turned up twelve things the drawings do not say. All are measured; none is fixed here.

The hardware does not go together on the joint axes dog.xml declares. A hip servo bolted to its own two M2 bores puts its output shaft 2.5 mm from the thigh's 7 mm boss hole at the closest point of the posture range, opening to 4.5 mm at full crouch; at the knee the miss is 5.3 mm. A 7 mm boss cannot enter a 7 mm hole 2.5 mm away. Either the MJCF's joint positions are approximate, or those holes are not for bosses, or the servo is a variant this model has not identified -- the geometry alone cannot separate the three, and a builder holding the real parts would settle it in seconds.

The bolt pattern says SG90 and the shaft position does not. 22.95 mm of bay, a 28.000 mm bolt span, 2.003 mm bores and a 12.2 mm rail are the SG90/MG90S footprint, agreeing to three decimals across four independent bays. But the design wants the output shaft within a millimetre of the midpoint between the mounting bores, where an SG90's sits about 5.5 mm off it.

The hip and knee servos are fitted a quarter turn apart. The trunk's four bays present their 28 mm span along x; the four shin plates present theirs along z.

• The trained foot is not the printed foot. The printed shin reaches 57 mm below the knee. dog.xml puts the touch site at 42 and collides on a capsule bottoming at 50, so the policy learned to walk on a leg 7 mm shorter than the one that gets printed.
• 30% of the published trajectory commands are outside the joint limits. neutral + table/10000 leaves the declared ctrlrange on 4126 of 13752 commands. The left gait's front-left knee is outside on 397 of its 400 steps; the trot commands 1.400 rad on a hip limited to 0.998. MuJoCo clamps silently, so the motion that was rewarded is the clamped motion — but firmware that reads the headers and drives servos directly would command angles the joints do not have.
• All four thighs are buried in the trunk shell, 2.597 mm deep, in MuJoCo's own meshes at MuJoCo's own hip positions. Assembled from the printed parts it survives at 12.1 and 17.5 mm³.
• The printable trunk is 3 mm narrower in y than the one MuJoCo renders, on both plates. It relieves the thigh interference above, and it costs the two shells 31.5% of the lip they close on: the printed pair engage over 807.5 mm³ where the rendered pair engage over 1178.1.
• The hip, not the knee, limits how low Albert can crouch. Below 67.4 mm the hip passes its declared limit with the knee still 2.4° from its own. Solved on dog.xml's own 42 mm foot site the hip never binds anywhere — which is why the design does not record it.
• The trained neutral pose does not stand the foot under the hip, and which way it misses depends on which foot you mean: 3.83 mm forward on the printed shin, 3.36 mm aft on the MJCF's own site.
• The same thigh is fitted to all four corners unmirrored, and there is no abduction joint. The right legs bury 5.4 mm³ deeper into the trunk than the left ones, and the feet stand 126 mm apart across a 110 mm hip track and cannot be brought in.
• RL/meshes/lower_leg_.stl is not watertight — it carries a second, zero-volume degenerate body. Nothing uses it; it is left alone.

The measurements behind all of these are in docs/measurements.md and the reasoning is in openspec/changes/archive/.

How it is put together

simulation/
  albert.py            the machine: two assemblies, four drivers, seven buttons
  printed.py           everything Albert prints
  sourced.py           everything its builder buys, mirroring the same joints
  leg.py               Leg -> UpperLeg -> LowerLeg, nested as dog.xml nests them
  trunk.py             the two printed shells
  parts.py             the 18 plate bodies, one class each
  hardware.py          servo, horn, M2 screw, M2 nut
  layout.py            every dog.xml number, the registration, the stance
  hardware_layout.py   every mounting feature, measured off the printed parts
  gaits.py             the trained tables as a formula the browser can evaluate
  gait_tables.py       GENERATED from the ESP32 headers; regenerate, do not edit
  seats.py             where printed solids overlap on purpose, and by how much
  mjcf.py              dog.xml, parsed — for the tests only
  tools/               probe.py (read the plate and its features), gen_gaits.py

Two things are worth knowing if you change it. The gait is a formula, not a table lookup, because a viewer driver is a symbolic expression and cannot index 400 rows: a piecewise-linear curve is exactly a + b·u + Σ wᵢ|u−i|, and |x| is sqrt(x·x). It is exact at all 1719 samples. And gait_tables.py is committed rather than parsed at runtime, because the framework tracks Python imports and would not notice an edited .h; test_albert.py re-parses the headers and fails on any drift.

License

MIT — free to use, modify and redistribute, including commercially, as long as the copyright notice and licence text travel with it.