Loading…
Loading…
AI learns to drive is a Unity ML-Agents experiment in reward design. I built a small driving task, then changed observations and rewards after the car repeatedly became stuck or collided with obstacles. The existing video shows selected runs in which the agent reaches the target. It is an implementation demonstration; it does not measure generalization or reliable driving.
The initial agent repeated the same failed behavior around one region. A final goal reward gives little feedback when an episode ends before the goal. I tried intermediate signals to make progress visible to the learner.
I replaced manual raycasts with RayPerceptionSensorComponent3D. The configured sensor uses 20 rays over 180 degrees. It reports nearby obstacles through the ML-Agents observation path. A sensor helps the policy observe the environment; it cannot by itself prove that the policy uses those observations correctly.
I tried a reward for reducing distance to the target, with a bonus for heading toward it. I also added a penalty when the car made little progress over time. A separate version used six reward checkpoints before the goal, with a penalty based on missed checkpoints when the agent failed.
Distance rewards can favor a locally short route even when an obstacle blocks it. An alignment bonus can encourage pointing at the goal while making little useful progress. A stuck penalty can encourage motion without successful navigation. Checkpoint rewards can teach the training route while leaving unseen routes difficult.
I enabled and disabled combinations of these terms while experimenting. The selected runs improved enough to reach the goal around obstacles. This was an exploratory process. No fixed-seed ablation table, episode count or held-out layout result is available in the current record.
My work was the Unity task configuration, observation changes and reward experiments. Unity and ML-Agents provide the simulation and learning framework. I followed Jack McKew's installation guide when the toolkit initially failed on my Apple M4 environment. I do not claim to have designed a new reinforcement-learning algorithm.
Freeze the scene, action space, training budget and evaluation procedure. Compare the starting reward, each added term and the combined reward with the same seed set. Keep training and evaluation layouts separate. Freeze the policy before evaluation and disable exploration during scored runs.
For each episode, record success, collisions, time to finish and whether the car remains stuck. Include the number of episodes and uncertainty across seeds. Show successful and failed clips from the same scored evaluation. A useful comparison holds all other settings fixed when changing one reward term.
The remaining evidence is a reproducible Unity source snapshot, saved policies, environment and package versions, training logs and that fixed evaluation. Those files are not included in this case study. The existing demo should be read as selected behavior until they are available.