Google's AI analyses a cornhole player's video, and the Apollo humanoid lands seven throws in a row

According to Google, the model helped Apollo land seven consecutive throws at cornhole, a bag-tossing game, after analysing a video of a human playing it. The in-house trial serves as a showcase for Argon, which has been open to developers and businesses worldwide since 13 October.

According to Google's launch post, Argon analysed the raw video, identified the coordinates of the scene in space and calculated the movements of the joints across the robot's whole body. The model then wrote the code for a simulation on its own, and the result was transferred to Apollo for the real-world trial.

Google calls this method “Video-to-Sim-to-Real” and says it cuts weeks of manual modelling down to a few hours. The trial remains an in-house demonstration covering a single throwing task, not a general measure of the robot's abilities.

Google also reports a score of 87% on the ERQA test, which evaluates the understanding of physical and spatial problems in 3D environments.