Year a Robot Passes the Coffee Test

This was a large divide. Respondents who predict an early resolution tend to believe the ingredients (beyond coffee!) for success largely exist today, and the challenge is primarily one of integration and engineering: "This is a case of 'it will happen whenever a team decides to invest a few months to do it' [given that] the technical capability is already here."; "I've already observed robots that come remarkably close to this, so I don't think this is very far away." Whereas late-resolution respondents typically see a wide chasm between current systems and the test's requirements, with one writing, "We do not have robots that are even close to that level of capability, not even in laboratory settings. We are only beginning to develop robotic hands with enough tactile sensitivity to avoid breaking the objects they handle and to adapt to different shapes and materials."

Late-resolution respondents often treat fine-motor-skill manipulation as a significant, and unsolved, obstacle that will need to be overcome before the test can be successfully performed: "Humans (hands) making coffee combine strength, fine control, tactical feedback, and reflexive adjustment in ways that are deeply integrated into our hardware and are generally subconscious. Because AI systems still find it difficult to learn 'from touch' I sense progress on dexterous manipulation will continue to lag." Another adds that "grinding coffee beans, inserting paper filters, scooping loose grounds, and controlling pours all involve different grasps and forces, often under visual occlusion where the robot's own gripper blocks its camera view." Many early-resolution respondents push back on that narrative, with one writing that "coffee beans, filters, and mugs are rigid or semi-rigid objects that obey predictable physics, completely bypassing the adversarial chaos of manipulating deformable or living things…The hard part is perception and planning rather than dexterity."

Many early-resolution respondents believe Vision Language Action foundation models are close to being able to navigate new environments well. One argues that "current LLMs can already handle most of the reasoning and planning required," and adds that "there's no requirement that all processing happen on-device, so the robot can offload reasoning to powerful frontier models," while another points to sim2real transfer methods, as technology that is enabling the type of generalization capabilities that will be needed to pass the test. Other early-resolution respondents discount the need for robust generalization: "The task really doesn't seem that hard considering it should be around ~10 steps which should be trainable." Late-resolution respondents, however, tend to see generalization as another core, unsolved challenge: "Humans put the coffee filters in the wrong drawer, hide mugs in bizarre cabinets, leave the machine unplugged, use unfamiliar brewers, and create cluttered geometry that was clearly designed by a raccoon with a grudge. Passing this test three times in three different homes means the robot has to survive that entire carnival of entropy."

Several early-resolution respondents argue that the coffee test's status as a celebrated challenge might accelerate timelines. One speculates that "there may already be a company that is focused on developing a coffee making robot just for the sake of saying they were the company that created the coffee making robot, to gain publicity." Late-resolution respondents tend to point out that, prominence of the test notwithstanding, economic incentives point elsewhere: "This is also a use case that has marginal commercial applicability for an adequate return on investment. Why would a company that wants to make money seriously focus on this use case?"