A vision for robotics

THE UW MACS LAB BUILDS ROBOTS THAT PLAY GAMES, PERFORM TASKS AND ADVANCE RESEARCH

Human vision is a sense that’s deeply complex. The way our bodies see and interact with the world is layered, nuanced and difficult to mimic in machines.

However, ME Assistant Professor Xu Chen believes that improving the integration of machine vision, intelligence and manipulation is the key to making robotic systems that are better at assisting humans with complex tasks.

“An ultimate goal in robotics is to build a research path to autonomy so that robots can help people do their jobs more effectively,” says Chen. “We want to develop systems that will work alongside humans and remove the parts of a job that are strenuous, unsafe or wasteful. Improving their vision and perception is an especially important part of that.”

To advance robot vision, Chen and his students in the UW Mechatronics, Automation and Control Systems (MACS) Lab have developed a variety of experimental systems. Some are practical, others are more fun.

Solving a Rubik’s Cube

One of the first tasks Chen set his robotic eyes and arms to was solving a Rubik’s Cube.

“Solving a Rubik’s Cube is a great integration of machine learning, mechanical manipulation and vision,” says Chen. “There are computer programs that will let you solve any combination in 20 turns or less. Building from those instructions, our goal was to build robots that could manipulate intelligently enough to solve the Cube with two arms and no additional fixtures. It was a good way to check whether the eyes and the arms of our robot could work with its brain.”

A pair of MACS Lab robotic arms solve a Rubik’s Cube. Video courtesy of Xu Chen / MACS Lab

At first the mechanical systems had faults. “We broke a lot of Rubik’s Cubes,” recalls Hui Xiao, an ME Ph.D. student who worked with Chen on the project. But with fine-tuning of the hardware and software together, the robot was soon able to quickly solve each cube put in its grasp.

Mastering chess

To advance the concept, the MACS Lab next turned to chess. Mingyu Wang, another of Chen’s students, recently completed his ME master’s thesis by developing a vision-based robotic chess player.

“Building precision is very important for robotics, and each part of chess allowed us to build on what we learned with the Rubik’s Cube,” explains Wang.

The variations in chess demand greater artificial intelligence. The board is less fixed and more dimensional, therefore requiring a more sophisticated imaging system in a less structured environment. And the ability to pick up and move any given chess piece in an interactive environment, without knocking another one over, requires greater dexterity.

The MACS Lab robotic chess player is a demonstration for combining machine intelligence, vision and manipulation with both precision and flexibility. Video courtesy of Xu Chen / MACS Lab

Winning at air hockey

Chen interacting with air hockey pieces near the robot — Xu Chen demonstrates a robot they are training to play air hockey.

The last game (so far) that Chen’s team has been training robots to play is air hockey. In air hockey the robot’s eyes and arms must be able to sense and react quickly to beat a human opponent. The team was inspired by watching professional tennis players.

“Human vision isn’t very fast, at least when compared to vision in other creatures, such as insects,” says Chen. “But professional athletes, like tennis players or hitters in baseball, have extraordinarily fast reaction times. Athletes are able to react quickly, not because of the speed of their vision, but because of training. Our goal is to see if a robotic system can do the same.”

The ability to respond and learn to adapt quickly and appropriately is a crucial challenge in developing robotic systems with greater autonomy.

The MACS Lab wasn’t the first to build robots for these games, but each project was purposely designed to advance the lab toward important research questions. And the same systems can also be tailored to more productive tasks.

Inspection of complex metallic parts

Xu Chen controls a robot designed to carefully inspect manufactured aircraft parts for flaws or defects.

Over the past year, with support from the Advanced Robotics for Manufacturing (ARM) Institute, Chen was part of a team of engineers from the UW, the University of Connecticut and GKN Aerospace who put their robots to the task of inspecting manufactured parts for imperfections. Specifically, they wanted a system that could spot some of the most difficult to discern defects — those in curved and complex metallic parts like ones used in airplane turbines.

A curved metal part is hard to study closely. It’s shiny and bent, so the lighting and combinations of angles and distances affect what can be seen and focused on.

“When working with airplane parts, the smallest defect can lead to a critical structural integrity issue, so the stakes are high,” says Alex Strzelecki, a software and automation engineer at GKN Aerospace who worked with Chen on the project. “Inspecting these parts is eye-straining and labor-intensive work for a person, but also a major challenge for a robotic system.”

As with air hockey and chess, the researchers needed their system to be fast and precise, but they also needed it to be a bit flexible. It’s one thing to set up a robot inspector for a large-scale assembly process, like an automobile, but different for manufacturing processes with smaller runs of parts, like in aerospace or emerging 3D-printing technologies.

The team’s robot inspector was able to consistently spot defects 95% of the time, substantially better than most human inspectors. Thanks to the camera’s visual feedback, the robot can understand how it is doing on the fly, and automatically adapts its motion and even reconfigures its lighting to counteract differences in geometry and orientation of the parts along the curved, reflective surfaces. And because the process uses data, it can be tailored to quickly sort parts that may just need more machining or polishing versus those destined for the scrap heap.

Explore

Research Area

Mechatronics and Robotics

ME faculty are at the forefront of research in robot-human interaction, nanorobotics, automation and advanced manufacturing.

An overview of the MACS Lab robotic inspection of metal parts project. Video courtesy of Xu Chen / MACS Lab

Elegantly structured senses

All of these MACS Lab projects combine machine vision, intelligence and manipulation. In the big picture, Chen is most interested in improving how robotic sensing and mechanical systems are organized and controlled.

“As humans we take for granted how intuitively our brain makes choices about prioritizing vision, perception and cognition, but those choices matter a lot,” notes Chen. “We already have so many types of robotic sensors and inputs to choose from and in five years we will have even more, so the options are almost infinite.”

Chen is building a game to help visualize the huge number of possible connections in a robot and show the beauty of the resulting arrangements. By finding the most elegant architectures, Chen hopes his research team will learn important lessons about how a robot’s brain can better prioritize its senses.

In doing so, maybe they’ll learn something about our own brains as well.

Find out more

Take a virtual tour with Xu Chen.

Visit the Mechatronics, Automation and Control Systems (MACS) Lab website.

Story by Andy Freeberg
Lead photography by Dennis Wise
Videos courtesy of Xu Chen / MACS Lab
All photos and videos were taken following appropriate protocols at the time.