Robot Can Reproduce Handwriting and Sketches

For celebrities who are sick of signing autographs, we might have a solution for you. Computer scientists at Brown University have created an algorithm that enables a robot to copy what a person writes and what they draw.

Using deep learning networks, the algorithm analyzes handwritten words or sketches. It then determines the probable pen strokes that were used to make them. The algorithm was trained on handwritten Japanese characters learning to identify patterns and stroke styles.

Algorithm Analyzes Each Stroke to Recreate Image

To properly recreate the image, the algorithm analyzes each individual stroke. It uses two different models of the image as it attempts to replicate it. First, the algorithm looks at a global model which analyzes the image as a whole. It then tries to identify the likely starting point for making the first stroke. Finally, the algorithm looks at the image pixel by pixel to determine where that stroke should go and where it should end. It then repeats the process over again until the image is complete.

“Just by looking at a target image of a word or sketch, the robot can reproduce each stroke as one continuous action. That makes it hard for people to distinguish if it was written by the robot or actually written by a human,” said Atsunobu Kotani, an undergraduate student at Brown who led the algorithm’s development.

Handwriting Robot Was Able to Write Characters It Had Never Seen Before

The handwriting robot was able to write the word “hello,” which was written by several people, in 10 different languages including Greek, English, Hindi, Chinese, and French.  The impressive thing was that the robot had never seen most of these as the algorithm was only trained using Japanese characters. It was able to reproduce the unknown characters simply by analyzing the image. Along with reproducing the handwriting samples, it was also able to recreate some rough sketches including one of the Mona Lisa.

“A lot of the existing work in this area requires the robot to have information about the stroke order in advance. If you wanted the robot to write something, somebody would have to program the stroke orders each time. With what Atsu has done, you can draw whatever you want and the robot can reproduce it. It doesn’t always do the perfect stroke order, but it gets pretty close,” said Stefanie Tellex, an assistant professor of computer science at Brown.

Check out our articles on a robot wedding photographer and ones making deliveries on a Napa Valley golf course.