Comic books have had a bit of a renaissance in recent years with the popularity of franchises like Black Panther and the Avengers. Normally the frames of comics get brought to life in live action movies or tv shows. Researchers at the Warsaw University of Technology want to switch things up by using AI to turn live action videos into comics.

Comixify Uses AI and Machine Learning to Turn Videos Into Comics

Thanks to machine learning and artificial intelligence, Comixify analyzes each frame of a video and picks out the ones that would best tell the story. It then turns those frames into comic book style images.

The AI was trained on a dataset of videos which helped it learn which types of frames were most important to a video’s story. According to the paper, “considerable amount of research was carried out in the domain of content popularity estimation. Various aspects of popularity prediction were looked at such as analyzing the user-access patterns relation to popularity prediction of the peak popularity time”

AI Selects Most Visually Pleasing Images

Once the AI picks out the frames that best represent the video, it then analyzes them using an image aesthetic estimation engine. From there it narrows down things further by selecting the most visually pleasing images.  Once all the frames are selected, they are then processed and turned into comic book style images.

You can try out Comxify for yourself at https://comixify.ii.pw.edu.pl/

Check out our articles on AI identifying Civil War photos and Nvidia generating game worlds from videos