Researchers Use Augmented Reality to Turn Coloring Books into 3D Experience

Colouring App Developed – Disney Research

A colouring app developed by Disney Research could cause characters to spring from the page in a 3D wonderthrough augmented reality. A child tends to colour a character on the book page normally while a smartphone or tablet running the app tends to monitor the drawing.

Based on the colouring of the child, the app fills the colours in real time on a 3D animated version of the character which is visible on the screen of the device and integrates it into a video. The core focus on the traditional activity of the colouring is maintained by the app while providing the magical digital overlay which tends to improves the engagement.

User testing performed by adults rather than the children in the early study, researchers observed that most of the users informed that the app increased their enthusiasm to draw in colouring books while 80% stated that the app increased their feeling of linking with the character.

Disney researchers, ETH Zurich as well as the Swiss university EPFL presented the augmented reality app at the ISMAR 2015 – IEEE International Symposium on Mixed and Augmented Reality in Fukuoka, Japan.

Disney Color & Play

Though the work of the research was presented to scientific audience, it had gone through the tech transfer process already, motivating the commercial product known as `Disney Color and Play’ which was launched earlier in the year by Disney Publishing Worldwide and Bendon.

The work tends to fit in a huge initiative known as Augmented Creativity at Disney Research that aims on utilising augmented reality in improving creative play. Robert W, Sumner, principal research scientist leading the group on animation and interactive graphics at Disney Research stated that `augmented reality holds unique and promising capabilities in bridging between real world activities and digital experiences enabling user in engaging their imagination and boost their creativity.

They are thrilled to have the opportunity in presenting the scientific advances behind this technology and are particularly happy that it is available to consumers, thanks to the cooperation with Disney Publishing’.

App Functions on Device With Camera

The researchers, inorder to create a new experience, at first created animated 3D virtual characters and then utilised custom software in generating 2D line art representations of the characters for colouring book.

 The app functioning on a device with a camera that viewed the user and the colouring book automatically detects the character the user tends to colour, displaying the 3D version. As the child progresses with the colour to the 2D drawing, the app tends to apply the same colour to the 3D characters in the areas visible in the 2D drawing as well as to the remainder of the 3D form which is not visible in the book.

 Owing to the colouring occurring in real time, the illusion is developed that the user is also colouring the blocked areas with the same texturing of the colour. Defining how to apply colour to the blocked areas seems one of the most difficult issue, according to Sumner.

By mirroring the user’s strokes on colours does not seem to work since the patter of colours used for a character’s face would not be the same for the back of the character’s head. The colour also needs to be continuous in order that no seams are seen between the visible areas and the blocked areas or where dissimilar portions of the textures tend to meet.