Creating charismatic characters: The customizability of FaceCore

The face engine is the software that renders the image that gets projected onto the robot face mask. This is done in real-time and is governed by a set of parameters that gives full control of both what the face looks like (i.e. shape and texture) and what it does (facial expression, gaze, lip movements etc). Since the face is projected, one can drastically change the appearance of the robot in a split second (if one should want to) – and by switching the magnetically attached face mask, it is possible to further change what the robot looks like. With a careful combination of mask and projection, it is possible to create some very interesting characters. In fact, ever since the conception of the Furhat robot we have gotten plenty of requests to make the robot look like a particular person or character. Examples of characters developed include a clone of a Swedish politician, a Japanese anime girl, a pug, and a few other recognizable faces.

However, crafting convincing quality characters can be difficult – the above examples all required some custom tools and pipelines in their making. In designing FaceCore, Furhat’s new face engine, one of the guiding principles has been to make creation and customization of high-quality robot characters both faster and easier.

The two most important properties that define what Furhat’s face looks like are the texture and the shape. The texture, or the “skin” of the model, is essentially an image that gets mapped onto the underlying model. Modifying the texture has always been possible on the Furhat robot, by downloading the entire texture as an image file, modifying it e.g. in photoshop and uploading it to the robot again. This process has some drawbacks, because it is sometimes difficult to predict the effect of changes made to the image once projected onto the robot. With FaceCore comes the introduction of ‘texture overlays’ which adds a lot of flexibility. It allows individual elements of the face texture – such as eyebrows, eyelashes, lips, makeup and facial hair – to be specified independently and combined on top of a base texture. Additionally, the base texture and the overlays can be modified by specifying color and transparency. The overlays are themselves regular images, so it is still possible to edit these in an external program for even more control, but for many applications, the provided overlays represent the best way to quickly craft a customized texture for a character.

The shape of the face is complementary to the texture, but equally important for the character appearance. Note that shape relates both to the physical mask and the face engine – both are important and need to be aligned and tuned together to produce convincing results. FaceCore comes with a set of shape parameters that allow you to move around, resize and reshape different facial features. For example, you may move the eyes further apart or closer together, tilt them or change their size or shape. You may widen the mouth, make the lips thinner or thicker etc, in order to create radically different looks for a character. These parameters are also great for producing caricature or cartoon-style characters. Since facial features are movable, they can also be used to adjust the face to a new mask. It used to be the case that one Furhat mask required a dedicated model. With FaceCore, this requirement is loosened, making it possible to move around the eyes/mouth/nose so they match a given mask which gives a lot of freedom and potential for experimentation. The above character controls taken together allow for a wide range of flexibility in creating and customizing Furhat faces and matching them to different face masks, and saving everything as a character that can be paired with a voice and instantly recalled.

The final appearance of a Furhat character depends on a combination of many factors – the texture, the shape, the mask, as well as the rendering (virtual lighting and shading). FaceCore comes with an overall improvement of the graphical fidelity, in particular when it comes to the eye region, while allowing for different character styles to be produced, ranging from cartoon/manga style to realistic. The rendering engine also supports dynamic shadows and eye reflections that are synchronized with the head movements of the robot, in such a way that when the robot head rotates in physical space, the shadows and eye reflections on the face are updated accordingly – an effective illusion that helps to bridge the gap between physical and virtual realms that make up the robot.

Thankfully, the new FaceCore engine does not only run on the robot, but it is also part of Virtual Furhat – the robot simulation tool that produces a real-time 3D rendering of the entire robot, including 3 degrees of freedom head movements (pan, tilt, roll), and accurately simulates the projection of face animation onto a switchable mask surface. This makes it quick to verify the effect of different character parameters as well as switching between masks. The Virtual Furhat is part of the Furhat SDK, which is available for download and runs on Windows, MacOS and Linux. Apart from previewing custom characters, the Virtual Furhat can be used to preview animations and test interactions (skills) built with the SDK.

Taken together, the new features in FaceCore allow for a wide range of flexibility in creating and customizing Furhat characters: the shape parameters introduce a way of re-shaping and adapting face geometry that previously required a custom model. Texture overlays make it quick to customize the appearance, and with Virtual Furhat it is easy to visualize the final result. Designing convincing robot characters has never been an easy task. But we believe the new capabilities in FaceCore makes it more manageable than ever. Now, the question is… what do you want your robot to look like?

Read part 2 of the FaceCore blog series here.