Being proven “right” is a guilty pleasure for most people. As humans, we secretly enjoy the little victories that come with the phrase “I told you so.” I was in Germany when I first saw Star Citizen’s new face tracking software and as cool as it was, I sat there wondering if it wasn’t all a waste of time. The tech is a collaboration between Cloud Imperium Games and Faceware Technologies, who are looking to map a players facial animations onto their in game character. It sounds cool, the stuff you’d expect out of a Sword Art Online or Ready Player One future. But, Star Citizen is a game that everyone seems to know how to develop better than the developers themselves. I’m as eager to experience Star Citizen as the next guy and this seemed like a weird direction to spend development resources on. CitizenCon took place in Austin Texas this year and I had another chance to see the technology first hand. Let me tell you, if being right is a guilty pleasure, sometimes it’s so much better to be wrong.
Faceware Technologies have been featured in a number of different projects from movies to video games. Face Over IP, or FOIP may not be a new technology but I struggle to think of any games where it has been implemented as a core feature intended to be used player to player. In the case of Star Citizen, it works by using a webcam to map the players face with a number of key points that translate to your in game avatar. Then, in real time, the expressions on your face are displayed in game. Faceware claims that their system works even for players that wear glasses, a claim that has been tested and proven by the community. By combining this with voice, players can now speak to one another in game with fully mapped facial animations. If you’re like me, you’re impressed but not so much so that you’re looking to jump on the bandwagon right away. That’s understandable. I wasn’t completely sold until I saw how the technology is being used at this very minute, within the universe of Star Citizen.
“For the first time we’ll be able to deliver the full range of human emotion, not just voice. Our players’ facial expressions will be translated onto their avatars’ face. Combine that with a player’s voice correctly positioned in the virtual world, and you have the most lifelike player-to-player communication ever.”
-Chris Roberts, CEO of Cloud Imperium Games
During the keynote presentation, the player initiated a video call to another player several miles away. The player on screen had their face mapped using the software and the two of them discussed gearing up and going out to a mission together. Though the software didn’t work perfectly in this demo, you really did get the impression that this was Star Citizen’s version of calling up your buddy and seeing if they want to get drinks or go to a party. Later, we saw the same call be made through the ships computer with the player’s face being displayed on one of the cockpit screens. Sure, the classic private message will work to let your friends know what’s going on and I’m sure people will still heavily utilize external voice chat programs like Discord or Teamspeak. But these sorts of interactions can’t be understated. They quite literally give every player a face in game and the psychological ramifications of that are fairly significant.
There’s a tested theory that if you want to merge into traffic and the car next to you won’t let you over, you should make eye contact with them. In fact, you probably already do this and if they refuse to return eye contact you may find yourself having a very one sided expletive filled conversation with said person. Humans are built to respond to face to face interactions. When we don’t understand the context of a conversation, we look to the face to determine what is being said. In fact, it’s part of the reason I think so much hostility exists in online gaming, it’s easy to reduce the person you’re talking with to a faceless entity. With each player being personalized by their facial animations, our avatars will be more memorable to those around us. In addition, content creators on YouTube will have access to powerful tools to help create believable in universe stories. Imagine what an “in-universe” news report would look like with the player taking on the role of the caster. While we still have a long way to go in overcoming the “uncanny valley” of avatars, I believe FOIP will work to drastically change the way people interact with one another. It’s the sort of small revolution we’ve come to expect from Cloud Imperium.
By now it should be apparent that both Faceware Technologies and Cloud Imperium Games are serious about giving players this experience. In the coming weeks, we’ll learn more information about the camera Faceware has designed to work specifically with this software. It features a 60 FPS -3.4 megapixel sensor, designed to operate in low light. The latter part addresses the concerns of using software like this which normally requires a strong light source in order for the camera to capture everything at a high enough frame rate to produce accurate results. Users like myself who prefer a dark play area can rest assured their mugs will be seen across the stars. For those that aren’t quite sold yet, just about any webcam will do the trick and the software will debut with the 3.3 patch, coming Soon™.