Store Page
FaceRig
All Discussions Screenshots Artwork Broadcasts Videos Workshop News Guides Reviews

FaceRig

Store Page
View Stats:
Global Achievements
FaceRig > Suggestion Box > Topic Details
darthokkata 31 Aug, 2014 @ 1:28am
Partial face recognition for VR possible in future?
Would be nice if in a future version there was a way to simulate the top half of the facial expression based on the bottom half. Nothing fancy, just simple animations that fit what the mouth expression is so the top half of the face doesn't look paralyzed.

Full face tracking is great and will obviously be superior, but this could be a great tool for VR if you can figure that out. Possibly tracking neck and shoulders as well could help with that by creating a way to modify which expression is shown based on the position of both. For example: a sinister smile as opposed to a pleasant one if the neck is bent and shoulders are raised or a curious one if the head is tilted, etc.

It would be especially effective in a VR environment due to how the player perceives the environment and other players. The perspective is very different and this could be a game changer if it could be applied to Oculus Rift enabled titles. Especially in social VR programs like MMO games, Second Life, or RiftMax theater.

Is this feasible? If so, now is the time to get that ball rolling while the DK2 kits are out. If you can get this working well for VR, and if it isn't too difficult to implement into Unity or UE4 this could really take off in that community. Get it done and put the word out to developers that it's working I can see lots of devs looking to adopt this tech for their multiplayer games.

Immersion and presence is really important for VR developers in their games and a tool like this that works with HMDs could really improve both quite a bit.

Seriously try to figure this out. It could really give this tech a place to shine in a new industry of entertainment, communication, socialization, machinima animation, and games.
Last edited by darthokkata; 31 Aug, 2014 @ 1:35am
< >
Showing 1-3 of 3 comments
Arix 31 Oct, 2014 @ 7:41pm 
This is a couple months old, but man do I agree with it. If you can guess where eyes are pointing or how to smash this together with the positional tracking of the DK2 you'd be millionaires.

Some part of me fears that Facebook already has 50,000 devs working on this as we speak. Get your foot in the door!!
#1
Zeke 23 Nov, 2014 @ 9:05pm 
Would be a hell of a thing to implement but it would be awesome.
#2
Holotech Studios  [developer] 24 Nov, 2014 @ 2:31am 
First and foremost, my appologies for a short rant :)
The ball is the the court of HMD manufacturers on this one.

Image based head tracking (by a camera that is not head-mounted) is based heavily on recognizing some face landmarks that are typically obstructed by a VR headset).

However HMD manufacturers, having already overcome the barrier or convincing the user to strap something to their head, have an edge on this one. They should use that to pick up on the wearer's expressions ...

They can add sensors to the HMD that allow the computer to pick up the face muscle motions. The visor presses against the face, I am sure they could add sensors in there that can track the pupil motion, and they can infer eyebrow motion either with pressure/friction sensitive material or maybe by monitoring electric activity that drives the face muscles..., as the top of the device actually presses against the eyebrows themselves.

They can ave a mounted high speed camera filming the lower part of the head ( with its own diffuse light source), and track the mouth and lip movement (with a greater accuracy than a camera that is not head mounted will ever be able to).

Even if a webcam-only system is developed by a third part that would do partial face-tracking (based on a desk placed camera), something as simple as a change in the design of the HMD can mess that up, so that's is not a smart way to go about it. To avoid programming resource waste, the HMD manufacturers themselves should provide this, not a third party.


Sorry for the rant :P. But I highly believe that all current VR HMD manufacturers are focusing too much on one way human communication ( feeding high fidelity rendered input to the user), and very little on allowing the VR headset user to actually communicate naturally to someone else in the VR world (a staggering amount of human communication is non verbal, and too little effort is currently made to pick up on that).
Maybe we should make a petition :D.


We have already shared these ideas via twitter to Occulus guys a year ago, before the Facebook acquisition. An engineer there followed us back and that was about it, no further communication.

Anyway, if the big guys in VR fail to implement these in their HMD's, let's hope we at Holotech Studios, will grow large enough to have the financial means to do it ourselves :).


Again, sorry or the rant, :D
Florin , the FaceRig Project lead
Last edited by Holotech Studios; 24 Nov, 2014 @ 2:40am
#3
< >
Showing 1-3 of 3 comments
Per page: 1530 50

FaceRig > Suggestion Box > Topic Details
Date Posted: 31 Aug, 2014 @ 1:28am
Posts: 3

Discussions Rules and Guidelines