Take-Away – It is possible to animate the face of a mesh avatar using the Deformer. The details follow:
Yesterday after the Server Beta meeting White Rabbit showed me a full body avatar. It’s eyes blinked. I mean blinked like the SL Avatar blinks. The eyelids moved. Now that is novel.
Now that may not seem like much, but if you have made avatars with mesh heads, you know that the face cannot be animated to show expressions as the SL Avatar can. What White Rabbit is discovering is that the Mesh Deformer in the current Project Viewer is handling the facial morphs used for expression.
I didn’t get to see all that White Rabbit has discovered as the region we were in on ADITI kept crashing and restarting. But, I did find what I saw impressive.
White Rabbit included the information in the STORM-1716 JIRA item. (Reference – This is an item visible to most JIRA users. ) Quoting:
Just another little update from my mesh deformer tests. Currently using a anthropomorphic rabbit AV I built from the ground up ( mind you completely different vertex order from the default human AV ) The deformer is picking up on facial features on my model and the morph sliders are applying to all regions of the face (and body ) virtually perfectly. Breast and butt physics are applying to correct regions as well etc etc.
I’m currently tweaking it to see if I can get more stuff in tune just by fiddling with it, but as it stands it is functioning far beyond my prior expectations.
Again to reiterate : this avatar I’ve created shares absolutely no geometry whatsoever with the base SL avatar and the mesh deformer is working correctly on facial morphs… the eyes are blinking ( how cool is that?!! )
For those concerned with the calc time, this is a roughly 12K poly mesh and is deforming in less than 20 seconds.
PS: Marcthur Goosson, The deformer seems functional, but even with something like this a time frame is always difficult to estimate as there is frequently unforeseen situations that arise before it makes it into SL in a widespread fashion. Oz is currently on a well deserved vacation until sometime next week ( Dec 8th or so, rough guess ) so please be patient!
This opens up some interesting possibilities. I am certainly impressed. Also, that a 12k poly mesh is taking 20 seconds to initialize is interesting. It is not too bad but it is not ideal either. However, Siddean Munro is certainly impressed: (I broke this quote into paragraphs not in the original text.)
I have to say this and please forgive me “by George, I think you’ve got it.”
I’ve tested with a default avatar, and one I made myself with barely any consideration to the underlying geometry (loosely follows the standard sizing XS shape) and holy crap!
[I] spent quite a lot of time playing with the facial sliders and the emoter HUD. I have a little tweaking to do on the face so that my bottom lip doesn’t get deformed by the top lip but that’s user error. The body deforms very nicely, I am seriously impressed! Well done guys, I’d be calling this one ready to go! I think. I’d like to start re-uploading my inventory with deformer enabled [mesh].
To echo WhiteRabbit0 above… the eyes blink! And the physics works beautifully. It takes very little time to calculate the initial deformation. Any subsequent deformations on the shape editor are instantaneous.
When I get a moment, I’ll upload some of my previous work with the deformer enabled and take some photos! So excited to see this finally in working shape!!
I have to say, I love the improvements to the xml export function – being able to say where and what to save the file as, and having it handy in the advanced menu is awesome.
Sorry for the gushing but wow. This needs to go out asap! (if I had one nitpick, it would be that the breast size sliders seem to pick up a section of middriff just under the breast as well, but I think I need to do more testing on this one).
How To
At first I thought White Rabbit was using special weighting by using of collision bones or something, but apparently not.
It seems all one need do is align the new meshes eyes with the SL Avatar’s eyes and keep the existing weighting found in the SL Avatar. For the mouth to work it too must be aligned.
I am not sure how well this is going to work. I suspect the closer the new mesh’s vertices are to vertices in the SL Avatar’s mesh the better it will work. While it is not a matter of having a vertex-by-vertex match up, it is likely a matter of how close a vertex in the new mesh is to a vertex in the SL Avatar. The closest SL Avatar vertex will have the most influence on the vertex in the new mesh.
Limits
Having to establish proximity to the controlling vertices in the SL Avatar is going to limit what can be done with custom avatars. So, I see this as a very limited solution for facial animation of custom avatars.
Consider. If one made an elephant, the trunk would be extending over the lower lip. That is going to be a BIG problem. I’ve made an illustration. This is not the best possible arrangement of an elephant over the SL Avatar head, but I hope it will show the problem.
At each corner of a triangle in the Avatar head is a vertex. I’ve put some small green dots on the elephant to represent some possible vertex locations. The colored lines represent the shortest distance between an Avatar vertex and an elephant vertex. The Deformer is going to associate the closest vertices something like this illustration.
The ‘A’ side of the trunk is mostly influenced by the vertices in the upper part of the SL Avatar’s face. Look at the Cyan colored lines. The ‘B’ side of the trunk is going to be influenced mostly by the lower part of the face, the jaw. Look at the purple colored lines. When the mouth opens and the jaw lowers the ‘B’ side of the trunk is going to stretch away from the ‘A’ side in the direction of ‘B’. Not exactly what we want.
The elephant’s jaw, the ‘C’ side, is going to be pretty much influenced correctly. Look at the yellow lines. But the trunk is going to be a mess.
Latest Version
People are still having trouble finding the latest version of the Deformer Project Viewer. I think part of the problem is the Project Viewers page doesn’t provide the feedback some of us need. It does not show the version number for the file being downloaded.
I provided a link in the left column of my blog to the page where you can find the Project Viewer. You just have to have the faith that the page will take you to the latest version. For the more obsessive and skeptical use this link: Mesh Deformer Latest Build.
The version is 3.4.1.266373. Built: 10/30/2012 1:37:51 AM. Oz is going on vacation. So, I doubt there will be any new builds before 12m/7d/2012.
Summary
This feature in the Deformer is not new. I suspect even the first Deformer code would have handled this type of facial animation. I think we are just now finding out it will. I don’t expect it to be perfect. So, I am not sure this is a major solution to animating custom mesh avatars. I think the Lab and users will continue to see the need for a better animation system. But, for now this will be a handy work-around.
It also has a neat ‘look what I can do’ aspect. 🙂
Thanks White Rabbit (whiterabbit0).
Interesting. I’ll have to take a look at this.