Votes
0
Product:
iClone 7
Version:
7.73
Status:
Active
Issue 6792
Lip sync: delay, smoothing and expressiveness improvements
I tried the lip sync feature yesterday and found that the smoothing feature is great. However, I found that I had to deal with the consonants and vowels differently. I might be wrong, but smoothing the consonants seemed to smooth them too much. The consonants still need to be sufficiently expressed because if the mouth or tongue does not close, it is not possible to pronounce the consonant. However, this is not the case with vowels. So maybe consider a slider for each viseme that determines how strongly the smoothing affects it.

Also give us the opportunity to overdrive the expressiveness of a single viseme in the timeline to override the lip options. I had to make several cuts in the options so that I could increase the exressiveness of single visemes next to other visemes that I toned down.

There is also a slight latency between audio and animation. The animation must come earlier, as the movement precedes the sound. I tried it up to 6 frames latency here:
https://www.youtube.com/watch?v=2o-2g18ycvY&feature=youtu.be

In my preliminary testing, I think 6 frames works best, but maybe it depends on the visemes. The problem with 6 frames is that the mouth seems to close too early once he stops speaking so there should be some slow fade out.

Also consider variable smoothing. Smooth and decrease the expressiveness of the the vowels more strongly when there are several visemes in quick succession.

Third, it would be cool if we could provide alternative visemes and custom visemes. A round-robin algorithm with two or three slight variations of each viseme would be cool, but maybe more important are different visemes for different expressiveness. Instead of dialing one visueme up and down, you should consider blending between visemes with different expressiveness. Especially the ones with a lot of jaw movement could benefit from alternative visemes with less jaw movement.

I hope that the bending jaw bone is fixed in the future and moves like an actual bone.
OS: Windows 10
  •  0
  •  1041
Submitted byMikay²
COMMENTS (0)