Lipsync Animation

iClone provides the most intuitive way to generate procedural, editable lipsyncing animations, allowing user to easily create Accurate, Natural, and Smooth talking performances.

Accurate

Detect & Generate Precise Text and Visemes from Audio

Natural & Smooth

Make Characters Speak Like They Should

16 Talking Styles

Designed for Character Speaking in Different Situations

AccuLips
General Features

AccuLips - the lipsync that works

An AI-powered and automatically generated Lip-sync animation brings a smoother and more efficient way to animate and edit facial expressions. AccuLIPS detects and extracts text and viseme from audio, or imports a prepared script for precise lipsyncing. Create natural speak with the co-articulation design, and further fine-tune every word’s visemes and strength level. AccuLIPS contains an English dictionary with 200,000 default words, that are easily customized by adding new words, or changing others.

ACCURATE
Detect & Generate Precise Text and Visemes from Audio

The iClone AccuLIPS system can automatically generate accurate text data, and correct viseme timing from an imported voice. This design gives users a highly manageable approach to professional lip-sync animation. This method lets you generate text from audio, while automatically aligning words to their corresponding voice sections. If the system is further aided with a text script, then you can get a more accurate text-to-voice alignment.

Editable English Word Dictionary

AccuLIPS allows users to add new words with their corresponding lip shapes defined in the Lips Track, or modify existing words from the main dictionary ( 200,000 words). Users are also able to save and load custom dictionaries.

lipsync animation - customize dictionaries for lip animation
NATURAL
Talking Just How It Should

Enhanced Mouth Shapes

With enhanced mouth shapes for each viseme, the lipsync result is more precise and natural for both realistic and stylized characters.

lipsync animation - mouth shapes for lip animation
lipsync animation - mouth shapes for lip animation
lipsync animation - mouth shapes for lip animation

Co-Articulation

AccuLIPS is designed to simulate real human speech behavior. For true-to-life talking animations, each mouth shape takes on the qualities of other mouth shapes that precede or follow them. i.e - articulators either anticipate the next sound or carryover qualities from the prior sound.

Schwa

Schwa(/ə/) is the weak vowel sound in some syllables that is not emphasized in words.

Slightly Open

Mouth shape would be slightly opened when there’s a break between words and sentences.

Phoneme Pair

Mouth movements are governed by pairing 8 lip-dental shapes, 7 tongue positions, and mouth width; all blended with varied time curves to represent more natural transitions between a total of 240 possible phoneme combinations.

Mouth Shape Affected by the Succeeding Phonemes

When we pronounce words, the preceding phonemes will be affected by the succeeding phonemes, therefore the visemes of the preceding phonemes may be different, like KEY/COOL/CAR.

Mouth Shape Affected by the Preceding Phonemes

Same emphasis on the succeeding phonemes - the phonemes that follow will also be affected by the preceding phonemes, like CHIPS/KICK/LICK.

SMOOTH
Smart Downsampling

Though it’s good to see all the detected visemes correctly align to each word, highly frequent lips and jaw movements can cause an unwanted mechanical appearance. The smart downsampling tool can effectively alleviate the jitters by reducing less prominent lip keys, just like how humans tend to slur in a effortless manner.

Enhanced Phoneme Pair Curves

Phoneme pair design is the foundation of iClone’s Lipsync without abrupt transitions between certain viseme pairings, and with optimized transition curves for smoother lip blends, especially when talking fast.

Talking Styles

The strength of lips and jaw movement varies due to when and how people speak. It can be influenced by speed or situation, such as whispering is different from yelling, mumbling is different from enunciating. The Talking Style Presets give you the quick setup over lips, jaw and viseme strengths optimized for different talking occasions.

lipsync animation - talking style editor lipsync animation - talking style editor
Exquisite Talking Result with Dynamic Wrinkles  

Dynamic wrinkles can upgrade your CC characters and bring facial expressions to an exceptional level of realism by adding additional details to lipsync and expressions.

General Lip-sync Features

iClone Lip-sync technology realizes the character lip-sync animation with an easy and intuitive process. The system will align the visemes of phonemes to what the character says, and allow users to further fine-tune. Multiple ways to import the voice resources for characters, permits the lip-sync editing to be more flexible.

Methods to Import Voice Resource
lipsync animation - record voice

Record Voice

Record the voice directly in iClone, so that the voice clip will immediately show.

lipsync animation - tts - text to speech

TTS

Text to Speech. Through the text in the panel, the default voice will be generated according to what is typed.

lipsync animation - audio file

Audio File

Import an audio file, so that the system can generate their corresponding visemes in the Timeline.

GENERATE VOICE & VISEMES ON THE TIMELINE

When importing voice files, the system will auto generate a voice track with corresponding timeline visemes (with AccuLIPS there will also be a text track). Users can intuitively adjust the voice and animation.

lipsync animation - generate voice and visemes on timeline
Customization
Single (Lips Editor)

Adjust the selected viseme with the Lips Editor; including exchanging it for another, deleting it or adding new ones, and editing its Expressiveness strength.

lipsync animation - lips editor
Global (Lip Options)

Use Lip Options to adjust the Smooth Options and Clip Strength of a selected clip. You can also separately fine-tune smooth level for Lips, Tongue and Jaw.

lipsync animation - lip options

MULTI-LANGUAGES SUPPORT

By cleverly bypassing voice-to-text recognition, AccuLIPS can extend its capabilities to other languages beyond English. Lip detection and accuracy can be further improved with additional text scripts while free tools are readily available to convert non-English audio files into their romanized counterparts.

CAPTURE YOUR VOICE AND FACIAL ANIMATION IN ONCE WITH ACCUFACE

AccuFACE plugin for iClone is an AI facial tracker powered by NVIDIA RTX GPU, capable of capturing facial expressions from both webcams and conventional video files. Smoothly synchronize facial movements with voice input. It also fine-tunes lip and tongue articulation using proprietary AccuLIPS technology.