vidu/lip-sync

Docs
Schema

[Core Function] Vidu Lip Sync is a video-to-video audio synchronization model. [Strengths] It excels at reanimating lip movements in an existing video to precisely match a new replacement audio track, while preserving the original face identity. [Best For] Highly recommended for: dubbing videos into different languages, correcting spoken dialogue post-production, and creating realistic digital avatars. [Limitations] Do NOT use this model if you need to change body movements or generate a new video from scratch. It requires a pre-existing video and a clear audio track. [Routing] Use this model specifically when the user wants to change what a person in a video is saying. If the user wants to transfer body movements, use Vidu Motion Sync instead.

$0.0184/sec
video-to-video

Input

Source video URL containing the face to reanimate.
Hint: Drag and drop files, paste from clipboard (Ctrl/Cmd+V), or provide a URL.
Replacement audio track URL. The model will reanimate the speaker's lips to match this audio.
Hint: Drag and drop files, paste from clipboard (Ctrl/Cmd+V), or provide a URL.
Reference face image URL. When provided, the model uses this face to preserve identity consistency throughout the video.
Hint: Drag and drop files, paste from clipboard (Ctrl/Cmd+V), or provide a URL.

Result

No results yet

Run the model to preview the output here.

Next:

README

What Vidu Lip Sync Does

Vidu Lip Sync is a video-to-video audio synchronization model that reanimates the lip movements in a source video to match a new audio track while preserving the speaker’s face identity. It is ideal for dubbing, dialogue correction, and avatar creation where you need natural, accurate mouth movements aligned to replacement audio.

Key Features

  • Lip Reanimation: Adjusts mouth movements in an existing video to match a new audio track.
  • Face Identity Preservation: Keeps the original speaker’s face consistent throughout the synced output.
  • Optional Reference Face: Accepts a reference_face_image_url to reinforce identity consistency.
  • Audio-Driven Editing: Drives lip motion from a replacement audio_url for clean dubbing and corrections.
  • Asynchronous Architecture: Uses a “Create Task → Poll Result” API workflow for reliable processing.

How to Use Vidu Lip Sync

  1. Host your source video (containing the face to reanimate) as an accessible URL.
  2. Prepare the replacement audio track URL, and optionally a reference face image URL for identity consistency.
  3. Submit a POST request to the /api/v1/vidu/lip-sync endpoint with the required video_url and optional audio_url and reference_face_image_url.
  4. Receive an async response with a task_id and a “pending” or “processing” status.
  5. Poll the results endpoint (get_result.url) and download the synced video once the task succeeds.

Best Use Cases

  • Language Dubbing: Re-sync existing video to dubbed audio in another language with natural lip movement.
  • Post-Production Dialogue Correction: Fix or replace lines without reshooting.
  • Digital Avatars: Drive talking-head avatars from new audio while keeping a consistent face.
  • Localization at Scale: Adapt video content for multiple markets by swapping audio tracks.

Tips for Better Results

  • Don’t Use It to Change Body Motion: Lip Sync only reanimates the mouth — for body movement, use Vidu Motion Sync instead; it cannot generate a new video from scratch.
  • Start From a Clear Source Video: A well-lit, front-facing source with a visible face yields the most accurate sync.
  • Provide a Clean Audio Track: Use clear, well-recorded audio for the tightest lip alignment.
  • Add a Reference Face When Needed: Supply reference_face_image_url if you want stronger identity preservation.
  • Match Audio Length to Video: Keep the replacement audio aligned to the source video’s duration for the cleanest result.

Parameters

Name Description Type Required Enums
video_url Source video URL containing the face to reanimate. string Yes -
audio_url Replacement audio track URL. The model will reanimate the speaker’s lips to match this audio. string No -
reference_face_image_url Reference face image URL. When provided, the model uses this face to preserve identity consistency throughout the video. string No -

Pricing

Unit: $/sec

Pricing
$0.0184/sec
  • pixverse/lipsync: [Core Function] PixVerse Lip Sync drives a talking video so the subject’s lips match given audio or text-to-speech. [Strengths] Accurate lip synchronization for talking-head videos; supports either an existing audio track or TTS from a chosen speaker voice. [Best For] Dubbing, virtual presenters, character dialogue, localizing spoken video. [Limitations] Do NOT use this to generate new motion or change content; it only re-times the subject’s lips on an existing video. It requires an input video, and you must provide EITHER audio_url OR (speaker_id + tts_content), not both. [Routing] Use when the user has a video and wants the speaker’s lips to match speech. Use audio_url for an existing voice track; use speaker_id (a named voice code) + tts_content (max 140 chars) to synthesize speech from text.
  • skywork/sky-lipsync: [Core Function] SkyReels Lip Sync (retalking) re-drives a talking video so the subject’s lips match a given audio track. [Strengths] Accurate lip re-synchronization on an existing talking-head video. [Best For] Dubbing, re-voicing talking-head video, and localizing spoken video. [Limitations] Do NOT use this to generate new motion or content from scratch; it only re-times lips on an existing video. Requires video_url and audio_url. Output resolution is fixed at 720p. [Routing] Provide the source video_url and the target audio_url; optionally provide reference_char_url to guide the driven face.