Kling Lip Sync

Make characters in videos synchronize their lip movements with audio for audio-visual matching effects.

Workflow

The lip sync feature requires two steps:

1. Face Recognition - Identify faces in the video and get session_id
2. Create Lip Sync Task - Create a lip sync task using session_id and audio

---

1. Face Recognition

Endpoint: POST /kling/v1/videos/identify-face

Description: Identify faces in the video and return face information and session ID for lip sync.

Authentication: Bearer Token

Authorization: Bearer YOUR_API_TOKEN

Request Parameters (Body)

Field	Type	Required	Default	Description
video_id	string	Optional	-	Video ID generated by Kling AI. Choose either this or video_url, cannot be both empty or both provided.
video_url	string	Optional	-	Video URL. Choose either this or video_id, cannot be both empty or both provided.

Response Parameters

Field	Type	Description
code	integer	Error code (0 indicates success)
message	string	Error message
request_id	string	Request ID
data	object	Data object
data.session_id	string	Session ID, valid for 24 hours, used for creating lip sync tasks
data.face_data	array	Face data list
data.face_data[].face_id	string	Face ID
data.face_data[].face_image	string	Face preview image URL
data.face_data[].start_time	integer	Lip sync available interval start time (ms)
data.face_data[].end_time	integer	Lip sync available interval end time (ms)

Request Example

{
  "video_url": "https://example.com/video.mp4"
}

Response Example

{
  "code": 0,
  "message": "success",
  "request_id": "req_123456",
  "data": {
    "session_id": "session_abc123",
    "face_data": [
      {
        "face_id": "face_001",
        "face_image": "https://example.com/face_preview.jpg",
        "start_time": 0,
        "end_time": 5000
      }
    ]
  }
}

---

2. Create Lip Sync Task

Endpoint: POST /kling/v1/videos/advanced-lip-sync

Description: Create a lip sync video generation task.

Authentication: Bearer Token

Authorization: Bearer YOUR_API_TOKEN

Request Parameters (Body)

Field	Type	Required	Default	Description
session_id	string	Required	-	Session ID returned by face recognition endpoint
face_choose	array	Required	-	Face lip sync configuration, currently only supports single person
face_choose[].face_id	string	Required	-	Face ID returned by face recognition endpoint
face_choose[].audio_id	string	Optional	-	Audio ID from preview endpoint. Choose either this or sound_file.
face_choose[].sound_file	string	Optional	-	Audio file (Base64 or URL). Choose either this or audio_id.
face_choose[].sound_start_time	integer	Required	-	Audio clip start time (ms)
face_choose[].sound_end_time	integer	Required	-	Audio clip end time (ms)
face_choose[].sound_insert_time	integer	Required	-	Insert time for clipped audio (ms)
face_choose[].sound_volume	number	Optional	1	Audio volume, range [0, 2]
face_choose[].original_audio_volume	number	Optional	1	Original video volume, range [0, 2]
external_task_id	string	Optional	-	Custom task ID, must be unique per user
callback_url	string	Optional	-	Callback URL for task result notification

Response Parameters

Field	Type	Description
code	integer	Error code (0 indicates success)
message	string	Error message
request_id	string	Request ID
data	object	Data object
data.task_id	string	Task ID
data.task_status	string	Task status: submitted, processing, succeed, failed

Request Example

{
  "session_id": "session_abc123",
  "face_choose": [
    {
      "face_id": "face_001",
      "sound_file": "https://example.com/audio.mp3",
      "sound_start_time": 0,
      "sound_end_time": 5000,
      "sound_insert_time": 0,
      "sound_volume": 1.0,
      "original_audio_volume": 0.5
    }
  ]
}

---

3. Query Task

Endpoint: GET /kling/v1/videos/advanced-lip-sync/:task_id

Description: Query the status and result of a lip sync task by task ID.

Authentication: Bearer Token

Authorization: Bearer YOUR_API_TOKEN

Path Parameters

Field	Type	Required	Description
task_id	string	Required	Task ID returned by create task endpoint

Response Parameters

Field	Type	Description
code	integer	Error code (0 indicates success)
message	string	Error message
request_id	string	Request ID
data	object	Data object
data.task_id	string	Task ID
data.task_status	string	Task status
data.task_status_msg	string	Task status message
data.task_result	object	Task result (only returned on success)
data.task_result.videos	array	Generated video list
data.task_result.videos[].id	string	Generated video ID
data.task_result.videos[].url	string	Generated video URL
data.task_result.videos[].duration	string	Video duration (seconds)
data.created_at	integer	Task creation time (ms)
data.updated_at	integer	Task update time (ms)

Response Example

{
  "code": 0,
  "message": "success",
  "request_id": "req_123456",
  "data": {
    "task_id": "task_123456",
    "task_status": "succeed",
    "task_status_msg": "success",
    "task_result": {
      "videos": [
        {
          "id": "vid_lipsync_123",
          "url": "https://example.com/lipsync_video.mp4",
          "duration": "5"
        }
      ]
    },
    "created_at": 1722769557708,
    "updated_at": 1722769557708
  }
}