How to Make Midjourney Consistent Characters Talk Using Runway & Pika Lip Sync (AI Video Tutorial)

Midjourney can now create images with consistent characters, so I’m going to show you how to make a movie with those consistent characters using the new lip sync features in both Runway and Pika. CHAPTERS & LINKS ⬇️

🍿 Here's the Time Trouble movie created in this tutorial
   • Time Travel Trouble - An AI-generated...  

🧐 My comparison of Runway & Pika Lip Sync
   • Runway vs Pika Lip Sync - Compare the...  

🍿 And here's the original Gen-2 and Wav2Lip time travel movie from June 2023
   • AI-Generated Time Travel Episode (Hum...  

📖 CHAPTERS
00:00 Introduction
01:22 Creating character reference images in Midjourney
02:32 Spreadsheet to keep track of characters, styles, script
03:38 Creating style reference images in Midjourney
04:00 Combining a character and style (cref & sref)
05:16 Inpaint the styled image to change wardrobe
06:05 Inpaint the character reference to change wardrobe
07:15 Workarounds for framing (close-up, wide shots)
08:14 Two different consistent characters in the same shot!
09:33 Pika Lip Sync
10:34 Longer Pika lip syncs using video (CapCut hack)
11:46 “Add 4s” doesn’t work with lip sync in Pika
12:11 Use other image-to-video generators to create longer videos and motion (Stable Video hack)
12:28 Runway Lip Sync & Motion Brush section
12:38 Runway Motion Brush
13:28 Add motion before lip sync
13:37 Motion Brush for gestures
14:08 Runway Lip Sync (Generative Audio) using Gen-2 assets
15:04 Runway Camera Control
16:02 Recap of steps: Character, Style, Motion, Speech, Lip Sync
16:26 Our time travel movie with Midjourney consistent characters talking with Runway/Pika lip synch

🛠️ TOOLS
Midjourney: https://www.midjourney.com/
Pika: https://pika.art/
Runway: https://runwayml.com/
I also used these for the final video:
Stable Video: https://www.stablevideo.com/
ElevenLabs: https://elevenlabs.io/
Suno: https://www.suno.ai/
CapCut: https://capcut.com/

#midjourney #runwayml #pika #pikalabs #lipsync #runwaylipsync #pikalipsync #talkingcharacters #ailipsync #aivideo #aivideocreation #aitutorial #aitutorialforbeginners #filmmaking #howtomakeaivideo #consistentcharacters #motionbrush #runwaymotionbrush