making animals talk

Hi,
Im struggling on how to make my animals lips move in order to make it look like their talking.

Im finding that by cutting clips and making them faster or slower or reverse or forward is time consuming and the clips wind up being so small I can't do anything with them.

In order to do this do I use morphing, warping? Can you direct me with steps on how to accomplish this.
I bought Davinci in December of 2021, and I want to be able to do this function. But I cant find anyone to teach me this.

Please help. I really like Davinci and when I learn this. I would love to get the davinci keyboard to make it easier.

Thnak you
Julie Panasik

It's not a cheap solution, but if you really want this to work, you have to look at a 3rd party plugin.
Since morphing isn't as popular as it used to be, there aren't too many options.

This is the one that has existed the longest

https://revisionfx.com/products/reflex/resolve/#gallery

And it used to be very good. I haven't used it recently, but they have a demo version available that works in Fusion and Resolve, so by all means check it out.

You can also use the grid warp inside of Fusion/Resolve but that doesn't do morphing, just warping and uses a grid (hence the name) and not splines which is much more useful for situations like you're describing.

It's not a cheap solution, but if you really want this to work, you have to look at a 3rd party plugin.
Since morphing isn't as popular as it used to be, there aren't too many options.

This is the one that has existed the longest

https://revisionfx.com/products/reflex/resolve/#gallery

And it used to be very good. I haven't used it recently, but they have a demo version available that works in Fusion and Resolve, so by all means check it out.

You can also use the grid warp inside of Fusion/Resolve but that doesn't do morphing, just warping and uses a grid (hence the name) and not splines which is much more useful for situations like you're describing.

Does anyone else at Davinci know how to make an animal talk without morphing. Just by reverse, forward, and speed. If so can they tell me the steps to make it work?

If I understand correctly, you want to make an animal appear as if talking by playing the clip at varying positing and negative speeds? That will be very difficult because you won't necessarily have the right mouth positions to work with.

You really need to use warping for the lips, jaw and the cheeks, and you will also need to create and comp in parts of the mouth and teeth that are currently occluded by the lips.

If you really must approach this editorially instead of the VFX route, try using TimeStretcher in Fusion. It allows you to precisely map source time to record time while interpolating for you between the keyframes.

Does anyone else at Davinci know how to make an animal talk without morphing. Just by reverse, forward, and speed. If so can they tell me the steps to make it work?

Do you mean something like this?


Here is another way but you need to install the free reactor plugin and the "Suck Less Audio Modifier"

Just use CrazyTalk. I've been using it for several years. Place your voice on the audio track, set your lips, nose, eyes, etc. for motion and click render. It will sync all together. Much faster and looks great.

If you are an editor, I'm shocked that you think Crazy Talk is good. It looks like one of those dummy's ventriliquists use.

Check out Moon pie on either Tick Toc or You tube. They have two dogs talking. It looks so real. I believe they used Final Cut. I can't believe there isn't a way on Davinci. I don't have an apple computer so I can't use Final Cut which is what some of those people with talking dogs use.

Can't any one at Davinci show me the steps to do this?


Julie

If I understand correctly, you want to make an animal appear as if talking by playing the clip at varying positing and negative speeds? That will be very difficult because you won't necessarily have the right mouth positions to work with.

You really need to use warping for the lips, jaw and the cheeks, and you will also need to create and comp in parts of the mouth and teeth that are currently occluded by the lips.

If you really must approach this editorially instead of the VFX route, try using TimeStretcher in Fusion. It allows you to precisely map source time to record time while interpolating for you between the key frames.

Could you write out the steps for me on how to do this? I did buy the Davinci software. I wish one of the Davinci editors would make a tutorial on how to do this. There is no one to show you how on the internet, unless you use the ones that look so phony. Im looking for quality like Moonpie on YOU tube or Tick tock, who use two dogs. And they do it so well. Im so frustrated that I can't find someone, or pay soneone to show me how. My name is Julie Panasik.

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

Agreed. Was skeptical until I watched one of the videos. That's exactly what they're doing. It's a trick that's only going to work if there's nothing else moving in the shot, and the camera is still.

To do it in Resolve, you'd just use the retime controls. With the clip selected in the timeline, hit command/ctrl-R. Scrub to before the mouth movement begins, click on black downward-facing arrow and select "add speed point" from the dropdown menu. Scrub to the end of the mouth movement, repeat, then use the handles to reposition the video to line up with audio.

Dragging the top handle works differently than dragging the bottom one, so experiment until you get the hang of it, but basically, the top one stretches time faster/slower and stays "pinned" to the frame you're dragging; the bottom one changes the moment a speed ramp begins or ends. To reverse a segment, select "reverse" from same dropdown.

All you have to do is repeat this for every syllable you want the animal to utter... Yes, it's very tedious. Yes, people really do that tedious work by hand. Yes, BMD should add a hotkey to add speed points, it's a long standing feature request.

If I were planning to do much of this, I'd look for a machine learning solution, but you definitely need to consider the cost/benefit in either case, and maybe some life goal re-evaluation.

Could you write out the steps for me on how to do this? I did buy the Davinci software. I wish one of the Davinci editors would make a tutorial on how to do this. There is no one to show you how on the internet, unless you use the ones that look so phony. Im looking for quality like Moonpie on YOU tube or Tick tock, who use two dogs. And they do it so well. Im so frustrated that I can't find someone, or pay soneone to show me how. My name is Julie Panasik.

The warping method would be too involved to explain in an online tutorial. The rocking back-and-forth with TimeStretcher is pretty easy. I think several people here already explained it. Just take a clip into Fusion, connect the TimeStretcher to the MediaIn node, drag the TimeStretcher into the viewer and move the slider left or right to see what it does.

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

Agreed. Was skeptical until I watched one of the videos. That's exactly what they're doing. It's a trick that's only going to work if there's nothing else moving in the shot, and the camera is still.

To do it in Resolve, you'd just use the retime controls. With the clip selected in the timeline, hit command/ctrl-R. Scrub to before the mouth movement begins, click on black downward-facing arrow and select "add speed point" from the dropdown menu. Scrub to the end of the mouth movement, repeat, then use the handles to reposition the video to line up with audio.

Dragging the top handle works differently than dragging the bottom one, so experiment until you get the hang of it, but basically, the top one stretches time faster/slower and stays "pinned" to the frame you're dragging; the bottom one changes the moment a speed ramp begins or ends. To reverse a segment, select "reverse" from same dropdown.

All you have to do is repeat this for every syllable you want the animal to utter... Yes, it's very tedious. Yes, people really do that tedious work by hand. Yes, BMD should add a hotkey to add speed points, it's a long standing feature request.

If I were planning to do much of this, I'd look for a machine learning solution, but you definitely need to consider the cost/benefit in either case, and maybe some life goal re-evaluation.

Thank you for this. I did this with the Retime controller. The problem I ran across is when my clips are short. It doesn't take long for a dogs mouth to open and shut. So with the speed points placed close together the arrow on the bottom doesn't show. So I can't make the clip go faster, or reverse. How can I handle this part?

And what is a machine learning solution?
This is just a hobby for me. Om retired and enjoy doing tedious work.

Julie

If you are an editor, I'm shocked that you think Crazy Talk is good. It looks like one of those dummy's ventriliquists use.

Julie

Hmmm, been an DP,TD, Editor for 35 years. You use software that fits your job. I have used CrazyTalk for clients that were amazed by my work. You can move other parts of the head like eyes blinking, ears, nose, foreheads, etc., all kinds of movement. They also have the new 2D to 3D option. Most users don't know the full potential of CrazyTalk, but I've used it since ver.1 and have become very good with the software.

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

Agreed. Was skeptical until I watched one of the videos. That's exactly what they're doing. It's a trick that's only going to work if there's nothing else moving in the shot, and the camera is still.

To do it in Resolve, you'd just use the retime controls. With the clip selected in the timeline, hit command/ctrl-R. Scrub to before the mouth movement begins, click on black downward-facing arrow and select "add speed point" from the dropdown menu. Scrub to the end of the mouth movement, repeat, then use the handles to reposition the video to line up with audio.

Dragging the top handle works differently than dragging the bottom one, so experiment until you get the hang of it, but basically, the top one stretches time faster/slower and stays "pinned" to the frame you're dragging; the bottom one changes the moment a speed ramp begins or ends. To reverse a segment, select "reverse" from same dropdown.

All you have to do is repeat this for every syllable you want the animal to utter... Yes, it's very tedious. Yes, people really do that tedious work by hand. Yes, BMD should add a hotkey to add speed points, it's a long standing feature request.

If I were planning to do much of this, I'd look for a machine learning solution, but you definitely need to consider the cost/benefit in either case, and maybe some life goal re-evaluation.

Thank you for this. I did this with the Retime controller. The problem I ran across is when my clips are short. It doesn't take long for a dogs mouth to open and shut. So with the speed points placed close together the arrow on the bottom doesn't show. So I can't make the clip go faster, or reverse. How can I handle this part?

And what is a machine learning solution?
This is just a hobby for me. Om retired and enjoy doing tedious work.

Julie

Increase the zoom level on the clip and the arrows will appear

If you are an editor, I'm shocked that you think Crazy Talk is good. It looks like one of those dummy's ventriliquists use.

Julie

Hmmm, been an DP,TD, Editor for 35 years. You use software that fits your job. I have used CrazyTalk for clients that were amazed by my work. You can move other parts of the head like eyes blinking, ears, nose, foreheads, etc., all kinds of movement. They also have the new 2D to 3D option. Most users don't know the full potential of CrazyTalk, but I've used it since ver.1 and have become very good with the software.

I going by what you had sent me and it didn't look real at all. Maybe you can show me something that looks good.

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

Hi, CAn you help me with this? I need some of my speed points to be right up against each other at times. But when I do this I can't see the drop dowhn arrow to change the speed, or make it go in reverse.
How can I handle this?

Julie

If you are an editor, I'm shocked that you think Crazy Talk is good. It looks like one of those dummy's ventriliquists use.

Julie

Hmmm, been an DP,TD, Editor for 35 years. You use software that fits your job. I have used CrazyTalk for clients that were amazed by my work. You can move other parts of the head like eyes blinking, ears, nose, foreheads, etc., all kinds of movement. They also have the new 2D to 3D option. Most users don't know the full potential of CrazyTalk, but I've used it since ver.1 and have become very good with the software.

I going by what you had sent me and it didn't look real at all. Maybe you can show me something that looks good.

I didn't send you anything. Mixing me up with someone else. Just Google CrazyTalk. Most Youtube videos are Ho-Hum, but there are some good ones. Here's a nice talking dog. I like the eye movements

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

Hi, CAn you help me with this? I need some of my speed points to be right up against each other at times. But when I do this I can't see the drop dowhn arrow to change the speed, or make it go in reverse.
How can I handle this?

Julie

Increase the zoom level on the timeline/clip and the arrows will appear

I saw the videos of moonpie.starbox, they just play some parts of the video forward and backward multiple times at various speeds, they do not use any morphing or warping

Hi, CAn you help me with this? I need some of my speed points to be right up against each other at times. But when I do this I can't see the drop dowhn arrow to change the speed, or make it go in reverse.
How can I handle this?

Julie

Increase the zoom level on the timeline/clip and the arrows will appear

Thank you Dave for answering me. All you guys at Davinci are the best and so helpful.
Julie

If I understand correctly, you want to make an animal appear as if talking by playing the clip at varying positing and negative speeds? That will be very difficult because you won't necessarily have the right mouth positions to work with.

You really need to use warping for the lips, jaw and the cheeks, and you will also need to create and comp in parts of the mouth and teeth that are currently occluded by the lips.

If you really must approach this editorially instead of the VFX route, try using TimeStretcher in Fusion. It allows you to precisely map source time to record time while interpolating for you between the keyframes.

Igor can you do this step by step? I went into Fusion and found the node Time stretched and connected it to the Median node and the output node. But in the viewer the video did not move at all. What am I missing in these steps?

Thank you
Julie

If the MediaIn moves, the time stretcher moves. It won't make a still image move. It is very helpful for people trying to help you to help them by including screen grabs of what you're doing.

Another method might be to grab stills of each letter being 'spoken' and overlay those, syllable by syllable, over your dog's head. This is a pretty advanced process you're going for so best of luck. If I were doing this, I'd research shows like Pup Academy and even reach out to their vfx team for advice.

If the MediaIn moves, the time stretcher moves. It won't make a still image move. It is very helpful for people trying to help you to help them by including screen grabs of what you're doing.

Another method might be to grab stills of each letter being 'spoken' and overlay those, syllable by syllable, over your dog's head. This is a pretty advanced process you're going for so best of luck. If I were doing this, I'd research shows like Pup Academy and even reach out to their vfx team for advice.

Im using videos in Fusion. Do I need to add another node to the time stretcher? Can you explain the steps in how I can accomplish this? Other than watching tutorials Im new to Fushion.
I checked out Pup Academy it appears to be a place to go and train your dog. Am I missing something?
What do they have to do with making a dog talk?

Julie

Timestretcher initially has only one key frame, 0, this means that the frame 0 from mediaIn node is held forever... You need to change numbers on the slider in the inspector panel to make it work. Like this, you step through the timeline and enter the number of the frame from mediaIn that you want to show at that particular frame in the timeline.

Yes, it would have been be more logical perhaps, if the timestretcher was already animated by default to reproduce exact frames from mediaIn (like in After Effects), so that you know where you stand, and start modifying instead of starting from scratch. Or it would be more confusing?

Sent from my Mi 9T using Tapatalk

Pup Academy is a TV show.

https://www.imdb.com/title/tt10302548/

If the MediaIn moves, the time stretcher moves. It won't make a still image move. It is very helpful for people trying to help you to help them by including screen grabs of what you're doing.

Another method might be to grab stills of each letter being 'spoken' and overlay those, syllable by syllable, over your dog's head. This is a pretty advanced process you're going for so best of luck. If I were doing this, I'd research shows like Pup Academy and even reach out to their vfx team for advice.

Im using videos in Fusion. Do I need to add another node to the time stretcher? Can you explain the steps in how I can accomplish this? Other than watching tutorials Im new to Fushion.
I checked out Pup Academy it appears to be a place to go and train your dog. Am I missing something?
What do they have to do with making a dog talk?

Julie

Galin, I researched the Pup Academy and its cast. There is no possible way to contact the production people of that show. They don't want people bothering them for something so simple. Especially from their stand point. The show is adorable though. Do you know to be more exact with each syllable would I use retime curve? Or just retime controls? Which one would provide the detail I need. Also when I chose the clips I want to use from the media panel and drag those to my timeline, and I start using retime control it shows the footage from the media panel, which I don't want to used. Why does it do this?

Does anyone else at Davinci know how to make an animal talk without morphing. Just by reverse, forward, and speed. If so can they tell me the steps to make it work?

Do you mean something like this?


Can you show me in the Fushion time line how I can do this?

Here is another way but you need to install the free reactor plugin and the "Suck Less Audio Modifier"

[/

Can you show me in the Fushion time line how I can do this?

1. Drag the clip you want to animate in the timeline, if it has any audio attached, remove it.
2. Drag also the audio file which contains the voice you want to synchronize.
3. Select both the clip and the audio, right click and select New Compound Clip.
2. Open the compound clip in fusion page
4. Add a Time Stretcher node between MediaIn1 and MediaOut1

5. Select the Time Stretcher node, go to the inspector and set the Interpolation mode to Nearest.
6. Hit play and adjust the Source Time parameter back and forth in real time during playback, all your movements will be automatically recorded in keyframes.
7. If you are not satisfied with the recording, double click on the Source Time label to remove all keyframes, click the diamond icon to add a keyframe and repeat step 6.

The main problem is that the Source Time parameter allows frame numbers beyond the frame range of the clip, if you move the Source time beyond the frame range the video will appear as black or paused.
You can follow these extra steps to limit the range of the Source Time parameter

8. Right click on the TimeStretcher node and select Edit Controls
9. Click on the "ID" drop down menu and select the SourceTime parameter
10. Change the Range & Allowed values, for example if the clip is 200 frames long, set both Range & Allowed values 0 to 200
11. Change the Input Ctrl from ScrewControl to SliderControl and click OK.

12. The Source Time parameter will be moved from the Controls tab to the User Tab, you can't move the source time beyond the range you specified on the previous steps


If you want to use Optical Flow (for smoother motion on slowed parts), add an Optical Flow node before the Time stretcher node, set the Time stretcher node Interpolation mode to Flow.

Don't use optical flow before you finish the recording of the Source Time parameter because optical flow is very heavy and can't be used in real time.

Can you show me in the Fushion time line how I can do this?

1. Drag the clip you want to animate in the timeline, if it has any audio attached, remove it.
2. Drag also the audio file which contains the voice you want to synchronize.
3. Select both the clip and the audio, right click and select New Compound Clip.
2. Open the compound clip in fusion page
4. Add a Time Stretcher node between MediaIn1 and MediaOut1

5. Select the Time Stretcher node, go to the inspector and set the Interpolation mode to Nearest.
6. Hit play and adjust the Source Time parameter back and forth in real time during playback, all your movements will be automatically recorded in keyframes.
7. If you are not satisfied with the recording, double click on the Source Time label to remove all keyframes, click the diamond icon to add a keyframe and repeat step 6.

The main problem is that the Source Time parameter allows frame numbers beyond the frame range of the clip, if you move the Source time beyond the frame range the video will appear as black or paused.
You can follow these extra steps to limit the range of the Source Time parameter

8. Right click on the TimeStretcher node and select Edit Controls
9. Click on the "ID" drop down menu and select the SourceTime parameter
10. Change the Range & Allowed values, for example if the clip is 200 frames long, set both Range & Allowed values 0 to 200
11. Change the Input Ctrl from ScrewControl to SliderControl and click OK.

12. The Source Time parameter will be moved from the Controls tab to the User Tab, you can't move the source time beyond the range you specified on the previous steps


If you want to use Optical Flow (for smoother motion on slowed parts), add an Optical Flow node before the Time stretcher node, set the Time stretcher node Interpolation mode to Flow.

Don't use optical flow before you finish the recording of the Source Time parameter because optical flow is very heavy and can't be used in real time.

Thank you so very much for all of this. How can I take a picture of my timeline so you can see what Im trying to accomplish? And do you know of anyone who I could talk with for training on only this feature of making mouths move on animals? I don't want to take a whole video editing class. But no one addresses what Im trying to do. Im willing to pay for your time or others, if you know of someone.
who can help me. My email is j2panasik@yahoo.com. Next week I'll be back from vacation and I will attempt what you have given me so far. But I really need a one and one demonstration. So if you could tell me how to send you a picture of my timeline that be great.

Thank you
Julie

Juiie

Can you show me in the Fushion time line how I can do this?

1. Drag the clip you want to animate in the timeline, if it has any audio attached, remove it.
2. Drag also the audio file which contains the voice you want to synchronize.
3. Select both the clip and the audio, right click and select New Compound Clip.
2. Open the compound clip in fusion page
4. Add a Time Stretcher node between MediaIn1 and MediaOut1

5. Select the Time Stretcher node, go to the inspector and set the Interpolation mode to Nearest.
6. Hit play and adjust the Source Time parameter back and forth in real time during playback, all your movements will be automatically recorded in keyframes.
7. If you are not satisfied with the recording, double click on the Source Time label to remove all keyframes, click the diamond icon to add a keyframe and repeat step 6.

The main problem is that the Source Time parameter allows frame numbers beyond the frame range of the clip, if you move the Source time beyond the frame range the video will appear as black or paused.
You can follow these extra steps to limit the range of the Source Time parameter

8. Right click on the TimeStretcher node and select Edit Controls
9. Click on the "ID" drop down menu and select the SourceTime parameter
10. Change the Range & Allowed values, for example if the clip is 200 frames long, set both Range & Allowed values 0 to 200
11. Change the Input Ctrl from ScrewControl to SliderControl and click OK.

12. The Source Time parameter will be moved from the Controls tab to the User Tab, you can't move the source time beyond the range you specified on the previous steps


If you want to use Optical Flow (for smoother motion on slowed parts), add an Optical Flow node before the Time stretcher node, set the Time stretcher node Interpolation mode to Flow.

Don't use optical flow before you finish the recording of the Source Time parameter because optical flow is very heavy and can't be used in real time.

Thank you so very much for all of this. How can I take a picture of my timeline so you can see what Im trying to accomplish? And do you know of anyone who I could talk with for training on only this feature of making mouths move on animals? I don't want to take a whole video editing class. But no one addresses what Im trying to do. Im willing to pay for your time or others, if you know of someone.
who can help me. My email is j2panasik@yahoo.com. Next week I'll be back from vacation and I will attempt what you have given me so far. But I really need a one and one demonstration. So if you could tell me how to send you a picture of my timeline that be great.

Thank you
Julie

Juiie

I followed your steps and now my video flashed while my animal moves. What did I do wrong? And in the Source Time when I play and slide the dial making the animal talk. What does this do for me? I'm totally confused on this part.
Julie

I followed your steps and now my video flashed while my animal moves.

Did you follow steps 8 to 11?
If yes, did you set the range & allowed parameters correctly?
On my example I set the range from 0 to 200 but this numbers depends on the length of the video.
If your animal video is 50 frames long you must the range & allowed parameters 0 to 50.

I followed your steps and now my video flashed while my animal moves.

Did you follow steps 8 to 11?
If yes, did you set the range & allowed parameters correctly?
On my example I set the range from 0 to 200 but this numbers depends on the length of the video.
If your animal video is 50 frames long you must the range & allowed parameters 0 to 50.

Hi Im sorry to be such a pain in the ass. I bet everyone there hates me because I have so many questions. But I have no one else to turn to. I want to understand how this works. When Im in the time stretcher and I slide that wheel back and forth in the inspecter how does that work, what does it do for me. I don't understand it. Sorry.

I followed your steps and now my video flashed while my animal moves.

Did you follow steps 8 to 11?
If yes, did you set the range & allowed parameters correctly?
On my example I set the range from 0 to 200 but this numbers depends on the length of the video.
If your animal video is 50 frames long you must the range & allowed parameters 0 to 50.

Hi Im sorry to be such a pain in the ass. I bet everyone there hates me because I have so many questions. But I have no one else to turn to. I want to understand how this works. When Im in the time stretcher and I slide that wheel back and forth in the inspecter how does that work, what does it do for me. I don't understand it. Sorry.

I also followed your parameters. My frame rate is 0 to 100. I typed that in for the parameters and the allowance. But now the lips don't move at all. If you would show me how to send you my screen and see what Im doing wrong. Maybe you could show me how to do it. It would save us a lot of time. Just a suggestion.
Thank you.