Free Transcriptions in Resolve using OpenAI Whisper

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Mario69Rossi

  • Posts: 172
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Apr 11, 2023 1:40 pm

Videoneth wrote:
Mario69Rossi wrote:Do you have a "system clean-up" thing installed on your mac? Or anything that would clear cache?


No I dont. Also I have both 0.17.16 and the older 0.17.5 installed and this older works fine with or without wifi connection.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

mateuszp1990

  • Posts: 3
  • Joined: Sun Apr 16, 2023 5:43 pm
  • Real Name: Mateusz Petryka

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun Apr 16, 2023 5:48 pm

Thank you for this amazing tool. I would really like to use it because it is the only FREE tool that allows to transcribe Polish language. Unfortunately I can't get it work. I am a noob when it comes to IT/programming.

I am on Windows 11. I have the newest DR Studio version. I downloaded and installed Python 3.11.3 from the official Python website. Then I installed StoryToolkitAI 0.17.16.2. When I open it, a cmd window appears for a few seconds and then it disappears, nothing happens. I uploaded the screenshot as the attachment. What can I try to make this work?
Attachments
screen.png
screen.png (27.47 KiB) Viewed 3425 times
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 17, 2023 11:21 am

After testing the new transcription tool integrated in Resolve, it doesn't beat the accuracy of StoryToolkit/whisper with the large-model v2

There is one thing I like in their tool is the fact that it detects burps, crowd cheering, music, etc.. and it indicates it in the transcription. Which is very helpful for finding laugh moments or things like that.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 17, 2023 4:51 pm

mateuszp1990 wrote:I am on Windows 11. I have the newest DR Studio version. I downloaded and installed Python 3.11.3 from the official Python website. Then I installed StoryToolkitAI 0.17.16.2. When I open it, a cmd window appears for a few seconds and then it disappears, nothing happens. I uploaded the screenshot as the attachment. What can I try to make this work?

Whisper, the transcription module we're using for StoryToolkitAI only works up to Python 3.10 - anything beyond that is super experimental. We actually recommend using python 3.9, please see this info here: https://github.com/octimot/StoryToolkit ... md#windows

Also, there's a thread on GitHub about using the tool on Windows here: https://github.com/octimot/StoryToolkitAI/issues/25 - multiple Python versions will prevent the Resolve API using with the tool (but the tool itself can still work even without Resolve). Feel free to join the conversation there and maybe we can help you set things up.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 17, 2023 5:07 pm

Videoneth wrote:After testing the new transcription tool integrated in Resolve, it doesn't beat the accuracy of StoryToolkit/whisper with the large-model v2

There is one thing I like in their tool is the fact that it detects burps, crowd cheering, music, etc.. and it indicates it in the transcription. Which is very helpful for finding laugh moments or things like that.

Resolve 18.5 looks super exciting!! I'd love to test it, but can't afford to break our editing room until we deliver some stuff in the next weeks...

I think it would be amazing if we could import external transcriptions, maybe even via SRT. Have you seen any way to do that?
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline
User avatar

Paddywack0

  • Posts: 62
  • Joined: Wed Sep 01, 2021 6:04 pm
  • Real Name: Nick Elborough

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 17, 2023 8:07 pm

Looks good. Did a few tests today. Works well inside Resolve but the Export to text option is very basic. No Timecode!

I would still use Storykit for sending timecoded transcripts to an Edit Producer.

But it is fast, seems to be pretty accurate and integrates well into the timeline in terms of editing direct from the transcript and moving around the timeline.

What is a bit odd is that you seem to have to analyse separately for transcribing and subtitling. It doesn't seem able to use the same file once you have done one.
Windows 10 Pro - Version Build 19043.1766
HP z820 24core Dual 3.5 Xeon E5
128Gb Memory
RTX 2080ti graphics
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 17, 2023 8:54 pm

Another thing still missing from their transcription tools is the ability to import the text directly into a text or text+ node. They have everything in place to do it, so hopefully they will improve that gap in user-friendliness.
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Apr 20, 2023 11:18 am

Octavian Mot wrote:
Videoneth wrote:After testing the new transcription tool integrated in Resolve, it doesn't beat the accuracy of StoryToolkit/whisper with the large-model v2

There is one thing I like in their tool is the fact that it detects burps, crowd cheering, music, etc.. and it indicates it in the transcription. Which is very helpful for finding laugh moments or things like that.

Resolve 18.5 looks super exciting!! I'd love to test it, but can't afford to break our editing room until we deliver some stuff in the next weeks...

I think it would be amazing if we could import external transcriptions, maybe even via SRT. Have you seen any way to do that?


The tool is pretty basic for now, and it's buggy too. I'll certainly use it to edit based on text in the future, but I didn't test the accuracy of the transcriptions (that's the most important thing for me).

We can't import external subtitle files (for now)

When a clip is transcribed, any clip put on the timeline will constantly focus on the source viewer when it's selected, even if the clip selected doesn't have the transcription attached to it.

It's a beta so bugs are expected, but this one is pretty much stops any further tests (for me at least), since it constantly switch to the source viewer every time a clip is selected on the timeline. So you can't just make a cut, select what you want to delete and press DEL (or any other keyboard shortcuts).

Edit:
Honestly, I think that your tool will always have a place because for many reasons.

1) It's good to have alternatives that works with resolve, Storytoolkit will work better on some occasion, Resolve in others.
2) You (and the final users) are not limited, so you can constantly add new features.
3) Having (and using) alternative is always a good thing to keep the software diversity alive, in case one just goes down for any reason.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Mario69Rossi

  • Posts: 172
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Apr 20, 2023 12:21 pm

If DR would use Whisper models then the results would be similar but I think the model they are using is not as advanced as Whispers models. That's probably why it's only in English
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Apr 20, 2023 4:04 pm

Mario69Rossi wrote:If DR would use Whisper models then the results would be similar but I think the model they are using is not as advanced as Whispers models. That's probably why it's only in English

It depends on the model used, and how is the audio. A slight change in the audio can change the transcription.

I always have to use the voice isolation, and depending on its intensity, the results can differ.

On top of that, the whisper models don't "transcribe" the laughs/music/etc. With Resolve, it indicates what's happening when there is no voice between []

And I'm sure BMD trained their own model anyway
Windows 10
19b
nVidia 3090 - 552.22
Offline

Mario69Rossi

  • Posts: 172
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Apr 20, 2023 5:04 pm

Do you use StoryToolkit after voice isolation applied?
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

mateuszp1990

  • Posts: 3
  • Joined: Sun Apr 16, 2023 5:43 pm
  • Real Name: Mateusz Petryka

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 22, 2023 7:18 pm

Octavian Mot wrote:Whisper, the transcription module we're using for StoryToolkitAI only works up to Python 3.10 - anything beyond that is super experimental. We actually recommend using python 3.9, please see this info here: https://github.com/octimot/StoryToolkit ... md#windows

Thank you! I have installed python 3.9 and everything works perfectly.
Offline

eser.listana

  • Posts: 2
  • Joined: Mon Apr 24, 2023 3:46 pm
  • Real Name: Eser Erzurum

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 24, 2023 4:27 pm

Thank you for this great tool! I was able to complete all the steps but I keep getting this error. Does anyone know what I'm doing wrong or how I can fix it?

(venv) C:\StoryToolkitAI>py StoryToolkitAI\app.py --debug
DEBUG:
--------------
Platform: Windows 10
Platform version: 10.0.22621
OS: 10 10.0.22621 SP0 Multiprocessor Free ('', '', '')
running Python 3.9.13.final.0
--------------
DEBUG: All package requirements met.
DEBUG: Loading config file C:\Users\eserl\StoryToolkitAI\config.json.
INFO: Running StoryToolkitAI version 0.17.17
INFO: Using cuda for Torch / Whisper.
DEBUG: MotsResolve module initialized.
DEBUG: Trying to load DaVinciResolveScript module...
DEBUG: Found DaVinci Resolve at the default location: C:\Program Files\Blackmagic Design\DaVinci Resolve\
DEBUG: Unable to find module DaVinciResolveScript from PYTHONPATH - trying default locations next

(venv) C:\StoryToolkitAI>
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 24, 2023 4:56 pm

Mario69Rossi wrote:Do you use StoryToolkit after voice isolation applied?

Was the question for me? If so, yes.
I try to get the cleanest audio before transcribing the timeline.
I have to deal with videos with background noise almost every time.
Even if the voices are clear, I prefer to remove everything before using the tool.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed May 17, 2023 10:04 pm

Using the 0.18.4 right now.

I didn't test the "Export transcript as Fusion Text" yet, but that's something that can really help for those who want to add effects on subtitles.

There is a "Selection to markers" option I just tested now! It's excellent.

You search a word or a sentence, click on "select all", then "Selection to markers", and you have duration markers on the timeline for every single one of the word/sentence!

The tool is almost perfect (for my use) now.
Attachments
storytoolkit.jpg
storytoolkit.jpg (49.28 KiB) Viewed 2851 times
Windows 10
19b
nVidia 3090 - 552.22
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu May 18, 2023 4:36 am

The tool is almost perfect (for my use) now.

Cool, thanks! :D

The Export transcript as Fusion Text is available since 0.18.3. It basically takes your transcript and turns it into a Text and Merge node which you can paste and connect into your Fusion comp:

Screenshot 2023-05-18 at 06.20.06.png
Screenshot 2023-05-18 at 06.20.06.png (396.8 KiB) Viewed 2781 times


I highly recommend doing a Fusion Comp on an Adjustment Layer in Resolve: drag and drop an Adjustment Layer into a new video track, over your entire timeline, right click and Open in Fusion.

This way you can see the footage/effects/etc. underneath, plus you can hear the audio in case you want to adjust the text to sound further. More about the process here: https://github.com/octimot/StoryToolkit ... usion-text

Also, to make the text titling gapless, use the Prevent Gaps Shorter Than, for eg. 0.3, when doing the transcriptions. This way the AI will prevent leaving empty frames between titles that are very close to each other.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun May 21, 2023 6:52 pm

I'll do that the next time I'm working on a shorter video, that needs dynamic subtitles, thanks.
I'll try to hack one of the Titles I have to make it a fusion effect, so I can drag it on these fusion titles.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Diakon

  • Posts: 1
  • Joined: Sat Jul 22, 2023 11:58 am
  • Real Name: Krzysztof Żwirski

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Jul 22, 2023 12:51 pm

mateuszp1990 wrote:Thank you for this amazing tool. I would really like to use it because it is the only FREE tool that allows to transcribe Polish language. Unfortunately I can't get it work. I am a noob when it comes to IT/programming.

I am on Windows 11. I have the newest DR Studio version. I downloaded and installed Python 3.11.3 from the official Python website. Then I installed StoryToolkitAI 0.17.16.2. When I open it, a cmd window appears for a few seconds and then it disappears, nothing happens. I uploaded the screenshot as the attachment. What can I try to make this work?


Hej,

Udało Ci się zrobić by działało?
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Aug 16, 2023 4:11 pm

0.19.5 is out for everyone :o
I have a question, what's the expectations (generally speaking) in term of time when indexing a video? Let's say with a 3090.

For those interested, here a demo of the dev about how it works, it's pretty crazy.

Windows 10
19b
nVidia 3090 - 552.22
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Aug 16, 2023 10:56 pm

Videoneth wrote:0.19.5 is out for everyone :o
I


19.5 is not out for everyone, only if you pay him.

This person turned out to be manipulative and greedy. He used us for beta testing, then released all the fixed and upgraded versions which we fixed behind a $20 a month paywall, He did 1 upgrade for us, to 17.16.2 and that was to include popup ads and nags for the paid version

He promised it was always going to be free, and it's true, the alpha grade buggy version from March never got deleted, but there was never an upgrade either, you have to pay for them.

Beyond this this product really being pay-ware pretending to be free, it's the gaslighting and manipulation of this community that's most distasteful. If you want to sell your app for $20 a month say that, and do that, don't pretend it's anything else.
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Aug 16, 2023 11:33 pm

CougerJoe wrote:
Videoneth wrote:0.19.5 is out for everyone :o
I


19.5 is not out for everyone, only if you pay him.

This person turned out to be manipulative and greedy. He used us for beta testing, then released all the fixed and upgraded versions which we fixed behind a $20 a month paywall, He did 1 upgrade for us, to 17.16.2 and that was to include popup ads and nags for the paid version

He promised it was always going to be free, and it's true, the alpha grade buggy version from March never got deleted, but there was never an upgrade either, you have to pay for them.

Beyond this this product really being pay-ware pretending to be free, it's the gaslighting and manipulation of this community that's most distasteful. If you want to sell your app for $20 a month say that, and do that, don't pretend it's anything else.


First of all, stop LYING!
It is a FREE tool, and the 0.19.5 version is out for everyone to use! I'm using it right now, and I'm not a Patreon member.

He promised it was always going to be free... guess what, it's still free!

I very grateful for this free tool, and you should be! If you don't like how someone decides to do "business", go somewhere else! Who are you to tell another person who they should do things, when they literally give you free stuff!

And last thing, you don't pay a monthly fee! You could get on his Patreon, pay $20 bucks (for the very latest version) and cancel the patreon. But it doesn't matter, it's literally updated for everyone on their GitHub page!

And like I said in another response to you ... the Patreon members are technically the ones who are more likely to be testing, since they get the new version when they come out. It means that any bugs will be experienced by the Patreon members.

Those who are taking it from the GitHub page get literally the better version because they are available weeks after the release on Patreon.

This is one of the best way in my opinion to distribute free software, while getting revenu from it from people who wants to support them financially.
Last edited by Videoneth on Wed Aug 16, 2023 11:39 pm, edited 1 time in total.
Windows 10
19b
nVidia 3090 - 552.22
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Aug 16, 2023 11:39 pm

Videoneth wrote:.


First of all, stop LYING!
It is a FREE tool, and the 0.19.5 version is out for everyone to use! I'm using it right now, and I'm not a Patreon member.

[/quote]

I don't even see a mention of a 19.5, link to this free 19.5, if you mean 19.4, then he states it's $20 a month like every other version since he used us for beta testing
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Aug 16, 2023 11:45 pm

CougerJoe wrote:
Videoneth wrote:.


First of all, stop LYING!
It is a FREE tool, and the 0.19.5 version is out for everyone to use! I'm using it right now, and I'm not a Patreon member.



I don't even see a mention of a 19.5, link to this free 19.5, if you mean 19.4, then he states it's $20 a month like every other version since he used us for beta testing

[/quote]

Well, look better.

And no, you're disrespectful to the people developing this FREE tool, and you're whining about something you're not entitle to anyway. Why would anyone help you out, it blows my mind.

Lol, and you're you so entitled that you are asking me to take your hand and show you how to get and use this tool.

Get over it! Thanks...

And to the dev of StoryToolKit, I'll say it again... THANK YOU SO MUCH for what you've created!
I use it almost every single day, and it's the best (and more useful) plugin for Resolve I have.

Edit:
And BTW, $20 for this piece of software is NOTHING. And it's not $20 a month, anyone can cancel their Patreon membership after 1 month after downloading the tool if they just want the latest one, now.
So Maybe instead of calling the dev out, get a month on Patreon and get the very last version of it, if you can't figure out something else. How about that.
Last edited by Videoneth on Wed Aug 16, 2023 11:50 pm, edited 1 time in total.
Windows 10
19b
nVidia 3090 - 552.22
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Aug 16, 2023 11:48 pm

You seem unwell and everything you've said is untrue.
19.4 is the latest version and it's behind a paywall.

There is no 19.5, you can't link to a patreon only 19.5 or a free version because it doesn't exist.
I fear you've having a psychotic break with reality. You believe you've downloaded a paid product that doesn't exist for free. It's very concerning
Offline
User avatar

Joe Shapiro

  • Posts: 2771
  • Joined: Thu Jul 25, 2013 7:23 am
  • Location: Los Angeles CA USA

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 12:18 am

Ok, you guys have piqued my curiosity enough that I’m exploring this. So far I’ve found that if you’re unwilling to install from sources then yes, 0.17.16 is the most recent version available.

HOWEVER, one can install the latest sources from GitHub and build the latest version. There’s what looks like a very complete set of instructions to do this. I’m about to give it a shot. Claims it’s all free and I have no reason at this point to doubt such.
Director, Editor, Problem Solver. Been cutting indie features for 23 years. FCP editor from version 2 to 7.
Resolve 18.6.6
MacBook Pro 16" M1 Max 64GB RAM, macOS 14.2.1
MacBook Air 13" M1 8GB RAM, macOS 14.2.1
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 12:31 am

Joe Shapiro wrote:
HOWEVER, one can install the latest sources from GitHub and build the latest version. There’s what looks like a very complete set of instructions to do this. I’m about to give it a shot. Claims it’s all free and I have no reason at this point to doubt such.


We'd all be very interested to hear of your results and review, and any problems you run into with the installation. Yes it's the compiled windows software that he sells via monthly subscription.
Offline
User avatar

iddos-l

  • Posts: 799
  • Joined: Sat Mar 30, 2019 7:55 pm
  • Real Name: iddo lahman

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 5:06 am

Hi,
I haven't dove into the subject, but the way I see it, if the latest version source code is public on GitHub, I consider it free.
I have a question for users who use this plugin:
Why not use the built-in transcription of Resolve?
Does the plugin provide better results? If so, in what ways does it outperform the Resolve version?
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 5:10 am

I only have the free one, as said, it's old and buggy, but it does translations like other whisper clients, this is the obvious next step for Resolve, but it's not there yet.
Offline
User avatar

Joe Shapiro

  • Posts: 2771
  • Joined: Thu Jul 25, 2013 7:23 am
  • Location: Los Angeles CA USA

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 4:17 pm

Any reason you don’t want to install the latest from sources? It takes a bit of work but it sounds like you’re really chafing with the old version.
Director, Editor, Problem Solver. Been cutting indie features for 23 years. FCP editor from version 2 to 7.
Resolve 18.6.6
MacBook Pro 16" M1 Max 64GB RAM, macOS 14.2.1
MacBook Air 13" M1 8GB RAM, macOS 14.2.1
Offline
User avatar

Joe Shapiro

  • Posts: 2771
  • Joined: Thu Jul 25, 2013 7:23 am
  • Location: Los Angeles CA USA

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 5:19 pm

Installed on macOS and tried to run.
Got lots of CERTIFICATES errors until I ran this command:
/Applications/Python\ 3.10/Install\ Certificates.command

Then all was good and I was up and running v0.19.5
Director, Editor, Problem Solver. Been cutting indie features for 23 years. FCP editor from version 2 to 7.
Resolve 18.6.6
MacBook Pro 16" M1 Max 64GB RAM, macOS 14.2.1
MacBook Air 13" M1 8GB RAM, macOS 14.2.1
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 5:55 pm

Oh boy, so much hate @CougerJoe ... I always appreciate honest and raw feedback, but calling someone manipulative and greedy is really top notch raw!

As I mentioned before in a much lengthier post, the tool is free and open source.

Thank you @Videoneth for taking the time to explain this once more.

We give early access to features to some of our Patreon subscribers for supporting this project, but all features are or will be pushed to the free GitHub page for everyone.

We're currently testing the newest 0.19.5 patch in our editing room and will probably compile a standalone installer this or next week. We'd really want to add another optimization before that, but let's see if it doesn't break too much of the rest of the code...

Videoneth wrote:I have a question, what's the expectations (generally speaking) in term of time when indexing a video? Let's say with a 3090

On a 3090, the video indexing should take around 1-1.2x the length of the video irrespective of resolution, but it really depends on the number of scene changes - the tool runs an advanced scene detection algorithm that is interested to see if new stuff was added to the scene.


iddos-l wrote:Why not use the built-in transcription of Resolve?

The tool is evolving in handling more than transcriptions.

For eg.,
you can do semantic search on text - search by topic, question etc.,

there's a direct interface to ChatGPT that includes timecoded contexts - just ask AI anything about what's being talked about in your footage (or summarize, or ask it to translate or explain concepts),

And we recently added the video indexing feature which allows you to find anything in your videos using plain English - see the video link posted by Videoneth above.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 11:13 pm

Joe Shapiro wrote:Any reason you don’t want to install the latest from sources? It takes a bit of work but it sounds like you’re really chafing with the old version.


Python on windows doesn't really work, when you don't know python and use step by step guides. Each different piece of software tends to or often does break the previous piece of software. As I have used a lot of AI software running python they keep on breaking each other.

Octavian Mot your business model was overly aggressive and did not reward anyone that was helping you. And your last update of the free version being to add nothing but Nags and advertisements when you could have at least fixed the timing issues. You have all your AI features and plenty more that could attract people towards your monthly subscription model. Annoying your primary userbase is not such a great idea.

I do apologise for any offense
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Aug 17, 2023 11:51 pm

iddos-l wrote:Hi,
I haven't dove into the subject, but the way I see it, if the latest version source code is public on GitHub, I consider it free.
I have a question for users who use this plugin:
Why not use the built-in transcription of Resolve?
Does the plugin provide better results? If so, in what ways does it outperform the Resolve version?

For me, what's implemented in resolve is really not good enough, and unusable for my situation (except for quick subtitles).

Of course, the one in Resolve is well integrated, and has some perks, but StoryToolKit is way more advanced imo. The Large-V2 model I always use for the transcription gives way more accurate results.

On top of the the ability to search words, parts of words, mark them all with timeline markers, at once...
Link the transcription to the timeline while playing it, and many more.

And they added a video search tool not that long ago. Which is pretty cool and crazy imo. Being able to index a whole video, then find scenes with a red chair, or whatever, can save a ton of times.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 12:11 am

Octavian Mot wrote:Oh boy, so much hate @CougerJoe ... I always appreciate honest and raw feedback, but calling someone manipulative and greedy is really top notch raw!

As I mentioned before in a much lengthier post, the tool is free and open source.

Thank you @Videoneth for taking the time to explain this once more.
.

You're welcome.

I can't stand seeing people dunking on others who are literally giving their time (and money) away for free, to help others.

There is another piece of open-source software that use this same type of model. And I honestly it's the best model (imo) for sharing open-source software, while having a way for people to support the devs financially. Everyone wins.

It's called "Flowframes" (https://github.com/n00mkrad/flowframes).

The guy has the latest version on github, but if you want the compiled version for windows, it's on patreon. OR you can compile it yourself from the code on github and you'll get the last version tool, for free.

I didn't know anything about Visual studio, I just installed it, opened the project (downloaded from github), clicked on "compile", and it made me the latest version, for free.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 12:27 am

CougerJoe wrote:
Joe Shapiro wrote:Any reason you don’t want to install the latest from sources? It takes a bit of work but it sounds like you’re really chafing with the old version.


Python on windows doesn't really work, when you don't know python and use step by step guides. Each different piece of software tends to or often does break the previous piece of software. As I have used a lot of AI software running python they keep on breaking each other.


Python on windows works perfectly. Another piece of misinformation :/
And of course they "break each other", because you use them wrong and don't have separate virtual environnements (created with python itself, or anaconda/miniconda)

Anyone who never used python before had to experience that, myself included...

They ou realize that there are integrated tools (and 3rd party tools) that exist since day one to create environments for each piece of software, which use their own version of certain package, without messing with anything else.

And BTW, on the Storytookit install page. And MOST guides with a step by step approche show how to create a virtual environment. The ones that don't just assume people already know how to do it.
---
4. Set up a virtual environment
We recommend running the tool inside a virtual environment like virtualenv. This is not required, but it prevents messing up your Python installation.

First, create a virtual environment:

python3.10 -m venv venv
---

I wish you good luck
Windows 10
19b
nVidia 3090 - 552.22
Offline

CougerJoe

  • Posts: 346
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 12:30 am

Videoneth wrote:There is another piece of open-source software that use this same type of model. And I honestly it's the best model (imo) for sharing open-source software, while having a way for people to support the devs financially. Everyone wins.

It's called "Flowframes" (https://github.com/n00mkrad/flowframes).



It's not his model I have a problem with it's the gaslighting. He said, his software is free and a gift to the Resolve community, it will always be free, then he paywalls all future releases of the windows executable and abandons the free version. So he's technically correct, his software is free and always will be unless he deletes the old alpha windows release, but it's not morally defensible.

It's been brought up that the source code is available, but ethically and perhaps legally someone one here can't start releasing windows executable versions when the author has chosen to keep them behind a paywall, it may be his livelihood. He could stagger the free version release as he updates the paid versions, but has chosen not to do that.

@videoneth I already told you I don't know Python, so I follow the authors setup guides, and yet they continuously break older already installed python software. This is a very common problem due to different requirements and versions
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 1:35 am

CougerJoe wrote:
Videoneth wrote:There is another piece of open-source software that use this same type of model. And I honestly it's the best model (imo) for sharing open-source software, while having a way for people to support the devs financially. Everyone wins.

It's called "Flowframes" (https://github.com/n00mkrad/flowframes).



It's not his model I have a problem with it's the gaslighting. He said, his software is free and a gift to the Resolve community, it will always be free, then he paywalls all future releases of the windows executable and abandons the free version. So he's technically correct, his software is free and always will be unless he deletes the old alpha windows release, but it's not morally defensible.

It's been brought up that the source code is available, but ethically and perhaps legally someone one here can't start releasing windows executable versions when the author has chosen to keep them behind a paywall, it may be his livelihood. He could stagger the free version release as he updates the paid versions, but has chosen not to do that.

@videoneth I already told you I don't know Python, so I follow the authors setup guides, and yet they continuously break older already installed python software. This is a very common problem due to different requirements and versions


I didn't know much about python, but I forced myself to learn the basics so I could use all the AI tools I was interested in. That's often the (very small) price to pay for being able to use the latest tech, for free.

I read all this thread from the beginning, and Octimot never gaslighted people, he just corrected the wrong info. And it's pretty normal to get aggravated when someone tells you that you're lying, when you dont, or misrepresents what you say. On this subject, you are in the wrong. And we explained where and why you are wrong. The software (scripts) is free, anyone can download it now, and have it forever.

Octimot's model is the same as the other example I used.

Patreon members get a pre-build model of the "lego X-Wing". And the people who are not on patreon have to build it themselves (if they want it), but they get all the pieces to do so, for free. And the building process is free too.

If followed the install guide, and I have the 0.19.5 (or .4) that I use almost every day. I used the 0.17.x version for a long time. Most of the features are very useful but also very niche too.

There is nothing to compile either for storytoolkit. It's just a matter of Git Clone the repo and following the install guide once, then git update when someone wants to update it. It even checks if the packages needed are installed (and in their correct version). If not, the script now installs the packages by itself (it wasn't the case with the earlier version).

Anyway, I'm pretty passionate about this subject because what he developed saved me days/weeks of works. I use the transcription > Search > ... to markers 99% of the time. Being able to have a very precise transcription and being able to mark all the things that needs to be removed helps a ton.

AND we can even convert a srt file to a transcription file, and use the tool with it. (take note BMD :D )
Windows 10
19b
nVidia 3090 - 552.22
Offline

eryngium

  • Posts: 2
  • Joined: Fri Aug 18, 2023 2:41 pm
  • Real Name: Eryngium MARIO

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 2:54 pm

I have problem on MAC OS
I generate transcription from Davinci, Sctipt take audio, generate text and save to files.

In console I see this:
Code: Select all
INFO: Finished transcription for Transcription of Timeline 1.wav in 44 seconds
INFO: Linking to current timeline: Timeline 1


But Timline in Davinci is empty. Transctiption not imported to timeline automaticaly.

Before I tested this on Windows and works perfect without problems.

What is wrong?
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 3:55 pm

You should have the window queue with the finished transcription, you click on the transcription you want to open :
storytoolkit1.jpg
storytoolkit1.jpg (15.79 KiB) Viewed 1526 times


And then you should have the main transcription window opening :

storytoolkit2.jpg
storytoolkit2.jpg (193.69 KiB) Viewed 1526 times


If then you click on "Sync with playhead", when you select a word or move your playhead, you'll be at the same spot in the text or on the timeline. You can activate/deactivate it at any time, it's pretty flexible.

Technically, you could ingest 10-20 videos, transcribe them, then doing the timelines and linking them each one to each timeline (I've done that in the past on 20 videos files).
Windows 10
19b
nVidia 3090 - 552.22
Offline

eryngium

  • Posts: 2
  • Joined: Fri Aug 18, 2023 2:41 pm
  • Real Name: Eryngium MARIO

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Aug 18, 2023 5:21 pm

Yes, everything I have like you wrote.
Playhead moving in Davinci when I select text.
But text is not imported to my timeline.

I can import manualy SRT to bin only.

I remember when i install this on windows, everything was automaticaly, when i render trancscription then finaly everything i see in my timeline.
Offline

Videoneth

  • Posts: 1712
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun Aug 20, 2023 3:33 pm

eryngium wrote:Yes, everything I have like you wrote.
Playhead moving in Davinci when I select text.
But text is not imported to my timeline.

I can import manualy SRT to bin only.

I remember when i install this on windows, everything was automaticaly, when i render trancscription then finaly everything i see in my timeline.


Maybe I'm wrong, but I don't remember the tool putting the subtitles automatically on a subtitle track after the transcription. Before the "Import SRT to bin", I remember dragging the srt file manually.

Maybe something changed, but I don't see as a bad thing because the user can decide what's happening at each step. For example, I almost never use the tool for the subtitles feature (I don't add subtitles to most of my videos), but to. If it was creating a subtitle track and importing the srt file automatically, I wouldn't like that.

Maybe it's an option you could ask to the dev to be added in the settings.
Windows 10
19b
nVidia 3090 - 552.22
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Aug 21, 2023 6:40 am

eryngium wrote:I can import manualy SRT to bin only.

The tool can only import the SRT file into the bin. Placing the subtitles in the timeline is not available via Resolve API. If BM unlocks that, we'd be happy to implement it fast on our end...
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Aug 21, 2023 6:45 am

BTW, version 0.19.5 standalone is now publicly available for anyone who doesn't want to or has a hard time installing the tool from source: https://github.com/octimot/StoryToolkit ... ses/latest

Although, we always recommend giving the source installation a try since it's more frequently updated compared to the standalone, plus, in the future, it might contain features which we cannot compile into standalone versions due to packaging conflicts. In case you do have a hard time with the installation, just let us know via GitHub issues and we'll do our best to help...

Cheers!
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Mario69Rossi

  • Posts: 172
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Oct 07, 2023 9:55 am

Both 0.19.5 and 0.17.16 stopped working on my system. Any of you experience the same problem?

The app opens but if I click on any of the buttons nothing happens.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Mario69Rossi

  • Posts: 172
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Oct 07, 2023 10:45 am

Mario69Rossi wrote:Both 0.19.5 and 0.17.16 stopped working on my system. Any of you experience the same problem?

The app opens but if I click on any of the buttons nothing happens.


Okay I found the problem, I turned off superscale on the clips and it works.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

bbqrooster

  • Posts: 1
  • Joined: Wed Mar 27, 2024 4:56 pm
  • Real Name: Victor Kwong

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Mar 27, 2024 5:13 pm

First of all, thanks for developing this wonderful tool. I have been using Premier Pro speech to text for transcription. The quality of the transcription done by Storytoolkitai is much superior. I was running version 0.19.5 of Storytoolkitai on a machine with an Intel i9-10850K and a Nvidia RTX 4080 Super. I found out that if I queued up transcription jobs, the GPU load and GPU memory usage stayed at 100% starting from the 2nd job. If I queue only one job each time (queue and complete one job, then restart the program for the next job), then the GPU load would stay well below 100% and the GPU memory stayed at around 70%. Anything seeing the same thing as I do.
Previous

Return to DaVinci Resolve

Who is online

Users browsing this forum: Drasius Kupstys, ghost355, Leslie Wand, panos_mts, Uli Plank and 231 guests