Free Transcriptions in Resolve using OpenAI Whisper

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Feb 10, 2023 7:13 pm

Octavian Mot wrote:We pushed an update only for the non-standalone version that adds support for the new Whisper module and large-v2 model, but also includes a few bug fixes. The large-v2 model is better in some cases compared with the large-v1 model (especially for non-english languages). But, the large v1 model is also still available as an option in the Transcription Settings window. Feel free to give it a run and let us know if something doesn't work as it used to!

More updates are coming in the weeks ahead and hopefully we'll be able to pack them into standalone releases for MacOS and Windows soon!


I never used the large model, only the medium.
Do you see a real difference between the two in your experience?

Btw, thank you for this tool. I used it almost all the time now.
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

AssistantEditor1

  • Posts: 107
  • Joined: Tue Jan 31, 2023 5:30 am
  • Real Name: Milton Breslin

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun Feb 26, 2023 12:22 pm

First of all, let me start by saying thank you for such an amazing tool!

If I can just poll other users, about how long does it take for you to get a transcription?

Using the large model, the stand alone version seems to hang for at least 15-20 minutes for me on the "loading model" step.

Seems like a long time considering the file is only about 2 minutes long.

Is that about normal?
Offline

Asher123

  • Posts: 127
  • Joined: Mon Jan 18, 2021 6:11 pm
  • Real Name: David Asher

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Mar 01, 2023 4:51 pm

This is great OP. Thank you.

Is it possible to use OpenAI Whisper to create transcripts that translate the video from, say, English to French?
Offline

CougerJoe

  • Posts: 330
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Mar 08, 2023 11:13 am

Sander de Regt wrote:On the websit/git it says that the tool will also (maybe) run in standalone mode. But I can't seem to find how to use it in standalone mode. Am I overlooking some info somewhere? I'm pretty new to gitting things done. I downloaded the exe archive because I thought that would be the easiest way to get started, but apart from the CMD window opening and closing by itself I don't see much going on.


That's a stand alone version as opposed to standalone mode(using it without resolve), standalone versions haven't been updated since 19th November. Others have reported that problem. Is anyone using a different Whisper variant that has the large V2 model and gives good results. This is the only one I know with the Resolve integration but I increasingly find it not very useful for translation work, it stops listening for periods of time or after a period of time it gives up and repeats the same line

Not saying it got worse, just the more I use it, the more flawed I find it with translation work. The large V2 model is supposed to help but the standalone version needs to be updated
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Mar 23, 2023 5:57 pm

I am using version 0.17.5.M1 it looks like there is an update - Do I have to do the whole installation process again?
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

CougerJoe

  • Posts: 330
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 24, 2023 2:02 am

Mario69Rossi wrote:I am using version 0.17.5.M1 it looks like there is an update - Do I have to do the whole installation process again?


Looking now, I don't see any update
· octimot_StoryToolkitAI.png
· octimot_StoryToolkitAI.png (25.73 KiB) Viewed 5704 times
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 24, 2023 11:54 am

Sorry for the lack of response... a lot of stuff happening at the moment here!

Mario69Rossi wrote:I am using version 0.17.5.M1 it looks like there is an update - Do I have to do the whole installation process again?

If you are using the Git version, just do a git pull, see this: https://github.com/octimot/StoryToolkitAI/blob/main/INSTALLATION.md#updates-on-the-non-standalone-tool

CougerJoe wrote:Looking now, I don't see any update

The standalone version will be updated soon with all the updates available already in the Git version.

Asher123 wrote:Is it possible to use OpenAI Whisper to create transcripts that translate the video from, say, English to French?

Not directly using Whisper, but it's on the list of planned features: https://github.com/octimot/StoryToolkit ... d-features


Thanks again for all the feedback!
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 24, 2023 9:29 pm

Octavian Mot wrote:
Mario69Rossi wrote:I am using version 0.17.5.M1 it looks like there is an update - Do I have to do the whole installation process again?

If you are using the Git version, just do a git pull, see this: https://github.com/octimot/StoryToolkitAI/blob/main/INSTALLATION.md#updates-on-the-non-standalone-tool

CougerJoe wrote:Looking now, I don't see any update

The standalone version will be updated soon with all the updates available already in the Git version.


I have the standalone version. If you were able to add a check for update in the app itself would the great.

Can you add a donate link in the github page? I'd be happy to send a small donation
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Mar 27, 2023 5:52 am

Just uploaded the latest standalone release which adds a lot of new features, including:
    - increased time precision on transcriptions
    - speech pre-detection for more efficient transcriptions (also eliminates a lot of hallucinations)
    - support for the large-v2 Whisper model (this connected with time precision is pretty good BTW)
    - Advanced Search on Resolve markers and even .txt files
More details here: https://github.com/octimot/StoryToolkitAI/releases/latest

There's a lot of work in progress, for eg. we're currently testing how to efficiently introduce GPT3 and 4 into the tool, but also working on stuff that was suggested here, like character limits on subtitles.

Thanks a lot for your feedback - a lot of it went into the tool!
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Mar 27, 2023 7:33 pm

Octavian Mot wrote:Just uploaded the latest standalone release which adds a lot of new features, including:
    - increased time precision on transcriptions
    - speech pre-detection for more efficient transcriptions (also eliminates a lot of hallucinations)
    - support for the large-v2 Whisper model (this connected with time precision is pretty good BTW)
    - Advanced Search on Resolve markers and even .txt files
More details here: https://github.com/octimot/StoryToolkitAI/releases/latest

There's a lot of work in progress, for eg. we're currently testing how to efficiently introduce GPT3 and 4 into the tool, but also working on stuff that was suggested here, like character limits on subtitles.

Thanks a lot for your feedback - a lot of it went into the tool!

Have you consider the possibility of using Bing AI (vs GPT3/4)? I know GPT4 was recently released, but access seems to be somewhat restricted (for now at least).

On a side note...it is too bad the Nvidia Foundations AI system is purely a paid system. It appears to be superior to all other AI as of this time.
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Mar 27, 2023 7:40 pm

I like the allucinante part!!
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline
User avatar

Uli Plank

  • Posts: 21289
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Mar 28, 2023 2:40 am

Do we have a Stanislaw Lem fan here?
No, an iGPU is not enough, and you can't use HEVC 10 bit 4:2:2 in the free version.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Mar 30, 2023 4:58 pm

Speaking of an AI that is hallucinating while taking over the world...

We just released a Patreon-only non-standalone update which includes ChatGPT in the tool. Basically, you can chat with AI directly from the tool and use transcript lines as the base for the conversation:



I'm not mentioning this just to shamelessly plug our Patreon (patreon.com/StoryToolkitAI) :lol:
but, also to say that using this in our everyday work is kind of cool.

You can pretty much ask AI to do whatever on your transcripts, including summarize, list characters, detect emotions or the tone of the conversation, give you ideas about what to ask in the next interview, etc. etc... or even have it create a CSV file with a list of characters or events or whatever from your transcript. ChatGPT not only looks into your context, but also uses it's own training that includes... well... everything.

Not to mention using AI for therapy when the client / boss / teenager kid / etc. is stressing you out...

For eg.: we use it all the time to summarize transcript lines for markers, or to give us portions of the text that could be interesting soundbites on a particular theme, or even to explain the text as we were 5 year olds - which is really interesting when dealing with complicated topics in your scenes.

We're looking now for ways to optimize things, but need some community support to kick things off, by checking the tool and see how AI works in their workflow, and particularly to cover the costs of AI experiments which are not cheap - via Patreon.

If you can't support via Patreon, no worries, the update will be available public for free soon'ish too.

BTW, the tool will always remain free and open source, for as long as we can support it!
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Mar 30, 2023 6:10 pm

If you could also include a donate with paypal or revolut link I would be happy to send a donation.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 31, 2023 7:52 am

Mario69Rossi wrote:If you could also include a donate with paypal or revolut link I would be happy to send a donation.

Thank you! But we're not really looking for single donations.

We're using this in our everyday productions and it's not difficult to simply focus on updates according to what we need internally, but building something for a community that's in a constant feedback loop really gets us to a better tool.

The idea of having a Patreon account is to build a community with people that are genuinely interested in using AI in their everyday editing work and that want to commit to supporting the work and the research we're doing, plus the extra costs that come with compiling the standalone versions, apple certificates etc. etc. etc.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 31, 2023 9:30 am

I respect that at the same time you're missing out on donations from many people, if you think about many Davinci Revolve users switched from Adobe and similar to escape the subscription model. I personally never consider purchasing software/plugins that have a subscription model.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 31, 2023 9:45 am

Mario69Rossi wrote:I respect that at the same time you're missing out on donations from many people, if you think about many Davinci Revolve users switched from Adobe and similar to escape the subscription model. I personally never consider purchasing software/plugins that have a subscription model.


I appreciate your feedback and completely agree with the stupidity of the software as service subscription model and also decided to drop Adobe long time ago because of that!

However, the tool we're making is open source and free, so I don't think the comparison is valid. Everyone can download it without paying anything for it...

As a patron, you don't pay for the software, but you contribute to its development. We're also updating patrons with info/tips/etc. about what makes our editing process with AI easier and how to potentially transition to this interesting new paradigm...

Once you no longer believe that the development is going in the right direction or serving your goals/ideals/etc., you simply no longer contribute, but you could still choose to use the tool for free whenever you feel like it... or build your own on top of our work, because, again, there are no restrictions...
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 31, 2023 1:55 pm

I'm curious, is the transcription time of 5 minutes 49 (with Large V2) for a 32 minutes videos is what I should expect with a 3090?

Do you have stats for the 40xx series?
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 31, 2023 4:34 pm

Videoneth wrote:I'm curious, is the transcription time of 5 minutes 49 (with Large V2) for a 32 minutes videos is what I should expect with a 3090?

That sounds more or less right, depending if you use 'Pre-detect speech' and 'Increased time precision' features. Also, the large-v2 model is super heavy for the GPU.

For us this is kind of acceptable, considering that in the case of most cloud services, this is the time it takes to only to upload the file...

Do you find it slow or fast enough?

Videoneth wrote:Do you have stats for the 40xx series?

We haven't tried it yet on the latest generation of Nvidia GPUs, but you should probably expect a 1.2x - 1.7x speed increase, again... depending on the model and the features you're using next to it.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Jack Chavez

  • Posts: 9
  • Joined: Fri Sep 04, 2015 5:35 pm

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostFri Mar 31, 2023 9:55 pm

Hi. Thanks for this. Even though it’s not particularly fast on my 2019 MPB with 16 GB of RAM — not surprising given the GPU is not being used — it’s already proving useful and I’m giddy about the potential improvements to the workflow. (60 seconds of English content takes 180 seconds to process so that should give you a pretty good idea of the crunch factor.)

Questions:
- Is there a way to kill individual jobs in the queue? Quitting and restarting the program doesn’t seem to achieve that.

- Is there a way to manually flush the cache? (Also, does quitting the program release the cache memory or is a restart required?) I tend to work with multiple languages so only having the models currently needed would be helpful.

- Is being able to store language models locally and being able to load/unload them being considered?

Cheers.
Offline

CougerJoe

  • Posts: 330
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 01, 2023 1:46 am

It is still creating paragraphs instead of sentences. At this stage, 4 months later after people commented about this, I think you must just like it like that, but for many this is not ideal. Can you add an option to break the subs into shorter sentences, even if by default it does paragraphs .

- Is being able to store language models locally and being able to load/unload them being considered?


The models are downloaded and loaded in the future when using it in GPU processing mode. Maybe this is different with CPU processing.
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 01, 2023 4:13 am

Jack Chavez wrote:Questions:
- Is there a way to kill individual jobs in the queue? Quitting and restarting the program doesn’t seem to achieve that.

For now, you can just delete the transcription_queue.json file. You can find in by going to File -> Configuration folder.
A clear queue button and more granular control over transcription items in the queue in in the updates backlog and will come soon.

Jack Chavez wrote:- Is there a way to manually flush the cache? (Also, does quitting the program release the cache memory or is a restart required?) I tend to work with multiple languages so only having the models currently needed would be helpful.
- Is being able to store language models locally and being able to load/unload them being considered?

I feel the answer to these questions is connected: the tool does not need to switch between models when dealing with different languages, unless you choose one of the models ending with .en (English-only). Whatever model you choose in the transcription settings window, it's going to switch it only when it reaches an item in the queue that you selected a different model for.
The underlying system (torch) is pretty efficient at handling caches, processing AI stuff etc., so a restart is not needed. If you're asking this based on something you noticed, feel free to open an Issue on GitHub and we can dig deeper into it.
BTW, you can find the transcription models in your local .cache folder. On Windows, that's C:\Users\<username>\.cache\whisper\<model>, on macOS it's ~/.cache
Once a model is downloaded, you no longer need Internet access to transcribe.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 01, 2023 4:34 am

CougerJoe wrote:It is still creating paragraphs instead of sentences. At this stage, 4 months later after people commented about this, I think you must just like it like that, but for many this is not ideal. Can you add an option to break the subs into shorter sentences, even if by default it does paragraphs .

Short answer: starting with version 0.17.17 you can automatically limit the number of characters or words per line https://github.com/octimot/StoryToolkitAI/blob/main/CHANGELOG.md#01717---2023-03-28

We will also implement punctuation splitting in one of the next updates.

But, based on the feedback we've been getting, it would be ideal to be able to create a set of rules (eg. Netflix guidelines) and let AI do the splitting for you.

In the past few months we've been getting deeper and deeper into Whisper's algorithm, and gained quite granular access to all sorts of stuff - word-level timings being one of them. If there's enough interest and support, we could start to work on an update which does intelligent semantic splitting of transcript phrases, as soon as we clear some of the backlog.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Jack Chavez

  • Posts: 9
  • Joined: Fri Sep 04, 2015 5:35 pm

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 01, 2023 5:59 am

Once a model is downloaded, you no longer need Internet access to transcribe.


Thanks for the answers. I found them very helpful.
Offline

verial

  • Posts: 4
  • Joined: Mon Apr 03, 2023 11:37 pm
  • Real Name: Damon Verial

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 03, 2023 11:39 pm

Despite installing it exactly as guided, I cannot get this app to work:

Traceback (most recent call last):
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11345, in poll_resolve_data
self.on_resolve('timeline_markers_changed')
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11216, in on_resolve
self.save_markers_to_project_settings()
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11136, in save_markers_to_project_settings
self.stAI.project_settings = self.stAI.get_project_settings(project_name=project_name)
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11920, in get_project_settings
with open(project_settings_path, 'r') as json_file:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\veria\\StoryToolkitAI\\projects\\watts\\project.json'

WARNING: Resolve not reachable after 15 tries. Disabling Resolve API polling until tool restart.
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Apr 05, 2023 2:50 am

Octavian Mot wrote:Once a model is downloaded, you no longer need Internet access to transcribe.


With the new version it doesnt seem to be storing the model. It fails without internet connection
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Apr 05, 2023 4:39 pm

verial wrote:Despite installing it exactly as guided, I cannot get this app to work:

Traceback (most recent call last):
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11345, in poll_resolve_data
self.on_resolve('timeline_markers_changed')
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11216, in on_resolve
self.save_markers_to_project_settings()
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11136, in save_markers_to_project_settings
self.stAI.project_settings = self.stAI.get_project_settings(project_name=project_name)
File "C:\storytoolsai\StoryToolkitAI\app.py", line 11920, in get_project_settings
with open(project_settings_path, 'r') as json_file:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\veria\\StoryToolkitAI\\projects\\watts\\project.json'

WARNING: Resolve not reachable after 15 tries. Disabling Resolve API polling until tool restart.


I have these errors to.

Fortunately, I have multiple "project.json" files from other projets, so what I did is create the folder that Storytoolkit can't create (or reach), with the same name as the Resolve project, and copy/past this json file in it. It works for me.

I don't know if it's a bug, but I rely so much on this tool now :D I do whatever it takes to make it work.

so in the mean time, go in to your c:\Users\veria\StoryToolkitAI\projects - see if you have older folders (with the json file in them), just copy/past one of the folder, rename it with the Resolve project name, and it should work.

EDIT:
I did a Git pull just now, it looks like it was fixed
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostWed Apr 05, 2023 6:12 pm

Last version stopped working for me, it keepes failing. Not sure if it's due to a very limited wifi connectivity, I'm currently in a remote area. But old version works fine with pre loaded models.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Apr 06, 2023 5:41 pm

I installed the latest version and cannot get it to run. It opens, and displays the following before terminating.
Screenshot 2023-04-06 123616.png
StoryToolkitAI failure point
Screenshot 2023-04-06 123616.png (8.42 KiB) Viewed 4448 times


Originally, I did not have python installed on my system, but it was easy enough to hunt down the correct version and get it installed. Since, I have reinstalled python, reinstalled StoryToolkitAI, restarted...basically everything I could think of, or read about in this thread...and still it does nothing.

There isn't much to go on in the log file either. This is the report I got after trying it again today.
Code: Select all
2023-04-06 12:23:44,159 - StAI - DEBUG:
--------------
Platform: Windows 10
 Platform version: 10.0.22621
 OS: 10 10.0.22621 SP0 Multiprocessor Free  ('', '', '') 
 running Python 3.9.13.final.0
-------------- (app.py:147)
2023-04-06 12:23:44,444 - StAI - DEBUG: All package requirements met. (app.py:167)
2023-04-06 12:23:44,444 - StAI - DEBUG: Loading config file C:\Users\Anthony\StoryToolkitAI\config.json. (app.py:11553)
2023-04-06 12:23:46,119 - StAI - INFO: Running StoryToolkitAI version 0.17.16.2 (standalone) (app.py:11370)
2023-04-06 12:23:46,141 - StAI - INFO: Using cuda for Torch / Whisper. (app.py:8737)
2023-04-06 12:23:46,141 - StAI - DEBUG: MotsResolve module initialized. (mots_resolve.py:42)
2023-04-06 12:23:46,142 - StAI - DEBUG: py not found on this machine. Skipping Python version check. (mots_resolve.py:95)
2023-04-06 12:23:46,142 - StAI - DEBUG: Trying to load DaVinciResolveScript module... (mots_resolve.py:159)
2023-04-06 12:23:46,142 - StAI - DEBUG: Found DaVinci Resolve at the default location: C:\Program Files\Blackmagic Design\DaVinci Resolve\ (mots_resolve.py:224)
2023-04-06 12:23:46,142 - StAI - DEBUG: Unable to find module DaVinciResolveScript from PYTHONPATH - trying default locations next (mots_resolve.py:236)


Anyone have any ideas on things to try to get this to run?
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline

Sander de Regt

  • Posts: 3500
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostThu Apr 06, 2023 8:46 pm

The latest version of Acronis flags the latest Storykit as malware.generic
Does anyone know why?
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 08, 2023 9:51 pm

I just figured out something really important for anyone else having issues with this. Do NOT install python from the Microsoft store. EVEN THOUGH it is the correct python version, StoryTookitAI CANNOT find it. Python MUST be installed from the official python website.
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSat Apr 08, 2023 11:44 pm

Nice! I was expecting this to take a lot longer based on the earlier comments I have seen. I was able to do a 10:19 long interview in 86 seconds using the large-v2 set.
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun Apr 09, 2023 4:23 pm

Sander de Regt wrote:The latest version of Acronis flags the latest Storykit as malware.generic
Does anyone know why?

I had to completely deactivate the protection from Acronis a long time ago (I only use Bitdefender now), because it constantly had false positive.
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun Apr 09, 2023 4:27 pm

Octavian Mot wrote:
Do you find it slow or fast enough?


No I think it's in line with my system.
No complain about the speed, it uses Cuda.
I was just curious if I could squeeze more

I know I know, github.... but what would change my life is to be able to add simple markers after search a doing a search for a word : Mark it all
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostSun Apr 09, 2023 4:48 pm

Is anyone was able to update to 0.17.19?
Code: Select all
INFO: Running StoryToolkitAI version 0.17.17
INFO: Using cuda for Torch / Whisper.
WARNING: Found the following Python versions on this machine: 3.10, 3.9, 3.8
WARNING: Current installation runs on Python version 3.9.13.
WARNING: DaVinci Resolve API on Windows might not work if this is the case.
WARNING: Please uninstall the following Python versions: 3.10
WARNING: You can use --skip-python-check to skip this check, but the tool might not start.
WARNING: Or, install the tool using git into a virtual environment with the highest Python version available on your machine on your own risk.
INFO: Resolve API module found and loaded
update available 0.17.19
WARNING: A new version of StoryToolkitAI is available.

Use git pull to update.

With a git pull it says that I'm up to date
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

CougerJoe

  • Posts: 330
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 4:43 am

Videoneth wrote:Is anyone was able to update to 0.17.19?

update available 0.17.19
WARNING: A new version of StoryToolkitAI is available.

Use git pull to update.[/code]
With a git pull it says that I'm up to date


He used us to beta test then changed to monthly subscription payware.
If you are a patreon customer, and can't download the latest version contact him there
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 5:26 am

CougerJoe wrote:
Videoneth wrote:Is anyone was able to update to 0.17.19?

update available 0.17.19
WARNING: A new version of StoryToolkitAI is available.

Use git pull to update.[/code]
With a git pull it says that I'm up to date


He used us to beta test then changed to monthly subscription payware.
If you are a patreon customer, and can't download the latest version contact him there


Maybe it would be a good idea to simply read the text on the first page of the site...
Screenshot 2023-04-10 002428.png
storytoolkitai_site_snap
Screenshot 2023-04-10 002428.png (53.79 KiB) Viewed 3694 times
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 5:43 am

AnthonyReno wrote:I just figured out something really important for anyone else having issues with this. Do NOT install python from the Microsoft store. EVEN THOUGH it is the correct python version, StoryTookitAI CANNOT find it. Python MUST be installed from the official python website.

Yes! Glad you found this fix on your own - it's mentioned in the documentation, but I just made it clear in the release page too. Unfortunately, the Resolve API implementation via Python is really sensitive on Windows and we need to be very careful with the environment there. You don't need to have Python for the standalone release, but if you do have it on your machine things get weird and there's nothing we can do about it unless BM or someone smarter help us sort this out: https://forum.blackmagicdesign.com/viewtopic.php?f=21&t=178184...
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 5:44 am

Videoneth wrote:I know I know, github.... but what would change my life is to be able to add simple markers after search a doing a search for a word : Mark it all


We're planning a feature that does this in two steps after you search for something via find or advanced search. Basically, you'll be able to select all the segments that contain your found matches and then use key M or SHIFT+M to mark in Resolve.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 5:54 am

CougerJoe wrote:He used us to beta test then changed to monthly subscription payware.

And yet here you are @CougerJoe - still being able to use the tool for free for as long as you like... :lol: :lol:

I detect the sense of humour in your statement @CougerJoe! :D

But again, all updates will be released for free forever. Some Patreon members get Early access to said updates because we think it's only fair. If you look at it from another angle, all the community benefits from their support...

Thanks @AnthonyReno for pointing out that paragraph! I really don't know how to make this more clear...

We are not a software company, but just sharing this tool with everyone while taking care that it works as expected, offer support, and trying to implement what people need in their daily work. TBH, some of the features that we added (for eg. punctuation splitting - that you also requested I think), we don't actually need in our editing room, but simply wanted to make them available for people that do.
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

CougerJoe

  • Posts: 330
  • Joined: Wed Sep 18, 2019 5:15 am
  • Real Name: bob brady

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 5:59 am

AnthonyReno wrote:
CougerJoe wrote:
Videoneth wrote:Is anyone was able to update to 0.17.19?

update available 0.17.19
WARNING: A new version of StoryToolkitAI is available.

Use git pull to update.[/code]
With a git pull it says that I'm up to date


He used us to beta test then changed to monthly subscription payware.
If you are a patreon customer, and can't download the latest version contact him there


Maybe it would be a good idea to simply read the text on the first page of the site...
Screenshot 2023-04-10 002428.png


He's using a payment model others use, including software I subscribe to, but it's much more reasonably priced, 1/3 the price of this software per month. The delay from the Patreon release is currently 14months to public free release. I don't know how his delay works, but as he charges so much it wouldn't make sense for the delay to be short
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 7:07 am

CougerJoe wrote:The delay from the Patreon release is currently 14months to public free release. I don't know how his delay works, but as he charges so much it wouldn't make sense for the delay to be short

We released this project publicly around 6 months ago and our Patreon page is up for around 3 weeks - I think there's a typo in your post, but even so, I hope you don't mind if I say that your general assessment seems inaccurate, because:

The first Patreon-member-only update was version 0.17.18 released 11 days ago (see https://github.com/octimot/StoryToolkitAI/blob/main/CHANGELOG.md), so logically this is also the longest you had to wait if you're not a member of the Frequent User or Producer tiers.

Nevertheless, these posts feel like yelp reviews to free beer tasting and since I'm not good at reading them, and don't really understand the point you're trying to make, nor your proposal, I hope you don't mind if I stop responding to them...

Again, no one is forcing you to pay for anything, and you can use the tool for free for as long as you want and wait a bit for access to all features, or choose to support the project if you can afford it to be part of something, while getting some perks.

Fair enough?

Please see the post above and read the GitHub project page, it's all there... everything else are just... AI hallucinations... :lol: :lol:
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline
User avatar

AnthonyReno

  • Posts: 170
  • Joined: Mon May 23, 2022 9:58 am
  • Location: USA
  • Real Name: Mark Reno

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 7:29 am

CougerJoe wrote:
AnthonyReno wrote:
CougerJoe wrote:
He used us to beta test then changed to monthly subscription payware.
If you are a patreon customer, and can't download the latest version contact him there


Maybe it would be a good idea to simply read the text on the first page of the site...
Screenshot 2023-04-10 002428.png


He's using a payment model others use, including software I subscribe to, but it's much more reasonably priced, 1/3 the price of this software per month. The delay from the Patreon release is currently 14months to public free release. I don't know how his delay works, but as he charges so much it wouldn't make sense for the delay to be short


I don't think I have ever seen anyone use that particular payment model. Everyone I see has a payment model to get early access to stuff. That's not what this is. The payment model is to be PART OF the project. You get to work on it, put in feedback, etc. I think maybe you have misunderstood the explanations of the subscription...or maybe haven't seen them? It is definitely not what you seem to think it is.
DR & F Studio v18.1.1,Win11Pro, i9-13900K, 128GB RAM
GPUs:Intel UHD 770 & RTX 3090ti
OS:1.8TB SSD,P&C drives:2x2TB SSD
Speed Editor, Pen:Huion Inspiroy Dial 2 & XPPen Artist 13.3 Pro, Elecom HUGE Trackball
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 4:52 pm

Octavian Mot wrote:
Videoneth wrote:I know I know, github.... but what would change my life is to be able to add simple markers after search a doing a search for a word : Mark it all


We're planning a feature that does this in two steps after you search for something via find or advanced search. Basically, you'll be able to select all the segments that contain your found matches and then use key M or SHIFT+M to mark in Resolve.

Perfect! Thanks a lot for that.
I know this feature will save me a ton of time :o
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostMon Apr 10, 2023 5:22 pm

CougerJoe wrote:
Videoneth wrote:Is anyone was able to update to 0.17.19?

update available 0.17.19
WARNING: A new version of StoryToolkitAI is available.

Use git pull to update.[/code]
With a git pull it says that I'm up to date


He used us to beta test then changed to monthly subscription payware.
If you are a patreon customer, and can't download the latest version contact him there

I think there are some confusion here.

The patreon exists only for people who want to support the open-source project, which is free to download from the github page. It's a donation system.

I personally don't use the installer for the windows version, I just cloned the project and installed all the python packages required, and regularly do a "git pull" to get the latest changes.

I don't think it's cool and honest to say that he uses us as beta tester before putting it behind a paywall.
Because it's not true. And the patreon people would be the beta-testers in this case, if they get earlier access :D, so thanks to them lol.

This is a free tool and it's available to everyone. It works as intended. The development could stop today (or 2 months ago), it wouldn't matter, the core functionality covers 95%. The updates are just bonuses. And the project could be forked tomorrow by someone else anyway.

But coming back on the notification for the new version, I suspect that the changelog.md file was pushed before the script files that contain the changes for 0.17.19 (or maybe not, I don't know), for the people who download it from github. Since Patreon members get the latest versions first.

There are many open-source project like that, the "packaged" versions (with installer and everything included) are very often not the latest versions, and people donating get it first. BUT, everyone can clone the project, and use the latest version for free if they know what they are doing.

Myself, I installed Visual Studio just to be able to compile a piece of open source software from git hub.
I don't code, I don't know anything about Visual Studio, but I'm able to get the latest version of what the people on Patreon get. It's a good system imo. People who want to support creators get something in return, and the tools are still free to use by anyone. And if someone wants the latest shiny version, they just have to use some elbow grease.

But I think it's just a misunderstanding about the patreon/open-source system.
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Mario69Rossi

  • Posts: 166
  • Joined: Fri Jan 29, 2021 4:11 am
  • Real Name: Mario Rossi

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Apr 11, 2023 10:01 am

0.17.16 doesnt work for me if not connected to the net.
DaVinci Resolve Studio 18.6.5
MacBook Pro M1 (2020) | 16GB ram | 1TB SSD
macOS Sonoma 14.4
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Apr 11, 2023 10:45 am

Videoneth wrote:[But coming back on the notification for the new version, I suspect that the changelog.md file was pushed before the script files that contain the changes for 0.17.19 (or maybe not, I don't know), for the people who download it from github. Since Patreon members get the latest versions first.

tldr;
Users need to know if updates might be available for them.

--
Full explanation:

Because it would be complicated for the tool to determine if the user currently has access to the Patreon-only versions, the public repo always gets the newest CHANGELOG.md.

This way, not only we're making sure that everyone is aware that an update might be available, but we're also trying to be transparent about which updates are available earlier for Patreon members and everyone else.

Only exception, since most standalone users don't really want to bother with git versions:
- if you use the non-standalone (git version), the tool checks what is the latest non-standalone version here: https://api.storytoolkit.ai/version/
- if you use the standalone version, the tool checks if there's a new release on the releases page: https://github.com/octimot/StoryToolkit ... ses/latest

This way, we're not bothering standalone version users with non-standalone updates that are hard for them to pull.

This was a lot of info... maybe we can discuss this via GitHub issues next time @Videoneth :)
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Octavian Mot

  • Posts: 286
  • Joined: Mon Aug 25, 2014 2:42 pm
  • Location: Germany

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Apr 11, 2023 10:52 am

Mario69Rossi wrote:0.17.16 doesnt work for me if not connected to the net.

Sorry, but it's really hard to understand what is going on on your machine, maybe you can open an issue on the GitHub page and we can try to debug it together?

The tool does not need access to the Internet once the selected model has been downloaded to your cache folder for transcriptions. The download only happens once per user per machine (so it will still be there next time when you use the tool).
Trying to keep it together at mots.us
Taming AI for filmmaking at StoryToolkit.ai
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Apr 11, 2023 1:20 pm

Octavian Mot wrote:
Videoneth wrote:[But coming back on the notification for the new version, I suspect that the changelog.md file was pushed before the script files that contain the changes for 0.17.19 (or maybe not, I don't know), for the people who download it from github. Since Patreon members get the latest versions first.

tldr;
Users need to know if updates might be available for them.

--
Full explanation:

Because it would be complicated for the tool to determine if the user currently has access to the Patreon-only versions, the public repo always gets the newest CHANGELOG.md.

This way, not only we're making sure that everyone is aware that an update might be available, but we're also trying to be transparent about which updates are available earlier for Patreon members and everyone else.

Only exception, since most standalone users don't really want to bother with git versions:
- if you use the non-standalone (git version), the tool checks what is the latest non-standalone version here: https://api.storytoolkit.ai/version/
- if you use the standalone version, the tool checks if there's a new release on the releases page: https://github.com/octimot/StoryToolkit ... ses/latest

This way, we're not bothering standalone version users with non-standalone updates that are hard for them to pull.

This was a lot of info... maybe we can discuss this via GitHub issues next time @Videoneth :)

Yeah don't worry about it, it wasn't an issue ;)
I generally get what's happening while I'm I writing about a subject or after posting it.
I guess the process of writing it for others makes it more clear in my mind.
I do the same thing when I'm stuck about something in Resolve, and post questions on the forum.
Windows 10
18.6.6
nVidia 3090 - 537.42
Offline

Videoneth

  • Posts: 1615
  • Joined: Fri Nov 13, 2020 11:03 pm
  • Warnings: 1
  • Real Name: Maxwell Allington

Re: Free Transcriptions in Resolve using OpenAI Whisper

PostTue Apr 11, 2023 1:29 pm

Mario69Rossi wrote:0.17.16 doesnt work for me if not connected to the net.

Maybe your mac is deleting the cache folder containing the models for the transcription, and each time you launch the tool, it downloads it again. And if you're not online, it can't do that.

Do you have a "system clean-up" thing installed on your mac? Or anything that would clear cache?
Windows 10
18.6.6
nVidia 3090 - 537.42
PreviousNext

Return to DaVinci Resolve

Who is online

Users browsing this forum: 4EvrYng, Bing [Bot], Muath1982, peeceful and 132 guests