Blackmagic Forum

I use a 12TB external hard drive which contains about 8TB of raw video. Does anyone have a suggestion for cheap/free way to catalog the video on this hard drive with searchable keywords. Most of it is wildlife and I'd like to add keywords of which animals are in which folders. I'd really like to be able to type in a keyword like "Elk" and have any clip with that keyword pop up and able to play/preview the clips.

Ideas/Suggestions??

Wouldn't this be possible in a resolve project?

Jacob Danell wrote:Wouldn't this be possible in a resolve project?

This is how I do it.

My footage is always associated with a project. There isn't any just hanging around. I keep top level generic folders: Weddings, Stock, Personal, Performances, etc. Inside those folders, I have this structure.



For every project, I just copy that template then rename the folder. For example, 201118 Project Name. Anything I dump into the Audio or Video folders gets organized into further folders by date. If I need to back up the project, archive it or transfer it to another machine, all I need is that one folder. Everything is there.

In my Stock folder, which would be kind of like your example, every time I go and shoot something, I add a new folder under Video/Raw with the date that it was shot. If there's matching external audio, then it has the same date under the Ambient or Voiceover folders.

In Resolve, I have a project associated to the Stock folder. I import the footage, organize it into bins, keyword it, set metadata, etc. If I need to find something, I can just search the keywords and/or metadata.

It's a much different organization process for me than using Lightroom for stills but the end result is the same.

What OS? Searchable tags and comments are available within OSX a few ways.

Good Luck

Howard Roll wrote:What OS? Searchable tags and comments are available within OSX a few ways.

Good Luck

Tags.png

I've tried adding tags in MacOS and it could be a solution but they don't copy over to another drive. So when it's time to backup all my raw video (with tags added) the tags don't copy with the raw files. This means I'd have to re-add all the tags anytime I backup or replace the drive they're on. I have 12 TB of raw video files and re entering thousands of tags is... well, insane is the only word that comes to mind.

MarnMxxx wrote:I've tried adding tags in MacOS and it could be a solution but they don't copy over to another drive. So when it's time to backup all my raw video (with tags added) the tags don't copy with the raw files. This means I'd have to re-add all the tags anytime I backup or replace the drive they're on. I have 12 TB of raw video files and re entering thousands of tags is... well, insane is the only word that comes to mind.

What type of raw file? With the OG DNG files I can tag the root folder and/or apply individual tags frame by frame to the DNGs. The concept of volatile tags is antithetical, you may have some read/write permission issues.

Good Luck

smslavin wrote:My footage is always associated with a project. There isn't any just hanging around.

Same for me. Footage is always project specific.

Howard Roll wrote:What type of raw file? With the OG DNG files I can tag the root folder and/or apply individual tags frame by frame to the DNGs. The concept of volatile tags is antithetical, you may have some read/write permission issues.

Good Luck

My files are: .MP4, .MXF, .mov, .m4v, .MTS,

Jim Simon wrote:

smslavin wrote:My footage is always associated with a project. There isn't any just hanging around.

Same for me. Footage is always project specific.

I do mostly wildlife videos and the raw files are used over and over for many different projects.

smslavin wrote:

Jacob Danell wrote:Wouldn't this be possible in a resolve project?

This is how I do it.

My footage is always associated with a project. There isn't any just hanging around. I keep top level generic folders: Weddings, Stock, Personal, Performances, etc. Inside those folders, I have this structure.

Screen Shot 2020-11-18 at 7.11.10 AM.png

For every project, I just copy that template then rename the folder. For example, 201118 Project Name. Anything I dump into the Audio or Video folders gets organized into further folders by date. If I need to back up the project, archive it or transfer it to another machine, all I need is that one folder. Everything is there.

In my Stock folder, which would be kind of like your example, every time I go and shoot something, I add a new folder under Video/Raw with the date that it was shot. If there's matching external audio, then it has the same date under the Ambient or Voiceover folders.

In Resolve, I have a project associated to the Stock folder. I import the footage, organize it into bins, keyword it, set metadata, etc. If I need to find something, I can just search the keywords and/or metadata.

It's a much different organization process for me than using Lightroom for stills but the end result is the same.

I have a similar process and template. I do catalogue/key index the clips in Resolve and it stays with the project file. If I need to search and group things, I create smart bins based on metadata - keywords.

Okay but has anyone made a Resolve Project with 10 TB of video?

MarnMxxx wrote:Okay but has anyone made a Resolve Project with 10 TB of video?

Yes. I have done a few that's got 12TB-14TB of source clips, and the timelines are usually around 90 to 113 minutes long. What does that have to do with a Resolve Project? Not sure where the hurdle is.

Ellory Yu wrote:

MarnMxxx wrote:Okay but has anyone made a Resolve Project with 10 TB of video?

Yes. I have done a few that's got 12TB-14TB of source clips, and the timelines are usually around 90 to 113 minutes long. What does that have to do with a Resolve Project? Not sure where the hurdle is.

The hurdle is finding a way to catalogue and keyword search 10TB of raw video. It will take hundreds of hours to add searchable keywords to all of it and I don't want to have to do it more than once. I'm looking for suggestions for the best way to catalogue all these video files and only have to do it once. Then have that catalogue forever. So if I type in "Elk". I want every elk clip I keyworded to show up so I can pick the one to use for any particular project.

So, is the idea of using a Resolve project suggesting that I start one new Resolve project?
Then import all 10TB of raw video and keyword them in this one Resolve project?
What are the chances that future Resolve versions won't be able to read this older project?
I just keep thinking if I had done this back in Final Cut Pro 7. I'd never be able to upgrade to FCPX because FCPX does not read FCP7 projects.

Key wording 10TB of video and have it not future proof is a very daunting thought knowing how much time it's going to take to just do it once.

Maybe this?

https://primestream.com/news/press-rele ... ntegrated/

MarnMxxx wrote:The hurdle is finding a way to catalogue and keyword search 10TB of raw video. It will take hundreds of hours to add searchable keywords to all of it and I don't want to have to do it more than once. I'm looking for suggestions for the best way to catalogue all these video files and only have to do it once. Then have that catalogue forever. So if I type in "Elk". I want every elk clip I keyworded to show up so I can pick the one to use for any particular project.

So this is going to be independent of the NLE you want to use and more at the file system level. You need to look into Digital Asset Management (DAM) or Media Asset Management (MAM) software. How to integrate a DAM or MAM with an NLE is another thing to look for, maybe some sort of DAM or MAM that has support for NLE via say, plug-ins.

Try digiKam. It's free software.

Iñigo Figuero wrote:Try digiKam. It's free software.

This was hopeful however it looks like it's just for images not video. BUMMER!

Jim Simon wrote:Maybe this?

https://primestream.com/news/press-rele ... ntegrated/

This will be too expensive for me, thanks though,

I've been using Adobe Lightroom Classic to import, catalog, tag, and reference both images and video. It has its quirks, but ultimately it's my primary catalog to find media in my library (which is at about 30,000 files.)

All of my media is on an Unraid server and the connection between that and my workstation is 10GB, and I have had minimal issues editing in Davinci due to this. Of course, everything I do is still at 1080p.

The biggest challenge I face is that the keywords that I define in Lightroom Classic do not transfer to Davinci when I ingest the video. But a basic workflow goes like this:

I use Lightroom CC to import the video directly from the camera's memory card. Lightroom Classic files it into a folder structure that's date based by moving the file from the memory card to the server. I immediately tag the files in Lightroom Classic appropriately so they're easy to find later, and I make sure the lightroom database is backed up.

Then, I fire up Davinci Resolve and either create, or open a project that those video files are designed for and I bring them into the Media Pool and update the meta data on each clip.

If the keywords in lightroom could transfer to Davinci, it would save me some time, but that's where I'm at right now.

As I've posted in other topics, the real challenge I have is discarding unwanted footage while preserving the underlying EXIF data on the files...

MarnMxxx wrote:Okay but has anyone made a Resolve Project with 10 TB of video?

Yes. 14 terabytes in fact for a narrative feature film. I struggle with this too at time but I just have a system like the folks before me suggested - a directory structure for each project.

I do admit that not all the files and images in them are usable because there are good takes and many bad takes. So I have started to clean them up, getting rid of he bad takes and keeping only the good takes to save space and also catalog them. That 14Tb of endless files are now more controllable, accessible, and used less storage (<200Gb). I now then copy them onto 256Gb SSD to archive the usable files for the project. 256Gb 2.5” SSD are cheap.

Best one I found is EazyFlixPix, its not free but you can give it a try with trial version.

You can try using Vidine and Vee-Hive software.

christaelrod wrote:You can try using Vidine and Vee-Hive software.

These looked promising too but are Windows only. I use a Mac.

Footage

arretx wrote:If the keywords in lightroom could transfer to Davinci, it would save me some time, but that's where I'm at right now.

As I've posted in other topics, the real challenge I have is discarding unwanted footage while preserving the underlying EXIF data on the files...

Any news on this?
I'm looking for this kind of solution. Lightroom was very good at photo managing.
Now I use Capture One for photo develop but still can be doable to use Lightroom in bundle with Photoshop to catalogue stuff.

The best thing will be to do everything in Resolve itself, I was looking at the possibility to produce small size cache files hoping to speed up the searching tasks.

Any updates?