BMD Camera Shootout

Joe, I'd really love to be there when you do the tests and assist, but we are oceans apart

I wish I was with you on the summer side of the equator.

PS. You made me realize that it has been over 20 years since I was in Australia. About a year for every hour of the flight!

I am in Austria, Europe, not Australia. Last week we had -12°C.
From the country of Mozart and Schwarzenegger

Joe, I'd really love to be there when you do the tests and assist, but we are oceans apart

I wish I was with you on the summer side of the equator.

PS. You made me realize that it has been over 20 years since I was in Australia. About a year for every hour of the flight!

I am in Austria, Europe, not Australia. Last week we had -12°C.
From the country of Mozart and Schwarzenegger

Oh, ha! That's what I get for typing before making coffee.

I am in Austria, Europe, not Australia. Last week we had -12°C.
From the country of Mozart and Schwarzenegger ;-)

;- )

short form would be "austria - no kangaroos" +g*

Joe, I'd really love to be there when you do the tests and assist, but we are oceans apart

I wish I was with you on the summer side of the equator.

PS. You made me realize that it has been over 20 years since I was in Australia. About a year for every hour of the flight!

I am in Austria, Europe, not Australia. Last week we had -12°C.
From the country of Mozart and Schwarzenegger

Rob I'm prepared to sponsor you for Australian Citizenship.

JB

Does he have to get a job first? Maybe JB can speak to Grant and get Rob a job at BMD

I am in Austria, Europe, not Australia.

Hopefully I'll pay better attention to details on Sunday.

I am in Austria, Europe, not Australia. Last week we had -12°C.
From the country of Mozart and Schwarzenegger

Oh, ha! That's what I get for typing before making coffee.

So when they did the entrance for the Eurovision Song Contest did you think someone was typing and making coffee at the time

Rob I'm prepared to sponsor you for Australian Citizenship.
JB

Oh man, you guys are already planning my career path
Any day now I should expect a Blackmagic branded van stopping in front of my house with Captain Hook behind the steering wheel, trying to lure me with a secret new camera into the car, then blindfold and kidnap me to Australia.

Now that my UM4.6k has died and my Pocket 6k Pro is expected to arrive next week will I not be missing anything? I'm toying the idea of getting another Ursa Mini maybe a G1 or another UM4.6k so I could keep all the existing rig and EVF but would the Pocket 6k Pro supercede everything the original UM4.6k can offer, with Braw Gen 5 vs 15 stop cDNG?

I don't know if we'll have an appreciable PQ winner...but I know which one I'd rather lug around. I kid, I'm actually surprised by how compact and comfortable the 12K + Sigma 35 are in hand. Especially the Sigma, I've never used it before and I just assumed it was much larger.

As a pre-test, I'm going to see how much the Sigma shifts color as the iris is stopped down.

Most of the 12k from the hinge of the monitor back is cooling too.

I use this body hand held all the time with primes.

Here’s a clip of me operating with a G2 (same body) on the great. You can see I’m agile. This body is also carrying a 2 stage matte box remote focus and MDR as well as a transmitter and timecode clock.

Still very small.

JB

https://www.flickr.com/gp/johnbrawley/3pqABM

Most of the 12k from the hinge of the monitor back is cooling too.

I use this body hand held all the time with primes.

Here’s a clip of me operating with a G2 (same body) on the great. You can see I’m agile. This body is also carrying a 2 stage matte box remote focus and MDR as well as a transmitter and timecode clock.

Still very small.

JB

https://www.flickr.com/gp/johnbrawley/3pqABM

Yeah, I'm kind of bummed that I won't have time to put the 12K on my shoulder and shoot something fun. I had actors lined-up to run a scene, but as the number of cameras and lenses we are testing has grown, I had to let that go.

It really is "mini" compared to the ENG beasts of my youth.

Hey John,
I could watch those BTS clips all day. Thanks for sharing.

Hey Joe,
Yeah, it really is a very manageable size. I have it balanced with no issue on an RS2 (the whole setup including the ronin is around 9.5 lbs).

Hey John,
I could watch those BTS clips all day. Thanks for sharing.

Hey Mark just for you.

360 camera on top of my G2 on The resident doing a oner.



Password is

Resident

JB

Hey John,
I could watch those BTS clips all day. Thanks for sharing.

Hey Mark just for you.

360 camera on top of my G2 on The resident doing a oner.



Password is

Resident

JB

Now we're talking! thanks

I want to match FOV and DOF between our cameras different sensor sizes as much as possible for the blind test. In order to do so, aperture must be adjusted. Captain Hook raised the concern that changing the iris on the lens could impact color rendition. So, as a pre-test to the test, I tested.

http://humcrush.com/12kshootout/index.html

While this is just one quick test, I'm satisfied that whatever differences might exist between T4 and T11 (the range we'll be playing within to match) will be negligible.

Joe, in this thread from Jan 2017 I posted a test series I did with AWB and different T stops on different lenses.
viewtopic.php?f=2&t=55434

While the color temperature did remain quite stable, the tint shifted a lot.

Joe, in this thread from Jan 2017 I posted a test series I did with AWB and different T stops on different lenses.
viewtopic.php?f=2&t=55434

While the color temperature did remain quite stable, the tint shifted a lot.

I'm not doubting that it happens, but at least with the copy of the Sigma 35mm we're using Sunday the differences are lower than the margin of error. (I planned to run the test 3-5 times to find a reasonable margin of error, but when the I saw how small the difference was, I just called it.)

Out of curiosity, were you changing anything other than iris and shutter angle?

Joe, in this thread from Jan 2017 I posted a test series I did with AWB and different T stops on different lenses.
viewtopic.php?f=2&t=55434

While the color temperature did remain quite stable, the tint shifted a lot.

I'm not doubting that it happens, but at least with the copy of the Sigma 35mm we're using Sunday the differences are lower than the margin of error. (I planned to run the test 3-5 times to find a reasonable margin of error, but when the I saw how small the difference was, I just called it.)

Out of curiosity, were you changing anything other than iris and shutter angle?

I think I did change the focus on the lamps a bit for the widest open aperture. But it seems I didn't note down what I exactly changed and honestly can't remember. But I also did a test in sunlight.

Joe, in this thread from Jan 2017 I posted a test series I did with AWB and different T stops on different lenses.
viewtopic.php?f=2&t=55434

While the color temperature did remain quite stable, the tint shifted a lot.

I'm not doubting that it happens, but at least with the copy of the Sigma 35mm we're using Sunday the differences are lower than the margin of error. (I planned to run the test 3-5 times to find a reasonable margin of error, but when the I saw how small the difference was, I just called it.)

Out of curiosity, were you changing anything other than iris and shutter angle?

Joe, do you then plan to show the results without changing tint in camera when changing aperture? What's the margin of error?

Hey John,
I could watch those BTS clips all day. Thanks for sharing.

Hey Mark just for you.

360 camera on top of my G2 on The resident doing a oner.



Password is

Resident

JB

LOL super cool. A fun way to watch it is to get it so I can see you and the lcd screen. Then I flipped my laptop upside down to watch what you're seeing right side up.

Joe, do you then plan to show the results without changing tint in camera when changing aperture?

Any given camera will only be at one aperture, and will be custom white balanced at that aperture. No other adjustments will be made to color other than that.

What's the margin of error?

On the test above? We don't know, because I only ran it once. It's kind of moot, though, because the difference is so tiny. Take the "difference" image I shared and double the contrast. Triple it. Quadruple it. It's still not meaningful.

On the test above? We don't know, because I only ran it once. It's kind of moot, though, because the difference is so tiny. Take the "difference" image I shared and double the contrast. Triple it. Quadruple it. It's still not meaningful.

Well, it's helpful to know you are going to auto white balance the test scene after you are changing the aperture. I can see a warm / cool difference overall in the two images you have in the large gray patch, it would affect skin tones to my eye if published without the adjustment. It seems in line with Robert's results too that the Sigma gets greener stopped down.

On the test above? We don't know, because I only ran it once. It's kind of moot, though, because the difference is so tiny. Take the "difference" image I shared and double the contrast. Triple it. Quadruple it. It's still not meaningful.

Well, it's helpful to know you are going to auto white balance the test scene after you are changing the aperture. I can see a warm / cool difference overall in the two images you have in the large gray patch, it would affect skin tones to my eye if published without the adjustment. It seems in line with Robert's results too that the Sigma gets greener stopped down.

I think your eyes, or your monitor, are playing tricks on you because one is slightly darker than the other. (It is really common on even very good monitors for the greyscale to alternate between green and magenta casts throughout.)

T9.7 on the right, T1.5 on the left. If you measure them, they are both essentially neutral.

Download the difference file and look at the histogram:

That's about as close to no difference as you're gonna see between two non-synthetic images.

[CaptainHook]

I grabbed the JPGs from your link above and the bigger neutral patch on the left very much so looks visibly different in colour but also measuring with photoshop using a 31x31 average at the same point of both showed the following:




The difference in the red channel is not insignificant to me where would I would consider one of them "neutral" (enough) but the other is not (based purely on the exports) - more importantly I think you're already discovering even with this "simple" pre-test you are finding people questioning your conclusions and results.

As mentioned before, good luck!

[CaptainHook]

By the way, the PNG's on your link are noted as 16bit but appear to be 8bit

Captain Hook, I think something is going awry in the export.

[CaptainHook]

Captain Hook, I think something is going awry in the export.

Tests are hard to perform and to present - so often something goes wrong!
You can see now why I and others are apprehensive about them.

Exporting for the web is causing the issue. I'll have to find away around that if I post any more stills.

Here are the actual values:

Captain Hook, I think something is going awry in the export.

Tests are hard to perform and to present - so often something goes wrong!
You can see now why I and others are apprehensive about them.

Certainly, especially when color management systems are in the mix. But we'll work through it, version by version, kinda like contributing to open source.

[CaptainHook]

You can see how problematic it could be to some though that you presented some files as "evidence" and then when they were scrutinised its now said they don't reflect the "truth" and to trust your numbers without others having the source files. Your png files are noted as 16bit but appear to actually be 8bit. None of this inspires confidence and I'm not saying/implying you're not trustworthy - just this is exactly what was warned could happen and why John etc was questioning how you were going to choose to present things and that giving your editorial opinion will also likely be controversial.

And yes colour management etc complicates things but that is really part of the whole topic.

I won't get involved anymore and leave you guys to it (plus its the weekend here) but I hope its obvious now just how difficult of a task you set out for yourself it is. I hope you get something out of it either way.

The page should now be linking to **hopefully** unmolested 16-bit pngs.

You should be able to download these files and duplicate my results.

You can see how problematic it could be to some though that you presented some files as "evidence" and then when they were scrutinised its now said they don't reflect the "truth" and to trust your numbers without others having the source files. Your png files are noted as 16bit but appear to actually be 8bit. None of this inspires confidence and I'm not saying/implying you're not trustworthy - just this is exactly what was warned could happen and why John etc was questioning how you were going to choose to present things and that giving your editorial opinion will also likely be controversial.

And yes colour management etc complicates things but that is really part of the whole topic.

I won't get involved anymore and leave you guys to it (plus its the weekend here) but I hope its obvious now just how difficult of a task you set out for yourself it is. I hope you get something out of it either way.

Some may think that. And some may see that "evidence" was presented and when peer reviewed, problems were noted, discussed and resolved. Please do challenge my opinions, assumptions and findings...indeed, hopefully every aspect will be challenged, and where flaws are found, if it's not too late, I'll correct them. If it is too late (like if I screw-up something in the actual test) then I'll make note when discovered.

Some may think this is all silly, and some may think it is all bs. Some may find it useful. As they say in the great state of Louisiana, "It's all good."

Joe, you kind of shot yourself in the foot mate.

You aspire to do “conclusive” camera test and that’s a great thing to want to do. When we discussed in an open and collegiate way the many many pitfalls of attempting to do this you were kind of. “I got this...my wife is a researcher”....

You were going to use aperture to vary exposure mismatches between different sensors.

The problem is that you’re already making assumptions that have then been proven wrong in this very thread.

You’ve shown that you weren’t aware that the response can be different depending on aperture.

You tested for it and kind of announced in a way that didn’t seem to leave any wriggle room that you’d tested for it and there was a difference but it was negligible. (Hey you learned something!)

Now you’ve done your tests at fixed apertures.

Then you kind of messed up again because the methodology was wrong, and people could immediately see something that you were dismissing as negligible.

You see where I’m going with this ?

I think you can easily address this.

Stop making claims.

Do your tests.

Put them out there.

Discuss.

Every time you make these kinds of statements that speak with the authority of a test, you devalue your own status and opinion, especially as you seem to have a side hustle making LUTs and speaking with authority on colour correction.

If you want credibility, then you really do need to know what you’re talking about.

I’ve done many many sets of these kinds of tests. For many reasons, often for my own work and knowledge and understanding, and often for others to give feedback.

I’ve screwed it up more times than I got it right.

The very best thing is to share your results, but draw your own conclusions privately first, rather than making sweeping statements. Because I’m pretty sure, like what’s happened here, you’re going to end up de-valuing your credibility.

And like many of us warned you. You really can’t cover all the bases, you will make mistakes and it’s best to be open, share the results and discuss it, rather than make a conclusion and put it out there.

You’ll notice a lot of the tests I make public, I very deliberately do not publish any conclusion. Just the tests and a discussion of the methodology. Sometime I WILL VEAR INTO OPNION, but it’s just subjective a lot of the time.

I say this as someone that has been exactly where you are right now. Don’t make an ass of yourself.

JB

All of us can be guilty of trying hard to defend a position, and using various means at our disposal to support it. Joe can take it to the extreme but I respect that. It's just a way of trying harder than most, or as some may think, harder than he should. But because the test seems intended to support his own premise, that there is nothing particularly special about the 12K images and that cameras can be matched to be virtually indistinguishable is seen differently than an open competition. I have no doubt he can succeed in matching already close cameras in a way that is virtually distinguishable. The keyword is "virtually" as some will be able, some won't and it comes down where you draw the line. But we're here for fun, and learning. At the end of the day, just as at the beginning, the 12K will have all the appeal to me that it ever did, which is substantial. I like the images it makes, I like the frame rates, the mounts, the value. I love my P6K too, it's probably the more appropriate tool for me but it's my right to lust after the 12K and that shall not be infringed!

You aspire to do “conclusive” camera test and that’s a great thing to want to do. When we discussed in an open and collegiate way the many many pitfalls of attempting to do this you were kind of. “I got this...my wife is a researcher”....

I'll make mistakes, and I can't magically avoid the difficulty of this, I was simply stating that I have an awareness of valid vs shoddy methodology.

You were going to use aperture to vary exposure mismatches between different sensors.

The problem is that you’re already making assumptions that have then been proven wrong in this very thread.

You’ve shown that you weren’t aware that the response can be different depending on aperture.

You tested for it and kind of announced in a way that didn’t seem to leave any wriggle room that you’d tested for it and there was a difference but it was negligible. (Hey you learned something!)

1. I had a problem presenting the results.
2. The results do show, well, what they show. The proper files are now available. You can recreate those results.

Now you’ve done your tests at fixed apertures.

No, I haven't. I didn't re-do the test at all, I simply changed to way I saved the files I uploaded to my website. The edison bulbs in the background clearly reveal the differing apertures.

Then you kind of messed up again because the methodology was wrong, and people could immediately see something that you were dismissing as negligible.

Again, no. The exported files had a color shift. (Common problem/mistake.) I later posted files "closer to the iron." that, I think, stay true.

John, I won't address the rest of your comments other than to say:
1. Thank you for your involvement. I've rubbed you the wrong way, that was never my intent.
2. Even if it doesn't appear so, I'm listening, and I actually do admit when I'm proven wrong. (Plenty of record of that on this very forum.) I actually kind of find that fun.
3. The scientific method starts with a hypothesis (or a "claim").
4. I'll be busy prepping and shooting the tests tomorrow, and then it will take some time for me to make-ready for the online free-for-all, so don't expect much more on this topic for several days. That's not me slinking away...just doing the thing.

All of us can be guilty of trying hard to defend a position, and using various means at our disposal to support it. Joe can take it to the extreme but I respect that. It's just a way of trying harder than most, or as some may think, harder than he should. But because the test seems intended to support his own premise, that there is nothing particularly special about the 12K images and that cameras can be matched to be virtually indistinguishable is seen differently than an open competition. I have no doubt he can succeed in matching already close cameras in a way that is virtually distinguishable. The keyword is "virtually" as some will be able, some won't and it comes down where you draw the line. But we're here for fun, and learning. At the end of the day, just as at the beginning, the 12K will have all the appeal to me that it ever did, which is substantial. I like the images it makes, I like the frame rates, the mounts, the value. I love my P6K too, it's probably the more appropriate tool for me but it's my right to lust after the 12K and that shall not be infringed!

Hey Tom,
So I do want to clarify some things...

1. I do not doubt that there are visual differences between the cameras we are going to test. (Which right now looks like: P4K, P6K, UMP12K, GH5, R5, A1.) There are visible differences.

What I doubt is that when presented an unidentified moving image at 4K, people – even highly trained people – will be able to reliably identify any given camera. I doubt this because of previous tests conducted by other people on even more disparate cameras.

2. I will not be changing anything to "match" the cameras other than:
-a. Reduce tell-tale signs of sensor size. (By changing distance to subject and iris.)
-b. Custom white balance to grey card.
-c. Scale to 4K DCI.

3. I genuinely want to learn what advantages to color reproduction the 12K offers, if any. Truly. This isn't a "gotcha" test. I want there to be improvements. I love that BMD pushed sensor tech in the way that they did. What I have said is, to date, I haven't seen a clear advantage with the limited amount of footage I've seen.

Oh, jeez.... Isn't the question whether the 12K sensor does something of interest or value that the standard technology on other BMD cameras doesn't?

Do this in a studio, and odds are you won't see any significant difference even if it's there, in no small part because of the subject matter, which is of no personal significance to anyone and which has no aesthetic value. "Measurable" is not going to save you, because you've already seen that differences you consider insignificant may not be regarded that way by others.

Meanwhile, after all these years, you still can't get anyone to explain in scientific detail why Alexa color is "better" than everything else, other than people like or claim to like it better.

Good luck, but if it were me, I'd just take it out and shoot some s*** and see if I like it better than what I'm used to. That won't leave you with anything you can tell the world, but at least you know or think you know for yourself.

1. I had a problem presenting the results.
2. The results do show, well, what they show. The proper files are now available. You can recreate those results.

1. If you wanted to be scientific you would explain what went wrong and how if you processed and exported with the same settings the two images had a difference people measured

2. Explain what you did to fix it

3. Most importantly make the raw files available so we can recreate your results properly that would be the scientific approach.

If you want some creditability back do the above 3 things.

This isn't a "gotcha" test.

shady.png (196.72 KiB) Viewed 4075 times

This you? Cause that sounds like a gotcha test and hella shady sorry
Are you gonna do some video with just as bad title and description with your conclusions and then if there are mistakes it wont matter cause the video is done and watched by people?
If you want to do open source peer review like you say then that happens BEFORE you publish the test. Give people the source files with no opinion first and do the revisions if needed and post the video later.
Or keep it shady cause "It's all good" like they say in louisiana

1. I had a problem presenting the results.
2. The results do show, well, what they show. The proper files are now available. You can recreate those results.

1. If you wanted to be scientific you would explain what went wrong and how if you processed and exported with the same settings the two images had a difference people measured

2. Explain what you did to fix it

3. Most importantly make the raw files available so we can recreate your results properly that would be the scientific approach.

For example, the in-camera white balance and tint could be published in the image caption if those were the only changes made to images.

And . . checking your own work before telling someone else to get their eyes checked helps.

We've measured a bunch of lenses adjusting aperture and yes it will shift which for my tests at work invalidates any colour comparisons. Just changes in CA can also be problematic here.

Wow, that's fascinating! I can see why CA would change, but I don't have a hypothesis about any other way color could shift. Do you know what's happening on a scientific level?

Do this in a studio, and odds are you won't see any significant difference even if it's there, in no small part because of the subject matter, which is of no personal significance to anyone and which has no aesthetic value. "Measurable" is not going to save you, because you've already seen that differences you consider insignificant may not be regarded that way by others.

What does aesthetic value have to do with it? If a "studio" test doesn't reveal differences:
A. The test wasn't demanding enough (highly possible.)
B. The differences aren't there, or are not noticeable. (also possible.)

2. I will not be changing anything to "match" the cameras other than:
-a. Reduce tell-tale signs of sensor size. (By changing distance to subject and iris.)
-b. Custom white balance to grey card.
-c. Scale to 4K DCI.

Thanks for clarifying that. I don't think it would be cheating to match them because people are interested in knowing how well that can be done too. For example, Matteo Bertoli has been promoting his buttery luts to show how closely he could match P6K to Alexa Mini. Is there $48,000 of difference? Well anytime you make the competition into a value proposition there are going to be means-based differences of opinion. I think it's entertaining and generally enjoy seeing the ox gored.

1. If you wanted to be scientific you would explain what went wrong and how if you processed and exported with the same settings the two images had a difference people measured

I did, but I'll do it again: I originally used Photoshop's "Export for web" feature which reduced the images to 8-bit and shifted colors even though I had the "convert to srgb" option off. I was thinking it terms of posting a reasonably sized "preview" to my website, not sharing a source file, and this lead to confusion. I should have known better, as I would have done and reacted the same way.

2. Explain what you did to fix it

Uploaded and linked to the unmolested 16-bit pngs. Y

3. Most importantly make the raw files available so we can recreate your results properly that would be the scientific approach.

Will do. Remember, this was a quick pre-test test. It's gonna take me some time to not only shoot the real test, but to let the poll run...after that, and only after that...will I share source files.

This isn't a "gotcha" test.

shady.png

This you? Cause that sounds like a gotcha test and hella shady sorry

Shady how? Right there out in the open. I've also shared my assumption/hypothesis/claims right here.

I have an opinion, I have an open bias and I have an agenda. All freely admitted. Some or all of which might change after Sunday.

Are you gonna do some video with just as bad title and description with your conclusions and then if there are mistakes it wont matter cause the video is done and watched by people?
If you want to do open source peer review like you say then that happens BEFORE you publish the test. Give people the source files with no opinion first and do the revisions if needed and post the video later.

I can't give people the source first, because, obviously, that will ruin the blind test. I will endeavor to get feedback as I move through the process though. In doing so, by "showing my ass" I will annoy some and discredit myself to others, but I'll do it anyhow so that when I do publish the blind test, it will be the best that it can be given all things considered.

Thanks for clarifying that. I don't think it would be cheating to match them because people are interested in knowing how well that can be done too. For example, Matteo Bertoli has been promoting his buttery luts to show how closely he could match P6K to Alexa Mini.

it's not that I think matching them is unfair...it just isn't what I want to test. I'm actually trying to give whatever differences there may be a chance to show-up.

Cameras are so close these days, and color matching tools are so good, that it isn't hard to match any two modern cameras. I guess since it is so easy, I might as well also post "matched" files for those that want to see them.

Is there $48,000 of difference? Well anytime you make the competition into a value proposition there are going to be means-based differences of opinion. I think it's entertaining and generally enjoy seeing the ox gored

Indeed. And this is where I have a strong opinion for a given use case. I will try to do a better job differentiating "presenting the fact" from "editorializing" ... but I am gonna editorialize because I have an agenda. A lot of other people have the same agenda, and even those that don't may find some value in the tests.

Now I need to go buy fruit and cloth!

Question for those experienced with the UMP12K
Is there a good reason to conduct the test exposing for ISO 800 vs 400?

The reason I ask is twofold:

1. It will reduce the amount of variability in the test, making it easier to include motion in the tests.
2. From my very, very quick pre-test tests, there's a lot of noise in the shadows at ISO 800 that cleans-up to be more comparable to Pocket4K/6K levels at ISO400.

I did, but I'll do it again: I originally used Photoshop's "Export for web" feature which reduced the images to 8-bit and shifted colors even though I had the "convert to srgb" option off.

If it shifted the colors it should have shifted the same for both pngs, not make them different. Makes no sense.

Remember, this was a quick pre-test test. It's gonna take me some time to not only shoot the real test, but to let the poll run...after that, and only after that...will I share source files.

Why won't you share the sources files for this pre test what are you trying to hide?

I have an opinion, I have an open bias and I have an agenda.

So you admit this isn't a scientific test by any stretch just an attempt for views and internet clout. Waste of time.
Looking forward to your fb posts that start with your above statement yeah right.

I can't give people the source first, because, obviously, that will ruin the blind test

You couldn't even prepare files for your own simple pre-test and want people to trust your blind test give me a break. Then you accused people of needing their eyes checked but it was you who made the mistake. Share the pre-test original files and stop being shady.

What does aesthetic value have to do with it? If a "studio" test doesn't reveal differences:
A. The test wasn't demanding enough (highly possible.)
B. The differences aren't there, or are not noticeable. (also possible.)

Way, way OT I guess, just ignore me, but you're effectively claiming that human perception can be divorced from ... human perception.

To make a layman's criticism of Steve Yedlin's test(s), not yours: of course the same non-dramatic shot of a guy sitting in front of a bookcase is going to look more or less "the same" out of more or less comparable cameras. Discerning distinctions among moving pictures of anything that uninteresting and aesthetically null will exceed anyone's attention span, even a DP's -- and even more so, after the material has been through post meant to render everything the same. So for all practical purposes there's no difference, because nobody sees any. Under other circumstances or with other subjects, that might not be true.

Or take a tortured analogy: you wouldn't compare two loudspeakers by playing tones and trying to decide which tone you liked better or which was more accurate. (You might perform that test in an anechoic chamber and let a computer assess accuracy, but that won't tell won't tell you which sound you prefer.) You'd have to listen to actual music, and maybe various kinds of music.

In a word, can your studio tests actually satisfy their stated purpose? I wonder.

@Steven Abrams

I would be interested in seeing your tests too, but don't say you don't have time for them if you can spend it parsing every word, every phrase in search of code for bias and agenda. He's already owning that.

I would be interested in seeing your tests too, but don't say you don't have time for them if you can spend it parsing every word, every phrase in search of code for bias and agenda. He's already owning that.

I'm not the one making threads and fb posts claiming my "tests" are scientific when they are not and that they would be in any way useful to anyone else thats the whole point people have been making here. Have you not been paying attention?