Jump to: Board index » General » Fusion

Fusion Spherical Camera Rendering doesn´t line up

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline
User avatar

dr_roberto

  • Posts: 2
  • Joined: Mon Jan 22, 2018 6:54 pm
  • Real Name: Robert Wegemund

Fusion Spherical Camera Rendering doesn´t line up

PostMon Jan 22, 2018 7:31 pm

Hello,

I am using Maya/Vray and Fusion to create a stereo panoramic(latlong) video.

as a test i am using a geosphere at 0,0,0 and a camera at 0,0,0.

1st problem: when i render the objects i use in 3d(maya) with the fusion spherical camera(latlong) and put both renderings on top of each other they don´t line up. for test purposes everything is just monoscopic. there is a slight offset (more than one pixel) towards the top and the bottom and a little bit (a pixel) everywhere (lets say this due to different AA algos... doesn´t matter for now). i marked the differences in red using a difference keyer

Image

2nd: when i turn on stereo on the SPH-cam in fusion, set it to parallel and set the eye separation to 6.5 units (i assume its fusion world units) it get even worse and i get something that looks like fusion would render interally as a cubic pano a then somehow converting it to latlong by itself but offsetting the texture on the different sides of the cube its using.
again i marked the differences in red using a difference keyer

Image

long story short: did anyone ever used the spherical cam stereo render in fusion with 3d package rendered stereo panoramic images and had the same issues?

please help, i have no idea where to find the magic button that will make it work. :D
a two SPH camera setup manually offset to -3.25 on X on the left cam and +3.25 on x on the right cam is unfortunatly not doing the trick either. i am clueless.

thank you
best
Robert
Offline
User avatar

Chad Capeland

  • Posts: 3017
  • Joined: Mon Nov 10, 2014 9:40 pm

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 6:36 am

Expected. There's no "correct" way to render stereoscopic spherical projections.
Chad Capeland
Indicated, LLC
www.floweffects.com
Offline
User avatar

Andrew Hazelden

  • Posts: 536
  • Joined: Sat Dec 06, 2014 12:10 pm
  • Location: West Dover, Nova Scotia, Canada

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 7:08 am

dr_roberto wrote:long story short: did anyone ever used the spherical cam stereo render in fusion with 3d package rendered stereo panoramic images and had the same issues?

please help, i have no idea where to find the magic button that will make it work. :D


Hi. Every single renderer uses their own custom math for the 360° lens shader implementation for things like stereo convergence, neck offsets, stereo pole merging, etc.. This is why Chad said there was no *one* singular mathematically exact approach from the process when you bring stereo into LatLong rendering.

The magic button you want would require you to have the same rendering engine + stereo 360° lens shaders available in both Maya + Fusion.

At this point in time there are no 3rd party renderer integrations under development for Fusion so you won't be able to get what you are after easily. In the distant past (Fusion 6.x) there were experimental Fusion Renderman and 3Delight renderer integrations but no one has taken that development challenge on in a long time.

Your next best approach would be to use a homebrew multi-camera in Maya and re-create the effect of a 360° stereo video camera rig in CG renderings. About 6 cameras are needed at minimum, more if you go for a circular ring shaped rig layout similar to the Google Jump/ Yi Halo physical camera rigs.

Then you would export the Maya camera transform data to Fusion and re-make the rig settings in Fusion by hand and match the scene scale. You'd have to get the camera film back and lens setting matched between Maya and Fusion.

You would then need to render out the two sets of multi-camera images (from Maya and Fusion), and run them through the same 360° stereo video stitching template to get an identically processed and stitched stereo 360° output.

This is slow and painful to do and would likely still have stitching artifacts in your output - especially in the zenith and nadir pole regions that would need to be manually cleaned up.
Last edited by Andrew Hazelden on Tue Jan 23, 2018 12:51 pm, edited 1 time in total.
Mac Studio M2 Ultra / Threadripper 3990X | Fusion Studio 18.6.4 | Kartaverse 6
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 9:27 am

Chad Capeland wrote:Expected. There's no "correct" way to render stereoscopic spherical projections.

Yes there is. Pixel-wide slicing with stereo rig rotating on the point between the cameras. This gives the best possible solution for 360 panoramic stereo with minimal artifacts. There are technicalities involved, but overall it is the correct way compared to other ways (cubic renders stitched, wide slices or whatnot).
I do stuff.
Offline
User avatar

Andrew Hazelden

  • Posts: 536
  • Joined: Sat Dec 06, 2014 12:10 pm
  • Location: West Dover, Nova Scotia, Canada

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 11:59 am

Hendrik Proosa wrote:
Chad Capeland wrote:Expected. There's no "correct" way to render stereoscopic spherical projections.


Yes there is. Pixel-wide slicing with stereo rig rotating on the point between the cameras. This gives the best possible solution for 360 panoramic stereo with minimal artifacts. There are technicalities involved, but overall it is the correct way compared to other ways (cubic renders stitched, wide slices or whatnot).


Hi Hendrik.

Pixel wide slicing is what a raytraced omni-directional stereo lens shader does but with extra refinements added on top vs doing a brute force approach of rendering 3840 vertical strips as 3840 separate video frames and then merging them together manually for each sequence of a video for the left and right eyes...

Chad has used Domemaster3D and other lens shaders on commercial projects so he knows how to do panoramic stereo rendering.

I have programmed 360/VR stereo lens shaders in the past for fulldome and LatLong stereo imagery that work in mental ray, V-Ray, and Arnold so I have also done that task as a developer. ;)

The original poster was asking for a 100% consistent pixel accurate alignment between a V-Ray for Maya, and Fusion 360/VR Stereo rendering. He even did frame differencing comparisons. That is very thorough in what he wants to see as the end output matching quality.

If you don't have both packages using the same exact stereo rendering approaches internally you won't get the same results. The formulas that are used for 2D mono LatLong output will be able to give you a pixel accurate alignment.

For 360/VR LatLong stereo output it is completely arbitrary how each renderer's built-in omni-directional stereo lens shader computes the stereo offsets and math between the left and right eye views when you want to layer on extra levels of complexity like zero parallax adjustments, and pole fading.

IMHO pixel-wide slicing by itself without stereo fadeout pole corrections applied from an actual compiled lens shader won't give you a 100% satisfactory result in the zenith and nadir regions as you will get a swirl like artifact that shows up on textured surfaces like a carpet in an interior scene. :)
Mac Studio M2 Ultra / Threadripper 3990X | Fusion Studio 18.6.4 | Kartaverse 6
Offline
User avatar

dr_roberto

  • Posts: 2
  • Joined: Mon Jan 22, 2018 6:54 pm
  • Real Name: Robert Wegemund

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 12:50 pm

dear chad, andrew and henrik,

thank you for your fast reply and the detailed explanation why its like this (andrew).

For now i ll stick with maya/vray and keep using fusion for comping. everything has been already rendered in maya and i want to update elements using fusion, thats why it needs to match precisely. if its foreground and background separately it wouldnt matter if its a little off.

thank you again andrew for the in depth view on things!

best regards!
robert
Last edited by dr_roberto on Tue Jan 23, 2018 7:46 pm, edited 1 time in total.
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 1:21 pm

I understand what OP asked and I also understand why different results occur. But that does not mean that every solution out there is as good as any other. The wild stepping in OP's second difference image isn't with any stretch of imagination just as correct as result from slicing method, with pole corrections applied or not. The fact that different softwares implement the technicalities differently does not mean that the basic idea is just one of many equally good ones.

If there is some better way to render stereo panoramas than slicing, I'd be very interested to know what it is. And by slicing I don't mean rendering a bunch of actual one pixel wide slices to be merged later but the idea of changing the camera transform with enough granularity to hide the merging errors that occur with wider slices.
I do stuff.
Offline
User avatar

Chad Capeland

  • Posts: 3017
  • Joined: Mon Nov 10, 2014 9:40 pm

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 1:48 pm

Hendrik Proosa wrote:I understand what OP asked and I also understand why different results occur. But that does not mean that every solution out there is as good as any other. The wild stepping in OP's second difference image isn't with any stretch of imagination just as correct as result from slicing method, with pole corrections applied or not. The fact that different softwares implement the technicalities differently does not mean that the basic idea is just one of many equally good ones.

If there is some better way to render stereo panoramas than slicing, I'd be very interested to know what it is. And by slicing I don't mean rendering a bunch of actual one pixel wide slices to be merged later but the idea of changing the camera transform with enough granularity to hide the merging errors that occur with wider slices.


It doesn't matter which solution is worse, the problem is they aren't the same. So you can't render something in software A and composite it in software B if you need to create a similar projection.

Spherical stereoscopic projections don't work in a general sense. There aren't any "good" ones, just unequally bad ones. The issue isn't merging errors, it's that what works for stereo when you are facing north and looking at the nadir is backwards when you're facing south. It's like a cowlick, but with disparity vectors.

Some solutions that do work are view dependent projections, like from textured 3D geometry or warp maps, or from light fields. The first two don't handle reflections and transparency well, but they're the easiest to use across multiple applications.
Chad Capeland
Indicated, LLC
www.floweffects.com
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 3:29 pm

Stating the known problems of stereo panoramas does not answer my question of whether there is a better way of rendering them than slicing. Because if there is, I'd like to know what it is. Realtime rendering or light fields are of zero help when one needs a pre-rendered stereo panorama in the format OP is showing.

The best bet for future is to go with the best available method and standardize it either formally or informally. It is the only chance to ever get to the point where one can render it from one software and match the other. And I don't think anyone is going to jump onto the wagon that produces results visible on ops second image.
I do stuff.
Offline
User avatar

Chad Capeland

  • Posts: 3017
  • Joined: Mon Nov 10, 2014 9:40 pm

Re: Fusion Spherical Camera Rendering doesn´t line up

PostTue Jan 23, 2018 4:07 pm

Hendrik Proosa wrote:Stating the known problems of stereo panoramas does not answer my question of whether there is a better way of rendering them than slicing. Because if there is, I'd like to know what it is. Realtime rendering or light fields are of zero help when one needs a pre-rendered stereo panorama in the format OP is showing.


Pick one?

There's no right answer here. You can either not do stereo, have bad stereo, or have dynamic view dependent projection. That's your choices. It's not a standardization issue, it's a problem with the math not working right. It's a cowlick that doesn't go away just because you want to standardize.
Chad Capeland
Indicated, LLC
www.floweffects.com
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Fusion Spherical Camera Rendering doesn´t line up

PostWed Jan 24, 2018 2:05 pm

I get your point. But in practical sense, if one needs a pre-rendered lat-long (or some other layout) stereo panorama for simple stereo rendering, something must be chosen in the end and some ways give better results than others. Just like pulling a matte from chroma keying is a hack, it still is possible to get reasonable results from this method. Telling client that this thing sucks and no, you won't get what others are getting doesn't fly very far. Its the same with non-panoramic stereo footage. It won't work when you flip the glasses, it won't work when you roll your head, you won't start seeing behind the corners when you move your head. But still it is used, used successfully, and it is possible to say that this is ok and that is not.

So, getting back to the spherical stereoscopic rendering, what I would define as a correct solution is a rendering where:
- parallax induced errors are minimized (disjoints/stepping where elements move relative to each other due to camera position difference);
- orientation induced errors are minimized (we can't prevent users looking upside down, but we can make sure that looking up or down will not produce flipped stereo effect by fading the stereo effect away towards the poles);
- there are no visual staircasing/distjoint merges;
- single view (left or right eye) will work visually (technically it can't be equal) as a 360 panoramic monoscopic image.
And to my knowledge, thin slicing with pole merging is the only solution that gives acceptable results.
I do stuff.
Offline
User avatar

Chad Capeland

  • Posts: 3017
  • Joined: Mon Nov 10, 2014 9:40 pm

Re: Fusion Spherical Camera Rendering doesn´t line up

PostWed Jan 24, 2018 4:58 pm

But outside of having user-defined lens shaders in Fusion (which would break a lot of stuff), there's no way to match the lens from another application, which is the issue here. There's no way to make valid stereo at the poles, so each attempt at doing so is just making different compromises. You won't be able to render two images in two different applications and composite them together unless you have explicit control over the lens shaders used (and almost no one is giving that out).

Your solution involves defining your own implicit or explicit function and entering that as a parameter for all of the applications you use to make the image. It's a perfectly fine idea, but it's not really happening outside of customized pipelines. I don't think BMD is interested in adding a button for every available lens function. Just not part of their philosophy.
Chad Capeland
Indicated, LLC
www.floweffects.com
Offline

Josef Heks

  • Posts: 6
  • Joined: Fri Aug 25, 2017 2:04 am

Re: Fusion Spherical Camera Rendering doesn´t line up

PostSun Feb 11, 2018 4:09 am

Are you guys also finding that the stereo mode of Fusions spherical camera is broken? I get broken stereo in each eye where I can see an invisible outline of a cubemap - as though the camera is not actually implementing any sophisticated technique to calculate stereo in 360 at all, just creating a stereo image for all 6 directions of a cube (comparable to a physical rig like this: Image).

Are you guys also experiencing this?

(I've posted a more complete individual thread about this issue, waiting for the moderators to let it though.)
Offline
User avatar

Chad Capeland

  • Posts: 3017
  • Joined: Mon Nov 10, 2014 9:40 pm

Re: Fusion Spherical Camera Rendering doesn´t line up

PostMon Feb 12, 2018 10:16 pm

That's exactly what's happening. Don't use the built in stereoscopic camera, either use mono or build your own stereo rig.
Chad Capeland
Indicated, LLC
www.floweffects.com
Offline

Josef Heks

  • Posts: 6
  • Joined: Fri Aug 25, 2017 2:04 am

Re: Fusion Spherical Camera Rendering doesn´t line up

PostMon Feb 12, 2018 11:03 pm

Have you built your own stereo rig before Chad?
I'm assuming a slit-scan approach?
Curious to know any details if you'd care to share.
Offline

paulwolf

  • Posts: 8
  • Joined: Wed Feb 11, 2015 9:51 pm

Re: Fusion Spherical Camera Rendering doesn´t line up

PostMon Apr 30, 2018 8:35 am

I'm experiencing the same problem with eyes flipping in the "back" of the spherical camera. I have used a 6 camera rig in another software (Flame) but there is a problem in stereo because the "seams" between the lenses don't get "stitched" - optical flow style like in mistikaVR. So its no good.

What you are saying is that the spherical camera in FUSION is just 6 cameras rig, no post-STITCHING, to deal with the seams between the cameras?

That would be very disappointing. and render the camera useless for Stereo VR.

:cry:

BTW in nuke there is a physical spherical camera...
Offline

Syro74

  • Posts: 5
  • Joined: Fri May 10, 2019 4:12 pm
  • Real Name: Sylvain Robert

Re: Fusion Spherical Camera Rendering doesn´t line up

PostWed May 15, 2019 7:03 pm

And that BlackMagic from this fake advertising? I still have the same problem with the Fusion Studio 16 version. No updates and I paid the pack due to this feature of 360VR and I can not use it. I only have a month left to find a solution to my VR rendering for my client, mounted entirely on Fusion Studio 9... :oops: :x :cry:

Return to Fusion

Who is online

Users browsing this forum: No registered users and 43 guests