Low latency 1080p on HDMI-USB 3.0

Hello I plan to have live processing of image, can you give me any hints what problems to avoid and how to have best latency?

From HD (1080p30) camera outputing to HDMI, through
Intensity Shuttle for USB 3.0
connected to a PC.

It would be best if this can work both with linux and windows OS on the PC. What kind latency should I expect?

I cant be specific about your setup but I can share my experiences. I am a mac user and I use the thunderbolt capture devices.

There are many things that will slow you down, it is best to look carefully at your entire chain as the whole delay may just be an accumulation of a lot of small acceptable delays.

First, if you are using a camera check the output delay as many non professional (and some professional) cameras have a delay on the output. You can do this with 2 monitors and 2 cameras- play timecode on one monitor and film that monitor with the camera you are testing, feed the output from that camera into a second monitor that is next to the first and then record both monitors with the second camera. Compare the timecode readings and see the difference. Make sure you use the same connection type as you would in capturing (use HDMI out of the camera if you are going to capture HDMI etc).

You can do a similar test with your output chain, play a timecode film on the computer screen and through the output system at the same time and compare the timecodes. It is important to make sure you have the real and complete output chain you would use- same make and model of display or projector, same cables and any other things in the chain- incomplete tests are useless.

I found I had very low latency input HDMI into the blackmagic software- you can test the input latency of the card using one computer to play the timecode movie and another to capture it and display it live. Here it can get a little sticky. Different capture libraries (quicktime vs qtkit vs direct show vs gstreamer etc) deal with the video differently, also your capture codec can play into this. Test the latency of the capture proces you are using with the same method.

Last check the latency of your image processing.

I found using a display port edition matrox triple head to go and large size beamers (2k native) gave me a very large delay (about 3 frames extra).

If you are writing the software yourself make sure it is done in the most efficient way for your machine. I found the blackmagic capture software had a lower latency than quicktime (which is slower than the current qtkit library) so if you can build from the SDK you could save a bit of delay.

If you are super careful with your chain you can get low latency input, but to get it really low you will probably have to write the software yourself and do it well (not to mention having a very powerful computer- examine the motherboard architecture and check which has the most efficient bus method- where are the USB ports, - on board or from a pcie card?).

Every little bit of time will add up for you in this scenario, it is not as simple as getting a good capture card, you will have to take care of every detail to really get the delay down.

Here is what we could get away with, 5 frames in to out with processing - full frame open cv live tracking of 600 points. Qtkit capture from SDI OSX 10.8.2. Almost a 3rd of the delay comes from our camera.

Best of luck- if you do do these tests let us know the results, it is a good resource for others.

Cheers

Fred