Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

The place for questions about shooting with Blackmagic Cameras.
  • Author
  • Message
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostSun Oct 02, 2022 11:15 pm

Hello everyone.

I want to share my first successes with using an ESP32 Microcontroller to integrate the Tilta Nucleus-Nano Wheel (with no motors) and my BMPCC6kPro.

After spending a couple of days deciphering the messages coming from the Nucleus-Nano Wheel via Bluetooth LE, now the system is extremely solid and precisely reproducible, smooth and flawless. I look forward to sharing a couple of test videos with all of you shortly.
Attachments
2022-10-02 22.25.50.jpg
2022-10-02 22.25.50.jpg (609.42 KiB) Viewed 11901 times
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostTue Oct 04, 2022 12:52 am

This is a video demonstrating how the Tilta Nucleus-Nano Wheel controls the ESP32 microcontroller. Since the Nucleus-Nano sends its current absolute position, it was pretty straightforward to normalize that value to control the focus of my BMPCC6kPro.

The little label that reports the lens distance displays the value the camera notifies on the control message 12.12 through the characteristic with UUID B864E140-76A0-416A-BF30-5876504537D9. That detail, together with the running timecode updated by the camera itself, proves that the ESP32 microcontroller is simultaneously connected to both the camera and the Nucleus-Nano and that the data coming from the wheel is successfully sent to the camera.

In my next video, I will show how accurate the focus control is: very smooth with slow movements, very reactive with quick ones.

Offline
User avatar

Xtreemtec

  • Posts: 5387
  • Joined: Wed Jan 02, 2013 11:48 am
  • Location: The Netherlands

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostTue Oct 04, 2022 9:59 am

Do these nano wheels also be able to control the Tilta Nucleus M motors? Or are they only able to control the Nano motor?

Just wondering because i have a different project were i want to control the Nucleus M motors.. :)
Daniel Wittenaar .:: Xtreemtec Media Productions ::. -= www.xtreemtec.nl =-
4K OBV Trailer, ATEM TVS HD, 4M/E Broadcast Studio 4K, Constelation 8K, Hyperdeck Studio 12G, Ursa Broadcast 4K, 4K fiber converters with Sony Control
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostTue Oct 04, 2022 11:32 am

I am not able to confirm that information since my project is intentionally motorless (either N- or M-): the purpose of my test is to control the focus of the lens thanks to the lens's internal motor (AF) through the Camera Control via Bluetooth.

Tilta doesn't seem to have any documentation on its website confirming or disproving your question. Nevertheless, I found a merchant selling a package of one Nucleus-Nano Wheel + one Nucleus-N Motor + one Nucleus-M Motor, claiming the Nucleus-Nano wheel can be paired with the M-motor as well.
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostSun Oct 09, 2022 4:35 pm

I finally had a chance to make a very basic focus test video.
If the camera and the Tilta Nucleus-Nano Wheel are active and with Bluetooth enabled, resetting the TTGO lets it connect to both devices. Once connected, the TTGO reads the running timecode from the camera and transmits the absolute position of the wheel as the absolute focus through the camera control protocol.

The Nucleus-Nano hand unit uses a range from 0 to 9,999 (00 00 to 27 0F) for the absolute position of the wheel, while the camera control protocol expects the absolute focus position to be in the range from 0 to 2048 (00 00 to 08 00). The TTGO also converts the values appropriately.

Offline

Domingo Olmo Martin

  • Posts: 69
  • Joined: Mon Aug 27, 2012 3:09 pm

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostMon Oct 10, 2022 6:07 am

Hello,
Have a look at this site, I think you might be interested.
https://github.com/err4o4/tilta-nucleus
Domingo Olmo Martin
Author - Director
Co-founder of Studio Barsoom
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostMon Oct 10, 2022 10:10 am

Hi, Domingo,

I had already seen that GitHub repository, which doesn't seem to focus on a Bluetooth connection like the one I am describing here. I am not sure whether Tilta has changed its communication protocol since err4o4 first attempted to reverse-engineer the Nucleus-Nano, a couple of years ago, or the commands the Nucleus-Nano transmits via a wired connection are structured differently than the ones it sends via Bluetooth.

In my own Bluetooth test, the data structure is different compared to the one err4o4 describes in their repository. I was able to determine that the commands are sent as 6-byte packets, with the following structure: NN MM DD DD DD SS where NN MM are two bytes determining the command, DD DD DD is the value being sent as a 3-byte big-endian and SS is the checksum of the 5 preceding bytes.

For example, the record button sends the following commands:
Code: Select all
C9 00 00 00 00 37   (record off)
C9 00 00 00 01 36   (record on)

while the opposite ends of the wheel position correspond to:
Code: Select all
01 05 00 00 00 FA     (focus near)
01 05 00 27 0F C4     (focus far)


The last byte of each packet is always the checksum of the 5 preceding bytes. That is what conclusively confirms that we are dealing with 6-byte long packets, even when the data is transmitted as a train of 20-byte long data (the maximum data packet BLE allows) because fast movements of the wheel produce a flurry of packets being sent. In that case, we are receiving ten 6-byte packets for every three 20-byte packets. Therefore, the checksum byte is what guarantees we are reading the data correctly.

With the previous assumptions, the ESP32 microcontroller I programmed to read the commands from the Bluetooth connection, translate them according to the Blackmagic Camera Control protocol and send them to the camera works flawlessly as one can see in the latest video I posted.

I cannot speak for a wired connection as my main goal was being able to control the BMPCC from the Nucleus via Bluetooth. In my case, the record/focus commands definitely follow the structure I described above.
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostSat Oct 15, 2022 9:31 pm

I further improved the logic governing the ESP32 microcontroller.
The major improvement is that the Bluetooth connections are now automatically re-established in case one of the two devices (the camera and the Nucleus-Nano wheel) gets disconnected. This allows operating the camera without worrying too much about the microcontroller.

The new implementation also includes:

  • two color-coded Bluetooth icons identifying the camera and the wheel respectively (white=waiting for first connection, blue=connected, red=disconnected/reconnecting)
  • auto shutdown if one of the two devices gets disconnected and doesn't reconnect in 30 seconds
  • battery level metering
  • a method using the two buttons of the TTGO to input the pin code when the camera is connected for the first time (or after the previously connected device are cleared/reset)

05.jpg
The ESP32 OLED display when both devices are connected
05.jpg (334.64 KiB) Viewed 11182 times


I have already run a shooting session with this new system and everything was extremely smooth with the focus pulling from the Nucleus-Nano wheel.

BMPCC6kPro+ScorpPro+Nucleus-N-Wheel.jpg
My gimbal system: BMPCC6kPro, Sigma 18-35, Nucleus-N-Wheel, custom ESP32 microcontroller all assembled on a FeiyuTech Scorp-Pro
BMPCC6kPro+ScorpPro+Nucleus-N-Wheel.jpg (667.89 KiB) Viewed 11182 times
Offline

mohd2244

  • Posts: 5
  • Joined: Wed Oct 12, 2022 5:57 am
  • Real Name: Mohd Romeli

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostTue Oct 18, 2022 11:25 am

Any chance this can be integrated with any lidar unit to aid auto-focus ?
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostWed Oct 19, 2022 9:31 pm

Technically speaking, integrating this ESP32 microcontroller with a LiDAR sensor would be pretty straightforward. The real problem would be identifying the correct strategy and calibration to make the data coming from the LiDAR sensor meaningful from a cinematic point of view.
Offline
User avatar

LuigiPulcini

  • Posts: 10
  • Joined: Sun Jul 24, 2022 12:59 pm
  • Real Name: Luigi Pulcini

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostWed Oct 19, 2022 9:52 pm

I run into some strange electrical behaviors of the Tilta Nucleus-Nano Wheel and was very disappointed to learn that Tilta has a disclaimer about the Micro USB port integrated with the wheel being only dedicated to firmware updates but unable to power the unit. This FAQ clarifies that.

My final configuration has the TTGO powered up through its USB-C port and the Nucleus-Nano connected to the 3.3V pin of the TTGO through a custom cable attached directly to the power pins of the wheel.

Nucleus-Nano-Wheel-power.jpg
The Tilta Nucleus-Nano Wheel powered from the 3.3V pin of the TTGO
Nucleus-Nano-Wheel-power.jpg (692.63 KiB) Viewed 10941 times


What makes this discovery even more disappointing is that the TTGO microcontroller costs around $15 and integrates a 5V to 3.3V step-down converter in addition to a 3.7V Li-Ion battery charger that can simultaneously charge the battery while powering up the microcontroller (and now even the Nucleus-Nano Wheel!). All that is missing in the Tilta Wheel, which sells for $150... an inexplicable engineering decision! I completely disassembled the Nucleus-Nano Wheel and could see there would have been plenty of space inside to integrate a $2 TP4056 circuit that would take care of charging the battery while powering up the wheel through the integrated Micro USB port.
Offline

ikester

  • Posts: 46
  • Joined: Sun Sep 03, 2017 7:26 pm
  • Real Name: Ike Arias

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostThu Dec 22, 2022 5:50 am

Luigi,

This is a very cool demonstration! How precise/repeatable are the focus positions on the Sigma lens? Can you consistently hit focus targets over several cycles?

Have you considered open sourcing your work to your GitHub?

I'm actually interested in the other side of the equation :) Controlling the Nucleus Nano motor programmatically, so that it can be driven from an "intelligent" object tracking system, like an OpenCV-based face recognition with distance estimation running on a mobile phone or small SBC and a webcam. Or maybe combined with Lidar.

I would love to be able to use an existing, commercial follow focus motor like the Nucleus, if I can figure out how to control it. Hopefully it's as straight forward as sending those 6-byte codes with the encoded distance over bluetooth.
Offline
User avatar

paffogorriones

  • Posts: 7
  • Joined: Sun Jul 03, 2022 10:55 am
  • Real Name: gabriele Passeri

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostFri Feb 03, 2023 2:32 pm

Hello Luigi,
My Name is Gabriele,

I came across your post while I was starting to research how to achieve what you have already accomplished - More specifically my goal is integrating a DJI Focus Lidar Sensor and calibrating the distance readings with lens positions to achieve smooth focus changes.

I was wondering if you would be interested in collaborating with me on this project. I have a DJI Focus Lidar Sensor and an ESP32 microcontroller and I am willing to perform the necessary testing and experimentation to understand the sensor's outputs. Together, we could figure out how to translate the sensor readings into smooth focus changes.

Let me know if this is something you would be interested in.

Alo if anyone else is interested in the project im keen to make everything transparent and open source :)
Offline

jonglass

  • Posts: 1
  • Joined: Mon Oct 16, 2023 11:31 pm
  • Real Name: Jonathan Glass

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostMon Oct 16, 2023 11:40 pm

Hi Luigi

It would be fantastic if you could open-source your code for this, I'm sure there are many people (myself included) who would find it incredibly useful!

Many thanks
Offline

DeepSOIC

  • Posts: 4
  • Joined: Sat May 25, 2019 3:00 pm
  • Real Name: Viktor Titov

Re: Focus control: ESP32 + Nucleus-Nano Wheel (no motors)

PostTue Feb 13, 2024 4:42 pm

I have managed to control a nucleus nano 2 motor.
See the firmware here: https://github.com/strawlab/tilta-n2-control

Return to Cinematography

Who is online

Users browsing this forum: No registered users and 45 guests