Jetson Nano Jetbot tutorial Camera problems

I’m following jetbot tutorials and trying to get Object Following example working, but have problems with the camera freezing after starting inference. The camera works when initialized and shows image in the widget, but when I try to start inference with following commands:

execute({'new': camera.value})
camera.unobserve_all()
camera.observe(execute, names='value')

The camera gets stuck, not showing updates in the widget and robot is stuck reacting to that one frame e.g.

  • I am using barrel connection for power with 12V lipo battery and 5V 5A dc-dc converter and running in 10w power mode.
  • I have piCamera V2
  • wifi dongle and arduino connected to usb ports
  • Tried to 5W mode and same problem

Any tips for troubleshooting?

EDIT:
Okay so I tested without activating the robot motors same time and it works: so my theory is the high start current for motors makes voltage drop to nano which compromises the camera power supply and makes it stuck which makes the execute function not work properly. I dont have reference motor driver and motors, Im using
DFRbot 12V 122rpm dc gear motors and motor driver that works up to 15A. As a bonus question, any electric engineers can recommend me power filtering options, what size capasitor i should put between the dc-dc 5v converter output and nano input?

EDIT2:
Motors were not the problem, camera still freezes while motors are disabled.

Hi, i think you’d better to measure the specific value of voltage drop and current draw when power on system, and then calculate capacitor accordingly. Or maybe you can delay the motors startup timing to avoid the peak power consumption?

I’m still having this problem and I guess it was not related to motors drawing current necessary. Even when motors are disabled the camera freezes after this line:

update({'new': camera.value})  # we call the function once to intialize

I’m having the same problem. I have built 9 jetbots and they all experience frozen frames in the object_following example at least 50% of the time. It seems unlikely that a bad camera or WiFi module could be to blame with every bot having the same problem. We just keep rebooting/restarting until it works and then hope it keeps working long enough to do our application (it sometimes freezes in the middle of a run). We are running the original distribution of the OS available in July of 2019. Are there updates that might fix this? The vision system is basically unusable in this state (and we chose the Jetson platform specifically for its vision system).

Hi dmackay93re1,

Thanks for sharing.

I believe the issue is power stability given the heavy workload in the object following example.

We’ve noticed before that changing the data that is fed to each neural network to fp16 precision has helped improve the stability in some instances.

This is straightforward for the collision avoidance model, since it is in PyTorch. However, the object following model has already been optimized to have fp32 input/output bindings, so it will require more consideration.

Hopefully we can resolve this issue soon, but I do not have a timeline.

In the meantime, it may be possible to reduce the power consumption over time by lowering the frame-rate of the camera. Have you tried this with any success?

Please let me know if you have any questions.

Best,
John

We did play with the framerate a bit but didn’t notice a marked improvement. The default seems to be 20 fps and we’ve tried 8 and 4 fps. Even at 4 fps, it still freezes. But it’s nice to know there may be a solution down the road. I’ll be doing this project again next semester with my students so a more robust fix would be great.

We want our jetbot to recognize a specific set of objects of our choosing (about 20 in all) and plan to train a new model from scratch. Could we address the fp16/fp32 issue at that time? How exactly should we do the training to minimize the load?

Hi John,
I’ve been reading that you can supply more power by utilizing the Barrel Jack Connector (4A @ 5V) instead of the 2A USB port. Do you think that would possibly solve the power stability issue here?

@dmackay93re1
I have used many different power supply options e.g. LiPo battery with dc-dc converter up to 5v 5A through barrel jack with no help.
Not sure if this board is usable in real life without some modifications.

Can anyone recommend similar boards like Jetson Nano but would be more suitable to use with Deep Learning inference and robotics?

Did you try attaching a fan to the heat sink? With that much power the system may overheat.

Didn’t try but the temps are getting not too high, nor is the nano pulling that much power

Hi @jaybdub,
We have tried the following things to increase power/decrease load:

  1. Splice 2 USB cables each coming from two USB outlets of a powerbank and then input into the barrel plug on the nano (did not help)
  2. Sprayed canned air on the heat sink to cool down the processor (the temperature of the CPU and GPU were 57C before cooling and 37C after but no effect on the camera problem).
  3. Decrease the frame rate to 8 and 4 (may have helped on one of the bots but not a general solution).
  4. We are now trying a fan on the heatsink but the fan doesn’t spin up on powerup. Is there a setting to enable the fan header?

We will be trying to create fp16 models with our own objects for the object following case. Can you direct me to any examples where these models were created?
Don

I fixed this problem by initializing a small motion of the motors before going to the update() function initialization. This should solve the sudden current peak draw issue at the beginning of update function, when ML and motor both start for the first time. The fault seems to be at the motor driver board.

Try this

from jetbot import Robot
import time
robot = Robot()
robot.left(speed=0.3)
time.sleep(0.2)
robot.stop()