Jetson orin 64gb or rtx 3090 24gb

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Hello from Ukraine!

I have a 7-inch quadcopter. I want to install Jetson ORIN 64GB on it to work with neural networks. I will be working with ros2, gazebo, and rviz.

I need to train neural networks, but I only have a MacBook Pro M1 Pro 16GB available. Gazebo does not start in a virtual machine on it, and I think the training speed will be too slow. Therefore, the task is to choose a computer for training and gazebo+rviz simulations. There are 2 options:

  1. A desktop computer with an RTX 3090 24GB and, probably, 64 or 128GB of RAM.
  2. Jetson ORIN 64GB

So, the question is. Can I just buy Jetson ORIN 64 and train my NNs on it? How much slower will it be, how practical is it? Is orin not a very good option for training and it is better to choose a desktop PC? Because if you buy orin, then you can install it on a quadcopter, thereby saving money on a desktop PC.

2

You will need an Ubuntu 22.04 64 bit x86-64/amd64 PC anyways just for the initial flashing of any Jetson module. No virtual machine. Apple Silicon machines won’t work. Without a native Ubuntu Linux PC hardware environment Jetson modules are useless. The required PC hardware can be on the low end, though.

For training the AI specialists in my company prefer AMD Threadrippers with one or more 3090.

2 Likes
3

Welcome!

Some information might be useful to you (and this is just another way of agreeing with @fchkjwlsq)


Jetsons have an integrated GPU (iGPU), and this means they are wired directly to the memory controller. A desktop PC uses a discrete GPU (dGPU), which goes through the PCI bus. This means the iGPU shares memory with the system, but a dGPU has its own memory (VRAM). The iGPU must share memory with the rest of the system.

When an iGPU shares memory, it has some requirements. One of those is that it tends to want to use unfragmented memory which is a single large block, and probably has some alignment requirements. Even if you have a lot of free system memory it is possible some of that cannot be used simply because it isn’t a single contiguous block, or perhaps because it starts at an inconvenient address. The dGPU does not have that problem since the memory is its own. Plus the dGPU VRAM is far faster than the usual system RAM.

Video and probably most every driver of hardware requires operating at a physical address level, which means virtual memory translating address blocks won’t work for those drivers. The iGPU connects to the memory controller, but the memory controller is capable of setting up direct 1:1 access to addresses without translation. The dGPU has no need for this. Regardless, the main point is that swap files and swap partitions help user space if there is not enough RAM, but in this case, if you don’t have enough memory for running some model (and training takes more memory), then you can’t simply add swap.

For training you will almost always be better off with a dGPU that has lots of VRAM. There are good reasons why some people will purchase lower performance GPUs with very high amounts of VRAM. That 3090 is a very good choice for training or execution of a model.

Of all embedded systems the AGX Orin is the top of the food chain. It has a lot of memory, doesn’t take a lot of power, and is rather fast. On top of that, Orin has newer software releases coming up, whereas Xavier and others are either about to reach end of new feature stage, or have been there for some time.

Incidentally, I am also a fan of Threadripper. Besides being cost effective, Threadripper has much higher data I/O than many units, and I’m not just considering within the CPU. I’m considering the number of PCIe lanes and the speeds they run at. If you have only a single 3090, then you don’t really need this, but if you want to add multiple disks, and don’t want to share (slow) some of the shared I/O, then Threadripper is a good choice (and it will cost less than Xeon).

All said though, the Orin is probably capable of training, but typically it is used for edge computing of preexisting models.

3 Likes
4

Hey guys, thank you very much for your answers. Especially for the details about the iGPU and dGPU. I understand how it works and I understand that for training, a 3090 is better, and on the orin, it is better to use a trained model. But, if you take into account a fixed budget, when you cannot afford to spend money on both a PC and an orin, will the orin be able to train models at all?

Because the orin is needed for the drone in any case, but without a PC, it seemed to me that you could do without and train models on the orin. Of course, with a slower speed than on a 3090.

5

You absolutely MUST have a PC. There is NO way around. Your M1 Macbook is useless here. Get a used PC that has 64 bit, UEFI, Quad Core, at least 8G of RAM, at least 128G of SSD just for being able to install your Orin or Xavier.

Training is possible on Orin as well. You could also use a Xavier - it’s slower but also cheaper. You won’t get Jetpack 6 with Xavier, though.

6

Once again I will agree with @fchkjwlsq, but use some alternate information to say the same thing


Jetsons don’t have an actual BIOS, and so the things normally done via a BIOS are performed in software. This includes things like setting up power rails and clock timings, maybe training RAM (adjusting for best clock speeds), so on. Only then do you get to the boot software. By doing this in software you get these changes:

  • Less hardware required.
  • Smaller system.
  • Lighter system.
  • Lower cost.
  • Can’t be “bricked” in most cases. Recovery mode of a Jetson simply turns it into a custom USB device understood by the “driver package” (which is the heart of flash software), and so there is no BIOS to fail.

The down side to this is that a Jetson cannot self-flash. It has to have an Ubuntu host PC to be flashed (for eMMC models, e.g., the AGX Orin, this is to eMMC; some people don’t realize it, but even the SD card dev kit models have QSPI memory on the module itself, and QSPI would contain that same “BIOS equivalent in software”). You could use a rather low performance Ubuntu desktop PC for flash, and it wouldn’t even need an NVIDIA video card (although that would be very useful, it could be as simple as a 3060 or any less expensive GPU). The primary concern is probably if it has enough VRAM on cheaper models.

If you have a low end Ubuntu desktop PC (the flash software runs on Linux, not Windows), then in theory you could do all of your training on an Orin, but you’d still need that PC for flashing (and installing some optional software is simpler through the flash software, which is SDK Manager/JetPack). Really though, I think someone with experience training on an AGX Orin would still need to answer (such as @dusty_nv ) just how practical training on AGX Orin would be.

There might be other discrete GPUs you could use on a desktop PC for training and flash of the Jetson that are not as expensive as the 3090, but the 3090 24GB of VRAM would be a good choice.

Incidentally, the eMMC models of Jetsons can have a security fuse burned, and when burned, all of the non-rootfs (non-o/s) software is put into eMMC partitions with a key, and if that key does not match the fuse, then boot will be refused. Jetson flash software always signs non-rootfs content, and that content must match the key before it is allowed as boot content. Before the fuse is burned that content still must be signed by the flash software, but it uses a NULL key. You couldn’t do that with SD card models (SD card models have the signing requirement, but the fuse cannot be burned and is always a NULL key). The boot content is not “hidden” by this, but the key to sign it is, and so once burned only the person with the key can modify that Jetson (at least not the boot content, so things like the kernel and device tree can be trusted).

7

As for a PC to use for deploying jetpack
 Check out this listing on [eBay]. It’s a ThinkPad T430, similar to the one I use, and it’s perfectly suited for the job. You can even flash your system over Ethernet, as it has a built-in port. Plus, you might save some money if you choose a model without a Windows license, like this one. The RAM and SSD are easily upgradeable, but the base configuration should be enough in my honest opinion.

8

Incidentally, I used to use a laptop with a Core 2 Duo, which is absolutely terrible on performance. It did work, but I had to forward graphics to my desktop system because the JetPack GUI wouldn’t fit the little 10" monitor 😚

I think the laptop you are showing has much higher performance, but the 120 GB disk would be cutting it close. I don’t know if the monitor is large enough.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.