Creating the link libdrm.so.2 → libdrm.so.2.4.0 solves the problem temporarily until the next time ldconfig runs (which will be triggered by e.g. an apt-get install).
The problem seems to be that the link libdrm_nvdc.so → tegra/libdrm.so.2 will cause ldconfig to overwrite libdrm.so.2 with link to libdrm_nvdc.so.
To solve this permanently, I removed the link libdrm_nvdc.so → tegra/libdrm.so.2, and then ran ldconfig:
sudo rm /usr/lib/aarch64-linux-gnu/libdrm_nvdc.so
sudo ldconfig
This created the link libdrm.so.2 → libdrm.so.2.4.0.
Of course this could potentially result in other kinds of failures, but I have not experienced any yet.