Nvcr.io advertises a different token realm after a token expires, dead-ending anonymous token refresh

AI & Data Science NGC Catalog Feedback
1

When pulling from nvcr.io with the standard Docker Registry v2 bearer-token flow, we noticed that the token realm the registry advertises changes across the auth lifecycle, which leaves an anonymous client unable to refresh once its token expires.

  1. On the first request with no token, the registry returns 401 with a challenge whose realm is https://nvcr.io/proxy_auth.
  2. Following that realm anonymously returns 200 OK with a bearer token (a pull grant for public repos), and the pull succeeds.
  3. When that token expires and the client retries, the registry returns 401 — but now the challenge points to a different realm, https://authn.nvidia.com/token with service="registry" and error="invalid_token".
  4. A client normally refreshes by following the realm in the challenge it just received. Following this second realm anonymously returns 401 with no token — whereas the first realm (nvcr.io/proxy_auth) still issues one. So the client, having correctly followed the advertised realm, has no way to obtain a fresh token and the pull dead-ends.

In other words, the realm advertised when unauthenticated (nvcr.io/proxy_auth, which issues tokens anonymously) is not the realm advertised after a token is presented and rejected (authn.nvidia.com/token, which does not) — so anonymous token refresh after expiry terminates on a 401.

We also noticed the registry keeps accepting an expired token for a short grace window (still 200 shortly past the token’s exp, rejected a few minutes later) — a secondary inconsistency, though the realm switch above is the part that breaks refresh.

Expected behavior

  • The registry should advertise a consistent token realm/service across the auth lifecycle — the realm in the post-expiry 401 should be the same one advertised on the initial 401, so a client that follows the advertised realm can obtain a fresh token and retry.
  • If the intended realm is authn.nvidia.com/token, it should be the one advertised from the start (and it should issue tokens for anonymous pulls of public content the same way nvcr.io/proxy_auth does).

*To Reproduce

# 1) no token -> 401, realm A = nvcr.io/proxy_auth
curl -sS -D - -o /dev/null -H "Accept: application/vnd.oci.image.index.v1+json" \
  https://nvcr.io/v2/nvidia/tritonserver/manifests/sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794
#   401 ; WWW-Authenticate: Bearer realm="https://nvcr.io/proxy_auth",scope="repository:nvidia/tritonserver:pull"

# 2) follow realm A anonymously -> 200 + token (pull grant); the pull works
curl -sS "https://nvcr.io/proxy_auth?scope=repository:nvidia/tritonserver:pull"
#   200 ; {"token":"<jwt, ~10 min lifetime>"}

# 3) after the token expires, retry -> 401, realm B = authn.nvidia.com/token (DIFFERENT)
curl -sS -D - -o /dev/null -H "Authorization: Bearer <expired token from step 2>" \
  -H "Accept: application/vnd.oci.image.index.v1+json" \
  https://nvcr.io/v2/nvidia/tritonserver/manifests/sha256:58df7489...
#   401 ; WWW-Authenticate: Bearer realm="https://authn.nvidia.com/token",service="registry",scope="repository:nvidia/tritonserver:pull",error="invalid_token"

# 4) follow realm B anonymously -> 401, no token issued (refresh dead-ends)
curl -sS -D - -o /dev/null "https://authn.nvidia.com/token?service=registry&scope=repository:nvidia/tritonserver:pull"
#   401

# (contrast) realm A still issues a token anonymously
curl -sS -D - -o /dev/null "https://nvcr.io/proxy_auth?scope=repository:nvidia/tritonserver:pull"
#   200
2

This looks like a clear authentication flow issue rather than a client-side problem. The token realm changing between initial access and refresh breaks the normal Docker Registry v2 behavior. Hopefully the registry team can keep the advertised realm consistent or ensure both endpoints support anonymous token refresh. The reproduction steps are well documented and should make debugging easier.