What you’re seeing is the result of driver chaining which is how Citrix and the Windows OS manages the sharing of the single GPU across multiple sessions on the single host.
As for performance, have you made the registry changes to get DirectX to use all GPU’s in the host?
When panning and zooming how are you measuring the experience? Are you’re measuring it at the client based on what you see, or are you measuring framerate in the session and and the client?
Have you optimised the HDX policy and do you have a client capable of decoding HDX at at least 30fps?