文章目录
- `nvidia-smi`查看所有GPU的简要信息状态
- `nvidia-smi –i xxx`指定查看某个GPU
- `nvidia-smi –l xxx`动态刷新信息(默认5s刷新一次),按Ctrl+C停止,可指定刷新频率,以秒为单位
- `nvidia-smi –f xxx` 将查询的信息输出到具体的文件中,不在终端显示
- `nvidia-smi -q`查看所有GPU的详细信息状态
- `nvidia-smi –q –u`显示单元而不是GPU的属性
- `nvidia-smi –q –i xxx`指定具体的GPU或unit信息
- `nvidia-smi –q –f xxx`将查询的信息输出到具体的文件中,不在终端显示
- `nvidia-smi –q –x`将查询的信息以xml的形式输出
- `nvidia-smi -q –d xxx`指定显示GPU卡某些信息,xxx参数可以为MEMORY, UTILIZATION, ECC, TEMPERATURE, POWER,CLOCK, COMPUTE, PIDS, PERFORMANCE, SUPPORTED_CLOCKS, PAGE_RETIREMENT,ACCOUNTING
- `nvidia-smi –q –l xxx`动态刷新信息,按Ctrl+C停止,可指定刷新频率,以秒为单位
- `nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv`选择性查询选项,可以指定显示的属性选项
- 设备修改选项
- `nvidia-smi –pm 0/1`设置持久模式:0/DISABLED,1/ENABLED
- `nvidia-smi –e 0/1`切换ECC支持:0/DISABLED, 1/ENABLED
- `nvidia-smi –p 0/1`重置ECC错误计数:0/VOLATILE, 1/AGGREGATE
- `nvidia-smi –c`设置计算应用模式:0/DEFAULT,1/EXCLUSIVE_PROCESS,2/PROHIBITED
- `nvidia-smi –r`GPU复位
- `nvidia-smi –vm`设置GPU虚拟化模式
- `nvidia-smi –ac xxx,xxx`设置GPU运行的工作频率。e.g. nvidia-smi –ac2000,800
- `nvidia-smi –rac`将时钟频率重置为默认值
- `nvidia-smi –acp 0/1`切换-ac和-rac的权限要求,0/UNRESTRICTED, 1/RESTRICTED
- `nvidia-smi –pl`指定最大电源管理限制(瓦特)
- `nvidia-smi –am 0/1`启用或禁用计数模式,0/DISABLED,1/ENABLED
- `nvidia-smi –caa`清除缓冲区中的所有已记录PID,0/DISABLED,1/ENABLED
- `nvidia-smi dmon`设备监控命令,以滚动条形式显示GPU设备统计信息。(略)
nvidia-smi
查看所有GPU的简要信息状态
cd C:\Program Files\NVIDIA Corporation\NVSMI
nvidia-smi
结果:
Microsoft Windows [版本 10.0.16299.248]
(c) 2017 Microsoft Corporation。保留所有权利。
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi
Wed Jan 15 16:32:13 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 441.08 Driver Version: 441.08 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GT 710 WDDM | 00000000:05:00.0 N/A | N/A |
| 50% 35C P8 N/A / N/A | 262MiB / 2048MiB | N/A Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 108... WDDM | 00000000:0E:00.0 Off | N/A |
| 0% 36C P8 13W / 280W | 149MiB / 11264MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
| 1 14724 C+G ...ense SDK 2.0\tools\realsense-viewer.exe N/A |
+-----------------------------------------------------------------------------+
C:\Program Files\NVIDIA Corporation\NVSMI>
不对啊,我只安装了CUDA8.0和9.0,我这哪来的10.2?
C:\Program Files\NVIDIA Corporation\NVSMI>nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:32_Central_Daylight_Time_2017
Cuda compilation tools, release 9.0, V9.0.176
C:\Program Files\NVIDIA Corporation\NVSMI>
实际跑六个摄像头轮询时的GPU使用情况:
nvidia-smi –i xxx
指定查看某个GPU
示例:
查看GT 710
nvidia-smi –i 0
查看1080ti
nvidia-smi –i 1
nvidia-smi –l xxx
动态刷新信息(默认5s刷新一次),按Ctrl+C停止,可指定刷新频率,以秒为单位
就是不断刷新打印信息,如:
3秒打印一次信息
nvidia-smi –l 3
nvidia-smi –f xxx
将查询的信息输出到具体的文件中,不在终端显示
不过试了一下不知道怎么输出。。。显示失败
后来试着输出到D盘,成功了。。。
在D盘新建一个名为print的文件夹,将文件输出到里面,也成功了:
nvidia-smi -q
查看所有GPU的详细信息状态
Microsoft Windows [版本 10.0.16299.248]
(c) 2017 Microsoft Corporation。保留所有权利。
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q
==============NVSMI LOG==============
Timestamp : Wed Jan 15 17:20:51 2020
Driver Version : 441.08
CUDA Version : 10.2
Attached GPUs : 2
GPU 00000000:05:00.0
Product Name : GeForce GT 710
Product Brand : GeForce
Display Mode : N/A
Display Active : N/A
Persistence Mode : N/A
Accounting Mode : N/A
Accounting Mode Buffer Size : N/A
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-29317d9c-e1b8-0f8d-1c0e-3ca99f2dbf3d
Minor Number : N/A
VBIOS Version : 80.28.A6.00.2F
MultiGPU Board : N/A
Board ID : N/A
GPU Part Number : N/A
Inforom Version
Image Version : N/A
OEM Object : N/A
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization Mode : N/A
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x05
Device : 0x00
Domain : 0x0000
Device Id : 0x128B10DE
Bus Id : 00000000:05:00.0
Sub System Id : 0x8C931462
GPU Link Info
PCIe Generation
Max : N/A
Current : N/A
Link Width
Max : N/A
Current : N/A
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : N/A
Rx Throughput : N/A
Fan Speed : 50 %
Performance State : P8
Clocks Throttle Reasons : N/A
FB Memory Usage
Total : 2048 MiB
Used : 277 MiB
Free : 1771 MiB
BAR1 Memory Usage
Total : N/A
Used : N/A
Free : N/A
Compute Mode : Default
Utilization
Gpu : N/A
Memory : N/A
Encoder : N/A
Decoder : N/A
Encoder Stats
Active Sessions : N/A
Average FPS : N/A
Average Latency : N/A
FBC Stats
Active Sessions : N/A
Average FPS : N/A
Average Latency : N/A
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Temperature
GPU Current Temp : 45 C
GPU Shutdown Temp : N/A
GPU Slowdown Temp : N/A
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : N/A
Power Draw : N/A
Power Limit : N/A
Default Power Limit : N/A
Enforced Power Limit : N/A
Min Power Limit : N/A
Max Power Limit : N/A
Clocks
Graphics : N/A
SM : N/A
Memory : N/A
Video : N/A
Applications Clocks
Graphics : 954 MHz
Memory : 800 MHz
Default Applications Clocks
Graphics : 954 MHz
Memory : 800 MHz
Max Clocks
Graphics : N/A
SM : N/A
Memory : N/A
Video : N/A
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes : N/A
GPU 00000000:0E:00.0
Product Name : GeForce GTX 1080 Ti
Product Brand : GeForce
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-f8f659c0-b441-5918-e7a5-5afa44563670
Minor Number : N/A
VBIOS Version : 86.02.40.00.1A
MultiGPU Board : No
Board ID : 0xe00
GPU Part Number : N/A
Inforom Version
Image Version : G001.0000.01.04
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x0E
Device : 0x00
Domain : 0x0000
Device Id : 0x1B0610DE
Bus Id : 00000000:0E:00.0
Sub System Id : 0x36021462
GPU Link Info
PCIe Generation
Max : 3
Current : 3
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 160000 KB/s
Rx Throughput : 2797000 KB/s
Fan Speed : 38 %
Performance State : P2
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
FB Memory Usage
Total : 11264 MiB
Used : 9268 MiB
Free : 1996 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
Compute Mode : Default
Utilization
Gpu : 43 %
Memory : 20 %
Encoder : 0 %
Decoder : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Temperature
GPU Current Temp : 64 C
GPU Shutdown Temp : 96 C
GPU Slowdown Temp : 93 C
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : Supported
Power Draw : 250.52 W
Power Limit : 280.00 W
Default Power Limit : 280.00 W
Enforced Power Limit : 280.00 W
Min Power Limit : 140.00 W
Max Power Limit : 330.00 W
Clocks
Graphics : 1949 MHz
SM : 1949 MHz
Memory : 5005 MHz
Video : 1645 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 1974 MHz
SM : 1974 MHz
Memory : 5505 MHz
Video : 1620 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes
Process ID : 1448
Type : C
Name : D:\20191031_tensorflow_yolov3\python\python.exe
Used GPU Memory : Not available in WDDM driver model
C:\Program Files\NVIDIA Corporation\NVSMI>
我们发现,最下面打印进程占用GPU容量时,显示Not available in WDDM driver model
,可参考如下解释:
Not available in WDDM driver model => WDDM stand for Windows Display Driver Model. You can switch to TCC and obtain the information with the command: nvidia-smi -dm 1, however this operation can only performed if you do not have any display attached to the GPU. So... It's mostly impossible...
在WDDM驱动程序模型中不可用=> WDDM代表Windows Display Driver Model。 您可以使用以下命令切换到TCC并获取信息:nvidia-smi -dm 1,但是仅当您未在GPU上连接任何显示器时,才能执行此操作。 所以...这几乎是不可能的...
By the way, don't worry about a high memory usage, Tensorflow reserve as much GPU memory as it can to speed up its processes. If you prefer a finer grained control on the memory taken use the following (it may slow down a little bit your computations):
顺便说一下,不用担心内存使用率高,Tensorflow会保留尽可能多的GPU内存,以加快处理速度。 如果您希望对所使用的内存进行更细粒度的控制,请使用以下命令(它可能会降低您的计算速度):
# 动态申请显存
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
You can create a dual boot on Ubuntu or just forget about this.
您可以在Ubuntu上创建双重引导,也可以忽略它。
nvidia-smi –q –u
显示单元而不是GPU的属性
示例:
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q -u
==============NVSMI LOG==============
Timestamp : Thu Jan 16 09:16:33 2020
Driver Version : 441.08
CUDA Version : 10.2
HIC Info : N/A
Attached Units : 0
C:\Program Files\NVIDIA Corporation\NVSMI>
nvidia-smi –q –i xxx
指定具体的GPU或unit信息
示例:
查看1080ti详细信息
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q -i 1
==============NVSMI LOG==============
Timestamp : Thu Jan 16 09:25:13 2020
Driver Version : 441.08
CUDA Version : 10.2
Attached GPUs : 2
GPU 00000000:0E:00.0
Product Name : GeForce GTX 1080 Ti
Product Brand : GeForce
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-f8f659c0-b441-5918-e7a5-5afa44563670
Minor Number : N/A
VBIOS Version : 86.02.40.00.1A
MultiGPU Board : No
Board ID : 0xe00
GPU Part Number : N/A
Inforom Version
Image Version : G001.0000.01.04
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x0E
Device : 0x00
Domain : 0x0000
Device Id : 0x1B0610DE
Bus Id : 00000000:0E:00.0
Sub System Id : 0x36021462
GPU Link Info
PCIe Generation
Max : 3
Current : 3
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 302000 KB/s
Rx Throughput : 0 KB/s
Fan Speed : 35 %
Performance State : P2
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
FB Memory Usage
Total : 11264 MiB
Used : 9268 MiB
Free : 1996 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
Compute Mode : Default
Utilization
Gpu : 42 %
Memory : 20 %
Encoder : 0 %
Decoder : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Temperature
GPU Current Temp : 64 C
GPU Shutdown Temp : 96 C
GPU Slowdown Temp : 93 C
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : Supported
Power Draw : 267.13 W
Power Limit : 280.00 W
Default Power Limit : 280.00 W
Enforced Power Limit : 280.00 W
Min Power Limit : 140.00 W
Max Power Limit : 330.00 W
Clocks
Graphics : 1949 MHz
SM : 1949 MHz
Memory : 5005 MHz
Video : 1645 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 1974 MHz
SM : 1974 MHz
Memory : 5505 MHz
Video : 1620 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes
Process ID : 1448
Type : C
Name : D:\20191031_tensorflow_yolov3\python\python.exe
Used GPU Memory : Not available in WDDM driver model
C:\Program Files\NVIDIA Corporation\NVSMI>
nvidia-smi –q –f xxx
将查询的信息输出到具体的文件中,不在终端显示
参考上面,示例略
nvidia-smi –q –x
将查询的信息以xml的形式输出
略,就是打印的格式变成xml了
nvidia-smi -q –d xxx
指定显示GPU卡某些信息,xxx参数可以为MEMORY, UTILIZATION, ECC, TEMPERATURE, POWER,CLOCK, COMPUTE, PIDS, PERFORMANCE, SUPPORTED_CLOCKS, PAGE_RETIREMENT,ACCOUNTING
示例:
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q -d MEMORY
==============NVSMI LOG==============
Timestamp : Thu Jan 16 09:33:08 2020
Driver Version : 441.08
CUDA Version : 10.2
Attached GPUs : 2
GPU 00000000:05:00.0
FB Memory Usage
Total : 2048 MiB
Used : 370 MiB
Free : 1678 MiB
BAR1 Memory Usage
Total : N/A
Used : N/A
Free : N/A
GPU 00000000:0E:00.0
FB Memory Usage
Total : 11264 MiB
Used : 9268 MiB
Free : 1996 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
C:\Program Files\NVIDIA Corporation\NVSMI>
nvidia-smi –q –l xxx
动态刷新信息,按Ctrl+C停止,可指定刷新频率,以秒为单位
xxx单位为秒
示例略
nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv
选择性查询选项,可以指定显示的属性选项
可查看的属性有:timestamp,driver_version,pci.bus,pcie.link.width.current等。(可查看nvidia-smi–help-query–gpu来查看有哪些属性)
示例:
Microsoft Windows [版本 10.0.16299.248]
(c) 2017 Microsoft Corporation。保留所有权利。
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv
name, pci.bus_id, vbios_version
GeForce GT 710, 00000000:05:00.0, 80.28.A6.00.2F
GeForce GTX 1080 Ti, 00000000:0E:00.0, 86.02.40.00.1A
C:\Program Files\NVIDIA Corporation\NVSMI>
设备修改选项
可以手动设置GPU卡设备的状态选项
nvidia-smi –pm 0/1
设置持久模式:0/DISABLED,1/ENABLED
nvidia-smi –e 0/1
切换ECC支持:0/DISABLED, 1/ENABLED
nvidia-smi –p 0/1
重置ECC错误计数:0/VOLATILE, 1/AGGREGATE
nvidia-smi –c
设置计算应用模式:0/DEFAULT,1/EXCLUSIVE_PROCESS,2/PROHIBITED
nvidia-smi –r
GPU复位
nvidia-smi –vm
设置GPU虚拟化模式
nvidia-smi –ac xxx,xxx
设置GPU运行的工作频率。e.g. nvidia-smi –ac2000,800
nvidia-smi –rac
将时钟频率重置为默认值
nvidia-smi –acp 0/1
切换-ac和-rac的权限要求,0/UNRESTRICTED, 1/RESTRICTED
nvidia-smi –pl
指定最大电源管理限制(瓦特)
nvidia-smi –am 0/1
启用或禁用计数模式,0/DISABLED,1/ENABLED
nvidia-smi –caa
清除缓冲区中的所有已记录PID,0/DISABLED,1/ENABLED
nvidia-smi dmon
设备监控命令,以滚动条形式显示GPU设备统计信息。(略)
参考文章1:Windows用命令查看nvidia的GPU使用情况
参考文章2:windows如何查看nvidia显卡(GPU)的利用率和温度