Some notes on ESXi `smartinfo` throwing `error Cannot open device`

I wrote about the built-in smartinfo command before at NVMe and SATA health data on ESXi: some links to investigate.

It is at /usr/lib/vmware/vm-support/bin/smartinfo, which is not in the path. I used it recently after having trouble with some 2.5-inch SSD devices, of one which is listed below.

Some note on this error what smartinfo from ESXi throwed on some devices over time on various ESXi machines:

Failed to get SMART stats for t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S3Z2NB0M529545P_____ error Cannot open device 
----------------------------------------------------- 

The same SSD sometimes had very big latency times:

2021-05-01T16:23:20.306Z cpu7:2097783)WARNING: ScsiDeviceIO: 1564: Device t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S3Z2NB0M529545P_____ performance has deteriorated. I/O latency increased from average value of 59147
2021-05-01T16:23:20.306Z cpu7:2097783)WARNING: microseconds to 1274466 microseconds.
2021-05-01T16:23:20.306Z cpu7:2097783)WARNING: ScsiDeviceIO: 1564: Device t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S3Z2NB0M529545P_____ performance has deteriorated. I/O latency increased from average value of 59150
2021-05-01T16:23:20.306Z cpu7:2097783)WARNING: microseconds to 2664445 microseconds.
2021-05-01T16:23:20.847Z cpu7:2097190)ScsiDeviceIO: 1530: Device t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S3Z2NB0M529545P_____ performance has improved. I/O latency reduced from 2664445 microseconds to 526814 microseconds.
2021-05-01T16:23:38.477Z cpu5:2097783)ScsiDeviceIO: 1530: Device t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S3Z2NB0M529545P_____ performance has improved. I/O latency reduced from 526814 microseconds to 116553 microseconds.
2021-05-01T16:24:01.380Z cpu9:2097674)NMP: nmp_ResetDeviceLogThrottling:3586: last error status from device t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S3Z2NB0M529545P_____ repeated 1 times

[Wayback] “smartinfo” “error Cannot open device” – Google Search delivered mainly results around the old smartinfo.sh tool that is now replaced by the smartinfo binary:

• [Wayback] Rapin Blog: check disk Wmware
• [Wayback] Cannot access disk S.M.A.R.T data – VMware Technology Network VMTN
• [Wayback] VmWare ESXi SMART info HDD – trianglesis blog

[Wayback] “ESXi” “smart” “error Cannot open device” – Google Search returned no relevant results.

I think an 860 EVO 2.5-inch SSD is not on par running half a dozen Windows VM’s that have regular write actions, but I will later give this a try, though I doubt it will help as this is on a VMware 6.7 update 3 based system which should not have the vmw_ahci driver slowness:

[Wayback] NVMe SSD Fast, but SATA SSD slow ! – VMware Technology Network VMTN (hyphen fixes mine):

I have done the esxcli system module set --enabled=false --module=vmw_ahci and rebooted, and I got a small performance increase.  But still not what I thought.

It refers these other posts:

• [Wayback] HomeLab – SuperMicro 5028D-TNT4 Storage Driver Performance Issues and Fix – VIRTUALIZATION IS LIFE!
• [Wayback] Easy fix for Supermicro and other Xeon D systems experiencing SATA3/AHCI slowdown on ESXi 6.5 | TinkerTry IT @ Home
• [Wayback] AHCI (vmw_ahci) performance issue resolved in ESXi 6.5 Update 1 (hyphen fixes mine):
  

  With the release of ESXi 6.5 Update 1, I am happy to report the observed performance issues with the Native AHCI driver have now been resolved! I have been running on earlier release of ESXi 6.5 Update 1 build for couple of weeks now and have not seen any of the problems I had before. For those interested, the official fix went is in version 1.0.0-37vmw or greater of the vmw_ahci driver.

  …

  If the Native AHCI driver is disabled as shown in the previous command, then you can re-enable it by running the following ESXCLI command:

  esxcli system module set --enabled=true --module=vmw_ahci

The system with this behaviour is on these versions:

# esxcli software vib list | grep ahci
sata-ahci                      3.0-26vmw.670.0.0.8169922             VMW     VMwareCertified   2021-04-03
vmw-ahci                       2.0.7-2vmw.670.3.143.17700523         VMW     VMwareCertified   2021-04-04

–jeroen