<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Sat, Jul 31, 2021, at 16:26, Khalid Baheyeldin wrote:<br></div><blockquote type="cite" id="qt" style=""><div>[ 128.001364] nvme nvme0: Device not ready; aborting initialisation<br></div><div>[ 128.002041] nvme nvme0: Removing after probe failure status: -19<br></div><div><br></div><div>That NVME was a 1.6TB Micron 9200 MAX, if that matters.<br></div><div><br></div><div>There was no device file under /dev/ for that disk anymore.<br></div></blockquote><div><br></div><div>The only nvme failure I've had so far failed into r/o mode, so I could actually pull data off of it, but it would fail to write. That said, it was a consumer m.2 nvme drive, rather than an enterprise u.2 drive. If you don't even get a device file to inspect, you're likely out of luck diagnosing the issue.<br></div><div><br></div><blockquote type="cite" id="qt" style=""><div>- Why would a device be functional before a reboot but totally go away<br></div><div>after, and not being even detected by the operating system?<br></div></blockquote><div><div><br></div><div>I know Intel Enterprise SSDs, for example, had a target for media wearout, and once that was reached, the drives would brick themselves entirely, instead of potentially introducing random errors over time. I believe the goal was to allow you to replace the drive immediately (and recover via array or backups), instead of potentially introducing smaller issues later which may propagate into backups.<br></div><div><br></div><div>Admins should be replacing those drives preemptively -- either by monitoring the stats using smartctl/nvme tools, or possibly the iLO/similar system will flag it as well.<br></div><div><br></div><div>Reference: <a href="https://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte/">https://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte/</a><br></div><div><br></div><div>It may be that Micron does something similar, but one of the metrics only gets checked at initialization time?<br></div><div><br></div></div><blockquote type="cite" id="qt" style=""><div>- Are NVME as unreliable as SSDs or better? Or are they just faster?<br></div></blockquote><div><br></div><div>I haven't heard any particular concerns about NVMe or SSD reliability, unless your device is full and/or you're not doing using trim/discard (both of which would limit the effectiveness of wear levelling).<br></div><div><br></div><div id="sig91988184"><div class="signature">-- <br></div><div class="signature"><b>Chris Irwin</b><br></div><div class="signature"><br></div><div class="signature"><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;">email: </span><a href="mailto:chris@chrisirwin.ca"><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;">chris@chrisirwin.ca</span></a><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;"></span><br></div><div class="signature"><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;"> xmpp: </span><a href="mailto:chris@chrisirwin.ca"><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;">chris@chrisirwin.ca</span></a><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;"></span><br></div><div class="signature"><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;"> web: </span><a href="https://chrisirwin.ca"><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;">https://chrisirwin.ca</span></a><br></div></div><div><br></div></body></html>