Last update: 04/07/2018
Intel® Optane™ SSD DC P4800X Series
375 GB Capacity HHHL (CEM3.0) Form Factor PCIe NVMe 3.0 x4 Interface
Technical specifications from the vendor's website: https://www.intel.com/content/www/us/en/products/memory-storage/solid-state-drives/data-center-ssds/optane-dc-p4800x-series/p4800x-375gb-aic-20nm.html
olsky03 were used for benchmarking the drives.
- Intel Optane(TM) SSD DC P4800X Series PHKS73350091375AGN - Bootloader : EB3B0306 DevicePath : /dev/nvme0n1 DeviceStatus : Healthy Firmware : E2010324 FirmwareUpdateAvailable : The selected Intel SSD contains current firmware as of this tool release. Index : 0 ModelNumber : INTEL SSDPED1K375GA ProductFamily : Intel Optane(TM) SSD DC P4800X Series SerialNumber : PHKS73350091375AGN
- Intel Optane(TM) SSD DC P4800X Series PHKS73350091375AGN - AggregationThreshold : 0 AggregationTime : 0 ArbitrationBurst : 0 Bootloader : EB3B0306 CoalescingDisable : 1 DevicePath : /dev/nvme0n1 DeviceStatus : Healthy DirectivesSupported : False DynamicMMIOEnabled : The selected drive does not support this feature. EndToEndDataProtCapabilities : 17 EnduranceAnalyzer : Workload did not induce any wear on the drive. Expected life calculation is invalid. Ensure that workloads run for an hour or more for valid calculations. ErrorString : Firmware : E2010324 FirmwareUpdateAvailable : The selected Intel SSD contains current firmware as of this tool release. HighPriorityWeightArbitration : 0 IOCompletionQueuesRequested : 30 IOSubmissionQueuesRequested : 30 Index : 0 Intel : True IntelGen3SATA : False IntelNVMe : True InterruptVector : 0 IsDualPort : False LBAFormat : 0 LatencyTrackingEnabled : The selected drive does not support this feature. LowPriorityWeightArbitration : 0 MaximumLBA : 732585167 MediumPriorityWeightArbitration : 0 MetadataSetting : 0 MetadataSize : 0 ModelNumber : INTEL SSDPED1K375GA NVMeControllerID : 0 NVMeMajorVersion : 1 NVMeMinorVersion : 0 NVMePowerState : 0 NVMeTertiaryVersion : 0 NamespaceId : 1 NamespaceManagementSupported : False NativeMaxLBA : 732585167 NumErrorLogPageEntries : 63 NumLBAFormats : 6 NumberOfNamespacesSupported : 1 OEM : Generic PCILinkGenSpeed : 3 PCILinkWidth : 4 PLITestTimeInterval : The selected drive does not support this feature. PhySpeed : The selected drive does not support this feature. PhysicalSectorSize : The selected drive does not support this feature. PhysicalSize : 375083606016 PowerGovernorAveragePower : The desired feature is not supported. PowerGovernorBurstPower : The desired feature is not supported. PowerGovernorMode : 0 Product : Coldstream ProductFamily : Intel Optane(TM) SSD DC P4800X Series ProductProtocol : NVME ProtectionInformation : 0 ProtectionInformationLocation : 0 ReadErrorRecoveryTimer : Device does not support this command set. SMARTEnabled : True SMARTHealthCriticalWarningsConfiguration : 0
With no clear pattern, we were able to observe that the drive sometimes enters in a frozen state after two consecutive low-level format commands launched through the
isdct (Intel SSD) tool. Intel is helping us investigating the issue.
Intel® Memory Drive Technology (IMDT) is a software that extends system memory transparently. Combined with an Intel® Optane™ Solid State Drive (SSD), Intel® Memory Drive Technology transparently integrates the SSD into the memory subsystem and makes it appear like DRAM to the OS and applications. Intel® Memory Drive Technology increases memory capacity beyond DRAM limitations and delivers DRAM-like performance in a completely transparent manner to the operating system and application. In addition, no changes are required to the OS or applications.
Enabling IMDT on
olsky03.cern.ch, we were able to see ~500GB of DRAM from the operating system, compared to the 64GB physically installed. IMDT does not allow to use the full capacity of the drives as system memory, since some of the capacity is kept internally by the software to implement pre-fetching and caching algorithms.
In order to test the IMDT capabilities, we performed a test using code from the GeneROOT project, a collaboration aiming at adopting high-energy physics data analysis tools on large genomics datasets. In particular, we ran a software with the goal of realigning genomic files in another format, a task which is very memory and storage consuming.
We compared the execution on
olsky03.cern.ch with IMDT enabled with a normal execution on a quad-socket Haswell server with 512GB of DRAM. All the files were read from local SSD storage.
The times reported below concern the realignment of a 75GB genomic file. Two runs were performed, limiting the amount of memory consumption to 200GB, 400GB, as well as limiting the number of threads:
|Memory limit (GB)||# threads||Wall time|
|IMDT||200||8||17h 39m 26s|
|DRAM||200||8||15h 03m 10s|
|IMDT||400||16||9h 26m 55s|
|DRAM||400||16||8h 31m 05s|
Although the DRAM still performs better (as expected) for this usecase, is very interesting to notice how the execution times are comparable. This is relevant, considering that the price/GB for the Optane drives might be much lower in the near future than the price/GB of a classic DDR4 memory DIMM.
Also, no optimizations were applied for the IMDT tests, leading to the possibility of a performance improvement.
Interesting presentation by Intel/Moscow State University: https://software.intel.com/sites/default/files/managed/ca/71/132980_hpcdevcon2017_benefits-of-intel-memory-drive-technology-for-scientific-applications.pdf
Luca Canali (IT/DB): still waiting for feedback to openlab, but he already discussed with Intel directly.
Fabrice Le Goff (ATLAS, CERN) Slides about Optane tests: https://docs.google.com/presentation/d/1uwKi5AAU380HWrYZxfJ9zmQAzP7wwU864AZ9Modrp6I/edit?usp=sharing