NUC 折腾笔记 - 储存能力测试

NUC 折腾笔记 - 储存能力测试

前文提到过,我计划折腾下这种小盒子,作为日常开发过程中的拓展设备。在短暂使用之后,我为它拓展了两块容量更高的硬盘,开始正式使用。

因为使用了一些网上颇有争议的国产硬件,所以我觉得或许可以记录下来,为后来者作为选择参考。

测试环境

硬件测试环境还是上篇文章提到的 NUC8,存储硬件选择了最近国货之光长江存储和光威,软件环境则选择了 Ubuntu 20.04 LTS 版本。

硬件环境

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-r2cHLFcV-1612279164603)(https://attachment.soulteary.com/2021/02/02/gloway-zhitai.jpg)]

下面的列表可以更直观的看到本次测试都用到了哪些“东西”:

  • NUC8 i5beh(这货不是国货,但是是国产,233)
  • 【国货】长江存储 ZHITAI PC005 Active 1TB - NVME
  • 【国货】长江存储 ZHITAI SC001 Act 1TB - SATA3
  • 【国货】光威 战将 16GB DDR4 2666
  • 【国货】光威 弈Pro 8GB DDR4 2666
  • 【搭配测试】雷克沙 16G DDR4 2666

软件环境

文章中测试使用的软件,则包含了下面的内容:

  • Ubuntu 20.04.2 LTS
  • dd (coreutils) v8.3
  • hdparm v9.58
  • nvme v1.9
  • dmidecode v3.2

查看存储设备基础信息

在开始测试之前,我们先来了解下磁盘和内存的基础信息,以及如何获取它们。

查看 SATA SSD

非 NVME 协议的 SATA 盘的信息查看,除了使用 smart 外,还可以使用 hdparm 工具来查看,相比较 smart ,你可以通过它直观看到这个磁盘具备的各种能力:

hdparm -I /dev/sda
/dev/sda:

ATA device, with non-removable media
  Model Number:       ZHITAI SC001 Active 1TB SSD             
  Serial Number:      ZTB11T0KA20[masked]M  
  Firmware Revision:  ZT013200
  Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
  Supported: 9 8 7 6 5 
  Likely used: 9
Configuration:
  Logical   max current
  cylinders 16383 16383
  heads   16  16
  sectors/track 63  63
  --
  CHS current addressable sectors:    16514064
  LBA    user addressable sectors:   268435455
  LBA48  user addressable sectors:  2000409264
  Logical  Sector size:                   512 bytes
  Physical Sector size:                   512 bytes
  Logical Sector-0 offset:                  0 bytes
  device size with M = 1024*1024:      976762 MBytes
  device size with M = 1000*1000:     1024209 MBytes (1024 GB)
  cache/buffer size  = unknown
  Form Factor: 2.5 inch
  Nominal Media Rotation Rate: Solid State Device
Capabilities:
  LBA, IORDY(can be disabled)
  Queue depth: 32
  Standby timer values: spec'd by Standard, no device specific minimum
  R/W multiple sector transfer: Max = 1 Current = 1
  DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
       Cycle time: min=120ns recommended=120ns
  PIO: pio0 pio1 pio2 pio3 pio4 
       Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
  Enabled Supported:
     *  SMART feature set
        Security Mode feature set
     *  Power Management feature set
     *  Write cache
     *  Look-ahead
     *  WRITE_BUFFER command
     *  READ_BUFFER command
     *  NOP cmd
     *  DOWNLOAD_MICROCODE
     *  48-bit Address feature set
     *  Mandatory FLUSH_CACHE
     *  FLUSH_CACHE_EXT
     *  SMART error logging
     *  General Purpose Logging feature set
     *  WRITE_{DMA|MULTIPLE}_FUA_EXT
     *  64-bit World wide name
     *  WRITE_UNCORRECTABLE_EXT command
     *  {READ,WRITE}_DMA_EXT_GPL commands
     *  Segmented DOWNLOAD_MICROCODE
     *  unknown 119[6]
        unknown 119[8]
        unknown 119[9]
     *  Gen1 signaling speed (1.5Gb/s)
     *  Gen2 signaling speed (3.0Gb/s)
     *  Gen3 signaling speed (6.0Gb/s)
     *  Native Command Queueing (NCQ)
     *  Host-initiated interface power management
     *  Phy event counters
     *  Host automatic Partial to Slumber transitions
     *  Device automatic Partial to Slumber transitions
     *  READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
     *  DMA Setup Auto-Activate optimization
     *  Device-initiated interface power management
     *  Software settings preservation
        Device Sleep (DEVSLP)
     *  SMART Command Transport (SCT) feature set
     *  SCT Features Control (AC4)
     *  SCT Data Tables (AC5)
     *  SANITIZE_ANTIFREEZE_LOCK_EXT command
     *  SANITIZE feature set
     *  BLOCK_ERASE_EXT command
     *  Data Set Management TRIM supported (limit 8 blocks)
Security: 
  Master password revision code = 65534
    supported
  not enabled
  not locked
    frozen
  not expired: security count
    supported: enhanced erase
  2min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000000000000000
  NAA   : 5
  IEEE OUI  : 000000
  Unique ID : 000000000
Device Sleep:
  DEVSLP Exit Timeout (DETO): 240 ms (drive)
  Minimum DEVSLP Assertion Time (MDAT): 31 ms (drive)
Checksum: correct

查看 NVME SSD

NVME 协议的磁盘则需要借助 nvme 来进行信息查看,默认系统中不会包含这个软件包,需要先进行安装:

apt install nvme-cli

安装完毕之后,使用 list 命令查看 nvme 设备的名称:

nvme list

Node             SN                   Model                                    Namespace Usage                      Format           FW Rev  
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     ZTA11T0JA2[masked]EK   ZHITAI PC005 Active 1TB                  1          92.90  GB /   1.02  TB    512   B +  0 B   ZT00D212

然后使用 smart-log 命令查看硬盘详细信息:

nvme smart-log /dev/nvme0n1

Smart Log for NVME device:nvme0n1 namespace-id:ffffffff
critical_warning                    : 0
temperature                         : 29 C
available_spare                     : 100%
available_spare_threshold           : 10%
percentage_used                     : 0%
data_units_read                     : 83,560
data_units_written                  : 221,989
host_read_commands                  : 362,055
host_write_commands                 : 810,814
controller_busy_time                : 8
power_cycles                        : 9
power_on_hours                      : 25
unsafe_shutdowns                    : 4
media_errors                        : 0
num_err_log_entries                 : 0
Warning Temperature Time            : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1                : 29 C
Temperature Sensor 2                : 26 C
Temperature Sensor 3                : 30 C
Thermal Management T1 Trans Count   : 0
Thermal Management T2 Trans Count   : 0
Thermal Management T1 Total Time    : 0
Thermal Management T2 Total Time    : 0

查看内存基础信息

查看内存基础信息需要借助 dmidecode 软件的能力,执行 dmidecode -t memory 后,便能看到内存的主要信息了:

dmidecode -t memory

# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.2.1 present.
# SMBIOS implementations newer than version 3.2.0 are not
# fully supported by this version of dmidecode.

Handle 0x003B, DMI type 16, 23 bytes
Physical Memory Array
  Location: System Board Or Motherboard
  Use: System Memory
  Error Correction Type: None
  Maximum Capacity: 32 GB
  Error Information Handle: Not Provided
  Number Of Devices: 2

Handle 0x003C, DMI type 17, 40 bytes
Memory Device
  Array Handle: 0x003B
  Error Information Handle: Not Provided
  Total Width: 64 bits
  Data Width: 64 bits
...

虽然信息很长,但是我们需要关注的数值其实很少,只需要知道下面两项就好:

Type: DDR4
  Speed: 2667 MT/s
  Configured Memory Speed: 2400 MT/s

Speed 代表了设备本身的速度,而 ** Configured Memory Speed** 则代表了系统实际使用的速度。

使用 hdparm 进行基础读测试

使用 hdparm 测试硬盘缓内缓外速度很容易,但是要注意,如果要测试硬件实打实的性能,需要添加 --direct 参数,激活 O_DIRECT 标志,不走页面缓存。

使用 hdparm 测试 NVME 磁盘读取速度

使用 hdparm 分别对 NVME 盘进行 三次读测试:

hdparm -Tt --direct /dev/nvme0n1

测试结果如下:

/dev/nvme0n1:
 Timing O_DIRECT cached reads:   4130 MB in  2.00 seconds = 2067.10 MB/sec
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 Timing O_DIRECT disk reads: 5142 MB in  3.00 seconds = 1713.82 MB/sec

/dev/nvme0n1:
 Timing O_DIRECT cached reads:   2974 MB in  1.99 seconds = 1490.91 MB/sec
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 Timing O_DIRECT disk reads: 5574 MB in  3.00 seconds = 1857.85 MB/sec

/dev/nvme0n1:
 Timing O_DIRECT cached reads:   2976 MB in  2.00 seconds = 1491.63 MB/sec
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 Timing O_DIRECT disk reads: 5074 MB in  3.00 seconds = 1690.87 MB/sec

可以看到文件系统读取速度基本保持在 1G 以上,磁盘数据读速度在 1.5G 左右。

使用 hdparm 测试 SATA 磁盘读取速度

同样使用 hdparm 对 SATA 磁盘进行测试,和之前的命令类似,只需要修改目标设备名称即可:

hdparm -Tt --direct /dev/sda

测试结果也类似:



 Timing O_DIRECT cached reads:   710 MB in  2.00 seconds = 355.07 MB/sec
 Timing O_DIRECT disk reads: 1278 MB in  3.00 seconds = 425.43 MB/sec

/dev/sda:
 Timing O_DIRECT cached reads:   710 MB in  2.00 seconds = 354.84 MB/sec
 Timing O_DIRECT disk reads: 1278 MB in  3.00 seconds = 425.49 MB/sec

/dev/sda:
 Timing O_DIRECT cached reads:   710 MB in  2.00 seconds = 355.41 MB/sec
 Timing O_DIRECT disk reads: 1278 MB in  3.00 seconds = 425.66 MB/sec

不过因为系统总线限制,可以看到文件系统读取速度在 350 M/s 左右徘徊,而直接读取设备则稳定在 425 M/s。

使用 dd 进行顺序文件写测试

先聊聊写文件。

日常开发最常见的需求之一便是写文件,尤其是写大文件。所以测试连续读写性能非常重要。为了方便测试,这里可以写一个简单的脚本,分别测试不同场景下的写性能。

echo "test 10G"
time sh -c "dd if=/dev/zero of=./ddfile bs=1M count=10240 oflag=direct && sync";
rm ./ddfile;

echo "test 1G/1M"
time sh -c "dd if=/dev/zero of=./ddfile bs=1M count=1024 oflag=direct && sync";
rm ./ddfile;

echo "test 1G/512K"
time sh -c "dd if=/dev/zero of=./ddfile bs=512k count=2048 oflag=direct && sync";
rm ./ddfile;

echo "test 1G/256K"
time sh -c "dd if=/dev/zero of=./ddfile bs=256k count=4096 oflag=direct && sync";
rm ./ddfile;

echo "test 1G/256K"
time sh -c "dd if=/dev/zero of=./ddfile bs=256k count=8192 oflag=direct && sync";
rm ./ddfile;

echo "test 1G/4K"
time sh -c "dd if=/dev/zero of=./ddfile bs=4k count=262144 oflag=direct && sync";
rm ./ddfile;

在 NVME 磁盘上进行读取测试

在 NVME 磁盘上执行后,你会看到结果会类似这样:

test 10G
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 10.9002 s, 985 MB/s

real  0m10.933s
user  0m0.070s
sys 0m4.581s


test 1G/1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.50586 s, 713 MB/s

real  0m1.554s
user  0m0.013s
sys 0m0.498s


test 1G/512K
2048+0 records in
2048+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.54981 s, 693 MB/s

real  0m1.593s
user  0m0.008s
sys 0m0.572s


test 1G/256K
4096+0 records in
4096+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.64218 s, 654 MB/s

real  0m1.684s
user  0m0.020s
sys 0m0.660s


test 1G/256K
8192+0 records in
8192+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.23154 s, 665 MB/s

real  0m3.272s
user  0m0.044s
sys 0m1.261s


test 1G/4K
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.76955 s, 225 MB/s

real  0m4.808s
user  0m0.154s
sys 0m2.056s

除了4K 随文件读写外,大于 256K 的测试性能表现都在 650 M/s 以上,1M 的文件读写的速度甚至能够持续在 900 M/s。

在 SATA3 磁盘上进行测试

test 10G
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 30.3082 s, 354 MB/s

real  0m30.372s
user  0m0.136s
sys 0m7.296s


test 1G/1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.15482 s, 340 MB/s

real  0m3.174s
user  0m0.013s
sys 0m0.816s


test 1G/512K
2048+0 records in
2048+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.32807 s, 323 MB/s

real  0m3.346s
user  0m0.043s
sys 0m0.871s


test 1G/256K
4096+0 records in
4096+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.49806 s, 307 MB/s

real  0m3.512s
user  0m0.039s
sys 0m0.946s


test 1G/256K
8192+0 records in
8192+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.95447 s, 309 MB/s

real  0m7.021s
user  0m0.060s
sys 0m1.886s


test 1G/4K
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.54626 s, 142 MB/s

real  0m7.610s
user  0m0.228s
sys 0m2.311s

SATA 盘写入保持在 300M/s 以上,4K写入也有 140 MB/s,对于 S ATA 盘来说,真的可以了。

使用 dd 进行顺序文件读取测试

和写入一样,为了省事,我们同样可以准备一个脚本。但是和写测试相反,我们需要先准备一个比较大的文件作为读取数据。

dd if=/dev/zero of=./ddfile bs=1M count=10240 oflag=direct && sync

time sh -c "dd if=./ddfile of=/dev/shm/ddfile bs=1M count=10240";
rm ./ddfile;

在 NVME 磁盘上进行读取测试

这里我们仅做一种即可,因为读的下限远远比写高的多:

10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 8.71309 s, 1.2 GB/s

real	0m9.332s
user	0m0.021s
sys	0m5.544s



10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 3.23808 s, 3.3 GB/s

real	0m3.843s
user	0m0.012s
sys	0m3.826s



10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 2.98388 s, 3.6 GB/s

real	0m3.599s
user	0m0.010s
sys	0m3.585s

在 SATA 磁盘上进行读取测试

对于 SATA 盘也是一样,相同的命令执行三次:

10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 22.571 s, 476 MB/s

real	0m23.195s
user	0m0.086s
sys	0m13.272s



10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 3.27516 s, 3.3 GB/s

real	0m3.877s
user	0m0.004s
sys	0m3.869s



10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 3.00661 s, 3.6 GB/s

real	0m3.619s
user	0m0.006s
sys	0m3.608s

内存读写测试

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-h8bEZWPw-1612279164605)(https://attachment.soulteary.com/2021/02/02/dram.jpg)]

内存读写和写测试一样,跑个开心就好,因为下限远比硬盘要来的高,一般情况都能满足需求,如果满足不了,那么就涉及到改软件架构了:

先跑一个简单的写测试:

time sh -c "dd if=/dev/zero of=/dev/shm/fill bs=1M count=10240"
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 2.44928 s, 4.4 GB/s

real  0m2.456s
user  0m0.015s
sys 0m2.441s

再跑一个读测试:

time sh -c "dd if=/dev/shm/fill of=/dev/null bs=1M count=10240"
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 1.15888 s, 9.3 GB/s

real	0m1.164s
user	0m0.011s
sys	0m1.154s

关于光威两种不同类型内存是否能混用

有群友说,两条相同品牌相同容量的光威内存搭配使用会出现问题。我这里因为没有相同规格的内存,所以没有遇到这个问题:

  • 实测光威奕Pro和光威战将、以及和雷克沙混搭组合使用没有问题。

奕 Pro 和战将组合插上,使用 dmidecode 筛选查看内存信息命令:

dmidecode -t memory

完整信息如下:

# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.2.1 present.
# SMBIOS implementations newer than version 3.2.0 are not
# fully supported by this version of dmidecode.

Handle 0x003B, DMI type 16, 23 bytes
Physical Memory Array
  Location: System Board Or Motherboard
  Use: System Memory
  Error Correction Type: None
  Maximum Capacity: 32 GB
  Error Information Handle: Not Provided
  Number Of Devices: 2

Handle 0x003C, DMI type 17, 40 bytes
Memory Device
  Array Handle: 0x003B
  Error Information Handle: Not Provided
  Total Width: 64 bits
  Data Width: 64 bits
  Size: 16384 MB
  Form Factor: SODIMM
  Set: None
  Locator: SODIMM1
  Bank Locator: Memory Channel A
  Type: DDR4
  Type Detail: Synchronous
  Speed: 2667 MT/s
  Manufacturer: 0813
  Serial Number: D330011A
  Asset Tag: 9876543210
  Part Number: WAR4S2666[masked]C    
  Rank: 1
  Configured Memory Speed: 2400 MT/s
  Minimum Voltage: 1.2 V
  Maximum Voltage: 1.2 V
  Configured Voltage: 1.2 V

Handle 0x003D, DMI type 17, 40 bytes
Memory Device
  Array Handle: 0x003B
  Error Information Handle: Not Provided
  Total Width: 64 bits
  Data Width: 64 bits
  Size: 8192 MB
  Form Factor: SODIMM
  Set: None
  Locator: SODIMM2
  Bank Locator: Memory Channel B
  Type: DDR4
  Type Detail: Synchronous
  Speed: 2667 MT/s
  Manufacturer: 0813
  Serial Number: 96C374D7
  Asset Tag: 9876543210
  Part Number: YCT4S2666[masked]C    
  Rank: 1
  Configured Memory Speed: 2400 MT/s
  Minimum Voltage: 1.2 V
  Maximum Voltage: 1.2 V
  Configured Voltage: 1.2 V

对两条光威内存进行写测试,性能结果如下:

free -g
              total        used        free      shared  buff/cache   available
Mem:             23           0          21           0           1          22
Swap:             1           0           1



dd if=/dev/zero of=/dev/shm/testfile bs=1M count=1000

1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.249176 s, 4.2 GB/s

dd if=/dev/zero of=/dev/shm/testfile bs=1M count=10000

10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 2.56671 s, 4.1 GB/s

dd if=/dev/zero of=/dev/shm/testfile bs=10M count=1000

1000+0 records in
1000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 3.32415 s, 3.2 GB/s

dd if=/dev/zero of=/dev/shm/testfile bs=100M count=100

100+0 records in
100+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 3.76005 s, 2.8 GB/s

dd if=/dev/zero of=/dev/shm/testfile bs=1000M count=10

10+0 records in
10+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 4.01882 s, 2.6 GB/s

最后

简单说下结论,以上测试都是在模拟密集计算场景、高负载的使用的情况下,我认为作为本地低延时的开发补充,NUC也好,国货内存、SSD也罢都能够满足我们对于 IO 的需求,在它的价格档位,可以谈得上是“真香”。

如果你是作为一般日常使用,那么可以忽略上面的数值,因为这些数值对你来说是高不可攀的天花板。你只需要根据自己的预算情况,选择性入手即可。

如果你不是某些品牌的簇拥的话,可以考虑适当给国产以机会。

–EOF


我现在有一个小小的折腾群,里面聚集了一些喜欢折腾的小伙伴。

在不发广告的情况下,我们在里面会一起聊聊软件、HomeLab、编程上的一些问题,也会在群里不定期的分享一些技术沙龙的资料。

喜欢折腾的小伙伴欢迎扫码添加好友。(请注明来源和目的、实名入群,否则不会通过审核)

关于折腾群入群的那些事


本文使用「署名 4.0 国际 (CC BY 4.0)」许可协议,欢迎转载、或重新修改使用,但需要注明来源。 署名 4.0 国际 (CC BY 4.0)

本文作者: 苏洋

创建时间: 2021年02月02日
统计字数: 12629字
阅读时间: 26分钟阅读
本文链接: https://soulteary.com/2021/02/02/nuc-notes-storage-ability-test.html

上一篇:使用 Zendesk maxwell 对接 kinesis (include producer and consumer)


下一篇:Kafka Producer Notes