磁盘是块存储设备,读写连续的一整块磁盘数据其实还是很快的,平时我们感觉磁盘比内存慢是因为没有顺序地读磁盘,而是在随机读,大部分时间都浪费在寻道和旋转上面。
做个试验,对比一下顺序读磁盘和读写内存的速度,看看到底差多少。
试验方法:创建一个长度为10M的字节数组,执行3种操作--顺序地访问该数组里的元素,随机地访问该数组里的元素,把该数组里的元素顺序地写入磁盘。
试验结果:
结论:
1. 顺序读写内存比随机读写内存快几十倍
2.顺序读写磁盘比随机读写内存快几倍
CPU读写磁盘的最小单位是块,一块通常是4K,连续整块整块的读写会非常快。搜索引擎要存储海量的文档,它充分利用了磁盘的这一特性,提高文档读取的速度。对搜索引擎感兴趣的同学可以扫码进入我的课程--《Go语言实现工业级搜索引擎》。
最后附上测试代码:
package serialize import ( "fmt" io "io" "math/rand" "os" "testing" "time" ) const SIZE = int(1e7) //不能再大了,否则会报错runtime: goroutine stack exceeds 1000000000-byte limit var ( arr [SIZE]byte indexes [SIZE]int ) func init() { for i := 0; i < SIZE; i++ { indexes[i] = i } rand.Seed(time.Now().UnixNano()) rand.Shuffle(SIZE, func(i, j int) { indexes[i], indexes[j] = indexes[j], indexes[i] }) } //顺序读内存 func TestReadRAMOrderly(t *testing.T) { begin := time.Now() for _, ele := range arr { _ = ele } fmt.Printf("read RAM unorderly %d ms\n", time.Since(begin).Milliseconds()) } //go test -v go_search_engine/forward_index/serialize -run=TestReadRAMOrderly //顺序写内存 func TestWriteRAMOrderly(t *testing.T) { begin := time.Now() for i := 0; i < SIZE; i++ { arr[i] = 1 } fmt.Printf("write RAM unorderly %d ms\n", time.Since(begin).Milliseconds()) } //go test -v go_search_engine/forward_index/serialize -run=TestWriteRAMOrderly //随机读内存 func TestReadRAMUnorderly(t *testing.T) { begin := time.Now() for _, i := range indexes { _ = arr[i] } fmt.Printf("read RAM unorderly %d ms\n", time.Since(begin).Milliseconds()) } //go test -v go_search_engine/forward_index/serialize -run=TestReadRAMUnorderly //随机写内存 func TestWriteRAMUnorderly(t *testing.T) { begin := time.Now() for _, i := range indexes { arr[i] = 1 } fmt.Printf("write RAM unorderly %d ms\n", time.Since(begin).Milliseconds()) } //go test -v go_search_engine/forward_index/serialize -run=TestWriteRAMUnorderly //顺序写磁盘 func TestWriteDiskOrderly(t *testing.T) { fout, err := os.OpenFile("arr.txt", os.O_RDWR|os.O_CREATE|os.O_TRUNC, os.ModePerm) if err != nil { panic(err) } begin := time.Now() fout.Write(arr[:]) //通过切片[:]把数组转slice fout.Close() fmt.Printf("write disk orderly %d ms\n", time.Since(begin).Milliseconds()) } //go test -v go_search_engine/forward_index/serialize -run=TestWriteDiskOrderly //顺序读磁盘 func TestReadDiskOrderly(t *testing.T) { fin, err := os.OpenFile("arr.txt", os.O_RDONLY, os.ModePerm) if err != nil { panic(err) } buf := make([]byte, 1024) begin := time.Now() for { _, err := fin.Read(buf) if err == io.EOF { break } } fin.Close() fmt.Printf("read disk orderly %d ms\n", time.Since(begin).Milliseconds()) } //go test -v go_search_engine/forward_index/serialize -run=TestReadDiskOrderly //为了避免缓存的影响,6个Test函数分6次执行