golang []byte和string的高性能转换
在fasthttp的最佳实践中有这么一句话:
Avoid conversion between
[]byte
andstring
, since this may result in memory allocation+copy. Fasthttp API provides functions for both[]byte
andstring
- use these functions instead of converting manually between[]byte
andstring
. There are some exceptions - see this wiki page for more details.
大概意思就是说,要尽量避免[]byte
和string
的转换,因为转换过程会存在内存拷贝,影响性能。此外在fasthttp中还提出了一个解决方案,用于[]byte
和string
的高性能转换。直接看下源码:
// b2s converts byte slice to a string without memory allocation.
// See https://groups.google.com/forum/#!msg/Golang-Nuts/ENgbUzYvCuU/90yGx7GUAgAJ .
//
// Note it may break if string and/or slice header will change
// in the future go versions.
func b2s(b []byte) string {
/* #nosec G103 */
return *(*string)(unsafe.Pointer(&b))
}
// s2b converts string to a byte slice without memory allocation.
//
// Note it may break if string and/or slice header will change
// in the future go versions.
func s2b(s string) (b []byte) {
/* #nosec G103 */
bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
/* #nosec G103 */
sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
bh.Data = sh.Data
bh.Cap = sh.Len
bh.Len = sh.Len
return b
}
可以看到上述实现中并没有内存拷贝,使用类似C语言的类型强转实现[]byte
和string
之间的类型转换。那么他们和一般使用的转换之间的性能差异有多大?看下如下性能测试:
var s = "adsfasdfadsfadsfasdfadfadfasdfasdfadsfasdfasdfasdfsadfas"
func BenchmarkB2sNew(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = s2b(s)
}
}
func BenchmarkB2sNormal(b *testing.B) {
var _ []byte
for i := 0; i < b.N; i++ {
_ = []byte(s)
}
}
var bt = []byte("adsfasdfadsfadsfasdfadfadfasdfasdfadsfasdfasdfasdfsadfas")
func BenchmarkS2BNew(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = b2s(bt)
}
}
func BenchmarkS2BNormal(b *testing.B) {
var _ []byte
for i := 0; i < b.N; i++ {
_ = string(bt)
}
}
测试结果如下:
goos: windows
goarch: amd64
pkg: awesomeProject5/lib
cpu: Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
BenchmarkB2sNew
BenchmarkB2sNew-8 1000000000 0.2401 ns/op
BenchmarkB2sNormal
BenchmarkB2sNormal-8 40813272 27.49 ns/op
BenchmarkS2BNew
BenchmarkS2BNew-8 1000000000 0.2393 ns/op
BenchmarkS2BNormal
BenchmarkS2BNormal-8 49520067 22.56 ns/op
PASS
可以看到在相同测试条件下,其性能差异竟然达到了100倍!
可见在高频网络访问中,如果直接在[]byte
和string
之间进行转换将会花费多大的性能!
需要注意的是这种方式也有弊端,在代码注释中可以看到它依赖golang中的string或slice的首部定义。如果后续golang版本对此有修改,则有可能导致代码无法运行。