业务中有这样一个struct:
type bizData struct{
A uint64
B uint64
C int32
D uint32
}
虽然没有实测,但我猜测这样的对齐方式,这个struct占24字节。
业务中用一个map指向这些对象: map[uint64]*bizData
这样的存储对象,一个容器开启10GB内存,最多存储60万左右的对象,容器内存就爆了。
于是自己实现一个简易的内存池:mempool.go
package mempool
//实现一个按块扩展的内存池
type memPool struct {
datas [][]bizData
freeNode chan uint32
countOfBlock int
}
//NewMemPool 构造内存池对象
func NewMemPool(countOfBlock int) *memPool {
out := &memPool{
datas: [][]bizData{make([]bizData, countOfBlock)},
freeNode: make(chan uint32, countOfBlock),
countOfBlock: countOfBlock,
}
for i := 0; i < countOfBlock; i++ {
out.freeNode <- uint32(i)
}
return out
}
func (pool *memPool) moreBlock() {
pool.datas = append(pool.datas, make([]bizData, pool.countOfBlock))
ch := make(chan uint32, len(pool.datas)*pool.countOfBlock)
close(pool.freeNode)
for item := range pool.freeNode {
ch <- item
}
start := (len(pool.datas) - 1) * pool.countOfBlock
for i := 0; i < pool.countOfBlock; i++ {
ch <- uint32(start + i)
}
pool.freeNode = ch
}
func (pool *memPool) Get() uint32 {
if len(pool.freeNode) == 0 {
pool.moreBlock()
}
select {
case idx := <-pool.freeNode:
return idx
default:
panic("impossible")
}
}
func (pool *memPool) Put(idx uint32) {
pool.freeNode <- idx
}
func (pool *memPool) Item(idx uint32) *bizData {
blockIdx := int(idx) / pool.countOfBlock
loc := int(idx) % pool.countOfBlock
return &(pool.datas[blockIdx][loc])
}
然后把对象的引用方式修改为: map[uint64]uint32 //值映射数据块的下标
使用上面的内存池后,分配250万小对象,进程内存始终未超过600mb!!
so, 大量小对象的情况下,自己实现内存池更好!