Kafak采用内存映射文件、硬盘顺序写入技术提示性能。即便是顺序写入硬盘，硬盘的访问速度还是不可能追上内存。所以Kafka的数据并不是实时的写入硬盘，
它充分利用了现代操作系统分页存储来利用内存提高I/O效率。
一、顺序写入
1. 机械硬盘
机械硬盘的结构你可以想象成一个唱片机，它有一个旋转的盘片和一个能沿半径方向移动的磁头。处理读取和写入请求时，
首先可以根据请求的开始地址算出要处理的数据在磁盘上的位置，之后要进行以下几步工作：
2. 固态硬盘
java io操作中通常采用BufferedReader，BufferedInputStream等带缓冲的IO类处理大文件，不过java nio中引入MappedByteBuffer操作大文件的方式，其读写性能极高。
File.read()将文件从硬盘拷贝到内核空间的一个缓冲区，再将这些数据拷贝到用户空间，实际上进行了两次数据拷贝。
FileChannal.map()直接将文件从硬盘拷贝到用户空间，只进行了一次数据拷贝。
1.
public class MapMemeryBuffer {
public static void main(String[] args) throws Exception {
ByteBuffer byteBuf = ByteBuffer.allocate(14 * 1024 * 1024);
byte[] bytes = new byte[14 * 1024 * 1024];
FileInputStream fis = new FileInputStream("d:\\java_transactions_book.pdf");
FileOutputStream fos = new FileOutputStream("d:\\java_transactions_book_copy.pdf");
FileChannel fileChannel = fis.getChannel();
long timeStar = System.currentTimeMillis();
// 读取
//fileChannel.read(byteBuf);
MappedByteBuffer mbb = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
long timeEnd = System.currentTimeMillis();
System.out.println("Read time :" + (timeEnd - timeStar) + "ms");
timeStar = System.currentTimeMillis();
// 写入
//fos.write(bytes);
// After a sequence of channel-read or put operations,
// invoke this method to prepare for a sequence of channel-write or relative get operations.
mbb.flip();
timeEnd = System.currentTimeMillis();
System.out.println("Write time :" + (timeEnd - timeStar) + "ms");
fos.flush();
fileChannel.close();
fis.close();
}
}
2.注意
A mapped byte buffer and the file mapping that it represents remain valid until the buffer itself is garbage-collected.
The content of a mapped byte buffer can change at any time, for example if the content of the corresponding region of the mapped file
is changed by this program or another. Whether or not such changes occur, and when they occur, is operating-system dependent and
therefore unspecified.
All or part of a mapped byte buffer may become inaccessible at any time, for example if the mapped file is truncated. An attempt to access an
inaccessible region of a mapped byte buffer will not change the buffer's content and will cause an unspecified exception to be thrown either at
the time of the access or at some later time. It is therefore strongly recommended that appropriate precautions be taken to avoid the manipulation
of a mapped file by this program, or by a concurrently running program, except to read or write the file's content.
Mapped byte buffers otherwise behave no differently than ordinary direct byte buffers.
解决：
AccessController.doPrivileged(newPrivilegedAction() {
publicObject run() {
try{
Method getCleanerMethod = buffer.getClass().getMethod("cleaner",newClass[0]);
getCleanerMethod.setAccessible(true);
sun.misc.Cleaner cleaner = (sun.misc.Cleaner)
getCleanerMethod.invoke(byteBuffer,newObject[0]);
cleaner.clean();
} catch(Exception e) {
e.printStackTrace();
}
returnnull;
}
});
参考：