NIO
non-blockuing io非阻塞IO
三大组件
Channel
channel有一点类似stream，他就是读写数据的双向通道，可以从channel将数据读入buffer中，也可以将buffer的数据写入channel中，而之前的stream要么是输入，要么是输出，channel比stream更为底层
常用的Channel有
FileChannelDatagramChannelSocketChannelServerSocketChannel
Buffer
用来缓冲读写数据，常用的buffer有
-
ByteBufferMapperedByteBufferDirectByteBufferHeapByteBuffer
Selector
多线程版设计
一个线程对应一个socket连接服务
多线程版缺点
- 内存占用高
- 线程上下文切换成本高
- 只适合连接数少的场景
线程池版设计
线程池版缺点
- 阻塞模式下，线程仅能处理一个socket连接
- 仅适合短连接场景
Selector版设计
selector的作用就是配合一个线程来管理多个channel，获取这些channel上发生的事件。这些channel工作在非阻塞模式下，不会让线程吊死在一个channel上。适合连接数特别多，但流量低的场景
调用selector的select()会阻塞直到channel发生了读写就绪事件，这些事件发生，select方法机会返回这些事件并交给thread来处理
DEMO
-
导入依赖
-
<dependency> <groupId>io.netty</groupId> <artifactId>netty-all</artifactId> <version>4.1.66.Final</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <version>1.18.20</version> <scope>provided</scope> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.76</version> </dependency> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>30.1.1-jre</version> </dependency> <dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-classic</artifactId> <version>1.2.4</version> </dependency>
-
-
创建一个只包含asc码的文件
-
代码演示
-
@Slf4j public class NettyTest { public static void main(String[] args) { // 通过FileInputStream获取Channel对象 try (FileChannel channel = new FileInputStream("test.txt").getChannel()) { // 获取ByteBuffer缓存对象 ByteBuffer byteBuffer = ByteBuffer.allocate(10); // 读取直到文件全部被读取完毕 while (channel.read(byteBuffer) != -1) { // 切换至读模式 byteBuffer.flip(); while (byteBuffer.hasRemaining()) { // 获取一个byte字节 final char c = (char) byteBuffer.get(); log.debug("读取字节：{}", c); } // 切换为写模式 byteBuffer.clear(); } } catch (IOException e) {} } }
-
ByteBuffer的一般使用步骤
- 向buffer写入数据，例如调用channel.raed(buffer)
- 调用flip()切换至读模式
- 从buffer读取数据，例如调用buffer.get()
- 调用clear()或compact()切换至写模式
- 重复1~4步骤
ByteBuffer结构
ByteBuffer有以下重要的属性
- capacity
- position
- limit
- 初始状态
- 写模式下，position是写入位置，limit等于容量
- flip动作发生后，position切换到读取位置，limit切换成读取限制
读取4个字节之后
- 执行clear(切换成写模式，清空缓存中的所有数据)操作之后
- compact方法，是把未读完的部分向前压缩，然后切换成写模式
测试工具类ByteBufferUtil
import java.nio.ByteBuffer;
import io.netty.util.internal.MathUtil;
import io.netty.util.internal.StringUtil;
/**
* netty工具类
*
* @author wuhunyu
* @version 1.0
* @date 2021-07-25 17:57
*/
public class ByteBufferUtil {
private static final char[] BYTE2CHAR = new char[256];
private static final char[] HEXDUMP_TABLE = new char[256 * 4];
private static final String[] HEXPADDING = new String[16];
private static final String[] HEXDUMP_ROWPREFIXES = new String[65536 >>> 4];
private static final String[] BYTE2HEX = new String[256];
private static final String[] BYTEPADDING = new String[16];
static {
final char[] DIGITS = "0123456789abcdef".toCharArray();
for (int i = 0; i < 256; i++) {
HEXDUMP_TABLE[i << 1] = DIGITS[i >>> 4 & 0x0F];
HEXDUMP_TABLE[(i << 1) + 1] = DIGITS[i & 0x0F];
}
int i;
// Generate the lookup table for hex dump paddings
for (i = 0; i < HEXPADDING.length; i++) {
int padding = HEXPADDING.length - i;
StringBuilder buf = new StringBuilder(padding * 3);
for (int j = 0; j < padding; j++) {
buf.append(" ");
}
HEXPADDING[i] = buf.toString();
}
// Generate the lookup table for the start-offset header in each row (up to 64KiB).
for (i = 0; i < HEXDUMP_ROWPREFIXES.length; i++) {
StringBuilder buf = new StringBuilder(12);
buf.append(StringUtil.NEWLINE);
buf.append(Long.toHexString(i << 4 & 0xFFFFFFFFL | 0x100000000L));
buf.setCharAt(buf.length() - 9, '|');
buf.append('|');
HEXDUMP_ROWPREFIXES[i] = buf.toString();
}
// Generate the lookup table for byte-to-hex-dump conversion
for (i = 0; i < BYTE2HEX.length; i++) {
BYTE2HEX[i] = ' ' + StringUtil.byteToHexStringPadded(i);
}
// Generate the lookup table for byte dump paddings
for (i = 0; i < BYTEPADDING.length; i++) {
int padding = BYTEPADDING.length - i;
StringBuilder buf = new StringBuilder(padding);
for (int j = 0; j < padding; j++) {
buf.append(' ');
}
BYTEPADDING[i] = buf.toString();
}
// Generate the lookup table for byte-to-char conversion
for (i = 0; i < BYTE2CHAR.length; i++) {
if (i <= 0x1f || i >= 0x7f) {
BYTE2CHAR[i] = '.';
} else {
BYTE2CHAR[i] = (char) i;
}
}
}
/**
* 打印所有内容
*
* @param buffer
*/
public static void debugAll(ByteBuffer buffer) {
int oldlimit = buffer.limit();
buffer.limit(buffer.capacity());
StringBuilder origin = new StringBuilder(256);
appendPrettyHexDump(origin, buffer, 0, buffer.capacity());
System.out.println("+--------+-------------------- all ------------------------+----------------+");
System.out.printf("position: [%d], limit: [%d]\n", buffer.position(), oldlimit);
System.out.println(origin);
buffer.limit(oldlimit);
}
/**
* 打印可读取内容
*
* @param buffer
*/
public static void debugRead(ByteBuffer buffer) {
StringBuilder builder = new StringBuilder(256);
appendPrettyHexDump(builder, buffer, buffer.position(), buffer.limit() - buffer.position());
System.out.println("+--------+-------------------- read -----------------------+----------------+");
System.out.printf("position: [%d], limit: [%d]\n", buffer.position(), buffer.limit());
System.out.println(builder);
}
private static void appendPrettyHexDump(StringBuilder dump, ByteBuffer buf, int offset, int length) {
if (MathUtil.isOutOfBounds(offset, length, buf.capacity())) {
throw new IndexOutOfBoundsException(
"expected: " + "0 <= offset(" + offset + ") <= offset + length(" + length
+ ") <= " + "buf.capacity(" + buf.capacity() + ')');
}
if (length == 0) {
return;
}
dump.append(
" +-------------------------------------------------+" +
StringUtil.NEWLINE + " | 0 1 2 3 4 5 6 7 8 9 a b c d e f |" +
StringUtil.NEWLINE + "+--------+-------------------------------------------------+----------------+");
final int startIndex = offset;
final int fullRows = length >>> 4;
final int remainder = length & 0xF;
// Dump the rows which have 16 bytes.
for (int row = 0; row < fullRows; row++) {
int rowStartIndex = (row << 4) + startIndex;
// Per-row prefix.
appendHexDumpRowPrefix(dump, row, rowStartIndex);
// Hex dump
int rowEndIndex = rowStartIndex + 16;
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2HEX[getUnsignedByte(buf, j)]);
}
dump.append(" |");
// ASCII dump
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2CHAR[getUnsignedByte(buf, j)]);
}
dump.append('|');
}
// Dump the last row which has less than 16 bytes.
if (remainder != 0) {
int rowStartIndex = (fullRows << 4) + startIndex;
appendHexDumpRowPrefix(dump, fullRows, rowStartIndex);
// Hex dump
int rowEndIndex = rowStartIndex + remainder;
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2HEX[getUnsignedByte(buf, j)]);
}
dump.append(HEXPADDING[remainder]);
dump.append(" |");
// Ascii dump
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2CHAR[getUnsignedByte(buf, j)]);
}
dump.append(BYTEPADDING[remainder]);
dump.append('|');
}
dump.append(StringUtil.NEWLINE +
"+--------+-------------------------------------------------+----------------+");
}
private static void appendHexDumpRowPrefix(StringBuilder dump, int row, int rowStartIndex) {
if (row < HEXDUMP_ROWPREFIXES.length) {
dump.append(HEXDUMP_ROWPREFIXES[row]);
} else {
dump.append(StringUtil.NEWLINE);
dump.append(Long.toHexString(rowStartIndex & 0xFFFFFFFFL | 0x100000000L));
dump.setCharAt(dump.length() - 9, '|');
dump.append('|');
}
}
public static short getUnsignedByte(ByteBuffer buffer, int index) {
return (short) (buffer.get(index) & 0xFF);
}
}
测试类NettyTest
@Slf4j
public class NettyTest {
public static void main(String[] args) {
// 构建一个容量为10的缓冲区
final ByteBuffer byteBuffer = ByteBuffer.allocate(10);
// 写入'A'
byteBuffer.put((byte) 0x41);
// 打印整个缓冲数组，下同
ByteBufferUtil.debugAll(byteBuffer);
// 写入'B', 'C', 'D'
byteBuffer.put(new byte[]{0x42, 0x43, 0x44});
ByteBufferUtil.debugAll(byteBuffer);
// 将position指针移到最开始的位置
byteBuffer.flip();
// 获取缓冲区position指向的字节，position指针向后移一位
log.debug("{}", byteBuffer.get());
ByteBufferUtil.debugAll(byteBuffer);
// 保留未读字节，position指针移到至未读字节的后一位
byteBuffer.compact();
ByteBufferUtil.debugAll(byteBuffer);
// 再次写入'E', 'F'
byteBuffer.put(new byte[]{0x45, 0x46});
ByteBufferUtil.debugAll(byteBuffer);
// 重置整个缓冲区的指针
byteBuffer.clear();
ByteBufferUtil.debugAll(byteBuffer);
}
}
打印结果
+--------+-------------------- all ------------------------+----------------+
position: [1], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 00 00 00 00 00 00 00 00 00 |A......... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [4], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
2021-07-25 20:21:34.676 main DEBUG com.wuhunyu.Test.NettyTest - 65
+--------+-------------------- all ------------------------+----------------+
position: [1], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [3], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 42 43 44 44 00 00 00 00 00 00 |BCDD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [5], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 42 43 44 45 46 00 00 00 00 00 |BCDEF..... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 42 43 44 45 46 00 00 00 00 00 |BCDEF..... |
+--------+-------------------------------------------------+----------------+
ByteBuffer常见方法
HeapByteBuffer |
DirectByteBuffer |
|---|---|
| java堆内存 | 直接内存 |
| 读写效率较低，受到GC影响 | 读写效率高，不会受到GC影响，但分配效率低 |
向buffer中写入数据
- 调用channel的read方法
- 调用buffer的out方法
从buffer读取数据
- 调用channel的write方法
- 调用buffer的get方法
get()方法会让position读指针向后走，如果向重复读取数据
- 可以调用rewind方法将position重新置为0
- 或者调用get(int index)方法获取索引i的内容，它不会移动读指针
- mark()可以标记当前position的位置，通过调用reset()方法可以使得position回到之前mark()标记的位置
// rewind()
final ByteBuffer byteBuffer = ByteBuffer.allocate(10);
byteBuffer.put(new byte[]{0x41, 0x42, 0x43, 0x44});
byteBuffer.flip();
ByteBufferUtil.debugAll(byteBuffer);
byteBuffer.get(new byte[4]);
ByteBufferUtil.debugAll(byteBuffer);
byteBuffer.rewind();
ByteBufferUtil.debugAll(byteBuffer);
// 打印结果
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [4], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
// get(int index)
final ByteBuffer byteBuffer = ByteBuffer.allocate(10);
byteBuffer.put(new byte[]{0x41, 0x42, 0x43, 0x44});
byteBuffer.flip();
ByteBufferUtil.debugAll(byteBuffer);
byteBuffer.get(new byte[4]);
ByteBufferUtil.debugAll(byteBuffer);
log.debug("{}", (char) byteBuffer.get(3));
ByteBufferUtil.debugAll(byteBuffer);
// 打印结果
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [4], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
2021-07-25 21:21:19.166 main DEBUG com.wuhunyu.Test.NettyTest - D
+--------+-------------------- all ------------------------+----------------+
position: [4], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
// mark(),reset()
final ByteBuffer byteBuffer = ByteBuffer.allocate(10);
byteBuffer.put(new byte[]{0x41, 0x42, 0x43, 0x44});
byteBuffer.flip();
byteBuffer.mark();
ByteBufferUtil.debugAll(byteBuffer);
byteBuffer.get(new byte[4]);
ByteBufferUtil.debugAll(byteBuffer);
byteBuffer.reset();
ByteBufferUtil.debugAll(byteBuffer);
// 打印结果
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [4], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [4]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 41 42 43 44 00 00 00 00 00 00 |ABCD...... |
+--------+-------------------------------------------------+----------------+