本文主要讲解Thrift的序列化机制, 看看thrift作为数据交换格式是如何工作的?
1.构造应用场景:
1). 首先我们先来定义下thrift的简单结构.
1
2
3
4
5
namespace java com.yangyang.thrift.api
struct Pair {
1: required string key
2: required string value
}
required修饰符你肯定能猜测到它的意义, 但是你是否有没有这样的疑惑, “1”, “2” 这些数字标识符究竟有何含义? 它在序列化机制中究竟扮演什么样的角色?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
private static String datafile = "1.dat";
// *) 把对象写入文件
public static void writeData() throws IOException, TException {
Pair pair = new Pair();
pair.setKey("key1").setValue("value1");
FileOutputStream fos = new FileOutputStream(new File(datafile));
pair.write(new TBinaryProtocol(new TIOStreamTransport(fos)));
fos.close();
}
// *) 从文件恢复对象
public static void readData() throws TException, IOException {
FileInputStream fis = new FileInputStream(new File(datafile));
Pair pair = new Pair();
pair.read(new TBinaryProtocol(new TIOStreamTransport(fis)));
System.out.println("key => " + pair.getKey());
System.out.println("value => " + pair.getValue());
fis.close();
}
public static void main(String[] args) throws Exception{
//writeData();
readData();
}
调用writeData(), 把pair{key=> key1, value=> value1} 写入文件1.dat中
1
2
3
4
5
namespace java com.yangyang.thrift.api
struct Pair {
2: required string key
1: required string value
}
评注: 这边2对应key, 1对应value.
2.thrift 数据格式描述
官网文档描述: http://thrift.apache.org/static/files/thrift-20070401.pdf
1
Versioning in Thrift is implemented via field identifiers. The field header for every member of a struct in Thrift is encoded with a unique field identifier. The combination of this field identifier and its type specifier is used to uniquely identify the field. The Thrift definition language supports automatic assignment of field identifiers, but it is good programming practice to always explicitly specify field identifiers.
翻译: thrift的向后兼容性(Version)借助属性标识(数字编号id + 属性类型type)来实现, 可以理解为在序列化后(属性数据存储由 field 大专栏 Thrift RPC实战(三) thrift序列化揭秘 _name:field_value => id+type:field_value), 这也解释了上述提到的场景的原因了.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
public void read(org.apache.thrift.protocol.TProtocol iprot, Pair struct) throws org.apache.thrift.TException {
org.apache.thrift.protocol.TField schemeField;
//读取结构开始标记
iprot.readStructBegin();
while (true)
{
// 读取Field属性开始标记
schemeField = iprot.readFieldBegin();
if (schemeField.type == org.apache.thrift.protocol.TType.STOP) {
break;
}
// field标记包含 id + type, switch根据(id+type)来分配相关的值
switch (schemeField.id) {
case 2: // KEY
if (schemeField.type == org.apache.thrift.protocol.TType.STRING) {
struct.key = iprot.readString();
struct.setKeyIsSet(true);
} else {
org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);
}
break;
case 1: // VALUE
if (schemeField.type == org.apache.thrift.protocol.TType.STRING) {
struct.value = iprot.readString();
struct.setValueIsSet(true);
} else {
org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);
}
break;
default:
org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);
}
// 读取Field属性结束标记
iprot.readFieldEnd();
}
// 读取结构体结束标记
iprot.readStructEnd();
// check for required fields of primitive type, which can't be checked in the validate method
struct.validate();
}
从恢复对象的函数中, 我们也可以对thrift定义的序列化对象有个初步的认识, 庖丁解牛,最终会被细化为readStructBegin, readFieldBegin, read(readString, readI32, readI64), readFieldEnd, readStructEnd的有组织有序调用.
3.数据交换格式分类
当前的数据交换格式可以分为如下几类:
thrift与常见数据交换格式的对比
4.向后兼容实践
Thrift官方文档, 也提到对新增的字段属性, 采用id递增的方式标识并以optional修饰来添加.