前言
由于要实现go服务端与嵌入式设备的通信打通,综合利弊选择golang与c++的grpc通信方式实现,GRPC由于原生不支持c语言(不可直接生成c的服务,但是可以生成序列化反序列化的代码,使用protoc-c),所以选用原生支持的c++,生成c++端的grpc序列化相关代码,grpc服务相关代码,至于grpc相关代码,若感兴趣可以试着自行尝试,但并不建议用在项目中,因为每次增加服务或者改变代码时,这部分都得自行做适配,且易出bug。
示例来源于官方代码的golang部分以及c++部分,实现现在的golang to c++、c++ to golang,至于前置的golang to golang可以看上两篇博文。
proto文件
syntax = "proto3";
/*
* 这里有一个坑,如果 option go_package="./;golang"; 没有设置,会出现下面的报错
* Please specify either:
• a "go_package" option in the .proto source file, or
• a "M" argument on the command line.
*/
option go_package="./;protofile";
option java_package = "ex.grpc";
package protofile;
message Req {
string message = 1;
}
message Res {
string message = 1;
}
service HelloGRPC {
rpc SayHi(Req) returns (Res);
}
这个proto文件非常简单,定义了两个message,分别为Req(请求message)和Res(响应message),定义了一个service,service虽然只定义了一个服务函数SayHi,但是可以有多个不同的服务函数,而且service也可以同时存在多个,自然而然的message也是支持多个。
当message中存在多个元素时,通过等号后边的编号作区分,由于grpc得到的是二进制流,而且不像json那样是key:value对,解析时水到渠成,所以要通过给编号,在解析时通过编号做出区分,如果说json类似于map(golang)\dict(python),那么这种按照编号定位的则类似于slice(golang)\list(python),但是编号是从1开始,1到15为高频,后面为低频。
具体详细含义看上两篇博文,这里不赘述。
golang端
通过sh脚本(windows可以采用bst脚本)将命令写进去,可以快速进行编译。
sh脚本
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative ./test.proto
protoc是官方的工具,如果想生成c代码可以使用protoc-c,但是无法提供grpc服务相关代码,只能生成序列化与反序列化代码。
通用参数–go_ou 与 --go-grpc_opt
主要的两个参数为 plugins 和 paths ,代表 生成 go 代码所使用的插件 和 生成的 go 代码的目录怎样架构。
paths 参数有两个选项,import和 source_relative。默认为 import ,代表按照生成的 go 代码的包的全路径去创建目录层级,source_relative 代表按照 proto 源文件的目录层级去创建 go 代码的目录层级,如果目录已存在则不用创建。
–go_out与–go_opt
–go_out指定根据proto文件生成的后缀为.pb.go
文件(例如test.pb.go)存放的位置,命令中给.
,代表生成的文件就存放在本文件目录内。
–go_opt可以看做是对命令--go_out
的一个参数设置,这里设置paths=source_relative
。
这里的后缀为.pb.go
文件就是序列化与反序列化相关的代码,如果使用protoc-c生成c代码,就只能生成这个部分的c/h代码,而不能生成下面的部分
–go-grpc_out 与 --go-grpc_opt
–go-grpc_out指定根据proto文件生成的后缀为.pb.go
的grpc服务相关文件例test_grpc.pb.go
存放的位置,命令中给.
,代表生成的文件就存放在本文件目录内。
–go-grpc_opt可以看做是对命令--go-grpc_out
的一个参数设置,这里设置paths=source_relative
。
go_server
golang服务端文件定义:
package main
import (
"GRPC/protofile"
"context"
"fmt"
"google.golang.org/grpc"
"net"
)
type server struct {
protofile.UnimplementedHelloGRPCServer
}
func (s *server) SayHi(ctx context.Context, req *protofile.Req) (res *protofile.Res, err error){
fmt.Println(req.GetMessage())
return &protofile.Res{Message: "服务端响应"}, nil
}
func main(){
listen, _ := net.Listen("tcp", ":8000")
s := grpc.NewServer()
protofile.RegisterHelloGRPCServer(s, &server{})
s.Serve(listen)
}
这里重点关注的几个点为:
1、 proto文件中定义的service会在生成的文件test_grpc.pb.go中做出struct定义:
Unimplemented+service名字(这里是HelloGRPC
)+Server
示例:
// UnimplementedHelloGRPCServer must be embedded to have forward compatible implementations.
type UnimplementedHelloGRPCServer struct {
}
然后在我们的服务端代码内,要定义一个struct,匿名继承这个struct,例如
type server struct {
protofile.UnimplementedHelloGRPCServer
}
2、proto文件中定义的service内部服务函数会在生成的文件test_grpc.pb.go中做出函数定义:
func (UnimplementedHelloGRPCServer) SayHi(context.Context, *Req) (*Res, error) {
return nil, status.Errorf(codes.Unimplemented, "method SayHi not implemented")
}
其实就是为上面提到的struct挂载方法,但是可以发现这个函数只是在初始阶段,即在服务里直接调用这个函数,也会正确返回,但是是初始值,而我们要在这里实现具体的服务,所以也要进行重写。
在服务端代码里做出重写:
func (s *server) SayHi(ctx context.Context, req *protofile.Req) (res *protofile.Res, err error){
fmt.Println(req.GetMessage())
return &protofile.Res{Message: "服务端响应"}, nil
}
这里只做了简单的示例,真正开发时可以根据实际需要实现更复杂功能
3、 启动服务
利用net包,监听目标端口,协议设定为tcp。
listen, _ := net.Listen("tcp", ":8000")
新建grpc服务
s := grpc.NewServer()
初始化grpc服务
protofile.RegisterHelloGRPCServer(s, &server{})
启动服务
s.Serve(listen)
到这里就完成了golang端的grpc服务,可以发现,调用的几乎都是生成的test_grpc.pb.go内部函数,而test.pb.go中的序列化反序列化函数,都是在test_grpc.pb.go内部完成调用的,所以如果使用protoc-c生成代码时,test_grpc.pb.go内部的所有代码,都需要手动完成,目前代码量大概在100行左右,后期功能增加,功能变复杂时,其工作量可想而知,所以建议使用c++,因为这部分已经由protoc自动生成。
golang客户端文件定义:
package main
import (
"GRPC/protofile"
"context"
"fmt"
"google.golang.org/grpc"
)
func main(){
conn,_ := grpc.Dial("localhost:8000", grpc.WithInsecure())
defer conn.Close() // 不这样做会一直无法关闭
client := protofile.NewHelloGRPCClient(conn)
req, _ := client.SayHi(context.Background(), &protofile.Req{Message: "客户端消息"})
fmt.Println(req.GetMessage())
}
可以发现,所有代码目前都放在了main函数内,因为真的是太简单了,但是不排除后期随着功能复杂时,结构会有改变,但是原理是相同的。
这里同样有需要重点关注的几个点:
1、grpc拨号
```go
conn,_ := grpc.Dial("localhost:8000", grpc.WithInsecure())
这里因为做示例,所以暂时使用不安全模式grpc.WithInsecure()
2、链接关闭
defer conn.Close()// 不这样做会一直无法关闭
3、初始化client
client := protofile.NewHelloGRPCClient(conn)
4、调用远端服务函数
req, _ := client.SayHi(context.Background(), &protofile.Req{Message: "客户端消息"})
这里虽然不论感觉上还是效果上都和直接调用远端函数是一样的,但是实质上也是告诉远端要调用哪个函数,给这个函数哪些参数,然后数据压缩,最后远端接受到数据解析执行后,再告诉远端执行的是哪个函数,返回了怎样的响应,然后数据压缩上报,服务端接收后解析得到数据结果。
只不过这个过程都在grpc内部完成了。
到此完成了golang端的服务端与客户端其执行结果可以看上两篇博文。
c++端
项目目录
c++端利用cmake得到makefile,再由makefile编译成可执行文件。
项目目录:
.
├── CMakeLists.txt
├── common.cmake
├── greeter_client.cc
├── greeter_server.cc
└── protos
└── protofile.proto
这里的protofile.proto内容和golang端的proto文件是相同的,在两端的proto必须确保完全相同!!!
,不然可能会造成调用时解析数据错乱出现异常不到的错误。
c++端主要参照了官方cmake进行的,common.cmake由官方项目得来,具体含义未研究,但不可缺少,CMakeLists.txt由官方项目中得来并根据实际情况进行改变。
greeter_client.cc与reeter_server.cc也根据官方项目和实际proto文件改变得到,重点关注内部创建grpc服务怎么实现。
cmake文件
protofile.proto
定义:
# Copyright 2018 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# cmake build file for C++ route_guide example.
# Assumes protobuf and gRPC have been installed using cmake.
# See cmake_externalproject/CMakeLists.txt for all-in-one cmake build
# that automatically builds all the dependencies before building route_guide.
cmake_minimum_required(VERSION 3.5.1)
set (CMAKE_CXX_STANDARD 11)
if(MSVC)
add_definitions(-D_WIN32_WINNT=0x600)
endif()
find_package(Threads REQUIRED)
if(GRPC_AS_SUBMODULE)
# One way to build a projects that uses gRPC is to just include the
# entire gRPC project tree via "add_subdirectory".
# This approach is very simple to use, but the are some potential
# disadvantages:
# * it includes gRPC's CMakeLists.txt directly into your build script
# without and that can make gRPC's internal setting interfere with your
# own build.
# * depending on what's installed on your system, the contents of submodules
# in gRPC's third_party/* might need to be available (and there might be
# additional prerequisites required to build them). Consider using
# the gRPC_*_PROVIDER options to fine-tune the expected behavior.
#
# A more robust approach to add dependency on gRPC is using
# cmake's ExternalProject_Add (see cmake_externalproject/CMakeLists.txt).
# Include the gRPC's cmake build (normally grpc source code would live
# in a git submodule called "third_party/grpc", but this example lives in
# the same repository as gRPC sources, so we just look a few directories up)
add_subdirectory(../../.. ${CMAKE_CURRENT_BINARY_DIR}/grpc EXCLUDE_FROM_ALL)
message(STATUS "Using gRPC via add_subdirectory.")
# After using add_subdirectory, we can now use the grpc targets directly from
# this build.
set(_PROTOBUF_LIBPROTOBUF libprotobuf)
set(_REFLECTION grpc++_reflection)
if(CMAKE_CROSSCOMPILING)
find_program(_PROTOBUF_PROTOC protoc)
else()
set(_PROTOBUF_PROTOC $<TARGET_FILE:protobuf::protoc>)
endif()
set(_GRPC_GRPCPP grpc++)
if(CMAKE_CROSSCOMPILING)
find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin)
else()
set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:grpc_cpp_plugin>)
endif()
elseif(GRPC_FETCHCONTENT)
# Another way is to use CMake's FetchContent module to clone gRPC at
# configure time. This makes gRPC's source code available to your project,
# similar to a git submodule.
message(STATUS "Using gRPC via add_subdirectory (FetchContent).")
include(FetchContent)
FetchContent_Declare(
grpc
GIT_REPOSITORY https://github.com/grpc/grpc.git
# when using gRPC, you will actually set this to an existing tag, such as
# v1.25.0, v1.26.0 etc..
# For the purpose of testing, we override the tag used to the commit
# that's currently under test.
GIT_TAG vGRPC_TAG_VERSION_OF_YOUR_CHOICE)
FetchContent_MakeAvailable(grpc)
# Since FetchContent uses add_subdirectory under the hood, we can use
# the grpc targets directly from this build.
set(_PROTOBUF_LIBPROTOBUF libprotobuf)
set(_REFLECTION grpc++_reflection)
set(_PROTOBUF_PROTOC $<TARGET_FILE:protoc>)
set(_GRPC_GRPCPP grpc++)
if(CMAKE_CROSSCOMPILING)
find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin)
else()
set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:grpc_cpp_plugin>)
endif()
else()
# This branch assumes that gRPC and all its dependencies are already installed
# on this system, so they can be located by find_package().
# Find Protobuf installation
# Looks for protobuf-config.cmake file installed by Protobuf's cmake installation.
set(protobuf_MODULE_COMPATIBLE TRUE)
find_package(Protobuf CONFIG REQUIRED)
message(STATUS "Using protobuf ${Protobuf_VERSION}")
set(_PROTOBUF_LIBPROTOBUF protobuf::libprotobuf)
set(_REFLECTION gRPC::grpc++_reflection)
if(CMAKE_CROSSCOMPILING)
find_program(_PROTOBUF_PROTOC protoc)
else()
set(_PROTOBUF_PROTOC $<TARGET_FILE:protobuf::protoc>)
endif()
# Find gRPC installation
# Looks for gRPCConfig.cmake file installed by gRPC's cmake installation.
find_package(gRPC CONFIG REQUIRED)
message(STATUS "Using gRPC ${gRPC_VERSION}")
set(_GRPC_GRPCPP gRPC::grpc++)
if(CMAKE_CROSSCOMPILING)
find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin)
else()
set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:gRPC::grpc_cpp_plugin>)
endif()
endif()
这个文件直接复制就可以,不需要改变,感兴趣可以自行研究。
protofile.proto
# Copyright 2018 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# cmake build file for C++ helloworld example.
# Assumes protobuf and gRPC have been installed using cmake.
# See cmake_externalproject/CMakeLists.txt for all-in-one cmake build
# that automatically builds all the dependencies before building helloworld.
cmake_minimum_required(VERSION 3.5.1)
project(Protofile C CXX)
include(./common.cmake)
# Proto file
get_filename_component(ty_proto "protos/protofile.proto" ABSOLUTE)
get_filename_component(ty_proto_path "${ty_proto}" PATH)
# Generated sources
set(ty_proto_srcs "${CMAKE_CURRENT_BINARY_DIR}/protofile.pb.cc")
set(ty_proto_hdrs "${CMAKE_CURRENT_BINARY_DIR}/protofile.pb.h")
set(ty_grpc_srcs "${CMAKE_CURRENT_BINARY_DIR}/protofile.grpc.pb.cc")
set(ty_grpc_hdrs "${CMAKE_CURRENT_BINARY_DIR}/protofile.grpc.pb.h")
add_custom_command(
OUTPUT "${ty_proto_srcs}" "${ty_proto_hdrs}" "${ty_grpc_srcs}" "${ty_grpc_hdrs}"
COMMAND ${_PROTOBUF_PROTOC}
ARGS --grpc_out "${CMAKE_CURRENT_BINARY_DIR}"
--cpp_out "${CMAKE_CURRENT_BINARY_DIR}"
-I "${ty_proto_path}"
--plugin=protoc-gen-grpc="${_GRPC_CPP_PLUGIN_EXECUTABLE}"
"${ty_proto}"
DEPENDS "${ty_proto}")
# Include generated *.pb.h files
include_directories("${CMAKE_CURRENT_BINARY_DIR}")
# ty_grpc_proto
add_library(ty_grpc_proto
${ty_grpc_srcs}
${ty_grpc_hdrs}
${ty_proto_srcs}
${ty_proto_hdrs})
target_link_libraries(ty_grpc_proto
${_REFLECTION}
${_GRPC_GRPCPP}
${_PROTOBUF_LIBPROTOBUF})
# Targets greeter_[async_](client|server)
foreach(_target
greeter_client greeter_server )
add_executable(${_target} "${_target}.cc")
target_link_libraries(${_target}
ty_grpc_proto
${_REFLECTION}
${_GRPC_GRPCPP}
${_PROTOBUF_LIBPROTOBUF})
endforeach()
本人对cmake不是太了解,就不好详细叙述了,但是仔细对照下目录文件和代码,我觉得几乎所有人都是可以抄出来自己的cmake文件的。
编译项目
使用cmake以下命令构建示例:
$ mkdir -p cmake/build
$ pushd cmake/build
$ cmake -DCMAKE_PREFIX_PATH=$MY_INSTALL_DIR ../..
$ make -j
$MY_INSTALL_DIR替换成自己grpc安装目录,例如我的安状在/usr/local/gRPC,则这里的命令为:
$ cmake -DCMAKE_PREFIX_PATH=$MY_INSTALL_DIR ../..
greeter_server.cc
文件定义:
/*
*
* Copyright 2015 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
#include <iostream>
#include <memory>
#include <string>
#include <grpcpp/ext/proto_server_reflection_plugin.h>
#include <grpcpp/grpcpp.h>
#include <grpcpp/health_check_service_interface.h>
#ifdef BAZEL_BUILD
#include "examples/protos/protofile.grpc.pb.h"
#else
#include "protofile.grpc.pb.h"
#endif
using grpc::Server;
using grpc::ServerBuilder;
using grpc::ServerContext;
using grpc::Status;
using protofile::Req; // 请求
using protofile::Res; // 响应
using protofile::HelloGRPC; // 服务
// Logic and data behind the server's behavior.
class HelloGRPCServiceImpl final : public HelloGRPC::Service {
Status SayHi(ServerContext* context, const Req* request,
Res* reply) {
std::string prefix("Hello ");
reply->set_message(prefix + request->message());
return Status::OK;
}
};
void RunServer() {
std::string server_address("0.0.0.0:8000");
HelloGRPCServiceImpl service;
grpc::EnableDefaultHealthCheckService(true);
grpc::reflection::InitProtoReflectionServerBuilderPlugin();
ServerBuilder builder;
// Listen on the given address without any authentication mechanism.
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
// Register "service" as the instance through which we'll communicate with
// clients. In this case it corresponds to an *synchronous* service.
builder.RegisterService(&service);
// Finally assemble the server.
std::unique_ptr<Server> server(builder.BuildAndStart());
std::cout << "Server listening on " << server_address << std::endl;
// Wait for the server to shutdown. Note that some other thread must be
// responsible for shutting down the server for this call to ever return.
server->Wait();
}
int main(int argc, char** argv) {
RunServer();
return 0;
}
greeter_client.cc
文件定义:
/*
*
* Copyright 2015 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
#include <iostream>
#include <memory>
#include <string>
#include <grpcpp/grpcpp.h>
#ifdef BAZEL_BUILD
#include "examples/protos/protofile.grpc.pb.h"
#else
#include "protofile.grpc.pb.h"
#endif
using grpc::Channel;
using grpc::ClientContext;
using grpc::Status;
using protofile::Req; // 请求
using protofile::Res; // 响应
using protofile::HelloGRPC; // 服务
class HelloGRPCClient {
public:
HelloGRPCClient(std::shared_ptr<Channel> channel)
: stub_(HelloGRPC::NewStub(channel)) {}
// Assembles the client's payload, sends it and presents the response back
// from the server.
std::string SayHi(const std::string& user) {
// Data we are sending to the server.
Req request;
request.set_message(user);
// Container for the data we expect from the server.
Res reply;
// Context for the client. It could be used to convey extra information to
// the server and/or tweak certain RPC behaviors.
ClientContext context;
// The actual RPC.
Status status = stub_->SayHi(&context, request, &reply);
// Act upon its status.
if (status.ok()) {
return reply.message();
} else {
std::cout << status.error_code() << ": " << status.error_message()
<< std::endl;
return "RPC failed";
}
}
private:
std::unique_ptr<HelloGRPC::Stub> stub_;
};
int main(int argc, char** argv) {
// Instantiate the client. It requires a channel, out of which the actual RPCs
// are created. This channel models a connection to an endpoint specified by
// the argument "--target=" which is the only expected argument.
// We indicate that the channel isn't authenticated (use of
// InsecureChannelCredentials()).
std::string target_str;
std::string arg_str("--target");
if (argc > 1) {
std::string arg_val = argv[1];
size_t start_pos = arg_val.find(arg_str);
if (start_pos != std::string::npos) {
start_pos += arg_str.size();
if (arg_val[start_pos] == '=') {
target_str = arg_val.substr(start_pos + 1);
} else {
std::cout << "The only correct argument syntax is --target="
<< std::endl;
return 0;
}
} else {
std::cout << "The only acceptable argument is --target=" << std::endl;
return 0;
}
} else {
target_str = "localhost:8000";
}
HelloGRPCClient greeter(
grpc::CreateChannel(target_str, grpc::InsecureChannelCredentials()));
std::string user("world");
std::string reply = greeter.SayHi(user);
std::cout << "Greeter received: " << reply << std::endl;
return 0;
}
greeter_server.cc 与 greeter_client.cc
c++不是太了解,但是流程与golang是相同的,都是_grpc.pb
文件内根据proto文件生成的grpc服务函数的各种struct(golang中为struct,进行struct继承,挂载方法)与class(c++中为class,进行class继承,class函数重写),的重写过程,然后调用从proto文件生成的_grpc.pb
文件中的服务。
个人比较习惯c,c++有些抵触,而且由于使用受体是c语言开发,所以计划是采用c++对外开放指针函数,外界在实现具体服务函数,然后注册到c++对应开放的指针函数。
具体效果就不演示了,但是这个已经是调通的示例了,拿过去可以直接运行。
至于golang与c++的配置gRPC环境过程,参考上两篇gRPC文章。