netcore实践：跨平台动态加载native组件

2021-12-18 01:54:39

缘起netcore框架下实现基于zmq的应用。

在.net framework时代，我们进行zmq开发由很多的选择，比较常用的有clrzmq4和NetMQ。其中clrzmq是基于libzmq的Interop包装，

NetMQ是100%C#的zmq实现（基于AsyncIO组件）。以上两种组件我都有过应用，孰优孰劣各有千秋，本文就不详谈了。

回归正题，netcore下使用zmq首先也是想到引用上述组件，实践后发现clrzmq暂时没有基于netstandard或者netcore的支持，而NetMQ做的比较好，已经基于

netstandard1.3进行了支持。

一番折腾，搭程序，配环境。。。而后 dotnet run ，在windows下正常呈现了zmq的各项功能。

于是继续dotnet publlish，通过Wnscp拷贝到centos7下执行。立即报错，

挺意外的，本以为NetMQ已经基于netstandard进行了支持，也应该对跨平台进行支持。可事实是AsyncIO的IOControl方法not supported on linux platform/

无奈，网上搜了会，也没找到任何关于netcore在linux下进行zmq的相关内容，事实上也没有看到AsyncIO或者NetMQ有关于跨平台支持的说明。

既然现有方式行不通那就只好自己动手了，自己操刀通过Interop包装libzmq来实现跨平台的zmq应用吧！

首先看下libzmq的组件目录：按x86和x64平台分为i386文件夹和amd64文件夹，且都包含windos下的dll组件和linux下的so组件

这挺难办了，要想做好还得考虑平台类型和操作系统类型，还要想想 netcore里有没有相关API提供。

先是网上搜了圈，也极少有关于netcore进行平台判断相关内容。根据以往在framework下的经验，直接到https://apisof.net搜索相关关键字

：OSPlatform

非常不错，netcore已经提供了，同理搜索了 OSArchitecture 、DllImport 等都发现netcore1.1版本已经实现了相关api （说白了是netstandard已经实现了相关API）

准备就绪，直接开干,首先是平台相关信息判断，并加载目标组件（具体方式请参照如下代码）

   class ZmqNative

    {

        private const string LibraryName = "libzmq";

        const int RTLD_NOW = ; // for dlopen's flags

        const int RTLD_GLOBAL = ;

        [DllImport(@"libdl")]

        static extern IntPtr dlopen(string filename, int flags);

        [DllImport("libdl")]

        protected static extern IntPtr dlsym(IntPtr handle, string symbol);

        [DllImport("kernel32.dll")]

        static extern IntPtr LoadLibrary(string filename);

        private static IntPtr LibPtr = IntPtr.Zero;

        static ZmqNative()

        {

            Console.WriteLine("OSArchitecture:{0}",RuntimeInformation.OSArchitecture);

            try {

                var libPath = @"i386";

                if (RuntimeInformation.OSArchitecture == Architecture.X86)

                {

                    libPath = @"i386";

                }

                else if (RuntimeInformation.OSArchitecture == Architecture.X64)

                {

                    libPath = @"amd64";

                }

                else

                {

                    Console.WriteLine("OSArchitecture not suported!");

                }

                if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))

                {

                    var libName = $"{AppContext.BaseDirectory}\\{libPath}\\{LibraryName}.dll";

                    Console.WriteLine("windows:{0}", libName);

                    LibPtr = LoadLibrary(libName);

                }

                else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))

                {

                    var libName = $"{AppContext.BaseDirectory}/{libPath}/{LibraryName}.so";

                    Console.WriteLine("linux:{0}", libName);

                    LibPtr = dlopen(libName, RTLD_NOW|RTLD_GLOBAL);

                    if(LibPtr!=IntPtr.Zero)

                    {

                        var ptr1 = dlsym(LibPtr, "zmq_ctx_new");

                        context = Marshal.GetDelegateForFunctionPointer<ZmqContext>(ptr1) ;

                        var ptr2 = dlsym(LibPtr, "zmq_socket");

                        socket = Marshal.GetDelegateForFunctionPointer<ZmqSocket>(ptr2);

                        var ptr3 = dlsym(LibPtr, "zmq_connect");

                        connect = Marshal.GetDelegateForFunctionPointer<ZmqConnect>(ptr3);

                    }

    }

                else

                {

                    Console.WriteLine("OSPlatform not suported!");

                }

                if (LibPtr != IntPtr.Zero)

                    Console.WriteLine("load zmqlib success!");

            }

            catch(Exception ex)

            {

                Console.WriteLine("load zmqlib error:\r\n{0}",ex);

            }

        }

        public delegate IntPtr ZmqContext();

        [DllImport(LibraryName, EntryPoint = "zmq_ctx_new", CallingConvention=CallingConvention.Cdecl)]

        public static extern IntPtr zmq_ctx_new();

        public static ZmqContext context = null;

        public delegate IntPtr ZmqSocket(IntPtr context, Int32 type);

        [DllImport(LibraryName, EntryPoint = "zmq_socket", CallingConvention = CallingConvention.Cdecl)]

        public static extern IntPtr zmq_socket(IntPtr context, Int32 type);

        public static ZmqSocket socket = null;

        public delegate Int32 ZmqConnect(IntPtr socket, IntPtr endpoint);

        [DllImport(LibraryName, EntryPoint = "zmq_connect", CallingConvention = CallingConvention.Cdecl)]

        public static extern Int32 zmq_connect(IntPtr socket, IntPtr endpoint);

        public static ZmqConnect connect = null;

        [DllImport(LibraryName, EntryPoint = "zmq_errno", CallingConvention = CallingConvention.Cdecl)]

        public static extern Int32 zmq_errno();

        [DllImport(LibraryName, EntryPoint = "zmq_strerror", CallingConvention = CallingConvention.Cdecl)]

        public static extern IntPtr zmq_strerror(int errnum);

    }

以上为测试代码，请自动忽略代码质量！

简单解释下，如上代码通过平台判断，动态加载组件，采用LoadLibaray的方式。有心的同学可能会发现几个delegate并且在Linux部分内通过dlsym获取了函数指针，具体原因下面会讲。

以上测试代码，在windows平台下同样正常无误，而在linux下还是遇到几个小坑~~容我慢慢道来：

1、通过DllImport进行Interop的时候，组件路径必须是确定的，这就引起了如何动态加载不同目录下组件的问题；

好在windows平台下通过LoadLibaray加载dll到进程空间后，DllImport标记的函数就从进程空间查找，不会重复import组件了。

而同样的原理在linux下用dlopen却不能实现，还是会提示找不到组件

2、初次部署centos7上时，报找不到libdl.so组件问题，主要原因是系统下没有glibc的原因，该问题可以通过yum安装glibc的方式解决;

//先查找系统内是否存在组件

$ sudo find / -name libdl*

//如不存在则安装glibc

# yum install glibc

#安装完毕后进行链接

$ sudo ln -s /usr/lib64/libdl.so. /usr/lib64/libdl

3、解决了libdl组件问题后，继续运行还是会发现报找不到libzmq组件的问题，实际就产生了问题1中所描述的，在linux系统下dlopen后，Interop过的函数并不会从进程空间查找。

为了解决上面遇到的问题，我们还有一条办法，就是创建 delegate ，并且通过LoadLibaray组件后通过GetProcAddress方式获取函数指针了。具体的解决方案在上述测试代码已经体现了，这里就不过多解释了。

以上，就全部解决了在 netcore框架基础上进行跨平台native组件应用的问题。真实测试结果如图：

请主动忽略初zmq应用外的其他信息，本次测试一同测试了通过App入口启动webapi + websockets + zmq ，api创建为aspnetcore在Web.dll内，websockets在Lib.dll内，zmq在App.dll内。

补充下完整测试代码：

using System;

using System.Collections.Generic;

using System.Text;

using System.Runtime.InteropServices;

namespace App

{

    class ZmqNative

    {

        private const string LibraryName = "libzmq";

        const int RTLD_NOW = ; // for dlopen's flags

        const int RTLD_GLOBAL = ;

        [DllImport(@"libdl")]

        static extern IntPtr dlopen(string filename, int flags);

        [DllImport("libdl")]

        protected static extern IntPtr dlsym(IntPtr handle, string symbol);

        [DllImport("kernel32.dll")]

        static extern IntPtr LoadLibrary(string filename);

        private static IntPtr LibPtr = IntPtr.Zero;

        static ZmqNative()

        {

            Console.WriteLine("OSArchitecture:{0}",RuntimeInformation.OSArchitecture);

            try {

                var libPath = @"i386";

                if (RuntimeInformation.OSArchitecture == Architecture.X86)

                {

                    libPath = @"i386";

                }

                else if (RuntimeInformation.OSArchitecture == Architecture.X64)

                {

                    libPath = @"amd64";

                }

                else

                {

                    Console.WriteLine("OSArchitecture not suported!");

                }

                if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))

                {

                    var libName = $"{AppContext.BaseDirectory}\\{libPath}\\{LibraryName}.dll";

                    Console.WriteLine("windows:{0}", libName);

                    LibPtr = LoadLibrary(libName);

                }

                else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))

                {

                    var libName = $"{AppContext.BaseDirectory}/{libPath}/{LibraryName}.so";

                    Console.WriteLine("linux:{0}", libName);

                    LibPtr = dlopen(libName, RTLD_NOW|RTLD_GLOBAL);

                    if(LibPtr!=IntPtr.Zero)

                    {

                        var ptr1 = dlsym(LibPtr, "zmq_ctx_new");

                        context = Marshal.GetDelegateForFunctionPointer<ZmqContext>(ptr1) ;

                        var ptr2 = dlsym(LibPtr, "zmq_socket");

                        socket = Marshal.GetDelegateForFunctionPointer<ZmqSocket>(ptr2);

                        var ptr3 = dlsym(LibPtr, "zmq_connect");

                        connect = Marshal.GetDelegateForFunctionPointer<ZmqConnect>(ptr3);

                    }

    }

                else

                {

                    Console.WriteLine("OSPlatform not suported!");

                }

                if (LibPtr != IntPtr.Zero)

                    Console.WriteLine("load zmqlib success!");

            }

            catch(Exception ex)

            {

                Console.WriteLine("load zmqlib error:\r\n{0}",ex);

            }

        }

        public delegate IntPtr ZmqContext();

        [DllImport(LibraryName, EntryPoint = "zmq_ctx_new", CallingConvention=CallingConvention.Cdecl)]

        public static extern IntPtr zmq_ctx_new();

        public static ZmqContext context = null;

        public delegate IntPtr ZmqSocket(IntPtr context, Int32 type);

        [DllImport(LibraryName, EntryPoint = "zmq_socket", CallingConvention = CallingConvention.Cdecl)]

        public static extern IntPtr zmq_socket(IntPtr context, Int32 type);

        public static ZmqSocket socket = null;

        public delegate Int32 ZmqConnect(IntPtr socket, IntPtr endpoint);

        [DllImport(LibraryName, EntryPoint = "zmq_connect", CallingConvention = CallingConvention.Cdecl)]

        public static extern Int32 zmq_connect(IntPtr socket, IntPtr endpoint);

        public static ZmqConnect connect = null;

        [DllImport(LibraryName, EntryPoint = "zmq_errno", CallingConvention = CallingConvention.Cdecl)]

        public static extern Int32 zmq_errno();

        [DllImport(LibraryName, EntryPoint = "zmq_strerror", CallingConvention = CallingConvention.Cdecl)]

        public static extern IntPtr zmq_strerror(int errnum);

    }

    class ZContext

    {

        private static IntPtr _context = IntPtr.Zero;

        static ZContext()

        {

            if (ZmqNative.context != null)

            {

                _context = ZmqNative.context();

            }

            else

            {

                _context = ZmqNative.zmq_ctx_new();

            }

            if (_context == IntPtr.Zero)

            {

                Console.WriteLine("zerror:{0}", ZError.GetLastError());

            }

        }

        public static IntPtr Current

        {

            get

            {

                return _context;

            }

        }

    }

    class ZError

    {

        public static string GetLastError()

        {

            var error = string.Empty;

            var no = ZmqNative.zmq_errno();

            if (no != )

            {

                var ptr = ZmqNative.zmq_strerror(no);

                error = Marshal.PtrToStringUTF8(ptr);

            }

            return error;

        }

    }

    enum ZmqSocketType :int {

        SUB =

    }

    class ClrZmq

    {

        private IntPtr zmq = IntPtr.Zero;

        public ClrZmq(string url) {

            Url = url;

        }

        public string Url { get; private set; }

        private bool _IsStarted= false;

        public bool IsStarted

        {

            get

            {

                return _IsStarted;

            }

        }

        public void Start() {

            if (IsStarted) return;

            try

            {

                if (ZmqNative.socket != null)

                    zmq = ZmqNative.socket(ZContext.Current, (int)ZmqSocketType.SUB);

                else

                    zmq = ZmqNative.zmq_socket(ZContext.Current, (int)ZmqSocketType.SUB);

                if (zmq == IntPtr.Zero)

                {

                    //error

                    Console.WriteLine("zerror:{0}",ZError.GetLastError());

                    return;

                }

                Console.WriteLine("create mq success!");

                var str =  Marshal.StringToHGlobalAnsi(Url);

                var ret = ;

                if (ZmqNative.connect != null)

                    ret = ZmqNative.connect(zmq, str);

                else

                    ret = ZmqNative.zmq_connect(zmq, str);

                if ( ret!= )

                {

                    //error

                    Console.WriteLine("zerror:{0}",ZError.GetLastError());

                    return;

                }

                Marshal.FreeHGlobal(str);

                Console.WriteLine("connect zmq success!");

            }

            catch (Exception ex) {

                Console.WriteLine(ex);

            }

        }

    }

}

码农公寓

相关文章