.NetCore下结合IdentityServer4构建自己的文件服务管理（UosoOSS）

2022-09-15 10:12:44

Web开发系统文件默认存储在wwwroot目录下面，现在越来越多的系统服务化了，UI也更加多元化，当然文件可以用第三方的文件服务，但是这里准备文件分离出来构建自己的文件服务配合数据库表来实现（UosoOSS）

思路：

1、构建自己的文件夹及文件管理（包括私有权限）这里需要结合IdentityServer4的客户端模式中的一些客户端ID设计到文件管理中去

2、实现一个类似 AliyunOSS的文件管理及客户端类库，帮助实现上传，以及文件地址的生成

3、处理文件的访问权限

首先来实现服务部分，构建自己的API文件服务

配置文件的根目录以及文件访问的根路径

 public class LYMFileServerOptions
    {
        /// <summary>
        /// 文件存储路径
        /// </summary>
        public string PhyicalFilePath { get; set; }
        /// <summary>
        /// 文件访问的路径 必须以"/"开始
        /// </summary>
        public string RequestPath { get; set; }
 
    }

 public static IServiceCollection AddUosoFile(this IServiceCollection services, Action<LYMFileServerOptions> serverOptions)
        {
//LYMFileServerOptions  配置处理


//相关服务处理
       }

接下来就是处理中间件了，这里需要用到UseFileServer，处理相关的路径就可以了

               var fileProvider = new PhysicalFileProvider(ppath);
                var fileServerOptions = new FileServerOptions();
                fileServerOptions.DefaultFilesOptions.DefaultFileNames = new[] { "" };
                fileServerOptions.FileProvider = fileProvider;
                fileServerOptions.RequestPath = options.RequestPath;
                applicationBuilder.UseFileServer(fileServerOptions);

1、接下来我们还需要构建自己的文件上次及文件访问限制设置

 applicationBuilder.UseEndpoints(endpoint =>
                {

                    endpoint.MapPost("/upload", async context =>
                    {
                        var clientId = context.User.Claims.Where(c => c.Type == "client_id").FirstOrDefault()?.Value;
                        var fileRepository = context.RequestServices.GetService<IFileRepository>();

                        #region FormData上传

                        var bucketname = context.Request.Form["bucketname"];
                        if (context.Request.Form.Files == null || context.Request.Form.Files.Count == 0)
                        {
                            context.Response.ContentType = "application/json";
                            await context.Response.WriteAsync(JsonConvert.SerializeObject(new { status = 1, message = "上传文件不存在", filename = "" }));
                            return;
                        }
                        try
                        {
                            #region 验证文件夹
                           //验证文件夹相关逻辑....#endregion
                            var file = context.Request.Form.Files[0];
                            var filename = context.Request.Form["filename"]; //可以没传服务端可以生成，为了客户端自定义的文件名称using (var stream = file.OpenReadStream())
                            {
                                const int FILE_WRITE_SIZE = 84975;
                                var basepath = PlatformServices.Default.Application.ApplicationBasePath;
                                var ppath = Path.Combine(basepath, options.PhyicalFilePath, bucketname);
                                if (!Directory.Exists(ppath))
                                {
                                    Directory.CreateDirectory(ppath);
                                }
                                using (FileStream fileStream = new FileStream(Path.Combine(ppath, filename), FileMode.Create, FileAccess.Write, FileShare.Write, FILE_WRITE_SIZE, true))
                                {
                                    byte[] byteArr = new byte[FILE_WRITE_SIZE];
                                    int readCount = 0;
                                    while ((readCount = await stream.ReadAsync(byteArr, 0, byteArr.Length)) > 0)
                                    {
                                        await fileStream.WriteAsync(byteArr, 0, readCount);
                                      
                                    }
                                }
                            };                        //添加文件数据 ......                           
                            context.Response.ContentType = "application/json";
                            await context.Response.WriteAsync(JsonConvert.SerializeObject(new { status = 0, message = "上传成功", filename = filename }));
                        }
                        catch (Exception ex)
                        {
                            context.Response.ContentType = "application/json";
                            await context.Response.WriteAsync(JsonConvert.SerializeObject(new { status = 1, message = "上传失败", filename = "" }));
                        }
                        #endregion



                    }).RequireAuthorization();
                });

2、获取访问文件的地址，根据自己的规则来获取，这里可以结合IdentityServer4的token来处理，但是有一点不好每次都需IdentityServer4去校验，所以这里还是自定义规则把前面生成到地址栏通过某种算法校验

  applicationBuilder.UseEndpoints(endpoint =>
                {

                    endpoint.MapPost("/getfileurl", async context =>
                    {
                      //根据自己的规则来获取，这里可以结合IdentityServer4的token来处理，但是有一点不好每次都需identityServer去校验，所以这里还是自定义规则 把前面生成到地址栏通过某种算法校验
                        context.Response.ContentType = "application/json";
                        await context.Response.WriteAsync(url);



                    }).RequireAuthorization();
                });

3、匹配文件访问的路由了

     applicationBuilder.UseEndpoints(endpoint =>
                {

                    //{**slug}
                    var builder = endpoint.MapGet(options.RequestPath + "/{filename}.{ext?}", async context =>
                        {
                            object filename = string.Empty;
                            context.Request.RouteValues.TryGetValue("filename", out filename);
                            object ext = string.Empty;
                            context.Request.RouteValues.TryGetValue("ext", out ext);
                            //处理文件夹权限以及文件访问权限算法验签业务逻辑
                           // 成功 await context.Response.SendFileAsync(fileurl);
                           //文件不存在或者 没有权限 可以返回404 401 

                        });
                });

4、服务端的配置基本结束了，接下来我们来处理客户端库的处理，包括文件上传、访问地址获取，文件夹的管理等等，这里需要注意文件上传带参数的坑

.NetCore下结合IdentityServer4构建自己的文件服务管理（UosoOSS）

 public class UosoFileOSS
    {
        private string authurl;
        private string ossserverurl;
        private string clientid;
        private string clientsecret;

        public UosoFileOSS(string _authurl, string _clientid, string _clientsecret, string _ossserverurl)
        {
            this.authurl = _authurl;
            this.clientid = _clientid;
            this.clientsecret = _clientsecret;
            this.ossserverurl = _ossserverurl;
        }

    


        /// <summary>
        /// 文件信息
        /// </summary>
        /// <param name="stream">文件流</param>
        /// <param name="bucketName"></param>
        /// <param name="fileName"></param>
        public async Task<HttpResponseMessage> UosoPutObjectAsync(Stream stream, string bucketName, string fileOldName, string fileName = "")
        {

            HttpMessageHandler httpMessageHandler = new HttpClientHandler
            {
                AutomaticDecompression =
    System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate
            };
            using (var client = new HttpClient(httpMessageHandler))
            {

                var requestoptions = new ClientCredentialsTokenRequest
                {
                    Address = this.authurl + "/connect/token",
                    ClientId = this.clientid,
                    ClientSecret = this.clientsecret,
                    Scope = "fileapi"
                };
                var request =await client.RequestClientCredentialsTokenAsync(requestoptions);
                if (request.IsError)
                {
                    throw new Exception(request.Error);
                }

                var content = new MultipartFormDataContent();
                var fileContent = new StreamContent(stream);
                fileContent.Headers.Add("Content-Disposition", "form-data; name=\"file\"; filename=\"" + fileOldName + "\"");
                content.Add(fileContent);
                var stringContent = new StringContent(bucketName);
                stringContent.Headers.Add("Content-Disposition", "form-data; name=\"bucketname\"");
                content.Add(stringContent);
                var stringContent1 = new StringContent(fileName);
                stringContent1.Headers.Add("Content-Disposition", "form-data; name=\"filename\"");
                content.Add(stringContent1);

                client.SetBearerToken(request.AccessToken);
                var respose =await client.PostAsync(this.ossserverurl + "/upload", content);
                return respose;
            }
        }

        public async Task<Uri> UosoGeneratePresignedUriAsync(UosoGeneratePresignedUriRequest req)
        {



            HttpMessageHandler httpMessageHandler = new HttpClientHandler
            {
                AutomaticDecompression =
   System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate
            };
            using (var client = new HttpClient(httpMessageHandler))
            {

                var requestoptions = new ClientCredentialsTokenRequest
                {
                    Address = this.authurl + "/connect/token",
                    ClientId = this.clientid,
                    ClientSecret = this.clientsecret,
                    Scope = "fileapi"
                };
                var request = await client.RequestClientCredentialsTokenAsync(requestoptions);
                if (request.IsError)
                {
                    throw new Exception(request.Error);
                }

                var dic = new Dictionary<string, string> {
                   { "Expires",req.Expiration.Ticks.ToString()},
                   { "Key",req.key}
               };

                var content = new FormUrlEncodedContent(dic);

                client.SetBearerToken(request.AccessToken);
                var respose =await client.PostAsync(this.ossserverurl + "/getfileurl", content);
                var result =await respose.Content.ReadAsStringAsync();
                return new Uri(result);
            }



        }

        /// <summary>
        /// 文件信息
        /// </summary>
        /// <param name="stream">文件流</param>
        /// <param name="bucketName"></param>
        /// <param name="fileName"></param>
        public HttpResponseMessage UosoPutObject(Stream stream, string bucketName, string fileOldName, string fileName = "")
        {

            HttpMessageHandler httpMessageHandler = new HttpClientHandler
            {
                AutomaticDecompression =
    System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate
            };
            using (var client = new HttpClient(httpMessageHandler))
            {

                var requestoptions = new ClientCredentialsTokenRequest
                {
                    Address = this.authurl + "/connect/token",
                    ClientId = this.clientid,
                    ClientSecret = this.clientsecret,
                    Scope = "fileapi"
                };
                var request = client.RequestClientCredentialsTokenAsync(requestoptions).Result;
                if (request.IsError)
                {
                    throw new Exception(request.Error);
                }

                var content = new MultipartFormDataContent();
                var fileContent = new StreamContent(stream);
                fileContent.Headers.Add("Content-Disposition", "form-data; name=\"file\"; filename=\"" + fileOldName + "\"");
                content.Add(fileContent);
                var stringContent = new StringContent(bucketName);
                stringContent.Headers.Add("Content-Disposition", "form-data; name=\"bucketname\"");
                content.Add(stringContent);
                var stringContent1 = new StringContent(fileName);
                stringContent1.Headers.Add("Content-Disposition", "form-data; name=\"filename\"");
                content.Add(stringContent1);

                client.SetBearerToken(request.AccessToken);
                var respose = client.PostAsync(this.ossserverurl + "/upload", content).Result;
                return respose;
            }
        }

        public Uri UosoGeneratePresignedUri(UosoGeneratePresignedUriRequest req)
        {



            HttpMessageHandler httpMessageHandler = new HttpClientHandler
            {
                AutomaticDecompression =
   System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate
            };
            using (var client = new HttpClient(httpMessageHandler))
            {

                var requestoptions = new ClientCredentialsTokenRequest
                {
                    Address = this.authurl + "/connect/token",
                    ClientId = this.clientid,
                    ClientSecret = this.clientsecret,
                    Scope = "fileapi"
                };
                var request = client.RequestClientCredentialsTokenAsync(requestoptions).Result;
                if (request.IsError)
                {
                    throw new Exception(request.Error);
                }

                var dic = new Dictionary<string, string> {
                   { "Expires",req.Expiration.Ticks.ToString()},
                   { "Key",req.key}
               };

                var content = new FormUrlEncodedContent(dic);

                client.SetBearerToken(request.AccessToken);
                var respose = client.PostAsync(this.ossserverurl + "/getfileurl", content).Result;
                var result = respose.Content.ReadAsStringAsync().Result;
                return new Uri(result);
            }

           

        }

    }

FileClientOSS

注意：MultipartFormDataContent 处理文件上传，如果段都传文件没问题，如果带有参数情况，可能写法就有点坑了，其他参数的描述不要直接加到 xxxContent中，需要单独加头文件描述，我也是Fiddler抓取了PostMan上传的描述解析到，应该比较靠谱

5、通过引用客户端类库来编写自己的文件上传帮助类，之类授权地址以及文件地址确定的情况下可以封装好，这里指定客户端的信息以及文件夹信息，这里我指定了OSSTest

 public class CustomOSSHelper
    {
        private readonly UosoFileOSS uosoFileOSS;
        private const string oauthurl = "http://localhost:3000";
        private const string ossserverurl = "http://localhost:3005";
        private const string clientid = "fileclient";
        private const string clientsecret = "fileclient";
        private const string cbucketName = "OSSTest";
        public CustomOSSHelper()
        {
            uosoFileOSS = new UosoFileOSS(oauthurl, clientid, clientsecret, ossserverurl);
        }

        public HttpResponseMessage Upload(Stream stream, string bucketName, string fileName)
        {
            var result = uosoFileOSS.UosoPutObject(stream, bucketName, fileName);
            return result;
        }
        public HttpResponseMessage Upload(Stream stream, string fileName)
        {
            var result = uosoFileOSS.UosoPutObject(stream, cbucketName, fileName);
            return result;
        }
        public HttpResponseMessage UploadKey(Stream stream, string oldfileName,string fileName)
        {
            var result = uosoFileOSS.UosoPutObject(stream, cbucketName, oldfileName,fileName);
            return result;
        }

        public string GetFileUrl(string bucketName, string fileName)
        {

            var req = new UosoGeneratePresignedUriRequest(ossserverurl, bucketName, fileName, SignHttpMethod.Get)
            {
                Expiration = DateTime.Now.AddHours(1)
            };
            return uosoFileOSS.UosoGeneratePresignedUri(req).AbsoluteUri;
        }
        public string GetFileUrl(string fileName)
        {

            var req = new UosoGeneratePresignedUriRequest(ossserverurl, cbucketName, fileName, SignHttpMethod.Get)
            {
                Expiration = DateTime.Now.AddHours(1)
            };
            return uosoFileOSS.UosoGeneratePresignedUri(req).AbsoluteUri;
        }
    }

CustomOSSHelper

6、接下来我们构建一个文件服务管理的界面来添加管理文件夹、文件信息，添加一个私用的OSSTest文件夹

7、接下来我们通过api来测试下，如果客户端指定一个没有的文件夹会提示 {"status":1,"message":"你不具有文件夹权限","filename":""}，这里必须上传自己管理的已有的文件夹，直接来正确的吧

提示上传成功了，得到文件名称，文件名称可以业务端自定义可以服务端生成

通过下面的文件管理可以看到文件的一些信息

8、通过文件名称来获取访问地址，这里是为了方便客户端获取到具有访问权限的地址的路径

http://localhost:3005/files/5ed3c5ab-e891-4a9d-b425-464cea4829f5.jpg?Expires=637307912138084906&OSSAccessKeyId=fileclient&Signature=01sqQMWse7NwSXQVEjEtloOaApRGEyWQzpWiV6GVgt8%3d

获取到指定生成地址的路径，下面把地址拷贝到浏览器中看下能不能正确访问且验证下自己的权限验证是否有效

下面我们来改一改路径将fileclient改成了fileclient1,然后访问，同时这个地址是带有过期时间的，过期后也不会有访问权限，这里不贴图了

接下来我们来验证下文件夹的权限，在这之前文件设置的私有，设置共有后，该文件夹下所有文件都可以访问，再用之前的fileclient1修改的地址访问，可以看到文件可以访问，后面的参数地址就会进入校验，所以http://localhost:3005/files/5ed3c5ab-e891-4a9d-b425-464cea4829f5.jpg 也是可以直接访问的

10、接下来我们来看下文件服务的文件也是按照我们预期生成的，访问路径也是ok的

总结：此文件服务虽不能文件同步，但是可以根据业务不同分离文件，同时此文件服务可以部署多个地址来减小服务器压力，这个文件服务模仿了AliyunOSS 文件管理来做的，也是为了公司的客户不想使用第三方的文件服务编写的

.NetCore下结合IdentityServer4构建自己的文件服务管理（UosoOSS）

码农公寓

相关文章