吐槽net下没有靠谱的FastDFS的sdk之使用thrift实现JAVA和C#互通-低调大师

吐槽net下没有靠谱的FastDFS的sdk之使用thrift实现JAVA和C#互通

2018-05-23 552

原文: 吐槽net下没有靠谱的FastDFS的sdk之使用thrift实现JAVA和C#互通

　事情是这样的，在一个新项目中引入了fastdfs，用这玩意做一些小数据的存储还是很方便的，然后在nuget上就找到了一个FastDFS的sdk，如下图：

一眼就看到了这个top1的sdk，应该会比较靠谱。。。简单的在项目中应用了一下没啥问题就忽悠上线了，然后就悲剧了，测试那边反馈说上传了一个

人群，拉下来的时候少了几个人，我的使用方式是将一批customerid按照bitmap的形式存到byte[]数组传到fastdfs，最后硬着头皮追踪下来发现是这个所谓

的sdk在upload的时候在bytes数组处理上出了bug，这下无语了，哎，nuget上这写sdk的估计也就是个人写着玩玩丢上去的，哪里敢用到生产上，还好在测

试环境发现了，不然又得出什么乱子了。

一：解决办法

　　问题还得要解决，不过庆幸的是，fastdfs是阿里的一个大牛YuQing写的，那应该有java的sdk更靠谱一点，用maven的话更方便。

        <dependency>
            <groupId>net.oschina.zcx7878</groupId>
            <artifactId>fastdfs-client-java</artifactId>
            <version>1.27.0.0</version>
        </dependency>

pull下来以后，这个sdk果然是fastdfs的作者写的，这下子安全感暴增，测试了一下，那个bug用这个sdk果然就没有问题了。。。开心~~~~

然后流程图大概就变成了这个样子。

二：解决C# 和 JAVA的互通问题

　　互通方式比较多，除了走rest这种面向http的方式，还可以使用thrift，grpc这种tcp的模式，最后我决定还是采用thrift走一遭，目前最新的版本是0.11了。

网址：http://thrift.apache.org/。看了一下C#的thrift sdk，貌似最高支持0.9.1，网址为：http://archive.apache.org/dist/thrift/0.9.1/ ，

有了这个thrift-0.9.1.exe之后，接下来就可以定义Thrift的DSL，这个DSL可以让thrift-0.9.1.exe 生成各个语言版本的sdk。

1. 定义Thrift DSL

service ThriftService
{
    string Upload(1: binary data),
    binary Download(1: string path),
    bool Remove(1: string path)
}

有人可能会问，这个DSL怎么写，这个大家可以看看官方的DSL的各个关键词描述的网址：http://thrift.apache.org/docs/idl 还是比较简单的，如果不清楚的

话，这个是示例大全： https://git-wip-us.apache.org/repos/asf?p=thrift.git;a=blob_plain;f=test/ThriftTest.thrift;hb=HEAD ，然后保存为1.thrift。

2. 通过thrift生成 C# SDK

生成的方式可以参考一下官网的模板：

thrift --gen <language> <Thrift filename>

C:\Users\hxc>cd C:\java\lib\thrift

C:\java\lib\thrift>thrift-0.9.1.exe -gen csharp C:\java\lib\thrift\1.thrift

可以看到，执行完之后，就多了一个gen-csharp文件夹，点进去看一下，会发现有一个文件名为DSL中定义的ThriftService.cs文件。

3. 通过thrift生成 JAVA SDK

执行完下面这条语句，你会发现你的文件夹又多了一份gen-java 。

C:\java\lib\thrift>thrift-0.9.1.exe -gen java C:\java\lib\thrift\1.thrift

三：SDK集成

　改造之后，我们使用JAVA作为服务端，C#作客户端，服务端要做的事情就是通过JAVA来封装FastDFS，然后让C#来调用。

1. JAVA服务端

《1》使用fastDFS 和 Thrift的Maven地址：

        <!-- https://mvnrepository.com/artifact/org.apache.thrift/libthrift -->
        <dependency>
            <groupId>org.apache.thrift</groupId>
            <artifactId>libthrift</artifactId>
            <version>0.9.1</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/net.oschina.zcx7878/fastdfs-client-java -->
        <dependency>
            <groupId>net.oschina.zcx7878</groupId>
            <artifactId>fastdfs-client-java</artifactId>
            <version>1.27.0.0</version>
        </dependency>

《2》 ThriftServiceImpl.java 实现类：

  1 package com.datamip.thrift;
  2 
  3 import java.io.IOException;
  4 import java.nio.ByteBuffer;
  5 import java.util.Date;
  6 
  7 import org.apache.log4j.Logger;
  8 import org.apache.thrift.TException;
  9 import org.csource.common.MyException;
 10 import org.csource.fastdfs.StorageClient;
 11 
 12 import com.fasterxml.jackson.databind.ObjectMapper;
 13 
 14 /*
 15  * thrift 服务端
 16  */
 17 public class ThriftServiceImpl implements ThriftService.Iface {
 18 
 19     public static Logger logger1 = Logger.getLogger(App.class);
 20     
 21     StorageClient client = null;
 22     
 23     ObjectMapper objectMapper=new ObjectMapper();
 24 
 25     public ThriftServiceImpl() throws IOException, MyException {
 26         client = new FastService().Create();
 27     }
 28 
 29     //上传文件
 30     public String Upload(ByteBuffer data) {
 31         
 32         byte[] bytes = data.array();
 33 
 34         logger1.info("已成功接受到upload请求: bytes.length="+bytes.length);
 35 
 36         if(bytes==null || bytes.length==0) return "";
 37         
 38         // 目前给的 “后缀名为 g1",以后可以动态变更，通过‘阿波罗’动态配置
 39         String[] result = null;
 40 
 41         try {
 42             result = client.upload_file(bytes, "g1", null);
 43             
 44             logger1.info("update 上传结果为： "+objectMapper.writeValueAsString(result));
 45             
 46             if (result.length < 2) return "";
 47 
 48         }catch (Exception e) {
 49             logger1.error("upload异常",e);
 50         }
 51 
 52         return result[1];
 53     }
 54 
 55     // 文件下载
 56     public ByteBuffer Download(String path) throws TException {
 57 
 58         logger1.info("已成功接受到download请求:"+path);
 59     
 60         if (path == null || path == "")
 61             return ByteBuffer.allocate(0);
 62 
 63         String[] arr = path.split("\\.");
 64     
 65         if (arr.length < 2)
 66             return ByteBuffer.allocate(0);
 67 
 68         String group_name = arr[1];
 69 
 70         try {
 71             byte[] bytes = client.download_file(group_name, path);
 72             
 73             logger1.info(String.format("根据path=%s，获取的bytes长度为:%s",path,bytes.length));
 74 
 75             return ByteBuffer.wrap(bytes);
 76 
 77         }catch (Exception e) {
 78             logger1.error("download异常",e);
 79         }
 80 
 81         // TODO Auto-generated method stub
 82         return ByteBuffer.allocate(0);
 83     }
 84 
 85     // 删除文件
 86     public boolean Remove(String path) throws TException {
 87 
 88         logger1.info("已成功接受到remove请求:"+path);
 89         
 90         if (path == null || path == "") return false;
 91 
 92         String[] arr = path.split("\\.");
 93         
 94         if(arr==null || arr.length<2) return false;
 95 
 96         String group_name = arr[1];
 97 
 98         try {
 99             int code = client.delete_file(group_name, path);
100             
101             logger1.info(String.format("当前path=%s, groupname=%s,返回状态值=%s",
102                                        path,group_name,code));
103             
104             if(code==0) {
105                 return true;
106             }
107             
108         }catch (Exception e) {
109             logger1.error("Remove异常",e);
110         }
111 
112         return false;
113     }
114 }

《3》 FastDFS的封装类

 1 package com.datamip.thrift;
 2 
 3 import java.io.IOException;
 4 
 5 import org.csource.common.MyException;
 6 import org.csource.fastdfs.ClientGlobal;
 7 import org.csource.fastdfs.StorageClient;
 8 import org.csource.fastdfs.StorageServer;
 9 import org.csource.fastdfs.TrackerClient;
10 import org.csource.fastdfs.TrackerServer;
11 
12 import com.datamip.utils.PropertiesUtils;
13 
14 public class FastService {
15 
16     public StorageClient Create() throws IOException, MyException {
17 
18         //读取配置文件
19         String path = PropertiesUtils.getProperties("setting.properties","fastdfs");
20         return this.Create(path);
21     }
22 
23     public StorageClient Create(String host) throws IOException, MyException {
24 
25         ClientGlobal.initByTrackers(host);
26 
27         // 3、创建一个TrackerClient对象。
28         TrackerClient trackerClient = new TrackerClient();
29 
30         // 4、创建一个TrackerServer对象。
31         TrackerServer trackerServer = trackerClient.getConnection();
32 
33         // 5、声明一个StorageServer对象，null。
34         StorageServer storageServer = null;
35 
36         // 6、获得StorageClient对象。
37         StorageClient storageClient = new StorageClient(trackerServer, storageServer);
38 
39         return storageClient;
40     }
41 }

《4》最后就是AppMain，Thrift开启19999端口。

 1 package com.datamip.thrift;
 2 
 3 import java.io.IOException;
 4 
 5 import org.apache.log4j.Logger;
 6 import org.apache.thrift.TProcessor;
 7 import org.apache.thrift.protocol.TBinaryProtocol;
 8 import org.apache.thrift.server.TServer;
 9 import org.apache.thrift.server.TSimpleServer;
10 import org.apache.thrift.transport.TServerSocket;
11 import org.apache.thrift.transport.TTransportException;
12 import org.csource.common.MyException;
13 
14 public class App {
15 
16     public static Logger logger1 = Logger.getLogger(App.class);
17      
18     public static void main(String[] args) throws IOException, MyException {
19          
20         try {            
21             TProcessor tprocessor = new ThriftService.Processor<ThriftService.Iface>(new ThriftServiceImpl());
22 
23             TServerSocket serverTransport = new TServerSocket(9999);
24             TServer.Args tArgs = new TServer.Args(serverTransport);
25 
26             tArgs.processor(tprocessor);
27             tArgs.protocolFactory(new TBinaryProtocol.Factory());
28 
29             logger1.debug("thrift 服务端开启，开放端口 19999");
30             
31             TServer server = new TSimpleServer(tArgs);
32             server.serve();
33         } catch (TTransportException e) {
34             e.printStackTrace();
35         }
36     }
37 }

2. C#客户端

《1》从negut上把dll拉下来，然后把生成的ThriftService.cs引入到我们的解决方案中

  public partial class ThriftService
    {
        public interface Iface
        {
            string Upload(byte[] data);
#if SILVERLIGHT
    IAsyncResult Begin_Upload(AsyncCallback callback, object state, byte[] data);
    string End_Upload(IAsyncResult asyncResult);
#endif
            byte[] Download(string path);
#if SILVERLIGHT
    IAsyncResult Begin_Download(AsyncCallback callback, object state, string path);
    byte[] End_Download(IAsyncResult asyncResult);
#endif
            bool Remove(string path);
#if SILVERLIGHT
    IAsyncResult Begin_Remove(AsyncCallback callback, object state, string path);
    bool End_Remove(IAsyncResult asyncResult);
#endif
        }

        public class Client : IDisposable, Iface
        {
            public Client(TProtocol prot) : this(prot, prot)
            {
            }

            public Client(TProtocol iprot, TProtocol oprot)
            {
                iprot_ = iprot;
                oprot_ = oprot;
            }

            protected TProtocol iprot_;
            protected TProtocol oprot_;
            protected int seqid_;

            public TProtocol InputProtocol
            {
                get { return iprot_; }
            }
            public TProtocol OutputProtocol
            {
                get { return oprot_; }
            }


            #region " IDisposable Support "
            private bool _IsDisposed;

            // IDisposable
            public void Dispose()
            {
                Dispose(true);
            }


            protected virtual void Dispose(bool disposing)
            {
                if (!_IsDisposed)
                {
                    if (disposing)
                    {
                        if (iprot_ != null)
                        {
                            ((IDisposable)iprot_).Dispose();
                        }
                        if (oprot_ != null)
                        {
                            ((IDisposable)oprot_).Dispose();
                        }
                    }
                }
                _IsDisposed = true;
            }
            #endregion



#if SILVERLIGHT
    public IAsyncResult Begin_Upload(AsyncCallback callback, object state, byte[] data)
    {
      return send_Upload(callback, state, data);
    }

    public string End_Upload(IAsyncResult asyncResult)
    {
      oprot_.Transport.EndFlush(asyncResult);
      return recv_Upload();
    }

#endif

            public string Upload(byte[] data)
            {
#if !SILVERLIGHT
                send_Upload(data);
                return recv_Upload();

#else
      var asyncResult = Begin_Upload(null, null, data);
      return End_Upload(asyncResult);

#endif
            }
#if SILVERLIGHT
    public IAsyncResult send_Upload(AsyncCallback callback, object state, byte[] data)
#else
            public void send_Upload(byte[] data)
#endif
            {
                oprot_.WriteMessageBegin(new TMessage("Upload", TMessageType.Call, seqid_));
                Upload_args args = new Upload_args();
                args.Data = data;
                args.Write(oprot_);
                oprot_.WriteMessageEnd();
#if SILVERLIGHT
      return oprot_.Transport.BeginFlush(callback, state);
#else
                oprot_.Transport.Flush();
#endif
            }

            public string recv_Upload()
            {
                TMessage msg = iprot_.ReadMessageBegin();
                if (msg.Type == TMessageType.Exception)
                {
                    TApplicationException x = TApplicationException.Read(iprot_);
                    iprot_.ReadMessageEnd();
                    throw x;
                }
                Upload_result result = new Upload_result();
                result.Read(iprot_);
                iprot_.ReadMessageEnd();
                if (result.__isset.success)
                {
                    return result.Success;
                }
                throw new TApplicationException(TApplicationException.ExceptionType.MissingResult, "Upload failed: unknown result");
            }


#if SILVERLIGHT
    public IAsyncResult Begin_Download(AsyncCallback callback, object state, string path)
    {
      return send_Download(callback, state, path);
    }

    public byte[] End_Download(IAsyncResult asyncResult)
    {
      oprot_.Transport.EndFlush(asyncResult);
      return recv_Download();
    }

#endif

            public byte[] Download(string path)
            {
#if !SILVERLIGHT
                send_Download(path);
                return recv_Download();

#else
      var asyncResult = Begin_Download(null, null, path);
      return End_Download(asyncResult);

#endif
            }
#if SILVERLIGHT
    public IAsyncResult send_Download(AsyncCallback callback, object state, string path)
#else
            public void send_Download(string path)
#endif
            {
                oprot_.WriteMessageBegin(new TMessage("Download", TMessageType.Call, seqid_));
                Download_args args = new Download_args();
                args.Path = path;
                args.Write(oprot_);
                oprot_.WriteMessageEnd();
#if SILVERLIGHT
      return oprot_.Transport.BeginFlush(callback, state);
#else
                oprot_.Transport.Flush();
#endif
            }

            public byte[] recv_Download()
            {
                TMessage msg = iprot_.ReadMessageBegin();
                if (msg.Type == TMessageType.Exception)
                {
                    TApplicationException x = TApplicationException.Read(iprot_);
                    iprot_.ReadMessageEnd();
                    throw x;
                }
                Download_result result = new Download_result();
                result.Read(iprot_);
                iprot_.ReadMessageEnd();
                if (result.__isset.success)
                {
                    return result.Success;
                }
                throw new TApplicationException(TApplicationException.ExceptionType.MissingResult, "Download failed: unknown result");
            }


#if SILVERLIGHT
    public IAsyncResult Begin_Remove(AsyncCallback callback, object state, string path)
    {
      return send_Remove(callback, state, path);
    }

    public bool End_Remove(IAsyncResult asyncResult)
    {
      oprot_.Transport.EndFlush(asyncResult);
      return recv_Remove();
    }

#endif

            public bool Remove(string path)
            {
#if !SILVERLIGHT
                send_Remove(path);
                return recv_Remove();

#else
      var asyncResult = Begin_Remove(null, null, path);
      return End_Remove(asyncResult);

#endif
            }
#if SILVERLIGHT
    public IAsyncResult send_Remove(AsyncCallback callback, object state, string path)
#else
            public void send_Remove(string path)
#endif
            {
                oprot_.WriteMessageBegin(new TMessage("Remove", TMessageType.Call, seqid_));
                Remove_args args = new Remove_args();
                args.Path = path;
                args.Write(oprot_);
                oprot_.WriteMessageEnd();
#if SILVERLIGHT
      return oprot_.Transport.BeginFlush(callback, state);
#else
                oprot_.Transport.Flush();
#endif
            }

            public bool recv_Remove()
            {
                TMessage msg = iprot_.ReadMessageBegin();
                if (msg.Type == TMessageType.Exception)
                {
                    TApplicationException x = TApplicationException.Read(iprot_);
                    iprot_.ReadMessageEnd();
                    throw x;
                }
                Remove_result result = new Remove_result();
                result.Read(iprot_);
                iprot_.ReadMessageEnd();
                if (result.__isset.success)
                {
                    return result.Success;
                }
                throw new TApplicationException(TApplicationException.ExceptionType.MissingResult, "Remove failed: unknown result");
            }

        }
        public class Processor : TProcessor
        {
            public Processor(Iface iface)
            {
                iface_ = iface;
                processMap_["Upload"] = Upload_Process;
                processMap_["Download"] = Download_Process;
                processMap_["Remove"] = Remove_Process;
            }

            protected delegate void ProcessFunction(int seqid, TProtocol iprot, TProtocol oprot);
            private Iface iface_;
            protected Dictionary<string, ProcessFunction> processMap_ = new Dictionary<string, ProcessFunction>();

            public bool Process(TProtocol iprot, TProtocol oprot)
            {
                try
                {
                    TMessage msg = iprot.ReadMessageBegin();
                    ProcessFunction fn;
                    processMap_.TryGetValue(msg.Name, out fn);
                    if (fn == null)
                    {
                        TProtocolUtil.Skip(iprot, TType.Struct);
                        iprot.ReadMessageEnd();
                        TApplicationException x = new TApplicationException(TApplicationException.ExceptionType.UnknownMethod, "Invalid method name: '" + msg.Name + "'");
                        oprot.WriteMessageBegin(new TMessage(msg.Name, TMessageType.Exception, msg.SeqID));
                        x.Write(oprot);
                        oprot.WriteMessageEnd();
                        oprot.Transport.Flush();
                        return true;
                    }
                    fn(msg.SeqID, iprot, oprot);
                }
                catch (IOException)
                {
                    return false;
                }
                return true;
            }

            public void Upload_Process(int seqid, TProtocol iprot, TProtocol oprot)
            {
                Upload_args args = new Upload_args();
                args.Read(iprot);
                iprot.ReadMessageEnd();
                Upload_result result = new Upload_result();
                result.Success = iface_.Upload(args.Data);
                oprot.WriteMessageBegin(new TMessage("Upload", TMessageType.Reply, seqid));
                result.Write(oprot);
                oprot.WriteMessageEnd();
                oprot.Transport.Flush();
            }

            public void Download_Process(int seqid, TProtocol iprot, TProtocol oprot)
            {
                Download_args args = new Download_args();
                args.Read(iprot);
                iprot.ReadMessageEnd();
                Download_result result = new Download_result();
                result.Success = iface_.Download(args.Path);
                oprot.WriteMessageBegin(new TMessage("Download", TMessageType.Reply, seqid));
                result.Write(oprot);
                oprot.WriteMessageEnd();
                oprot.Transport.Flush();
            }

            public void Remove_Process(int seqid, TProtocol iprot, TProtocol oprot)
            {
                Remove_args args = new Remove_args();
                args.Read(iprot);
                iprot.ReadMessageEnd();
                Remove_result result = new Remove_result();
                result.Success = iface_.Remove(args.Path);
                oprot.WriteMessageBegin(new TMessage("Remove", TMessageType.Reply, seqid));
                result.Write(oprot);
                oprot.WriteMessageEnd();
                oprot.Transport.Flush();
            }

        }


#if !SILVERLIGHT
        [Serializable]
#endif
        public partial class Upload_args : TBase
        {
            private byte[] _data;

            public byte[] Data
            {
                get
                {
                    return _data;
                }
                set
                {
                    __isset.data = true;
                    this._data = value;
                }
            }


            public Isset __isset;
#if !SILVERLIGHT
            [Serializable]
#endif
            public struct Isset
            {
                public bool data;
            }

            public Upload_args()
            {
            }

            public void Read(TProtocol iprot)
            {
                TField field;
                iprot.ReadStructBegin();
                while (true)
                {
                    field = iprot.ReadFieldBegin();
                    if (field.Type == TType.Stop)
                    {
                        break;
                    }
                    switch (field.ID)
                    {
                        case 1:
                            if (field.Type == TType.String)
                            {
                                Data = iprot.ReadBinary();
                            }
                            else
                            {
                                TProtocolUtil.Skip(iprot, field.Type);
                            }
                            break;
                        default:
                            TProtocolUtil.Skip(iprot, field.Type);
                            break;
                    }
                    iprot.ReadFieldEnd();
                }
                iprot.ReadStructEnd();
            }

            public void Write(TProtocol oprot)
            {
                TStruct struc = new TStruct("Upload_args");
                oprot.WriteStructBegin(struc);
                TField field = new TField();
                if (Data != null && __isset.data)
                {
                    field.Name = "data";
                    field.Type = TType.String;
                    field.ID = 1;
                    oprot.WriteFieldBegin(field);
                    oprot.WriteBinary(Data);
                    oprot.WriteFieldEnd();
                }
                oprot.WriteFieldStop();
                oprot.WriteStructEnd();
            }

            public override string ToString()
            {
                StringBuilder sb = new StringBuilder("Upload_args(");
                sb.Append("Data: ");
                sb.Append(Data);
                sb.Append(")");
                return sb.ToString();
            }

        }


#if !SILVERLIGHT
        [Serializable]
#endif
        public partial class Upload_result : TBase
        {
            private string _success;

            public string Success
            {
                get
                {
                    return _success;
                }
                set
                {
                    __isset.success = true;
                    this._success = value;
                }
            }


            public Isset __isset;
#if !SILVERLIGHT
            [Serializable]
#endif
            public struct Isset
            {
                public bool success;
            }

            public Upload_result()
            {
            }

            public void Read(TProtocol iprot)
            {
                TField field;
                iprot.ReadStructBegin();
                while (true)
                {
                    field = iprot.ReadFieldBegin();
                    if (field.Type == TType.Stop)
                    {
                        break;
                    }
                    switch (field.ID)
                    {
                        case 0:
                            if (field.Type == TType.String)
                            {
                                Success = iprot.ReadString();
                            }
                            else
                            {
                                TProtocolUtil.Skip(iprot, field.Type);
                            }
                            break;
                        default:
                            TProtocolUtil.Skip(iprot, field.Type);
                            break;
                    }
                    iprot.ReadFieldEnd();
                }
                iprot.ReadStructEnd();
            }

            public void Write(TProtocol oprot)
            {
                TStruct struc = new TStruct("Upload_result");
                oprot.WriteStructBegin(struc);
                TField field = new TField();

                if (this.__isset.success)
                {
                    if (Success != null)
                    {
                        field.Name = "Success";
                        field.Type = TType.String;
                        field.ID = 0;
                        oprot.WriteFieldBegin(field);
                        oprot.WriteString(Success);
                        oprot.WriteFieldEnd();
                    }
                }
                oprot.WriteFieldStop();
                oprot.WriteStructEnd();
            }

            public override string ToString()
            {
                StringBuilder sb = new StringBuilder("Upload_result(");
                sb.Append("Success: ");
                sb.Append(Success);
                sb.Append(")");
                return sb.ToString();
            }

        }


#if !SILVERLIGHT
        [Serializable]
#endif
        public partial class Download_args : TBase
        {
            private string _path;

            public string Path
            {
                get
                {
                    return _path;
                }
                set
                {
                    __isset.path = true;
                    this._path = value;
                }
            }


            public Isset __isset;
#if !SILVERLIGHT
            [Serializable]
#endif
            public struct Isset
            {
                public bool path;
            }

            public Download_args()
            {
            }

            public void Read(TProtocol iprot)
            {
                TField field;
                iprot.ReadStructBegin();
                while (true)
                {
                    field = iprot.ReadFieldBegin();
                    if (field.Type == TType.Stop)
                    {
                        break;
                    }
                    switch (field.ID)
                    {
                        case 1:
                            if (field.Type == TType.String)
                            {
                                Path = iprot.ReadString();
                            }
                            else
                            {
                                TProtocolUtil.Skip(iprot, field.Type);
                            }
                            break;
                        default:
                            TProtocolUtil.Skip(iprot, field.Type);
                            break;
                    }
                    iprot.ReadFieldEnd();
                }
                iprot.ReadStructEnd();
            }

            public void Write(TProtocol oprot)
            {
                TStruct struc = new TStruct("Download_args");
                oprot.WriteStructBegin(struc);
                TField field = new TField();
                if (Path != null && __isset.path)
                {
                    field.Name = "path";
                    field.Type = TType.String;
                    field.ID = 1;
                    oprot.WriteFieldBegin(field);
                    oprot.WriteString(Path);
                    oprot.WriteFieldEnd();
                }
                oprot.WriteFieldStop();
                oprot.WriteStructEnd();
            }

            public override string ToString()
            {
                StringBuilder sb = new StringBuilder("Download_args(");
                sb.Append("Path: ");
                sb.Append(Path);
                sb.Append(")");
                return sb.ToString();
            }

        }


#if !SILVERLIGHT
        [Serializable]
#endif
        public partial class Download_result : TBase
        {
            private byte[] _success;

            public byte[] Success
            {
                get
                {
                    return _success;
                }
                set
                {
                    __isset.success = true;
                    this._success = value;
                }
            }


            public Isset __isset;
#if !SILVERLIGHT
            [Serializable]
#endif
            public struct Isset
            {
                public bool success;
            }

            public Download_result()
            {
            }

            public void Read(TProtocol iprot)
            {
                TField field;
                iprot.ReadStructBegin();
                while (true)
                {
                    field = iprot.ReadFieldBegin();
                    if (field.Type == TType.Stop)
                    {
                        break;
                    }
                    switch (field.ID)
                    {
                        case 0:
                            if (field.Type == TType.String)
                            {
                                Success = iprot.ReadBinary();
                            }
                            else
                            {
                                TProtocolUtil.Skip(iprot, field.Type);
                            }
                            break;
                        default:
                            TProtocolUtil.Skip(iprot, field.Type);
                            break;
                    }
                    iprot.ReadFieldEnd();
                }
                iprot.ReadStructEnd();
            }

            public void Write(TProtocol oprot)
            {
                TStruct struc = new TStruct("Download_result");
                oprot.WriteStructBegin(struc);
                TField field = new TField();

                if (this.__isset.success)
                {
                    if (Success != null)
                    {
                        field.Name = "Success";
                        field.Type = TType.String;
                        field.ID = 0;
                        oprot.WriteFieldBegin(field);
                        oprot.WriteBinary(Success);
                        oprot.WriteFieldEnd();
                    }
                }
                oprot.WriteFieldStop();
                oprot.WriteStructEnd();
            }

            public override string ToString()
            {
                StringBuilder sb = new StringBuilder("Download_result(");
                sb.Append("Success: ");
                sb.Append(Success);
                sb.Append(")");
                return sb.ToString();
            }

        }


#if !SILVERLIGHT
        [Serializable]
#endif
        public partial class Remove_args : TBase
        {
            private string _path;

            public string Path
            {
                get
                {
                    return _path;
                }
                set
                {
                    __isset.path = true;
                    this._path = value;
                }
            }


            public Isset __isset;
#if !SILVERLIGHT
            [Serializable]
#endif
            public struct Isset
            {
                public bool path;
            }

            public Remove_args()
            {
            }

            public void Read(TProtocol iprot)
            {
                TField field;
                iprot.ReadStructBegin();
                while (true)
                {
                    field = iprot.ReadFieldBegin();
                    if (field.Type == TType.Stop)
                    {
                        break;
                    }
                    switch (field.ID)
                    {
                        case 1:
                            if (field.Type == TType.String)
                            {
                                Path = iprot.ReadString();
                            }
                            else
                            {
                                TProtocolUtil.Skip(iprot, field.Type);
                            }
                            break;
                        default:
                            TProtocolUtil.Skip(iprot, field.Type);
                            break;
                    }
                    iprot.ReadFieldEnd();
                }
                iprot.ReadStructEnd();
            }

            public void Write(TProtocol oprot)
            {
                TStruct struc = new TStruct("Remove_args");
                oprot.WriteStructBegin(struc);
                TField field = new TField();
                if (Path != null && __isset.path)
                {
                    field.Name = "path";
                    field.Type = TType.String;
                    field.ID = 1;
                    oprot.WriteFieldBegin(field);
                    oprot.WriteString(Path);
                    oprot.WriteFieldEnd();
                }
                oprot.WriteFieldStop();
                oprot.WriteStructEnd();
            }

            public override string ToString()
            {
                StringBuilder sb = new StringBuilder("Remove_args(");
                sb.Append("Path: ");
                sb.Append(Path);
                sb.Append(")");
                return sb.ToString();
            }

        }


#if !SILVERLIGHT
        [Serializable]
#endif
        public partial class Remove_result : TBase
        {
            private bool _success;

            public bool Success
            {
                get
                {
                    return _success;
                }
                set
                {
                    __isset.success = true;
                    this._success = value;
                }
            }


            public Isset __isset;
#if !SILVERLIGHT
            [Serializable]
#endif
            public struct Isset
            {
                public bool success;
            }

            public Remove_result()
            {
            }

            public void Read(TProtocol iprot)
            {
                TField field;
                iprot.ReadStructBegin();
                while (true)
                {
                    field = iprot.ReadFieldBegin();
                    if (field.Type == TType.Stop)
                    {
                        break;
                    }
                    switch (field.ID)
                    {
                        case 0:
                            if (field.Type == TType.Bool)
                            {
                                Success = iprot.ReadBool();
                            }
                            else
                            {
                                TProtocolUtil.Skip(iprot, field.Type);
                            }
                            break;
                        default:
                            TProtocolUtil.Skip(iprot, field.Type);
                            break;
                    }
                    iprot.ReadFieldEnd();
                }
                iprot.ReadStructEnd();
            }

            public void Write(TProtocol oprot)
            {
                TStruct struc = new TStruct("Remove_result");
                oprot.WriteStructBegin(struc);
                TField field = new TField();

                if (this.__isset.success)
                {
                    field.Name = "Success";
                    field.Type = TType.Bool;
                    field.ID = 0;
                    oprot.WriteFieldBegin(field);
                    oprot.WriteBool(Success);
                    oprot.WriteFieldEnd();
                }
                oprot.WriteFieldStop();
                oprot.WriteStructEnd();
            }

            public override string ToString()
            {
                StringBuilder sb = new StringBuilder("Remove_result(");
                sb.Append("Success: ");
                sb.Append(Success);
                sb.Append(")");
                return sb.ToString();
            }

        }

    }

View Code

《2》封装一个简单的CURD操作

  1     public class ThriftSHelper
  2     {
  3         private static string fastdfs = ConfigurationManager.AppSettings["fastdfs"];
  4 
  5         TTransport transport = null;
  6         TProtocol protocol = null;
  7         ThriftService.Client client = null;
  8 
  9         public ThriftSHelper()
 10         {
 11             var arr = fastdfs.Split(':');
 12             var host = arr[0];
 13             var port = Convert.ToInt32(arr[1]);
 14 
 15             transport = new TSocket(host, port);
 16             protocol = new TBinaryProtocol(transport);
 17             client = new ThriftService.Client(protocol);
 18         }
 19 
 20         public static ThriftSHelper Create()
 21         {
 22             return new ThriftSHelper();
 23         }
 24 
 25         public string UploadFile(BitArray bit)
 26         {
 27             string path = string.Empty;
 28 
 29             try
 30             {
 31                 var bytes = new byte[Convert.ToInt32(Math.Ceiling((double)bit.Length / 8))];
 32 
 33                 transport.Open();
 34 
 35                 bit.CopyTo(bytes, 0);
 36 
 37                 path = client.Upload(bytes);
 38             }
 39             catch (Exception ex)
 40             {
 41                 LogHelper.Error(ex);
 42             }
 43             finally
 44             {
 45                 transport.Close();
 46             }
 47 
 48             return path;
 49         }
 50 
 51         /// <summary>
 52         /// 下载文件
 53         /// </summary>
 54         /// <param name="fileName"></param>
 55         /// <returns></returns>
 56         public BitArray DownloadFile(string fileName)
 57         {
 58             BitArray bitArray = null;
 59 
 60             try
 61             {
 62                 transport.Open();
 63 
 64                 var bytes = client.Download(fileName);
 65 
 66                 return new BitArray(bytes);
 67             }
 68             catch (Exception ex)
 69             {
 70                 LogHelper.WriteLog(fileName, ex);
 71             }
 72             finally
 73             {
 74                 transport.Close();
 75             }
 76 
 77             return bitArray;
 78         }
 79 
 80         /// <summary>
 81         /// 删除文件
 82         /// </summary>
 83         /// <param name="fileName"></param>
 84         public void RemoveFile(string fileName)
 85         {
 86             try
 87             {
 88                 transport.Open();
 89 
 90                 client.Remove(fileName);
 91             }
 92             catch (Exception ex)
 93             {
 94                 LogHelper.WriteLog(ex);
 95             }
 96             finally
 97             {
 98                 transport.Close();
 99             }
100         }
101     }

　　好了，这个问题我们就这样完美解决了，跑在生产上还是蛮好的。

微信关注我们

原文链接：https://yq.aliyun.com/articles/678551

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

把项目中那些恶心的无处存储的大块数据都丢到FastDFS之快速搭建

原文: 把项目中那些恶心的无处存储的大块数据都丢到FastDFS之快速搭建在我们开发项目的时候，经常会遇到大块数据的问题（2M-100M），比如说保存报表中1w个人的ID号，他就像一个肿瘤一样，存储在服务器哪里都觉得恶心，放在redis，mongodb中吧，一下子你就会收到报警，因为内存满了。。。放在mysql吧？？？你还得建立一个text字段，和一些小字段混在一起，还是有点恶心，还得单独拆出来，还得怕有些sql不规范的人挺喜欢select * 的，这速度挺恶心的呀。。。直接放到硬盘吧，没扩展性，你1T大小的硬盘又能怎样，照样撑爆你，放在hadoop里面吧，对.net程序员来说，没有这个缘分，好不容易微软有一个.net hadoop sdk，说放弃就放弃了，兼具以上各种特性，最后目光只能落到FastDFS上了。一： FastDFS fastDFS的本意是一个分布式的文件系统，所以大家可以上传各种小文件，包括这篇和大家说到的那些一坨一坨的数据，同样你也可以认为是一些小文件，下面我画一下它的大概架构图：我来解释一下： 1. fastDFS是按照Group的形...

2018-05-24

578

Java基础巩固-关于Java序列化和反序列化

题外话: 从事IT要学习的东西太多了，有时候会比较浮躁，因为要学的东西太多但又无从下手，甚至有很多基础都还没有深入学习，这个时候应当静下心来，正所谓不忘初心，方能始终，之前一直听说过序列化，但也没有去深入一点点的了解过，这个时候，就当好好巩固下了～ java序列化常被称为持久化，将其写入磁盘中。对于一个存在于jvm的对象来说，内部的状态保存在内存中，当jvm停止时这些状态就丢失了，但有些时候对象的内部是需要持久保存的，对象序列化机制（object serialization）是Java语言内建的一种对象持久化方式，可以很容易的在JVM中的活动对象和字节数组（流）之间进行转换，该机制中对象可以表示为字节序列，该字节序列包括该对象的数据，有关对象的类型的信息和存储在对象中数据的类型。数据序列化就是将对象或者数据结构转化成特定的格式，使其可在网络中传输，或者可存储在内存或者文件中。反序列化则是相反的操作，将对象从序列化数据中还原出来。而对象序列化后的数据格式可以是二进制，可以是XML，也可以是JSON等任何格式。【整个过程在jvm独立的，在一个平台上序列化的对象可以在另外的平台反序列...

2018-05-25

663

发表评论

资源下载

更多资源

优质分享App

近一个月的开发和优化，本站点的第一个app全新上线。该app采用极致压缩，本体才4.36MB。系统里面做了大量数据访问、缓存优化。方便用户在手机上查看文章。后续会推出HarmonyOS的适配版本。

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题，腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构，目前腾讯云软件源站支持公网访问和内网访问。

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称，一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集，帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。