Silverlight访问摄像头和麦克风(2)视频对话

2021-10-18 10:33:47

今天使用wcf的duplex方式实现了视频对话，但是很卡，晚上准备改写为Socket方式或者将客户端定时请求服务器资源改变为服务器主动回调客户端取资源。简要将今天的尝试记录一下。

思路是文本聊天通过duplex方式进行，而视频部分则通过客户端定时将截屏发送到服务器，再由服务器转发到聊天对象。

编写WCF服务端

定义服务契约：

代码

[ServiceContract(CallbackContract=typeof(IChatServiceCallBack))]
    public interface IChatService
    {
        [OperationContract]
        bool Login(string user,string partner);

        [OperationContract]
        bool SendMessage(MessageInfo message);

        [OperationContract]
        List<UserVideo> GetVideosByte(string userName, string partnerName);

        [OperationContract]
        void SendVideoByte(UserVideo video);
    }


    [ServiceContract]
    public interface IChatServiceCallBack
    {
        [OperationContract(IsOneWay=true)]
        void Receive(List<MessageInfo> messages);
    }

实现服务：

代码

public class UserUpdate//记录用户上次取消息的时间
   {
       public string UserName{get;set;}
       public DateTime LatestTime{get;set;}
   }
   public class ChatService : IChatService
   {
       IChatServiceCallBack callBack;
       Timer timer;
       string _user;
       string _partner;
       public static List<UserUpdate> listUserUpdate = new List<UserUpdate>();//存放用户取得消息的最近时间
       public static List<MessageInfo> listMessages = new List<MessageInfo>();//模拟存放聊天信息
       public static List<UserVideo> listVideos = new List<UserVideo>();//临时存放视频信息
       public void StartTimer()
       {
           timer = new Timer(new TimerCallback(CallClientToReceiveMsg), null, 500, 500);//定时回调客户端，传送资源
       }
       public bool Login(string user,string partner)
       {
           try
           {
               _user = user;
               _partner = partner;
               if (listUserUpdate.Where(m => m.UserName == user).ToList().Count == 0)
               {
                   listUserUpdate.Add(new UserUpdate() { UserName = user, LatestTime = DateTime.Now });
               }
               callBack = OperationContext.Current.GetCallbackChannel<IChatServiceCallBack>();
               StartTimer();
               return true;
           }
           catch
           {
               return false;
           }
       }
       private void CallClientToReceiveMsg(object o)//客户端回调完成后
       {
           try
           {
               DateTime dt = listUserUpdate.Where(m => m.UserName == _user).ToList()[0].LatestTime;
               listUserUpdate.Remove(listUserUpdate.Where(m => m.UserName == _user).ToList()[0]);
               listUserUpdate.Add(new UserUpdate() { UserName = _user, LatestTime = DateTime.Now });
               callBack.Receive(listMessages.Where(m => (m.Sender == _user && m.ReceiveUser == _partner && m.SendTime > dt) || (m.ReceiveUser == _user && m.Sender == _partner && m.SendTime > dt)).ToList());
           }
           catch
           {
               timer.Dispose();
               StartTimer();
           }
       }
       public bool SendMessage(MessageInfo message)//发送消息方法
       {
           try
           {
               listMessages.Add(message);
               return true;
           }
           catch
           {
               return false;
           }
       }

       public List<UserVideo> GetVideosByte(string userName,string partnerName)//取得视频信息
       {
           List<UserVideo> list = new List<UserVideo>();
           list=listVideos.Where(m=>(m.UserName==partnerName&&m.PartnerName==userName)).ToList();
           if (list.Count > 0)
           {
               listVideos.RemoveAll(m => (m.UserName == partnerName && m.PartnerName == userName));
           }
           return list;
       }

       public void SendVideoByte(UserVideo video)
       {
           listVideos.Add(video);
       }
   }

消息契约和用户视频对象

代码

[DataContract]
    public class MessageInfo
    {
        [DataMember]
        public string ID { set; get; }
        [DataMember]
        public string Title { set; get; }
        [DataMember]
        public string Message { set; get; }
        [DataMember]
        public DateTime SendTime { set; get; }
        [DataMember]
        public DateTime? ReadTime { set; get; }
        [DataMember]
        public string Sender { set; get; }
        [DataMember]
        public string ReceiveUser { set; get; }
        [DataMember]
        public string ReceiveOrgan { set; get; }
        [DataMember]
        public string ReceiveMode { set; get; }
        [DataMember]
        public int State { set; get; }
        [DataMember]
        public int Receipt { set; get; }
        [DataMember]
        public string Source { set; get; }
    }

[DataContract]
    public class UserVideo
    {
        [DataMember]
        public string UserName { get; set; }
        [DataMember]
        public string PartnerName { set; get; }
        [DataMember]
        public byte[] VideoByte { set; get; }
    }

Silverlight客户端代码

首先需要登录服务器，使服务器开始监控消息

代码

address = new EndpointAddress("http://localhost:8752/ChatService.svc%22);
            binding = new PollingDuplexHttpBinding(PollingDuplexMode.MultipleMessagesPerPoll);
            proxy = new ChatServiceClient(binding, address);
            proxy.ReceiveReceived += new EventHandler<ReceiveReceivedEventArgs>(proxy_ReceiveReceived);
            proxy.LoginCompleted += new EventHandler<LoginCompletedEventArgs>(proxy_LoginCompleted);
            proxy.LoginAsync(User, Partner);

当链接到服务器之后呢，就开始监控对方视频资源，当然这里可以做成邀请-同意的模式另外触发这个事件

void proxy_LoginCompleted(object sender, LoginCompletedEventArgs e)
        {
            AddText("connected");
            //连接到服务器后开始接收视频
            ReceiveVideo();
        }

当连接到服务器之后，一旦服务器收到对方发给自己的信息就可以执行回调操作

代码

void proxy_ReceiveReceived(object sender, ReceiveReceivedEventArgs e)
        {
            if (e.Error == null)
            {
                foreach (MessageInfo msg in e.messages)
                {
                    AddText(msg.Sender + "  say:" + msg.Message);
                }
            }
        }

这里采用的方式比较简单，就是有客户端定时去服务器上取得数据。代码

void ReceiveVideo()
        {
            System.Windows.Threading.DispatcherTimer timerForReceive = new System.Windows.Threading.DispatcherTimer();
            timerForReceive.Interval = new TimeSpan(0, 0, 0, 0, 200);
            timerForReceive.Tick += new EventHandler(timerForReceive_Tick);
            timerForReceive.Start();
            AddText("start to receive video");
        }

        void timerForReceive_Tick(object sender, EventArgs e)
        {
            proxy.GetVideosByteCompleted += new EventHandler<GetVideosByteCompletedEventArgs>(proxy_GetVideosByteCompleted);
            proxy.GetVideosByteAsync(User, Partner);
        }

        void proxy_GetVideosByteCompleted(object sender, GetVideosByteCompletedEventArgs e)
        {
            if(e.Error==null)
            {
                foreach(ChatService.UserVideo video in e.Result)
                {
                    MemoryStream ms = new MemoryStream(video.VideoByte);
                    BitmapImage bitmap = new BitmapImage();
                    bitmap.SetSource(ms);
                    imagePartner.Source = bitmap;
                    ms.Close();
                }
            }
        }

而向服务器传递数据也是通过定时截图发送的方式进行

代码

void btnSendVideo_Click(object sender, RoutedEventArgs e)
        {
            System.Windows.Threading.DispatcherTimer timer = new System.Windows.Threading.DispatcherTimer();
            timer.Interval = new TimeSpan(0, 0, 0, 0,200);
            timer.Tick += new EventHandler(timer_Tick);
            timer.Start();
            AddText("start sending video");
        }

此方法为向服务器发送视频比特流

代码

void timer_Tick(object sender, EventArgs e)
        {
            WriteableBitmap bmp = new WriteableBitmap(this.rectangleUser, null);
            MemoryStream ms = new MemoryStream();
            EncodeJpeg(bmp, ms);
            UserVideo userVideo = new UserVideo();
            userVideo.PartnerName = this.Partner;
            userVideo.UserName = this.User;
            userVideo.VideoByte = ms.GetBuffer();
            proxy.SendVideoByteAsync(userVideo);
            AddText("send a video jpg");
        }

        //编码
        public static void EncodeJpeg(WriteableBitmap bmp, Stream dstStream)
        {
            // Init buffer in FluxJpeg format
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int[] p = bmp.Pixels;
            byte[][,] pixelsForJpeg = new byte[3][,]; // RGB colors
            pixelsForJpeg[0] = new byte[w, h];
            pixelsForJpeg[1] = new byte[w, h];
            pixelsForJpeg[2] = new byte[w, h];

            // Copy WriteableBitmap data into buffer for FluxJpeg
            int i = 0;
            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int color = p[i++];
                    pixelsForJpeg[0][x, y] = (byte)(color >> 16); // R
                    pixelsForJpeg[1][x, y] = (byte)(color >> 8);  // G
                    pixelsForJpeg[2][x, y] = (byte)(color);       // B
                }
            }
            //Encode Image as JPEG
            var jpegImage = new FluxJpeg.Core.Image(new ColorModel { colorspace = ColorSpace.RGB }, pixelsForJpeg);
            var encoder = new JpegEncoder(jpegImage, 95, dstStream);
            encoder.Encode();
        }
        //解码
        public static WriteableBitmap DecodeJpeg(Stream srcStream)
        {
            // Decode JPEG
            var decoder = new FluxJpeg.Core.Decoder.JpegDecoder(srcStream);
            var jpegDecoded = decoder.Decode();
            var img = jpegDecoded.Image;
            img.ChangeColorSpace(ColorSpace.RGB);

            // Init Buffer
            int w = img.Width;
            int h = img.Height;
            var result = new WriteableBitmap(w, h);
            int[] p = result.Pixels;
            byte[][,] pixelsFromJpeg = img.Raster;
            // Copy FluxJpeg buffer into WriteableBitmap
            int i = 0;
            for (int x = 0; x < w; x++)
            {
                for (int y = 0; y < h; y++)
                {
                    p[i++] = (0xFF << 24) // A
                                | (pixelsFromJpeg[0][x, y] << 16) // R
                                | (pixelsFromJpeg[1][x, y] << 8)  // G
                                | pixelsFromJpeg[2][x, y];       // B
                }
            }

            return result;
        }

此处为视频开始的事件

代码

void btnVideo_Click(object sender, RoutedEventArgs e)
        {
            if (source != null)
            {
                source.Stop();
                source.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
                VideoBrush vBrush = new VideoBrush();
                vBrush.SetSource(source);
                this.rectangleUser.Fill = vBrush;
                if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess())
                {
                    source.Start();
                }

                AddText("start video");
            }
        }

        void txtMessage_KeyDown(object sender, KeyEventArgs e)
        {
            if (e.Key == Key.Enter)
            {
                SendMsg();
            }
        }

这个Demo存在的问题有三个

1，视频流传输到服务器再转到对方客户端的过程有些复杂而且费时，打算采用两种方法解决，一是用Socket通信，二是用双通道推送方式。

2，目前的视频流应该经过压缩后传输，在客户端进行解析，否则如此大的数据量解析是个问题。

3，由于是IIS托管WCF服务，在这个DEMO中有WCF回调客户端的方式，这种方式一旦客户端掉线，服务端回调不到客户端就会出现异常，这种异常靠 try catch解决不了。

4，在某些电脑的IE8上打开视频时会卡死。

如果您看到这篇文章对这几个问题比较了解，兄弟我还是非常想请教一下的，谢谢。

本文转自wengyuli 51CTO博客，原文链接：http://blog.51cto.com/wengyuli/587816，如需转载请自行联系原作者

码农公寓

编写WCF服务端

相关文章