streaming live video from webcam

I am trying to develop a two-way real-time video conferencing system, and while I can capture raw audio samples and images from the webcam, and send them to another computer via a UDP socket, but I know this is not the most efficient way to send audio and video data between clients in real time. I'm looking for information on how to access the various audio and video codecs installed on the computer, and use them to send compressed audio/video over the network. I've spent hours on google looking for this information, and it's proving to be rather difficult to track down useful information.

I'm hoping one of you has worked on something similar and knows of some resources where I can find the information I need.

thanks in advance.

I never did it, but to stream video you need DirectShow. Microsoft, to my knowledge, hasn't still provided language bindings for C# or a .Net port/translation of this library. Fortunately you have DirectShow.Net.

I've used LEADTOOLS Multimedia Developer Toolkit for a project before. It's not free, but it was really slick and easy to use.

You need to look at the documentation for the development side of the Windows Media Player. If you are going to use DirectShow you will need to look at the documentation for the DirectX Video Mixing Renderer or the VMR. You may have to use P/invoke's and or C++/CLI code to talk to C/C++ code that does the heavy lifting. I am not aware of anything in .NET as of yet that encapsulates this functionality. There is code on the internet for this very thing and once you get the VMR up and running it should be a few lines of code to get the right video input pin and video source for the player.

that just seems like a lot of work when silverlight has streaming BUILT IN, but silverlight has too many restrictions to be useful in my application. I wish they would at least provide in the .Net framework everthing that silverlight has.

DirectShow has been mentioned in a lot of the web pages I've seen about this, but I haven't been able to find any good tutorials on how to do the things I'm trying to do. EVERYTHING assumes you want to save the video to a file, and I've been having a hard time finding code that shows how to just get the raw output from the codec so I can use a UDP socket instead.

Quote:

---

Originally Posted by Elkvis

DirectShow has been mentioned in a lot of the web pages I've seen about this, but I haven't been able to find any good tutorials on how to do the things I'm trying to do. EVERYTHING assumes you want to save the video to a file, and I've been having a hard time finding code that shows how to just get the raw output from the codec so I can use a UDP socket instead.

---

I've shown you a library that has all you need to interface with DirectShow. That library comes with documentation, so that's a start.

Quote:

---

that just seems like a lot of work when silverlight has streaming BUILT IN

---

What does Silverlight have to do with any of this?

Quote:

---

Originally Posted by Elkvis

that just seems like a lot of work when silverlight has streaming BUILT IN, but silverlight has too many restrictions to be useful in my application. I wish they would at least provide in the .Net framework everthing that silverlight has.

DirectShow has been mentioned in a lot of the web pages I've seen about this, but I haven't been able to find any good tutorials on how to do the things I'm trying to do. EVERYTHING assumes you want to save the video to a file, and I've been having a hard time finding code that shows how to just get the raw output from the codec so I can use a UDP socket instead.

---

Well that's just it, you will be "rewriting" the part of directshow that saves the video to a file and the part the reads from a file essentially. When you compose your filter graph you will need on the sending end a custom renderer filter that writes out your video/audio in a protocol of your choice, and a corresponding source filter that can read these packets on the reception end and pass it further upstream. Writing these filters is not trivial, but you can download the directshow collection of base classes that simplifies it some and has some examples from the directx extras package from msdn. Most of the filter examples are also in the directshow.net sdk depending on which language you would want to implement the filter in.

There is Morgan Multimedia - RTP DirectShow Filters that uses RTP as the transport mechanism, perhaps you will find that useful.

Quote:

---

Originally Posted by VirtualAce

What does Silverlight have to do with any of this?

---

I was pointing out that if silverlight was suitable for my needs I would be using it, but since it is very restrictive (for "security" reasons), it won't be adequate for my needs.

Quote:

---

Originally Posted by Elkvis

I was pointing out that if silverlight was suitable for my needs I would be using it, but since it is very restrictive (for "security" reasons)

---

It is safe to hold your breath on that. If you aren't in a hurry and don't mind waiting for Silverlight 5 beta sometime soon this year, you will no longer be required to run in a sandbox: The Future of Microsoft Silverlight | Microsoft Silverlight

From access to p/Invoke, to COM interoperability, to documents folder access, the only thing distinguishing a silverlight application from a windows .net application will be what device you are using to access it.

Sidenote: If you have any complaints about the fact they are basically removing all security features that made silverlight safe, you will waste your breath, like I did. It seems the idea many developers have of security goes more or less like this:

- Programmer A: Silverlight is great! It is very secure. You run it basically in a sandbox.
- Programmer B: Yeah. But I need to access an external API and I can't.
- Programmer A: Yeah, that part sucks. Too much security. Can't do anything.
- Programmer A: Wait...

silverlight 5 looks very interesting. perhaps it will be useful to me when it comes out.

on the subject of DirectShowNet, I have it working, but the documentation is basically nonexistent, except for the official MSDN documentation of the API, which of course is in C++, but hopefully it translates well enough for me to understand what's going on. I've never worked with directx before, and I really don't even know where to start, but I guess I'll just have to figure it out.

Quote:

---

on the subject of DirectShowNet, I have it working, but the documentation is basically nonexistent, except for the official MSDN documentation of the API, which of course is in C++, but hopefully it translates well enough for me to understand what's going on. I've never worked with directx before, and I really don't even know where to start, but I guess I'll just have to figure it out.

---

This is what I have found as well. Good documentation for both the managed and unmanaged sides of this beast are simply not out there. Fortunately there are some books on Amazon that actually do talk about how to go about correctly using DirectShow in applications and in games (with a full screen or windowed device and with multiple movies playing).

The video mixing renderer for DirectX9 is nice but is sorely lacking in documentation and it is not apparent from Microsoft code samples (which are very trivial implementations of the feature) exactly how to perform certain operations on the movies or how to communicate with the VMR and/or control it. Googling is equally as worthless since most of the samples are derivatives of the samples in the Windows SDK and several people recommend not using the VMR9. It is possible to put live webcam feeds onto DCs (IE: textures) and then to output or blit the DC to another DC for display. I cannot go into more detail about this but I can tell you it most certainly works. So if you can use the managed side of the VMR it may save you some work with the downside that you may need an XNA device.