MediaStreamSource sample for Silverlight

< Installing Silverlight Offline | Home | Silverlight and Web Service Errors >

via Twitter:

MediaStreamSource sample for Silverlight

posted @ Wednesday, October 01, 2008 10:29 AM | 16 Comments

You may be reading the title and wondering what is MediaStreamSource? MediaStreamSource is a piece of the Silverlight runtime that removes a the influence of a media file's container, giving developers direct access to APIs for manipulating encoded elementary audio and video streams.

Huh?

Basically it can enable you as the developer to implement file parsers/etc. in managed code instead of Silverlight, enabling support beyond the native built-in formats for media. There hasn’t been much information about these types of topics, and I’d agree that for the mainstream, they may be a bit more advanced media scenarios for when the default containers and formats aren’t enough for your use. Outside of the MSDN documentation there hasn’t really been any good samples of this use either.

Until now.

One of the program managers on the Silverlight media team, Larry Olson, has just provided a detailed public sample of the MediaStreamSource in action on the MSDN Code Gallery site. He calls the effort ManagedMediaHelpers. The project contains:

Silverlight class library (MediaParsers) which has helper classes for working with MP3 files, including being able to find the right point in an MP3 to begin playback.
Silverlight test project for NUnit
Silverlight class library (Mp3MediaStreamSource) which has logic for using MP3 file streams
Silverlight Application Demo which shows the interaction between a MediaElement and a MediaStreamSource

Why would you want this? Larry outlines in the project:

“For one thing, having access to elementary streams means that developers can now implement scenarios that other solutions haven't necessarily provided thus far. One example of this is adaptive streaming or multi-bitrate support as was seen during the 2008 Olympics.

For another reason, having access to elementary streams allows developers to implement scenarios that the Silverlight runtime hasn't had a chance to implement yet or that the runtime might not be able to implement in the same timeframe that a developer wants it. Examples of this could be, RTSP:T protocol support, SHOUTcast protocol support, seamless audio looping, ID3 v1 and ID3 v2 metadata support, and many other scenarios.”

This is a great sample and source for those working with media within Silverlight. Right now it is audio only, MP3 support, but gives you an idea of the MediaStreamSource API and functionality you could implement.

tags: silverlight, media, xaml, mediaelement, mediastreamsource, managedmediahelpers, adaptive streaming, mbr, shoutcast, mp3

This work is licensed under a Creative Commons Attribution By license.

Comments

Perry

10/1/2008 3:28 PM | # re: MediaStreamSource sample for Silverlight

Sounds pretty cool and definitely worth a look. Do you know if this can support basic digital signal processing? My use case would be to create an equalizer type function or to be able to read the pitches to create a those bouncing bars like in typical media players.
Parag Mehta

10/3/2008 2:35 AM | # re: MediaStreamSource sample for Silverlight

I noticed this was there, but the uses weren't documented before this.

This means technically we can create a FLV MediaStreamSource right ? If we get the parsing logic of FLV in C#, we can easily create a native FLV MediaStreamSource which can be used to play Flash Files natively in Silverlight :)

Cool. The only thing at this point preventing me to put hours into this is someone will do it eventually :) (hopefully as OpenSource project)!.

This is another problem Tim with new technology, people aren't willing to put many hours into new things because for eg, in this instance I will be scared to put too much resources on this since I don't know if Microsoft will come up from their own side pretty soon! and will make my code a waste.
cuco

10/3/2008 12:50 PM | # re: MediaStreamSource sample for Silverlight

Larry,

Thanks for the ManagedMediaHelpers project.

I have started the learning curve on Silverlight 2.0. I'm currently developing a project that will allow me to manipulate VC1 video within Silverlight. More specifically, I plan to have a Silverlight Media Control library, written in C# and C++/CLI, that will allow me to manage and display VC1 video using Direct3D. In order to do so, I need to write some functions into a library that will allow me to decode each frame of VC1 video.

Is there an API in Silverlight that will allow me to decode the VC1 video?

Regards,

Cuco
loarabia

10/3/2008 4:53 PM | # re: MediaStreamSource sample for Silverlight

To the first two comments:
The MediaStreamSource as it exists today actually doesn't quite enable your scenarios (DSP & FLV) unless you do some form of transcoding. IE: Mp3 -> RAW or FLV -> WMV. Today's MediaStreamSource requires encoded content in a format Silverlight supports (WMV/WMA/MP3). However, you are not the first people to call out these scenarios and we are listening.

To the third comment.
I am afraid that there actually isn't an API today in Silvelright that allows you to directly decode VC-1. My best recommednation would be to start with the ASF file format specification on MSDN and then move onto the actual decoding of the VC-1 content itself. However, I think I should also note that Media Foundation / DirectShow is likely a better candidate right this moment for doing a frame by frame decode of VC-1 and you can at least wrap that in Mananged code although you cannot wrap it in a way that will work in Silvelright. If you wrote a managed VC-1 decoder based on the SMPTE spec, you could go far towards this scenario inside of Silverlight but that would be a hard thing to do.

Regards,
Larry Olson
cuco

10/6/2008 9:03 AM | # re: MediaStreamSource sample for Silverlight

Larry,

Thanks for the info.

I think I'll try my hands at writing a managed decoder for VC1. I'm currently going through the SMPTE specs and test materials for VC1.

Questions:

1 - Are the threading classes from .Net part of the Framework in Silverlight 2.0? I would like to make my VC1 video libraries heavily threaded.

2 - Is Direct3D compatible with Silverlight 2.0? That is, will I be able to use Direct3D for displaying the decoded VC1 video from within a Silverlight control?

Regards,

Cuco
Sanj

10/8/2008 9:38 AM | # re: MediaStreamSource sample for Silverlight

Thanks Larry,

The unwrapping of the MP3 bitstream makes sense but obviously WMV demux and decode would be vastly more useful because that's going to be the typical use case. I'm sure you know that already and we're looking forward to seeing that sample. In advance of that, it would be great just seeing a breakdown in theory of the differences between the MP3 sample and WMV (besides the obvious).

Thanks!
S
Matthias

10/13/2008 12:41 AM | # re: MediaStreamSource sample for Silverlight

I there anybody knowing a way to implement FLV playback inside Silverlight. I would like to implement YouTube video playback inside my Silverlight app.
Cagdas Gerede

10/21/2008 9:56 PM | # re: MediaStreamSource sample for Silverlight

This feature is great to have a finer control on the media playback.

However, unless Silverlight increases the number of formats it supports such as H.264, AAC, PCM (see Flash for more), this will not be very useful. Not a lot of businesses can afford the costs of decoding media in MP3. It is a great feature, but please, we need more formats to be supported.
timheuer

10/22/2008 10:02 AM | # re: MediaStreamSource sample for Silverlight

Cagdas: We've announced support for H.264 and AAC coming in a future version soon. We simply couldn't put it in this version for the time being.
Raul

10/31/2008 11:36 PM | # bug in Position?

Hi,
Do I need MediaStreamSource?
I'm simply trying to use a MediaElement to play a MemoryStream from a specified position, like this:
mediaElement.SetSource(theMemoryStream);
mediaElement.Position = new TimeSpan(0, 0, 0, 0, fromTime);
mediaElement.Play();
Strangely, Position cannot be set. It always plays from position zero.
Is this a bug?
Do I need MediaStreamSource for a workaround?
Thanks very much for any help.
Raul
daniella

11/3/2008 4:33 PM | # re: MediaStreamSource sample for Silverlight

Yes I am getting this problem to with MediaElement position not working. I think it must be bug but there is not workaround that I have heard. :(
Uthay

12/30/2008 9:06 AM | # Silverlight & Authorization plugin for Windows Media Services - Referral URL

Hi

i dont know, it is the rite place to post it, anyway here is my problem....

I'm trying an Authorization plugin for WMS to validate referral URL from UserContext, it means, WMService will stream only to request come from particular website, plugin wrote in C# and its WORKING fine with embedded Windows Media Player 9 or later.

now testing the same plugin against embedded Silverlight (ver 2.0/ASP.Net 3.5) Mediaplayer (
i'm Authorizing only at WMS_EVENT_PLAY, its working fine with WMP but Silverlight Player throwing some error

System.Runtime.InteropServices.COMException (0x8002000B): Invalid index. (Exception from HRESULT: 0x8002000B (DISP_E_BADINDEX))
at Microsoft.WindowsMediaServices.Interop.IWMSContext.GetStringValue(String pstrName, Int32 lNameHint, String& pstrValue, Int32 lOptions)
at MyWMSNameSpace.MyWMSAuthorizationPlugin.Microsoft.WindowsMediaServices.Interop.IWMSEventAuthorizationPlugin.AuthorizeEvent(WMS_EVENT& i__Event, IWMSContext io__UserContext, IWMSContext io__PresentationContext, IWMSCommandContext io__CommandContext, IWMSEventAuthorizationCallback i__ServerCallback, Object i__Context)

i tried some other events like WMS_EVENT_OPEN , WMS_EVENT_CONNECT but WMP itself throwing error.

what i'm looking for is,

1) i want to clarify whether Silverlight supports Referral URL info to WMServices??
2) if yes then at Which Event is sending the info to WMService
3) or i'm making any mistake in Authorizing or in Event Handling because i'm getting error in my plugin for other Events

here is the main part of the code
------------------------------------------------------------------------------------

//Identify the events the plug-in can authorize
object IWMSEventAuthorizationPlugin.GetAuthorizedEvents()
{
WMS_EVENT_TYPE[] l__AuthorizeEvents ={WMS_EVENT_TYPE.WMS_EVENT_PLAY};
return (object) l__AuthorizeEvents;
}

void IWMSEventAuthorizationPlugin.AuthorizeEvent
(
ref WMS_EVENT i__Event,
IWMSContext io__UserContext,
IWMSContext io__PresentationContext,
IWMSCommandContext io__CommandContext,
IWMSEventAuthorizationCallback i__ServerCallback,
object i__Context
)
{

string l__URLRefererString = string.Empty;
io__UserContext.GetStringValue
(
WMSDefines.WMS_USER_REFERER,
WMSDefines.WMS_USER_REFERER_ID,
out l__URLRefererString,
0
);

if (l__URLRefererString == "MyWebsite")
{
// value of ACCESS.
}
else
{
// value of ACCESSDENIED.
}
}

-------------------------------------------------------------------------------------------------------------------------------------

Note *** : i think i have answer for my own question 1, When i checked the WMServices Client Log , i could see the Referral URL entry for Silverlight Embedded page.

Any suggestions

Thanks in advance

- Uthay
kf2

1/20/2009 6:24 AM | # re: MediaStreamSource sample for Silverlight

hey guys. thats a very cool project. but we need a WMV sample!!
guillaume

2/20/2009 5:34 AM | # re: MediaStreamSource sample for Silverlight

Hey guys,
This is great piece of project, i monitor this waiting until the streaming capabilities will be open for any content. In this approach and for specific purpose i had to spend some time (and not waste it because i'm pretty up to date now with the ASF format) to write a C# ASF reader/writer, it work pretty well into silverlight. If anyone found it usefull please let me know ! guillaume.pelletier at dotvision.com
Ian Jones

3/19/2009 8:26 AM | # re: MediaStreamSource sample for Silverlight

It looks like Silverlight 3 will enable the idea of FLV decoding with the Raw AV Pipeline.....
Mark Kestenbaum

9/17/2009 5:51 AM | # re: MediaStreamSource sample for Silverlight

Hi Tim,

I'm actually working with this in Silverlight 3, placing a raw PCM 16 bit audio stream in the MediaStreamSource and it plays great!

The problem is, as mentioned by someone above, that setting the position has no effect. Also, reading the current position (for example, to move a slider or display the current position in a TextBlock) gives a position which keeps increasing but is not correct. (It looks like it increments about 1 second for every 7 seconds of play). Is this a bug? Or could I be doing something wrong?

Mark