GStreamer for all, all for GStreamer.

By Georges Auberger Georges Auberger Permalink

GStreamer is one of the media core that Songbird currently supports. Today, it is packaged with Linux builds only. On other platforms we currently support VLC (Mac & Windows), Quicktime (Mac) and Windows Media (Windows). We’re considering standardizing on GStreamer as Songbird’s default media core across all the platforms we support.

Let me share some of our motivations for doing so. GStreamer is a media pipeline that Songbird can have full visibility into. Its architecture allows for community-generated plugins to be installed to enable features such as: equalizer, crossfade, gapless playback, volume normalization and effects. GStreamer provides a centralized media pipeline that has full control/visibility around what it can play and what it cannot. This would enable Songbird to detect media items that cannot be played back because of missing components and ask the user if Songbird can install those components.

Songbird_Media_Core_Architecture.png

Here is a list of criteria we took into consideration:

  • Robust architecture. GStreamer architecture is inspired by DirectShow and Quicktime, two proven media pipelines. It implements a push/pull model and has a well defined event model.
  • Graph-based structure to allow chaining of filters (decoding -> encoding). This is useful to minimize disk access during transcoding for instance.
  • Encoding & Decoding (audio, video, subtitles). Basic things to expect from a media core.
  • Support for monolithic plugins. Allows for a plugin to encapsulate many operations such as reading a file, decoding it and sending the output to a sound card. This is particularly useful for DRM implementations.
  • Content identification. GStreamer is based on mime/type, which integrates with the web well.
  • Network transports (http, https, rtsp, mms).
  • Cross platform. Windows, OS X, Linux.
  • Threadsafety, performance, low latency.
  • Source & Sink filter API. Useful for supporting things such as UPnP.
  • License terms friendly towards open source and commercial licensing.
  • Developpers and user community support.
  • Clean and well documented APIs.

We’d love your feedback on this. We will continue to support multiple media core (Quicktime, Windows Media, VLC), but by defaulting to GStreamer on all platform, we can obtain more fine grain control of the playback experience and enable community-generated plugins that can be cross platform.

Both comments and trackbacks are currently closed.

1 Trackback

  1. By Songbird 1.0 a review « Whitenoise Dec 14, 2008 9:52 pm

    [...] view.The indexing of songs was pretty fast. My 12G song library got indexed within 3 minutes!  Songbird1.0 uses GStreamer,a cross platform  multimedia engine for its [...]

24 Comments

Subscribe
  1. obe1kenobi Jan 30, 2008 5:47 pm Permalink

    I’m in favor of anything that allows more cool features, which this does.

  2. danburke Jan 30, 2008 6:32 pm Permalink

    I’m also in favor of using GStreamer. Primarily because of the potential new features that will help match the features of iTunes, but also because I feel like it would be better to use the same media core on all platforms, to keep consistency on all operating systems.

  3. Zero456 Jan 30, 2008 7:34 pm Permalink

    I agree as well. GStreamer works flawlessly in Songbird under Linux and the effort for standardization should make Songbird easier to keep up-to-date and build. My only concern is whether or not the restricted packages (Ones that potentially have license restrictions in certain parts of the world) will be included or not in the default install.

  4. erudagrian Jan 30, 2008 9:01 pm Permalink

    I think is a very good idea I am a newbie developer but this helps a lot on the propouse of this proyect to make easy and accesible the participation for all the comunity around the world, besides I think is great to use open source also in the very heart of songbird and not only in his feathers

  5. kourge Jan 30, 2008 9:25 pm Permalink

    If GStreamer can handle QuickTime and Windows Media files through decoders (which I’m assuming it can from the diagram) then GStreamer’s definitely the way to go. Visualizations FTW!

  6. jigarshah Jan 30, 2008 11:59 pm Permalink

    Its essential to have such thing to make it extensible. Like enhancing equalizer etc..

  7. skyul Jan 31, 2008 12:14 am Permalink

    It is a great idea to support GStreamer on all platforms because it will reduce maintainence costs.

    However, nobody still seems to use GStreamer on Windows yet, and there are known problems such as random crahses while playing MPEG4 video codecs (for example, XViD).

    On GStreamer mailing list, I heard that SongBird team is creating a directshowwrapper for GStreamaer. Is it almost finished? What’s your schedule on this issue?

    I am a developer working on Chameleo project (http://www.chameleo.org/). It is basically a video player (or media platform) using GStreamer as its core. Our team also works on GStreamer directshow wrapper. I hope we could avoid duplicate work if you’re almost done with it.

  8. Pino Jan 31, 2008 12:17 am Permalink

    Way to complex for me, all I can say is…

    Cool picture! (only the farting is stupid)

  9. cwilliams Jan 31, 2008 2:40 am Permalink

    This move would be excellent if it would allow Winamp style plugins (Input, Output, Visualization, DSP).

    There are several file formats that have input plugins for Winamp that I use. Foobar also has a wrapper plugin that allows me to use Winamp input plugins. There are of course excellent visualizations like Milkdrop.

    Songbird needs input/output/vis/dsp plugins. If the move to Gstreamer sets the stage for that, then great! Keep up the excellent work.

    Of course, Songbird would have to perform well with my 70k+ library first…

  10. dgmurdockiii Jan 31, 2008 10:17 pm Permalink

    Why not just use video lan on mac, Windows, and linux and update to latest version and u will be very surprised on what it can do and i mean the latest nightly version

  11. tylerstyle Feb 1, 2008 12:12 am Permalink

    A unified system is always the way to go. It helps easing the cross-platform usability and maintenance, which usually I’m stuck with doing for all people I know.
    One core, same Problems, same Solutions.

    I don’t know enough to either direct you to vlc, or gstreamer. But I am faithful, that you can do that on your own. I haven’t had unsolvable problems with both of ‘em.

    Cheers for open-source unification.

  12. stevel Feb 1, 2008 8:41 am Permalink

    VLC is GPL’d whereas GStreamer is LGPL’d. This means we wouldn’t be able to use VLC in our partner/licensee builds.

  13. dgmurdockiii Feb 1, 2008 2:43 pm Permalink

    why not just make a version that just for partner/licensee have it uses GStreamer and then all the other ones uses vlc

  14. stevel Feb 1, 2008 5:54 pm Permalink

    Because that wouldn’t be standardising on one media core. :) That would be multiple media cores still. I’m not clear, what do you think VLC gets us that GStreamer doesn’t?

  15. dgmurdockiii Feb 2, 2008 11:25 am Permalink

    VLC is good at handling incomplete files and so can be used to preview files being downloaded

    Input
    UDP/RTP unicast or multicast, HTTP, FTP, MMS, DVDs, VCD, SVCD, CD Audio, DVB (only on Linux and on Windows in development versions), Video acquisition (via V4l and DirectShow), RSS/Atom Feeds, and from files stored on your computer.
    Container formats
    3GP,[13] ASF, AVI, FLV, MKV, QuickTime, MP4, Ogg, OGM, WAV, MPEG-2 (ES, PS, TS, PVA, MP3), AIFF, Raw audio, Raw DV, FLAC. MXF, VOB.
    Video formats
    Cinepak, DV, H.263, H.264/MPEG-4 AVC, HuffYUV, Indeo 3,[14] MJPEG, MPEG-1, MPEG-2, MPEG-4 Part 2, Sorenson (thus enabling direct playback of the modified Sorenson H.263 encoded videos downloaded from YouTube), Theora, VC-1,[15] VP5,[15] VP6,[15] WMV.
    Subtitles
    DVD, SVCD, DVB, OGM, (partial)SubStation Alpha,[16] SubRip, (partial)Advanced SubStation Alpha,[16] MPEG-4 Timed Text, Text file, Vobsub, MPL2,[17] Teletext.[17]
    Audio formats
    AAC, AC3, ALAC, AMR,[13] DTS, DV Audio, FLAC, MACE, MP3, QDM2/QDMC, RealAudio,[18] Speex, Screamtracker 3/S3M, TTA, Vorbis, WavPack,[19] WMA.

    [edit] Output formats for streaming/encoding

    VLC can transcode into several formats depending on the operating system.

    Container formats
    ASF, AVI, FLV,[17] MP4, Ogg, Wav, MPEG-2 (ES, PS, TS, PVA, MP3), MPJPEG, FLAC, MOV
    Video formats
    H.263, H.264/MPEG-4 AVC, MJPEG, MPEG-1, MPEG-2, MPEG-4 Part 2, VP5,[15] VP6,[15] Theora, DV
    Audio formats
    AAC, AC3, DV Audio, FLAC, MP3,[20] Speex, Vorbis
    Streaming protocols
    UDP, HTTP, RTP, RTSP, MMS, File *

    VLC is popular for its ability to play the video content of incomplete, unfinished, or damaged video downloads before the files have been fully downloaded. (For example, files still downloading via BitTorrent or eMule) This is because it is a packet-based player.
    * The player also has the ability to use libcdio to access .iso files so that the user can play files on a disk image, even if the user’s operating system does not have the capability of working directly with .iso images.
    * VLC supports all codecs and all file formats supported by FFmpeg. This means that DVD Video and MPEG-4 playback as well as support for Ogg and Matroska (MKV) file formats works “out of the box”. However, this feature is not unique to VLC, as any player using the FFmpeg libraries, including MPlayer and xine-lib-based players, can play those formats without need for external codecs. VLC does, however, also support codecs that are not included in FFmpeg.
    * VLC media player has some filters that can distort, rotate, split, deinterlace, mirror videos, create display walls, or add a logo overlay. It can also produce video output as ASCII art.
    * VLC media player can play high definition recordings of D-VHS tapes duplicated to a computer using CapDVHS.exe. This offers another way to archive all D-VHS tapes with the DRM copy freely tag. An entire library of tapes (or about 50 D-VHS 2 hours long) can be placed on a single terabyte sized hard drive. However, they must be imported by playing the tapes in real time, and can’t simply be copied as data files.
    * Using a FireWire connection from cable boxes to computers; VLC can stream live, unencrypted content to a monitor or HDTV.
    * VLC media player can display the playing video as the desktop wallpaper.
    * On Microsoft Windows, VLC also supports the Direct Media Object (DMO) framework and can therefore make use of some third-party DLLs.

  16. dgmurdockiii Feb 2, 2008 11:30 am Permalink

    KDE developers have criticized GStreamer for not offering a stable ABI. This problem eventually led to the development of Phonon, a simplified multimedia-framework for KDE4, which provides wrappers for various multimedia frameworks, including GStreamer.

  17. tpm Feb 4, 2008 6:45 am Permalink

    This is not really true. GStreamer offers ABI stability for each stable series. However, it is likely that at some point (in 2-3 years time maybe, no one really knows) the GStreamer developers will abandon the current GStreamer-0.10 series and start a new stable series with a changed API/ABI, which would leave the old GStreamer-0.10 series unmaintained.

    The KDE folks were looking for something that is ABI stable and maintained for at least 5 years, which GStreamer currently can’t offer, since it’s still in the process of finding the perfect multimedia API that can do it all.

  18. Uraeus Feb 4, 2008 7:33 am Permalink

    @dgmurdockiii: You are aware that there is a ffmpeg plugin for GStreamer too right?

  19. mig Feb 14, 2008 5:09 pm Permalink

    VLC should be kept around as an optional add-on that you can install after the fact if you prefer it.

    This assumes we can convince the product people to agree that it’s important to do so, of course.

  20. Jeff McNeill Jun 17, 2008 2:49 pm Permalink

    So if I read this right, the only reason not doing VLC is because of the licensing, correct? Can VLC do a dual licensing option with you folks?

  21. stevel Jun 17, 2008 2:51 pm Permalink

    @Jeff: It’s mostly the licensing… even if VLC relicensed as LGPL instead of GPL (which would be cool, no doubt), there’s still the legality of its potentially-patent-infringing codecs built into it.

  22. IUnknown Aug 30, 2008 9:14 am Permalink

    GPL is sxxk! What freedom do you get from GPL? That license force you to join them once you touch it. That’s a stupid idea. It like a virus, that’s why people hate GPL, so do I.

  23. falzueta Jan 15, 2009 5:17 am Permalink

    Gstreamer… not reliable at this moment. I don’t think is a good move, mainly because of some errors playing some popular formats like .mp3 As you can see not few people have this problem. http://getsatisfaction.com/songbird/topics/songbird_has_encountered_the_following_media_core_error_internal_data_flow_error

  24. valeswararao Jun 25, 2009 2:06 am Permalink

    Is There is any player to view images using gstreamer plugins..?