MPEG4IP Project
MPEG4IP Project
The MPEG4IP project provides a standards-based system for encoding,
streaming, and playing MPEG-4 encoded audio and video. To achieve
this we've integrated a number of existing open source packages, and
also created some original code to fill in the gaps.
Please note this project is intended for developers who are interested
in MPEG-4 audio and video, and Internet streaming. It is not intended
for end-users. Please read all the legal information in the file COPYING!
Also note that the primary development focus of this project is the
Linux platform. If you're going to use the package on other platforms,
especially non-UNIX platforms, you'll probably have some work to do.
Please use the SourceForge site to report problems, suggest enhancements,
ask questions, etc. The URL is http://www.sourceforge.net/projects/mpeg4ip
There is also a project web site at http://www.mpeg4ip.net/ that has some general information on MPEG4IP.
We also have a guide to MPEG4IP donated by everwicked. See
doc/MPEG4IP_Guide.pdf.
Overview
There are two ways to use MPEG4IP to create content:
The older method assumes that you've somehow managed to capture raw audio
and/or video into a file. That's the starting point from which you can use
the MPEG4IP encoding tools to create an MP4 file. The simplest method being
to use the 'mp4encode' script. Detailed instructions for this script and the
individual tools that it uses are in doc/encoding/encoding.htm
The newer method is an integrated live encoding tool called mp4live. This
program is designed to make it easy to create MP4 files or transmit live
audio/video streams over the network. It can even do both things
simultaneously! The key requirement to use this tool is to have a video
capture device and a Video for Linux (v4l) driver for it. So far we've
tested with the bttv driver for Brooktree based video capture cards, and
the qce driver for Logitech QuickCam Express USB webcams. Both of these
solutions can be acquired for about $50 US! Please see the file
server/mp4live/README.html for more information about mp4live.
Once an MP4 file is prepared, it can be placed in the content
directory of a streaming server. We typically use Apple's Darwin Streaming
Server, but any server that understands MP4 files (or hinted Quicktime
files) can be used.
When the content is encoded and available on the server, you can run
the player. Start 'gmp4player' and then enter the RTSP URL to the server
and the content, (command line works too). E.g.
Although we're focussed on streaming, the player will also playback from
a local file. E.g
That's not all! The player is not limited to local playback of MP4 files.
It can also read AVI, CMP, DIVX, AAC, MP3, MPEG 1/2 and WAV. This is useful for
debugging since the encoded data can be check independently of the MP4
file container, and known good content, such as your favorite MP3, can be
used to verify that the player is working correctly with your hardware. E.g.
And that's still not all... You can set up your player to run a playlist
by creating a simple text file with the extension .mp4plist or .mxu, with
each item (file or stream) on it's own line.
Note: If you prefer a no UI version of the player, or your system does not
include GTK or GLIB, just the bare video window, 'mp4player' is available
to fit that need.
Legal:
Please see the file COPYING.
Building and Installing
We are now (as of release 0.9.5) using make dist for our tarball. To
build, you should only have to issue the following:
We no longer include lame in our distribution; however, we require
lame version 3.92 or later for mp4live. This may change at some point,
but for now, it is required. If you wish to make without mp4live, you
can give the
If you don't have root privileges, but still wish to install the
distribution to a directory to which you do have write permission,
then here is an example of how to do that:
If this process works for you, you can skip down to the next section on
Configure Script Options. If this does not work, or you have downloaded
from CVS, the below applies.
We've built the distribution using GNU autoconf, automake and libtool.
We have attempted to follow the GNU conventions for open source packages.
This is complicated by the fact that we build on many other packages. Where
the package was already using the GNU tools, we left things alone. Where
the package was using it's own Makefile, we left things alone if it was a
complicated Makefile. If it was straightforward we replaced it with an
equivalent automake file.
If you have to install any of these tools (for example, automake is
not included with Mac OS X), find out where the others are installed
(which autoconf). If the path does not start with /usr/local, use the
For example, if autoconf is in /usr/bin, use the
Note: libtool is frequently not installed on Linux systems (autoconf,
and automake generally are). You can download libtool from a GNU mirror
site, or http://download.sourceforge.net/mpegip/libtool-1.3.5.tar.gz
In order to compile mpeg4ip from CVS, we require the following tools:
If you don't have these tools and are trying to compile from CVS, don't
complain - update to these tools, then let us know.
Be sure to read the OS specific section later before continuing here.
In general, the code should be portable, but as someone once said
"There's no such thing as portable code, just code that has been ported."
When you find problems please be sure to use the SourceForge site to tell
us what you encountered, and hopefully how you fixed it.
To build:
The bootstrap script will pass any arguments to the configure scripts.
For the curious, the bootstrap script invokes the configure scripts of
the included packages that have them, and then our own top level configure
script is generated and run. At the end of this process all the Makefile's
are ready, and setup in the correct hierarchy. If this doesn't work for you,
you're free to hack as needed ;-)
If you are using cvs_bootstrap and are running into problems where your
GTK configurations are not found, you will need to find the aclocal files
associated with gtk and glib (in particular glib-2.0.m4, glib.m4, gtk-2.0.m4,
gtk.m4). These are usually in a share/aclocal directory somewhere on your
system.
We currently search /usr/local, /usr/contrib, /opt/gnome /usr /opt/local. If these
do not work for you, find the above files (find . -name glib-2.0.m4 -print), and add the path (without the
share/aclocal) to cvs_bootstrap.
Configure Script Options
Two options of potential interest are
If you are building on a system with an Intel x86 CPU clone, you may
need to specify the configuration option
To build with IPv6 support, use the
Darwin Quicktime Streaming Server
Please note that the Apple Darwin Quicktime Streaming Server is NOT distributed
with mpeg4ip. It can be downloaded from Apple, as either
source or pre-built binaries. For those who choose the source option, below is
the express version of the build and install process for the Darwin Streaming
Server:
To build the Darwin Streaming Server:
To install the Darwin Streaming Server from the build:
See the documentation that accompanies the server on how to configure it
for your environment.
Note the default content directory is /usr/local/movies and that you can
configure the darwin server by pointing your browser at port 1220. (ie: localhost:1220)
There are some sample mp4 files available on the mpeg4ip SourceForge
download area. Also Envivio, has some sample mp4
files. We suggest first downloading one of these samples and try opening
the file with gmp4player. If that works, then try copying the file to the
streaming server's content directory (e.g. /usr/local/movies), and enter
the appropriate RTSP URL in gmp4player.
These are described in more detail in doc/encoding/encoding.htm
A few debugging tools are also included:
For playback, you get:
If you installed the Darwin Streaming Server, those executables will end
up in /usr/local/sbin.
DarwinStreamingServer Provides streaming service for MP4 files
PlaylistBroadcaster Provides multicast playlist service for MP4 files
Executable Notes
FAAC and FAAD
In our desire to make the package smaller, we have removed SNDFILE.
This has the effect of allowing FAAC to only encode raw PCM files,
unless bootstrap detects that SNDFILE is installed. We do not have
the complete version of FAAD, so the faad standalone decoder will not
build.
If the existing translation behaviour of FAAC is required, please
obtain SNDFILE and install it, then re-run bootstrap and make for
*nux base platforms. For windows based platforms, obtain a fresh
copy of faac and use it instead of the one in our package.
An even better idea would be to obtain the complete faac/faad package
from http://www.audiocoding.com.
Directory Structure
If you're going to start hacking, a map of the territory may prove useful:
mpeg4ip - top level project directory
config - autoconf files
doc - the minimal doc we've written so far
include - project wide includes
common - shared code
lib - project wide libraries
player - player specific code
server - server specific code
util - generally useful utilities
Standards Compliance
We're not only supporters of open source, we're supporters of open standards!
We've attempted to use the publically defined standards as much as possible.
Here's what we believe we are following. If you find something non-compliant,
please let us know. We certainly will want to fix it.
Here are the citations:
ISO/IEC 14496-1:2001 MPEG-4 Systems (includes MP4 file format)
The ISO/IEC documents must be purchased from either ISO (www.iso.ch) or one of the national bodies.
In the US, ANSI is the representative body, and provides an online store under www.ansi.org
The IETF RFCs can be found in mpeg4ip/doc/ietf
Note the MP4 file format is derived from Apple's QuickTime file format.
That specification is Apple Computer QuickTime File Format, June 28 2000
and is available here, or through the
QT developer site.
We're also involved in the Internet Streaming Media Alliance (ISMA)
which seeks to standardize the protocols and formats used for streaming.
We believe MPEG4IP is interoperable with that organization's 1.0 Technical
Specification. For more information see http://www.isma.tv
Latency in mpeg4ip
There have been enough questions to merit a place in the README about
the latency in mpeg4ip.
First of all, mpeg4ip is not designed for video conferencing or real
time display of data. Most streaming products are not (look at Real
or Quicktime's buffering schemes - QT buffers 3.0 seconds of data,
while Real can buffer up to 30-40 seconds).
When you need to look at latency in a streaming environment, you need
to look at each potential piece. In the simplest case (mp4live to
mp4player), there are 5 potential places where latency can take place -
at mp4live, in the kernel IP transmit stack, in the network, in the
kernel IP receive stack and in the player.
Mp4live has very little latency. In video, it tends to be 1 frame. In
audio, it tends to be 3-4 audio frames (if using AAC, frames tend to
be 1024 samples. MP3 tends to be 1152 samples, but can change based on
sampling frequency and bit rate. Samples can be converted to seconds by
dividing by the sample rate).
The kernel IP stacks can have some latency built in, as well. Probably
not too much, but you should be aware that it can exist (look at a sniffer
trace of DSS output and you'll see what I mean).
Network latency can occur, as well. Collisions, etc, can happen, especially
if server and player are not on the same network.
Finally, the player has latency as well. For streaming, we tend to buffer
2 seconds for each stream. This is changeable by setting the
RtpBufferTimeMsec variable in the .gmp4player_rc file (for Windows users,
you'll have to change the registry to do so). The value is in milliseconds,
and you need to set it to a non-zero value (0 indicates the default value).
Player Information
See the new README.html in player/src.
Known Problems
This is due to an error in the thread scheduling code that causes a problem
with a delay of less than the thread scheduler quantum (200 msec in
some versions, 20 in others). Since the average delay used is 9
to 10 msec (less as we get closer to the video rendering time), this
can have a great effect on video playback.
You can get around this error by rebuilding your libc, after changing the
However, as the Windows run-time library only provides up to 1 millisecond
resolution (0.001 sec), such measurement on Windows machines on a LAN may
be very inaccurate.
A typical RRT between two hosts on a LAN is around 0.5 millisecond.
Measurements between Linux/Unix boxes have no such a problem.
Neither QT or Real appear to understand mpeg1/2 or mp3 content in a .mp4
container file for local playback.
=== END OF README ===
Overview
Legal
Building and Installing
Configure Script Options
Darwin Quicktime Streaming Server
OS Supported
Executables
Executables Notes
Directory Structure
Standards Compliance
Latency in mpeg4ip
Player Information
Known Problems
$ gmp4player rtsp://myserver.mydomain.com/mycontent.mp4
$ gmp4player mycontent.mp4
$ gmp4player mymusic.mp3
./bootstrap <params for configure>
make
make install (needs root privileges).
--disable-mp4live or --disable-server arguments. Neither
the bootstrap or cvs_bootstrap scripts will allow you to build without
lame installed or one of the above arguments.
mkdir -p $HOME/local/bin $HOME/local/lib
./bootstrap --prefix=$HOME/local
make
make install
configure --prefix=<path before /bin on other packages>
configure --prefix=/usr
command when installing automake or libtools.
gcc 3.2 or greater
libtool 1.4.3
autoconf 2.53
automake 1.6
./cvs_bootstrap <arguments to pass to configure scripts>
make
make install (optional, typically need root privileges)
--disable-server and --disable-player
which disable the building of the server and player respectively. By
default both server and player are built.
--enable-mmx=no. The
configuration script automatically detects an x86 target CPU and
enables MMX assembly code in the build (if the NASM assembler is available).
If your CPU doesn't support MMX instructions you will want to disable
this feature. The configure script will also check for the minimum
version of nasm supported; we require 0.98.19 or greater.
--enable-ipv6 command option.
cd DSS4
./Buildit
cd DSS4
mkdir Dist
./DSS_MakeRoot -f Dist
cd Dist
./Install (need root privileges)
mp4live
Integrated, live encoding to file or network - Linux only
avi2raw
Extracts raw audio/video tracks from an AVI file
lboxcrop
Vertically crops raw video to a new aspect ratio
faac
Encodes raw audio into MPEG-4 AAC encoded audio
mp4venc
Encodes raw video into MPEG-4 encoded video using ISO codec
mp4creator
Creates and hints audio/video tracks to an mp4 file. Can also be used to extract tracks and reconstruct AAC headers
xvidenc
Encodes raw video into MPEG-4 encoded video using the Xvid codec
mp4encode
Front-end script to the above encoding tools
mp4extract
Utility to extract tracks from an MP4 file (deprecated)
mp4dump
Utility to dump MP4 file meta-information in text form
mp4trackdump
Utility to dump MP4 file track information in text form
mp4info
Utility to display MP4 file summary
avidump
Utility to display AVI file summary
yuvdump
Utility to display a raw video file on the screen
gmp4player
Simple graphical interface player
mp4player
Bare video window with sync'ed audio
|-> encoding - how to encode content
|-> ietf - copies of the relevant RFC's
|-> mcast - how to multicast
|-> mp4v2 - man pages for mp4v2 library
|-> programs - man pages for core programs
|-> video
|-->
libmpeg32 - mpeg1/2 encoder/decoder
|-->
mpeg4 - ISO MPEG-4 video encoder/decoder
|-> SDL - Simple DirectMedia Layer
|-> avi - AVI file format
|-> bitstream - MPEG style low level bitstream utlity
|-> config_file - Configuration file utility
|-> gnu
|-->
getopt - gnu getopt routines
|-> mp4 - MP4 (aka MOV/Quicktime) file format library
|-> mp4v2 - new MP4 library written from scratch
|-->
test - contains some test programs
|-->
util - contains new mp4dump and mp4extract utilities
|-> mpeg2t - mpeg2 transport stream utilities
|-> msg_queue - SDL based Inter-thread messages utlity
|-> rtp - UCL RTP
|-> sdp - Our own SDP
|-> win32 - libary files need for MS Windows
|-> xvid - xvid video encoder/decoder
|-> lib - libraries specifically for the player
|-->
audio
|--->
faad - FAAD AAC decoder
|--->
mp3 - MP3 decoder
|-->
libhttp - Our own http client
|-->
rtsp - Our own RTSP client
|-> plugin - home of player plugins
|-->
audio - audio plugins
|--->
raw - raw audio plugin
|-->
rtp - rtp bytestream plugins
|--->
h261 - h261 rtp plugin
|--->
isma_audio - isma audio format plugin
|-->
video - video plugins
|--->
raw - raw video plugin
|--->
mpeg3 - mpeg1/2 video plugin
|--->
xvid - xvid video plugin
|--->
h261 - h261 decoder
|-> src - the player executable
|-->
codec
|--->
aa - aac plugin
|--->
mp3 - mp3 plugin
|--->
mpeg4 - mpeg4 ISO decoder plugin
|--->
wav - wav plugin
|-->
osx - Mac OS X UI
|-->
win_common - common windows code
|-->
win_client - windows client
|-->
win_gui - windows gui code.
|-> audio
|-->
faac - AAC encoder program
|-> mp4creator - create and hint A/V tracks to an mp4 file
|-> mp4live - mp4 live encoding interface
|-->
gui - gtk gui for mp4 live.
|-> util
|-->
avi2raw - extract raw A/V data from AVI files
|-->
avidump - dump AVI meta information.
|-->
lboxcrop - vertically crop raw video
|-->
mp4encode - front-end script to simplify encoding process
|-->
rgb2yuv - rgb to yuv converter
|-->
xvidenc - command line interface to Xvid MPEG-4 encoder
|-> video
|-->
H26L - ITU H.26L TML 9.4 reference video encoder (EXPERIMENTAL)
|-> iptv - read Cisco IP/TV programs from a content manager
|-> yuv - simple tools for examining raw video
ISO/IEC 14496-2:2000 MPEG-4 Video
ISO/IEC 14496-3:1999 MPEG-4 Audio (includes AAC)
ISO/IEC 11172-3:1993 MPEG-1 Audio (includes MP3)
ISO/IEC 13818-3:1998 MPEG-2 Audio (includes extensions to MP3)
ISO/IEC 13818-7:1997 MPEG-2 AAC
IETF RFC 1889 & 1890
RTP
IETF RFC 2326
RTSP
IETF RFC 2327
SDP
IETF RFC 2250
RTP Payload for MPEG-1/2
Note: we're just using the audio part for MP3 (and video reception for the player).
IETF RFC 3119
A More Loss-Tolerant RTP Payload Format for MP3 Audio
IETF RFC 3016
RTP Payload for MPEG-4 Audio/Visual
Note: we're implementing the MPEG-4 video part
IETF draft-ietf-avt-mpeg4-simple-06.txt
work in progress
Note: we're implementing the AAC audio part, and for the next
release will finalize implementation.
--aac-old-file-format parameter to mp4creator.
If you need to generate new aac files, you can use the mp4creator --aac-old-file-format, then extract the track with the mp4creator
--extract command.
RtpIpPortMin=<port> and
RtpIpPortMax=<port>
in the .gmp4player_rc file created in your home
directory. The IETF recommends a range of 6970 to 6999.
#define OLD_SURFACE in player/src/video.cpp.
THREAD_SCHED_USECS to 20000 (or lower) from 200000 in thread_private.h.
LimitAudioSdlBuffer config
variable to 1 in the .gmp4player_rc file. This seems to occur on a
Soundblaster Live, Red Hat 7.3 machine.
RRT = Time_RR - Time_SR - Time_delay_at_client.
-timescale=90000 when using mp4creator with
mp3 files. If you're using mp4live to stream audio, use the
rtpUseMp4RtpPayload14 config file option.