Sunday, May 12, 2013

Raspberry Pi: Streaming video and microphone from Microsoft LifeCam

  1. Wireless Video and Audio streaming using JPEG format WebcamInitially, I was using YUV format of PS3 Eye and encode the data to JPEG format to stream the video data over TCP/UDP, but it was taking large cpu usage of Rpi.
    I was thinking this could be due to the JPEG encoding...
    Microsoft LifeCam supports MJPEG format, and I am hoping this will reduce the cpu usage because PI does not need to deal with encoding.
    Well, let's see...
  2. Microsoft LifeCam support format
    I used Microsoft LifeCam Cinema 720p HD Webcam, and here is the support formats.

    v4l2-ctl --list-formats

    ioctl: VIDIOC_ENUM_FMT
            Index       : 0
            Type        : Video Capture
            Pixel Format: 'YUYV'
            Name        : YUV 4:2:2 (YUYV)

            Index       : 1
            Type        : Video Capture
            Pixel Format: 'MJPG' (compressed)
            Name        : MJPEG

    If you want to check details of each format, use "v4l2-ctl --list-formats-ext" command.

  3. Start Streaming!

    TX (RPi)
    gst-launch -v v4l2src ! "image/jpeg,width=320,height=240,framerate=30/1" ! multipartmux ! tcpserversink host=x.x.x.x port=5000 sync=false alsasrc device=hw:1 ! audioconvert ! audioresample ! 'audio/x-raw-int,rate=8000,width=16,channels=1' ! udpsink host=x.x.x.x port=5001
    RX (Android phone)
    Please refer to the previous work using Android smartphone streaming.(Link)
    Raspberry Pi
    Microsoft LifeCam Cinema 720p HD Webcam
    EW-7781Uh WiFi dongle
    Android phone (Droid X)

    Video and microphone streaming was tested using MJPEG format of Microsoft LifeCam.

    Video and audio latency was reasonable (< 1 sec). 
    RPi CPU usage was less than 30% (top command)




  1. This comment has been removed by the author.

  2. Hello Takeo,

    thank you for your great tutorials regarding the implementation of video and audio-streaming with gstreamer.
    Unfortunately, we were not able to accomplish the audio-streaming with our webcam (with a built-in-microphone). The video-streaming is just fine!

    Currently we use the following simple gstreamer pipeline command to stream the video:
    gst-launch-0.10 \
    v4l2src ! \
    video/x-raw-yuv,width=320,height=240,framerate=\(fraction\)20/1 ! \
    ffmpegcolorspace ! \
    jpegenc ! \
    multipartmux ! \
    tcpserversink host=x.x.x.x port=5000

    We followed your instructions from above to add the audio-stream, but get the following error-message: "WARNING: ... no element 'audio'"

    We have installed the gstreamer-0.10 as described in an earlier post!
    Can you please help us?

    Marc & Johannes

  3. Hi Johannes,

    Thank you for your comments.
    I am wondering if device is recognized.
    Can you please try the followings?
    1. Check if audio input is recognized by ALSA
    $ cat /proc/asound/cards
    2. Check if the microphone signal is coming out from RPi audio jack
    gst-launch -v alsasrc device=hw:X ! audioconvert ! volume volume=10 ! alsasink
    * hw:X --> X should be the device number listed by the step1.


  4. Can you share your android project file? I can't solve android side