odroid-m1s:application_note:mpp

Rockchip MPP (Media Process Platform) on Gstreamer

Tested on Ubuntu 20.04 with the Linux kernel 4.19.219 and several packages.
It is supported on the 5.10.z.
ODROID-M1 is faster then ODROID-M1S almost 10%.
ODROID-M1 and ODROID-M1S Decoder compare result is here. UHD, QHD and FHD.
On the MIPI display, you should use 'plane-id' property on the rkximagesink plugin.
target
$ ... ! rkximagesink plane-id=115

And, recommend to use 'width', 'height' and 'rotation' properties on the mppvideodec plugin.

target
$ ... ! mppvideodec rotation=720 height=1280 ! rkximagesink plane-id=115
video play layer and window is not matched.

Rockchip provides the Media Process managing packages for the Gstreamer.
So you can use Rockchip MPP with it on the Gstreamer.
On the page, you can know how to install and use some plugins and their properties.

Let start with with update your system

target
$ sudo apt update
$ sudo apt upgrade
$ sudo reboot

After rebooting, check the kernel version and build date. Your system must be the same or later version.

target
$ uname -a
Linux gnome-desktop 4.19.219-odroid-arm64 #1 SMP Fri, 06 Oct 2023 02:01:53 +0000 aarch64 aarch64 aarch64 GNU/Linux

Install

target
$ sudo apt update
$ sudo apt upgrade
$ sudo apt install librga2 librockchip-mpp1 gstreamer1.0-rockchip1 gstreamer1.0-plugins-bad gstreamer1.0-tools

The minimal version of required packages for media processing must be checked.

target
$ dpkg -l | grep -e librga2 -e librockchip-mpp1 -e gstreamer1.0-rockchip1
ii  gstreamer1.0-rockchip1                     5:1.14-4+202311281218~focal           arm64        The Gstreamer plugins for Rockchip platforms.
ii  librga2                                    5:2.2.0-1+202311151049~focal          arm64        Userspace interface to Rockchip RGA 2D accelerator
ii  librockchip-mpp1                           5:1.5.0-1+202311151047~focal          arm64        Media Process Platform

And, you can also check it with the gst command.

target
odroid@gnome-desktop:~$ gst-inspect-1.0 |grep Rockchip
rockchipmpp:  mppjpegdec: Rockchip's MPP JPEG image decoder
rockchipmpp:  mppvideodec: Rockchip's MPP video decoder
rockchipmpp:  mppjpegenc: Rockchip Mpp JPEG Encoder
rockchipmpp:  mppvp8enc: Rockchip Mpp VP8 Encoder
rockchipmpp:  mpph265enc: Rockchip Mpp H265 Encoder
rockchipmpp:  mpph264enc: Rockchip Mpp H264 Encoder

How to use

It is recommended to play on the HDMI Display. On the MIPI Display, Please try with advanced way.

You can play the media via playbin/playbin3 commands.

target
$ gst-launch-1.0 playbin3 url=file:///${PWD}/<video_file_name>

if you play above FHD resolution media, you should add more properties.

target
$ gst-launch-1.0 playbin3 video-sink=rkximagesink url=file:///${PWD}/<video_file_name>

You can play the media file via the commands.

target
$ gst-launch-1.0 files location=<file_path_n_file_name> \
        ! <qtdemux|matroskademux|tsdemux> name=demux \
        demux.audio_0 \
        ! queue ! decodebin ! audioconvert ! audioresample ! autoaudiosink \
        demux.video_0 \
        ! queue ! <h264parse|h265parse> ! mppvideodec ! rkximagesink

you have to find correct container of media file and codecs. then select the correct demuxed and parser. Here are example list for each demuxed's support extension files.

demuxer Support container
tsdemux MPEG-ts(.ts)
matroskademux Matroska(.mkv)
qtdemux MPEG4(.mp4, .mov)
avidemux Audio Video Interleave(.avi)

Each plugin has property to control it. you can find by gst-inspect-1.0 commands.

target
$ gst-inspect-1.0 rkximagesink
Factory Details:
  Rank                     secondary (128)
  Long-name                Video sink
  Klass                    Sink/Video
  Description              A standard X based videosink
  Author                   Julien Moutte <julien@moutte.net>
 
Plugin Details:
  Name                     rkximage
  Description              Rockchip X/DRM Video Sink
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstrkximage.so
  Version                  1.14.4
  License                  LGPL
  Source module            gst-rockchip
  Binary package           GStreamer Rockchip Plug-ins
  Origin URL               Unknown package origin
 
GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstRkXImageSink
 
Implemented Interfaces:
  GstNavigation
  GstVideoOverlay
 
Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
              framerate: [ 0/1, 2147483647/1 ]
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
 
Element has no clocking capabilities.
Element has no URI handling capabilities.
 
Pads:
  SINK: 'sink'
    Pad Template: 'sink'
 
Element Properties:
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096
...

Here is some useful properties.

mppvideodec

  • format - set output format.
    • target
      $ get-launch-1.0 \
      ...
      ! mppvideodec format=I420
      ...
  • width & height - scaling the video size. max input is 4096×2304. default value is 0.(no scaling)
  • rotation - rotate the video. 0, 90, 180, 270 are available.
    Please use it with width & height properties by correct size. if not you can see like this message from kernel log. and the video doesn't play.
    target
    ...
    Password: [  144.450925] rga2: unknown ioctl cmd!
    [  144.554357] [drm:vop2_plane_atomic_check] *ERROR* Invalid source: 1726x3840. max input: 4096x2304
    [  144.619220] [drm:vop2_plane_atomic_check] *ERROR* Invalid source: 1726x3840. max input: 4096x2304
    [  144.658526] [drm:vop2_plane_atomic_check] *ERROR* Invalid source: 1726x3840. max input: 4096x2304
    ...
  • crop-rectangle - Crop the video plane.
    • target
      ...
      ! mppvideodec crop-rectangle='<x, y, width, height>'
      ...

rkximagesink

  • plane-id - Assign the video play layer. when you using rkximagesink, it is required to have a exclusive hardware layer, and the plug-in will automatically find the layer to play, but the automatically found layer may not meet the requirements, so you have to manually specify the layer, the way is as follows:
    target
    $ gst-play-1.0 ... ! rkxiamgesink plane-id=115

    The 115 is the ID of the target layer, which can be confirmed through the /sys/kernel/debug/dri/0/state node. You can use the following command to list all layers:

    target
    odroid@gnome-desktop:~$ sudo -i
    [sudo] password for odroid:
    root@gnome-desktop:~# cat /sys/kernel/debug/dri/0/state|grep "plane\["
    plane[57]: Smart1-win0
    plane[79]: Smart0-win0
    plane[101]: Esmart1-win0
    plane[115]: Esmart0-win0
    plane[129]: Cluster0-win0
    plane[143]: Cluster1-win0

    This content refers to “rockchip user guide linux gstreamer”.

  • force-aspect-ratio - When enabled, scaling will respect original aspect ratio.
    when falsed, you can change aspect ratio via window resizing.
target
$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720, framerate=15/1 \
        ! mpph264enc ! h264parse ! mppvideodec ! fpsdisplaysink

Test Device - Logitech BRIO

target
$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,heigh=720, framerate=30/1 \
        ! mpph264enc ! h264parse ! mppvideodec ! fpsdisplaysink

Test Device - Logitech BRIO

target
$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! mpph264enc ! h264parse ! mppvideodec ! rkximagesink
target
$ gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! mppjpegdec ! mpph264enc ! h264parse ! mppvideodec ! glimagesink

How to get debugging log or profile performance

We suggest two ways.

Before play the gstreamer, let's set the environment parameter. then play the gstreamer.

now you can see the mpp library messages.

target
$ export mpi_debug=1 && export mpp_debug=1 &&  export h264d_debug=1 && export mpp_syslog_perror=1

You can also get the gstreamer debugging message.

target
$ GST_DEBUG=<debugging_level> gst_launch-1.0 ...

debugging_level value are 1 - 9. please check it

You can also profile decoding performance. we suggest to use “fpsdisplaysink”.

target
$ gst-launch-1.0 filers location=<file_path>
        ! <qtdemux|matroskademux|tsdemux|avidemux>
        ! <h264pasre|h265parse>
        ! mppvideodec
        ! fpsdisplaysink text-overlay=false video-sink=rkximagesink -v

text-overlay must be false. because it can cause performance bottleneck. and video-sink also must be rkximagesink on the rockchip board. '-V' options show command line debugging message and in this case, you can see like it.

target
...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2051, dropped: 18, current: 26.48, average: 23.74
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2060, dropped: 19, fps: 18.00, drop rate: 2.00
2023/10/05 11:25 · luke.go
  • odroid-m1s/application_note/mpp.txt
  • Last modified: 2023/11/14 11:02
  • by luke.go