4K Video Processing and Streaming Platform on TX1

[Pages:21]4K Video Processing and Streaming Platform on TX1

Tobias Kammacher Dr. Matthias Rosenthal

Z?rcher Fachhochschule

Institute of Embedded Systems / High Performance Multimedia Research Group

Zurich University of Applied Sciences

Goal

1. Live video streaming

o In 5 minutes

2. Bottlenecks

o GPU o Kernel

2

Z?rcher Fachhochschule

Live Video Streaming

Consumer-level YUV420. Seconds-latency.

Video Sources

Mixing

Encoding

Stream

Gbps Mbps

Color Space Conversion

Z?rcher Fachhochschule

4K 4Gbps Scaling

Target Bandwidth

Transport Protocol 3

Live Video Streaming - Dynamic

Video Sources

Mixing

Encoding

Stream

HLS

Gbps Mbps

RTP

4

Z?rcher Fachhochschule

Classic Approach

Video Sources Mixing

Encoding

FPGA

Pipeline

Encoding IP

Stream

CPU

Streaming Application

Fixed Implementation

Interface

5

Z?rcher Fachhochschule

Approach with TX1

Processing

GStreamer MM API CPU GPU DMAs

CODECs

H.264 H.265 VP8

Streaming

HLS Mpeg-TS RT(S)P

...

Interfaces

CSI PCIe USB Ethernet

6

Z?rcher Fachhochschule

Software Frameworks

? GStreamer ? Pipeline-based Multimedia Framework ? Very easy to use (one-liner) ? Open-Source

? L4T Multimedia API (since L4T 24.2) ? Low-level APIs for application development

? GPU Integration ? CUDA ? OpenGL / EGL

7

GStreamer is free software available under the terms of the LGPL license ZO?rpcheenr GFaLch?hoachnsdchuthlee oval logo are trademarks or registered trademarks of Silicon Graphics, Inc

Software Stack for Streaming

Sources

Sinks

Processing CODECs

Stream

High-Level: VisionWorks/OpenCV, TensorRT, cuDNN, Custom Application

User Space

v4l2, alsa, tcp/udp

xvideo, overlay (omx), tcp/udp

libargus, V4L2 API

NVOSD

Buffer utility

GStreamer

mix, scale, convert, cuda, openGL

Multimedia API

cuda, openGL

omx h264/h265, libav, mp3

NvVideoEncoder, NvVideoDecoder

rtp, rtsp, hls, mpeg-ts

Libraries

X11 OpenGL, EGL, Vulkan CUDA

OpenMAX (omx)

Kernel Space

V4L2, videobuf2

DRM/KMS/FB

ALSA

Linux Kernel

v4l2-subdev

GPU Driver Drivers / Modules

Host1x / Graphics Host

Sockets TCP/IP/UDP

Eth Driver

HW

VI (CSI)

Video Source

Z?rcher Fachhochschule

CPU

GPU

Display Ctrl

Convert

CODECs

PCIe Ctrl

Eth PHY

8

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download