FeM C3 Streaming and Recording Setup Documentation
Encoding Pipeline
Recording
Marking
The main goal of the marking process is to avoid the massive I/O that was necessary in earlier encoding setups. So the core concept is to use a custom FUSE filesystem that provides virtual files as concatenation of input files without copying large files and storing redundant data. There is one FUSE instance per lecture, not e.g. one single instance for a whole congress. The mountpoints of the FUSE instances are shared over network via glusterFS.
In 2012 there was a remake of the recording pipeline, especially in this process to support multiple audio tracks for 29C3 and maybe higher resolutions in the future. The pipeline used before 29C3 will be referenced as „v1“, the current version as „v2“.
Marking v1 workflow (< 29C3)
recording format: rawDV 720×576 (if source is 16:9 then it is recorded anamorph) with 1x stereo PCM 16bit 48ks/s
files containing 4500 frames each, split by simple perl script
custom FUSE file system:
fuse-vdv providing a file
uncut.dv
fuse-vdv provides a generated kdenlive project file
this file is opened by an operator on her workstation directly via the glusterFS network share.
in the provided kdenlive project the virtually concatenated recording is trimmed and the project is saved
on save, fuse-vdv parses the written project file and extracts the trim marks
using the trim marks and the knowledge of constant frame sizes in raw DV fuse-dvd provides three additional virtual files
a file cut.dv
which contains just the frames between the trim marks
a file cut-complete.dv
which contains the frames between the trim marks and a prepended intro as well as an appended outro file
a file cut.wav
which provides the demuxed audio data of cut.dv
subsequent processes (e.g. encoding the final files) just use cut-complete.dv
or cut.wav
(audio-only formats) as input file
Marking v2 workflow (>= 29C3)
(only the differences between v1 and v2 are shown here)
recording format:
MPEG-TS container, which can contain a vast number of video and audio tracks and different codecs for them
files are split by the ffmpeg segmenter every 3 minutes
custom FUSE file system:
fuse-ts providing a file
uncut.ts
fuse-ts also provides a kdenlive project file
operator trims like in workflow v1
after saving the kdenlive project file, fuse-ts does not provide trimmed versions of the virtual files, since it doesn't know anything about MPEG-TS or the contained tracks or their codecs
it just extracts the trim marks from the project file, converts them from frame numbers to fractioned seconds and stores them for later use
subsequent processes (e.g. encoding the final files) use the untrimmed uncut.ts
as input file and must do the trimming (via ffmpeg's -ss
and -t
options) and prepending intro etc. (via ffmpeg's concat
-video filter) by themselves
Ticket Tracker
Encoding Cloud
LiveCD image to boot multiple encoder instances
Recording segments available via central storage box
Encoding ticket processing with tracker connection (XML-RPC)
-
Streaming
Encoding
each room one RTMP encoder
sdi card, capture tool, ffmpeg pipeline
multiple data sinks using enhanced tee → supertee
-
each room one WMV encoder
each room one slide only encoder
Relay
RTMP and HTTP Relay using Nginx
WMV relay using Windows Server 200X with Media Services
CDN Controll software and config files