Even though most of our TV time is on streaming platforms these days, sometimes I watch TV programs with my wife, or F1 highlights on my own.

Often, we'll just watch the program earlier the following day, especially if it's on in the evening (my wife has to get up early).

We tend to watch Channel 4 programs the most, and sometimes ITV and the BBC. In the UK, the BBC runs programs without adverts, funded by the TV License - a unique set up in the world.

However, Channel 4 and ITV are funded by adverts and about 25% of watch time is adverts. Both have streaming solutions, but these also show adverts (unless you pay a fee) and some of them are very repetitive.

My solution is to just record the TV stream, then process it afterwards so it can be streamed locally using my home server to the TV.

The recording records the adverts, so the channels are still effectively paid for the viewing, but after processing the adverts are cut using a piece of software named comskip.

Comskip needs a configuration file to optimise its performance. My comskip.ini is here and this seems to work well on ITV and Channel 4.

This setup has taken some time of tweaks and optimisations to set up, but the result uses all free software and works quite well.

Now, my setup is a little unique. I use my server for various tasks and hosting Emby Media Server is one task so that I can stream Music and Movies to my TV, phone, PCs or tablet.

The server is at the front of the flat, which is in the middle of the building so a bad place for TV signal and nowhere near an aerial socket. This is why I use an old PC for doing the recording. That PC runs LibreElec on bare metal well and has good built in compatibility with the USB TV Tuner Stick Geniatech MyGica T230A. Should the signal be bad, it's convenient to restart without my server going down.

The old PC is connected to an even older 26" 720p TV via VGA D-SUB and 3.5mm audio - but it's a reasonable TV and audio is handled by a homemade amp, of course 😉

In the Living Room, I have a big TV and sound system, but no PC is connected to it since I use a Roku Streaming Stick+ for the steaming services. This conveniently supports H.264 (AVC) and H.265 (HEVC) decoding at a low power. An Emby app for it connects to my server so all material is available there.

LibreElec has TVHeadend added for TV support and recording, and the Kodi addon for showing the TV Guide and managing recordings. TVHeadend does have a web interface too, but it sadly ain't pretty!

I record in native format. This allows, if I want to, to watch a TV recording from the beginning, whilst it is still recording. I often do this with F1 so I can start watching about 45 minutes later than the start time and manually skip adverts, so it finishes close to the original end time, within Kodi itself.

For streaming to the Roku or elsewhere though, I do some processing. Now, technically HD TV signal in the UK is already H.264 but signal drops mean it does not stream that well, and running comskip on the original files also did not output correct timings. The recording is also interlaced so needs de-interlacing before streaming (bwdif is used to double the framerate). So, I re-encode the recording again using FFMPEG.

This is done on the server. The complete steps are:

1. Copy the files from the recording PC to the server (I also delete them from the source)
2. Shut down the recording PC
3. For each file:
   1. Use FFMPEG to convert the video to H.264 format mkv file
   2. Use comskip to output the start and end times for commercials in SCF format
   3. Convert this file to a concat.txt file that FFMPEG concat demuxer can read
   4. Use FFMPEG concat demuxer to concatenate start and end times of the same source encoded file to make a shorter version
   5. Move the result to a folder that Emby server can read

I wrote a bash and python script to do all of the above. This is scheduled at 1AM via a cron job (crontab) running on the server.

The server has a Core i7 10700 processor which is 8 Cores / 16 Threads CPU. It does support VAPI encoding too, but the quality/speed I found not competitive compared with CPU encoding. I found the medium preset and CRF of 25 a fair speed/compression/quality tradeoff for the output files. The bitrate cannot be too high, otherwise the Roku can't stream it. I also wanted the processing time to be fast to minimise energy usage and make a file available sooner in case I wanted to watch the show on the same day.

I've picked H.264 as the video codec here for maximum compatibility. There is not much difference on quality and file size for 1080p TV recordings compared to the newer H.265 (which is much better at UHD), but it means my older hardware can decode it without the Emby server transcoding.

To pick a preset, I tested encoding the first 2 minutes of a recording, with a CRF set to 25, with yadif filter (I later switched to bwdif). Here are the results:

Your mileage will of course vary. Yes, slow and fast resulted in the same size, and so did superfast and ultrafast. Veryfast produced the smallest file! The picture quality between them was not much.

For de-interlacing, which is necessary for most UK DVB broadcasts, the filter makes a difference:

Both yadif=1 (yadif 2x) and bwdif double the framerate so you get a 50fps file instead of 25fps, but both strangely result in a smaller file though add to processing time. I had a slight preference for bwdif for Formula 1 car movement.

The full native .ts recording is 90 minutes and 3.6GB in size.

Re-encoding with the veryfast preset takes 13 minutes 26 seconds and produces a file size of 2.4GB.

ffmpeg -y -i "recording.ts" -vf 'bwdif' -vcodec libx264 -preset veryfast -tune film -crf 25 -acodec copy -scodec copy -f matroska "recording.mkv"

Running comskip takes 2 minutes 2 seconds.

comskip --scf --hwassist --threads=8 --ts --ini=comskip.ini "recording.mkv"

Splitting it and concatenating the file without adverts takes just 3 seconds (NVMe SSD helps here!) and produces a file size of 1.9GB, and a runtime of 67 minutes.

ffmpeg -y -f concat -safe 0 -i "recording_concat.txt" -vcodec copy -acodec copy -scodec copy "recording-cut.mkv"

Ideally, the comskip chapters file would be supported natively in mkv in the Roku Emby player - but I couldn't find a way and I instead trust comskip to do a good job and slice the file and combine it again using FFMPEG. The final file is served without adverts, however my script does not delete the originals just in case.

Unfortunately, comskip does not produce a file format that works with FFMPEG so I made a python script scf_process.py to convert it:

import sys
import os
from datetime import datetime

filename = sys.argv[1]
filename_noext = os.path.splitext(filename)[0]
file_ext = os.path.splitext(filename)[1]
filenamesafe = filename.replace("'", "\'\\\'\'")

print(f"Processing file: {filename}")

# Using readlines()
file_scf = open(f"{filename_noext}.scf", 'r')
scf_lines = file_scf.readlines()

file_concat = open(f"{filename_noext}_concat.txt", 'w')


# Strips the newline character
line_str = ""
time_str = ""
chapter_count = 1
inpoint = 0
outpoint = 0

for line in scf_lines:
    line = line.strip()
    if line.split("=")[0] == f"CHAPTER{chapter_count:02}":
        print(f"CHAPTER{chapter_count:02}")
        time_str = line.split("=")[1]
    if line.split("=")[0] == f"CHAPTER{chapter_count:02}NAME":
        if line.split("=")[1] == "Commercial starts":
            pt = datetime.strptime(time_str, "%H:%M:%S.%f")
            outpoint = pt.hour*3600 + pt.minute*60 + pt.second + pt.microsecond / 1000000
            if (outpoint > 0):
                file_concat.write(f"file '{filenamesafe}'\n")
                file_concat.write(f"inpoint {inpoint}\n")
                file_concat.write(f"outpoint {outpoint}\n")
        if line.split("=")[1] == "Commercial ends":
            pt = datetime.strptime(time_str, "%H:%M:%S.%f")
            inpoint = pt.hour*3600 + pt.minute*60 + pt.second + pt.microsecond / 1000000
        chapter_count += 1

file_concat.write(f"file '{filenamesafe}'\n")
file_concat.write(f"inpoint {inpoint}\n")

# writing to file
file_concat.close()
 

Running:

python3 /path/to/scf_process.py "recording"

This converts the following example scf:

CHAPTER01=00:00:00.001
CHAPTER01NAME=Commercial starts
CHAPTER02=00:00:06.000
CHAPTER02NAME=Commercial ends
CHAPTER03=00:13:20.017
CHAPTER03NAME=Commercial starts
CHAPTER04=00:17:28.001
CHAPTER04NAME=Commercial ends
CHAPTER05=00:25:28.022
CHAPTER05NAME=Commercial starts
CHAPTER06=00:29:35.021
CHAPTER06NAME=Commercial ends
CHAPTER07=00:39:00.000
CHAPTER07NAME=Commercial starts
CHAPTER08=00:43:11.024
CHAPTER08NAME=Commercial ends
CHAPTER09=00:52:19.020
CHAPTER09NAME=Commercial starts
CHAPTER10=00:56:32.016
CHAPTER10NAME=Commercial ends
CHAPTER11=01:15:12.006
CHAPTER11NAME=Commercial starts
CHAPTER12=01:19:21.024
CHAPTER12NAME=Commercial ends
CHAPTER13=01:28:27.003
CHAPTER13NAME=Commercial starts
CHAPTER14=01:30:09.001
CHAPTER14NAME=Commercial ends

To the following recording_concat.txt:

file 'recording.mkv'
inpoint 6
outpoint 800.017
file 'recording.mkv'
inpoint 1048.001
outpoint 1528.022
file 'recording.mkv'
inpoint 1775.021
outpoint 2340
file 'recording.mkv'
inpoint 2591.024
outpoint 3139.020
file 'recording.mkv'
inpoint 3392.016
outpoint 4512.006
file 'recording.mkv'
inpoint 4761.024
outpoint 5307.003
file 'recording.mkv'
inpoint 5409.001

This txt format with the same file name but different in and out points can be used by ffmpeg to combine those sections into one file.

Putting it altogether is a bash script:

#!/bin/bash

# Sync recordings
rsync --remove-source-files -avh root@x.x.x.x:/media/sdb1-ata-ST500LM000-1EJ16/recordings/ "$1"
# Power off recorder
ssh root@x.x.x.x 'poweroff'

# Pick up files modified today
find "$1" -name "*.ts" -type f -mtime 0 -print0 | while read -d $'\0' pathnamefull; do
  echo "Processing $pathnamefull"
  dirname=$(dirname "$pathnamefull")
  fnameext=$(basename "$pathnamefull")
  fname=${fnameext%.*}
  pathname=${dirname}/${fname}
  echo "File $fname"
  # Convert to h264 and deinterlace
  echo "---------------h264------------------"
  < /dev/null ffmpeg -y -i "$pathnamefull" -vf 'bwdif' -vcodec libx264 -preset veryfast -tune film -crf 25 -acodec copy -scodec copy -f matroska "${pathname}.mkv"
  # Use comskip to produce an SCF file
  echo "--------------Comskip----------------"
  comskip --scf --hwassist --threads=8 --ts --ini=$(dirname "$0")/comskip.ini "${pathname}.mkv"
  # Convert SCF to concat format
  echo "----------------SCF------------------"
  python3 $(dirname "$0")/scf_process.py "${pathname}.mkv"
  # Split and remove ads
  echo "--------------concat-----------------"
  < /dev/null ffmpeg -y -f concat -safe 0 -i "${pathname}_concat.txt" -vcodec copy -acodec copy -scodec copy "${pathname}-cut.mkv"
  # Move to final folder
  echo "---------------move------------------"
  mkdir -p "${dirname//\/TV2\//\/TV\/}"
  mv "${pathname}-cut.mkv" "${dirname//\/TV2\//\/TV\/}/${fname}.mkv"
done

This script is scheduled daily using crontab -e:

0 1 * * * /mnt/home/dan/Videos/Scripts/TVProcess.sh /mnt/home/dan/Videos/TV2 >> /mnt/home/dan/Videos/Scripts/TVProcess.log 2>&1

The bash script needs a parameter of the directory files will be copied to and processed in (example /mnt/home/dan/Videos/TV2). It substitutes TV2 for TV to copy the final file to a directory that emby monitors (example /mnt/home/dan/Videos/TV).

It's very unlikely you'll be able to use the script as-is. If you are copying from another machine, you'll need to set up password-less SSH (ssh-keygen followed by ssh-copy-id) and correct the source directory as well as destination directory customisations.

The files scf_process.py and comskip.ini must be in the same location.

What's not working?

Subtitles. Currently they are retained in DVB format in the conversion to mkv, but DVB subtitles are not supported by Roku Emby media player (unless burned in via transcoding) and converting DVB to text compatible SRT (sub rip) format is hard to automate.

5.1 sound (AAC) is not working either. Everything I've recorded doesn't have it available in the original stream and it might be a limitation of the USB TV stick, PC or TVheadend.

• Emby Media Server
• LibreElec
• github.com - comskip
• superuser.com - Why does preset "veryfast" in FFmpeg generate the most compressed file compared to all other presets?
• superuser.com - How can I remove multiple segments from a video using FFmpeg?
• kaashoek.com comskip - UK Freeview