ICMP Redirect Host(New addr: X.X.X.X)

I'm setting up a VPN on my linux server and I wanted to have firewall rules to control communication between clients so in my OpenVPN configuration I removed client-to-client so OpenVPN pass all of the traffic to the kernel.

Once I did this and did a ping from one client to another, I would get these ICMP Redirect messages but the ping would work.

$ ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=63 time=59.198 ms
92 bytes from Redirect Host(New addr:
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 b2ac   0 0000  3f  01 f1d7

64 bytes from icmp_seq=1 ttl=63 time=54.229 ms
92 bytes from Redirect Host(New addr:
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 b854   0 0000  3f  01 ec2f

64 bytes from icmp_seq=2 ttl=63 time=55.497 ms
64 bytes from icmp_seq=3 ttl=63 time=54.718 ms

After a bit of Googling I found this forum post which mentions traffic that comes in from one interface and goes out the same interface on linux causes the redirects. I then found there's a sysctl that controls this behavour called net.ipv4.conf.all.send_redirects. So a quick sysctl net.ipv4.conf.all.send_redirects=0 and the redirects went away and all was right in the world.

Linux ZFS Root & Datasets & systemd-networkd

With my servers I prefer to have the root filesystem be a set of 2 SSDs in a ZFS mirror. That way you get bit rot detection, snapshots before significant changes, separate datasets, and redundancy. I follow the openzfs guide with a few tweaks to set this up. Then I create a dataset at /c where I prefer to put all of my configuration files and then syslink to them at their original locations; however, this leads to an issue during boot for some services.

The issue is that only the root pool is mounted early on and nested datasets are only mounted as part of the local-fs.target chain. This causes an issue if the service is loaded before this systemd unit. In my case, I wanted my systemd-networkd configration files to be stored in /c but when systemd-networkd runs, /c isn't mounted so the syslinks are bad and won't be configured.

The solution is fairly simple, we want systemd-networkd to run after local-fs.target. To accomplish this, you'll want to run sudo systemctl edit systemd-networkd which will open the override.conf file for editing. Add the following, save, and exit.


Now on your next boot, everything should work properly. This should work for most units but for generators (such as netplan.io), this won't work because they run very early in the systemd process

Plex GPU Transcoding in Docker on Debian 10

With Docker 19.03 adding native support for GPU passthrough and Plex support for GPU transcoding being reliable and stabe, it's now very easy to get both working together for some super duper GPU transcoding.

I installed an NVIDIA Quadro RTX 4000 in my 2U server recently and after installing all the packages required and one flag to docker, Plex was able to use the GPU.


Firstly, we'll install the latest nvidia drivers for Debian buster. If you're on stretch or earlier, you will have to install the nvidia drivers manually. ffmpeg (which Plex uses for transcoding) requires at least 418.30 so check what distro provides if you aren't on Debian.

echo "deb http://deb.debian.org/debian buster-backports main contrib non-free" | sudo tee /etc/apt/sources.list.d/buster-backports.list

sudo apt update

sudo apt install linux-headers-$(uname -r|sed 's/[^-]*-[^-]*-//')
sudo apt install -yt buster-backports nvidia-driver libcuda1 libnvidia-encode1 libnvcuvid1

Once you got that done you will have to restart to not have the nouveau get in the way, then once you're back up you should be able to run nvidia-smi and see your GPU.


Now onto nvidia and Docker. Install Docker if you haven't and then we'll add the nvidia-docker gpg key and apt repository.

curl -sL https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -sL https://nvidia.github.io/nvidia-docker/debian10/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

sudo systemctl restart docker


With docker restarted, you should be able to run docker run --gpus all nvidia/cuda:10.0-base nvidia-smi and see the same output as before on the host. Next step is to add the --gpus all (see usage here) to your Plex container. Note that docker-compose does not have support for GPUs yet so you will have to do this with docker run for your container. For example, my plex is launched with a command similar to this:

docker run --name plex --restart unless-stopped --gpus all --network=host --env VERSION=latest --volume /plex:/config --volume /media:/media linuxserver/plex

Once Plex is up you'll want to go to Settings > Transcoder and enable Use hardware acceleration when available and Use hardware-accelerated video encoding. Now start watching something and make sure it's transcoding. You should then be able to check nvidia-smi and see the process and (mostly memory) usage. You should see something like this:

| NVIDIA-SMI 440.82       Driver Version: 440.82       CUDA Version: 10.2     |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|   0  Quadro RTX 4000     On   | 00000000:03:00.0 Off |                  N/A |
| 45%   74C    P0    86W / 125W |   2159MiB /  7982MiB |     11%      Default |

| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|    0     32212      C   /usr/lib/plexmediaserver/Plex Transcoder    1501MiB |
|    0     32870      C   /usr/lib/plexmediaserver/Plex Transcoder     342MiB |
|    0     39082      C   /usr/lib/plexmediaserver/Plex Transcoder     302MiB |

As you can see, 4K streams are quite memory intensive. They are 4x a 1080p stream which is 4x a 720p stream. GPU transcoding is primarily memory limited. Secondarily limited by number of streams: consumer cards are limited to 2 by nVidia. There is a patch to remove that limit though. Thirdly, you're limited by the nvenc and nvdec FPS limit. This website has lots of information comparing various GPUs and their transcoding performance.

If you don't see Plex in there, you'll want to look at Plex Media Server.log and look for any debug messages that might show why it's unable to use your hardware for transcoding. In my case I had 2 error messages: Cannot load libnvidia-encode.so.1 and Cannot load libnvcuvid.so.1 which is why I you install those libraries earlier.

Enjoy your sweet GPU transcoding!

PhidgetSBC3 and D-Link DWA-160

There's actually a much easier way of doing this. Just apt-get install firmware-linux-free and boom! You get the carl9170 firmware and the DWA-160 is recognized.

I recently got my hands on a PhidgetSBC3 and plugged a D-Link DWA-160 into and found that it already had support built into the kernel but it had an error. So I went searching in the /lib/firmware directory and saw there was no carl9170-1.fw file. So I downloaded and when I plugged the DWA-160 back in it worked and correctly showed up in the web interface. Installing it is very easy:

apt-get install curl -y
curl -L http://linuxwireless.org/en/users/Drivers/carl9170/fw1.9.7?action=AttachFile&do=get&target=carl9170-1.fw > /lib/firmware/carl9170-1.fw

PhidgetSBC2 and D-Link DWA-160

So I've had a little project going on with the foosball table at work: a system to detect when someone scores a goal. I've built a prototype photogate that goes inside of the goal. My design is to have 2 photogates hooked up to my PhidgetSBC2 and write some ruby code to detect a goal and send a message to a connected iPad app to indicate who scored. Everything was going fine until I took the PhidgetSBC2 to our foosball table and discovered our 2.4GHz network had no coverage there. Which means the wifi usb adapter I had for the PhidgetSBC2 was useless. Undeterred I started on 2 different solutions.

Continue reading "PhidgetSBC2 and D-Link DWA-160"