Fixing Internet Speed on Virtualized pfSense

It’s been a few weeks since I set up my pfSense router inside Proxmox inside a HP desktop computer. After I set it up, I noticed my internet speeds weren’t quite what I was getting with the Orbi acting as the router. With the Orbi, I generally got somewhere around 650-700 Mbps for downloads and 700-800 Mbps for uploads. With pfSense, I was getting around 520-550 Mbps for both. My internet service should be 1Gbps in both directions (actually a theoretical maximum of 940Mbps due to the way the network hardware works). I set up pfSense as the Great 2020 Work From Home was in full swing, so I thought maybe Verizon’s network had more concurrent users during the day slowing me down. I didn’t really think anything of it until today, when I was downloading a 100 GB game.

When I first setup pfSense, I told Proxmox to give it two NICs of the VirtIO paravirtualized type. When I get pfSense set up, I noticed it told me the speed of the two interfaces was 10 Gbps, and my web page loading times were very long. I assumed this was a duplex mismatch, and changed the NIC type to Intel E1000. Pages loaded just fine after that. It turns out it was a mistake to change the NIC type. VirtIO was the correct type, and the 10 Gbps speed was referring to the link to the Proxmox virtual switch, not the link to the internet or my physical Cisco switch. I changed back to VirtIO and disabled all hardware offloading in the System > Advanced > Networking settings of pfSense.

I also happened upon a Reddit post describing the same issue I had. I followed the directions to install ethtool and add one line like

post-up ethtool -K vmbr0 tx off

for each virtual and physical interface in /etc/network/interfaces.

I also discovered that while pfSense CPU usage was only in the single digits when doing web browsing, during speed tests and large downloads, it hit close to 100%. I resolved that by adding another CPU core in the Proxmox hardware configuration. CPU usage is now 70-80% during big downloads.

I fixed everything up with these changes. My downloads and uploads now easily hit their maximum possible speeds of 940 Mbps, at least when other internet usage is kept to a minimum. I wish I did this last year when we first got Fios because I never got the advertised gigabit speeds with the Orbi router. I guess the Orbi wasn’t designed to handle a gigabit WAN connection. pfSense handles it with no trouble, at least once it’s properly configured.

So we finally got FiOS

After a couple false starts, we got our FiOS service going on Thursday. We got the gigabit internet and lifestyle and reality custom TV package. The download speeds have been a tad disappointing at about 330 mbps, while the uploads are an excellent 700 mbps. The download is three times better than we had before, but I’d like it to at least match the upload. I’m not sure what’s causing that problem, but I’ll have to try a few things to fix it this coming week.

The TV side of things is going very well so far. The Ceton PCI tuner is working just fine for now, but I’ll still probably upgrade to the new HD Homerun Prime when it comes out. I have the Ceton card in my living room computer, connected to the TV. It runs NextPVR which contains a client for local use and a server that other devices on the network can connect to.
It’s a very powerful program, with lots of options for customization. In the living room, we just use the local NPVR client. I didn’t realize that cable TV services don’t transmit program guide data like the over-the-air service does. The Schedules Direct service is supported by NPVR, with fairly easy set up. It costs $25 per year, but, to me, that’s very reasonable for the convenience.

I was originally planning to use Emby with the NPVR plugin on the server side and the Emby Roku app on the client side. Quite frankly, it sucked. It just dumped all 1200 channels the tuner card sees into a menu with no options to sort or filter them. When I clicked a channel to watch, it didn’t work either. It’s a small caveat, but to use the live TV function of Emby, you have to pay a subscription fee of $4.99 per month, $54 per year, or $119 for life. So I had to go back to the drawing board.

I decided to try out Kodi on my Raspberry Pi server. I installed the NPVR add-on and adjusted some settings. It kind of worked. The guide was perfect, but playback was a little wonky. The audio was fine, but the video would stutter or freeze. I figured it was good enough and I could fix the problem. I ordered a new Raspberry Pi 3B+ and set it up with OSMC. OSMC is a fork of Debian with Kodi preinstalled. I installed it on the Pi, and then velcro’d the Pi to the back of the TV. After a little bit of setup, it was working perfectly. The guide looked just like NPVR, live TV worked without any catches and watching recordings was great. I’m extremely satisfied with the setup.

There’s only one small thing left to do; add a remote control. In the living room I have a Logitech K400+ couch keyboard, which is great for changing app settings, launching games and light web browsing, but it’s really clunky and unintuitive for watching TV. A regular remote control would be ideal. I remembered the new computer my dad got for Christmas 2006. It was a Dell XPS 410, which had a novel for the time TV tuner card. This meant it came with Windows XP Media Center Edition, and a remote control. The remote had every button you need, but none you don’t, with a very solid and high quality feel. To my surprise, I was able to track down two new in box examples on eBay. Set up with NPVR should be a breeze, because it has native support for these media center edition remotes. I also discovered Kodi has an MCE remote add-on, so hopefully set up should be easy there too. The first one comes tomorrow, so I’ll find out soon.

While the internet service so far is a tad disappointing, I’m totally thrilled with the TV. This setup with a cable card tuner inside an HTPC with client computers around the house is something I’ve wanted since not long after we opened that new computer over a decade ago.