pfSense and mDNSResponder Madness

I’ve been having a bit of a name resolving problem at home. My home network is not the most straight-forward, but it’s certainly not too complicated either. There’s a pfSense gateway, and a Linux server behind it that I want accessible both internally as well as externally. That Linux sever should, of course, ideally be accessible by the same hostname regardless or where the connection comes from.

The way pfSense handles this is by defining a host override in the DNS Forwarder configuration. I have pfSense configured to update a Dynamic DNS service with the IP address of its WAN interface. The WAN interface thus has a DNS ‘A’ record pointing to its IP address.

My home network is NAT’ed by pfSense. The Linux server can be accessible from the outside through a port-forward configuration in pfSense. To address it by hostname, on the public DNS side, I use a ‘CNAME’ record to point its hostname to that of the gateway WAN interface. Clients from the public Internet will thus hit the pfSense gateway when they try to resolve the hostname of the Linux server.

On the LAN side, I use a host override in pfSense’s DNS Forwarder configuration. LAN clients will thus receive a ‘A’ record answer from pfSense directly for the Linux server’s IP address on the LAN.

It sounds very simple and straight-forward. Let me summarise again (with example names):

1. Gateway has a public DNS hostname gw.somewhere.com, ‘A’ record resolving to IP address of WAN interface, updated via Dynamic DNS.
2. Linux server has public DNS hostname linux.somewhere.com, ‘CNAME’ record that points to gw.somewhere.com.
3. pfSense has a host override for linux.somewhere.com that is an ‘A’ record for the Linux server’s actual LAN IP address.

I’ve no trouble getting to the Linux server from the public Internet. However, accesses from inside my home network is problematic.

The problem is that, using my MacBook Pro inside my LAN, I often hit the pfSense gateway when I try to access linux.somewhere.com. Typically, what I’m trying to do is to SSH to the Linux server. The first SSH attempt will hit the actual Linux server correctly. The next subsequent attempt, however, will hit the pfSense gateway (I’ll know it is wrong because the SSH host key verification will fail). If I wait a long while and try again, I’ll get the actual Linux server again, but then thereafter, it will be the pfSense gateway. Yeah, cycle repeats.

A temporary solution for me had been to HUP the mDNSResponder. Yah, I’ve that Unix server administrator background to know lots about caches. Cache flushes will solve lots of problems. In the old Mac days, you’d run dscacheutil -flushcache. Nowadays, it is superseded by sending SIGHUP to the PID of mDNSResponder.

Yes, that will flush its cache, and I will get back my Linux server. But not in the subsequent access. It’s not a real fix. The cache was somehow getting corrupted. Very irritating, very frustrating. It’s been bugging me a lot, and I think finally enough for me to really spend time to debug the problem.

So I found out what’s happening.

On my MacBook Pro, the very clever IPv6 friendly Mac OS X, when asked to resolve the name of linux.somewhere.com, will actually send one query each for the ‘A’ and ‘AAAA’ record. The pfSense responds immediately with the host override answer, giving me the correct LAN IP address of the Linux server. My SSH goes through correctly.

However, the ‘AAAA’ question can’t be handled by pfSense, so it gets forwarded out to the Internet. There, it receives an answer from public DNS with the CNAME record pointing to gw.somewhere.com. Yes, even though I have no ‘AAAA’ record setup, the ‘AAAA’ question does retrieve the ‘CNAME’ record. Now, having got the reference to gw.somewhere.com, mDNSResponder on my MacBook Pro happily goes on to put the WAN interface IP address of the pfSense gateway as a cached answer for linux.somewhere.com. Subsequent accesses, thus, hit the pfSense gateway, until that cache answer expires.

The fix that I’ve decided to go with is to configure another host override to the IPv6 address of my Linux server. At least, that will make sure that the ‘AAAA’ question will be answered by pfSense directly.

Another solution is to not have that ‘CNAME’ record in the public DNS. However, for me, the ‘CNAME’ fit my needs much better, so I want to continue using it.

I think, solving this issue, also helped fixed another nagging problem. My MacBook Pro’s network browsing (to find network shares) has always been very problematic. They aren’t always listed; or when they are, they somehow can’t be accessed (“opened”); and even when they’ve been connected to, I sometimes can’t disconnect from them!

I hope this has been helpful.