Daryl Hunter – Hubby, Daddy, Mentor, Friend

So, up until very recently – today actually I guess – it was technically possible to "hijack" DNS queries.  Someone would request www.domain.com and you could spoof/redirect that query to a website of your choosing.  Not that anyone would do that 🙂

Today, DNSSEC (secured DNS queries) rolled out across all 13 root DNS servers.  There are several articles on the interwebz talking about this, and IMO slashdot & wikipedia have some good info to read about.

For us at LifeChurch.tv, we were already hit with an eDNS "bug" a while back. I guess to be fair, it wasn't really a "bug" – but rather a "feature" that we weren't ready for.  And, with these DNSSEC changes, I've actually had to "undo" my bug fix from February.  Let's dive in a little more on testing & fixes that we had to do.

Problem Exists
So – in the most basic terms – DNS queries are 512 bytes long.  At least until DNSSEC – now they are bigger because they contain more data – you know, the "security" stuff.  To verify if you have a problem, you'll use the command "dig" to do this.  For me, I did this on a linux based webserver sitting behind our Cisco ASA cluster.  Here's an article describing how to use "dig" for this test & sample output:

Did you catch that?  The command tells me that my DNS queries are not supporting eDNS (enhanced) and my DNS query size limits at 512 bytes.  I need to fix that.  But where?

Cisco Firewall Changes
So – we run clustered Cisco ASA firewalls.  Our firewalls are running code version 8.2(1) – which is a little old.  It's not ancient, but not current either.  Here's a Cisco Support Forum article talking about this.  Apparently this particular eDNS setup is "fixed" on the 8.2(2) version of ASA code, but I'm not going to upgrade code right now.  So, I look further in that article & a fix is highlighted in the comments:

message-length maximum client auto

So, let's jump on our ASA cluster and take a look at what we need to do.

Sure enough – our DNS inspection is set to limit the DNS query size at 512 bytes.  Let's fix that.

Great, we've implemented the "fix" – let's test it on that linux server using the "dig" command.

Hmm.  That's weird.  The problem isn't fixed.  What's missing?

Windows Server DNS Changes
So, even though our Cisco ASA firewalls are now fully DNSSEC compliant – we're still somehow/someway "limited" to 512 bytes on DNS queries/responses.  Why?  Well – there are two issues at play here at LC.tv – maybe at your organization too:

• Most of our DNS Servers – about a dozen – are running Windows 2003 R2 – which has eDNS disabled by default – thus – locking in the 512 byte size limit
• Our "new" DNS Servers – about 5 of them – are running Windows 2008 R2 – which has eDNS ENABLED by default – but – if you remember this post – I disabled – thus forcing the 512 byte size limit.

So, let's fix it.  It's pretty easy.  You'll need the "dnscmd" tool – which is found by default on Server 2008, and found on the "Support Tools" for Server 2003.  The command you run is:

• dnscmd.exe /config /enableednsprobes 1

This command takes effect "immediately" – and doesn't require a reboot.  You can verify this has been done because it DOES put an entry in your registry:

See that?  The "EnableEDnsProbes" DWORD value of 1 shows the command was successful.

Okay, time to test again – back to linux / dig like before.

There ya go – that's better.  Look at that bottom test.  We've fixed the firewall, we've fixed the Windows DNS Server – now we're done.

Lather, rinse & repeat on all your affected firewalls / DNS servers.

Enjoy!