some of you may have seen my last blog post about the preparation of the Troopers network. Today I want to give you a little teaser on what to expect for the talk I will present during the IPv6 Security Summit. As the title implies, it’s not only about building a secure IPv6 WiFi, but also a reliable one. One might think that there aren’t many differences in comparison to IPv4, but the heavy reliance on multicast of IPv6 does have implications for Wi-Fi networks in general. Continue reading “#TR16 IPv6 Security Summit Teaser: Building a Reliable and Secure IPv6 WiFi Network”
I am currently preparing the Troopers network in a lab environment to ensure that we all will have a smooth Wi-Fi experience during Troopers. I wanted to spice things up a little bit for the Wi-Fi deployment (more on that in a following blogpost) and get rid of IPv4 wherever possible. Our Wi-Fi infrastructure consists of typical Cisco Access Points (1602) and a 2504 Wireless LAN Controller. Beginning with WLC image 8.0 it is finally supported to establish the CAPWAP tunnel between the AP and the WLC over IPv6, which is awesome and I wanted to implement it right away. Continue reading “DHCPv6 Option 52 on Cisco DHCPv6 Server”
Troopers is right around the corner and as I am responsible for the whole conference network I wanted to make sure that everything is working as expected. I went to the venue on Friday because of two things I wanted/needed to setup. Compared to last year’s setup we had a couple of changes in regards to the provider connection (resulting in some changes for our network setup). First, we now have a rather big pipe for the uplink and more importantly (well that depends on the point of view ;)) there is a native IPv6 connection. Before that I had to tunnel all IPv6 traffic from the venue to one of our gateways and to forward it out (as native IPv6) from there. As this step isn’t necessary anymore, and the staff on the venue isn’t that experienced with IPv6, I had in mind to setup and verify that IPv6 is working as desired. The router used over there is a Mikrotek Routerboard. As I haven’t worked with these devices before, I was curious whether everything works as it should ;).
After configuring the IPv6 address on the WAN interface I tried to install a default route pointing to the uplink’s Global Unicast Address. But to my surprise, the Mikrotek router kept stating that the next hop was unreachable. This was odd, as the provider’s device was happily answering to pings from the Mikrotek’s command line. Additionally, the Mikrotek router does not install a route when it can’t reach the next hop configured (which is actually not that bad as it at least prevents fat fingering the address). It still didn’t make any sense. After googling around (I found the Mikrotek documentation a little bit lackluster) and trying some other things it still didn’t work. As a last resort, I told myself “screw it and let’s try with the link local address of the provider router”, but how do I get this address as I only was provided with the GUA? Right, looking at the Neighbor Cache of the Mikrotek router I was able to quickly find the link local address of the next hop.
After using this address (together with the interface) as the next hop it started working, by magic. At least I can now sleep better as it is one less thing I have to worry about ;).
Moral of the story: Still in 2015 don’t expect a device to behave like it should when it comes to IPv6. Unfortunately, I wasn’t able to follow this strange behavior up due to time constraints, but it is working and you can enjoy for the first time native IPv6 in the conference network.
If you want to know more about the general conference setup please stop by for my talk at the IPv6 Security Summit.
Given that Enno and I are network geeks, and that I am responsible for setting up the Troopers Wifi network I was curious which components might be used at Cisco Live and which IPv6 related configuration was done for the Wifi network to ensure a reliable network and reduce the chatty nature of IPv6. Andrew Yourtchenko (@ayourtch) already did an amazing job last year at Cisco Live Europe explaining in detail (at the time session BRKEWN-2666) the intricacies of IPv6 in Wifi networks, and how to optimize IPv6 for these networks. He was also a great inspiration for me when setting up the Troopers Wifi network a couple of weeks later. Thank You!
I know I am a bit late with this post, but I was speaking on the North American IPv6 Summit in Denver three weeks ago. The focus of my talk was on Why IPv6 Security is hard – Structural Deficits of IPv6 & Their Implications (slightly modified/updated from the Troopers IPv6 Security Summit). We consider the NA IPv6 Summit as one of the most important IPv6 events at all and we were happy to contribute to the overall success. The conference was organized for the 7th time by the Rocky Mountain IPv6 Task Force and took place in the Grand Hyatt Denver (37th floor ;-)). Luckily the weather was perfect, and the view of the landscape from the conference rooms was just amazing. I really enjoyed the time in Denver, as the organizer sdid all they could to treat the speaker well J. The talks were of mix of regular research or case-study type talks and some sponsored talks ranging from deployment experience, security and statistics to SDN (Yes, I said it ;)) and the Internet of Things (I said it again ;)). The line-up was nicely put together.
I recently stumbled over a document from Microsoft which lists all services/applications that support IPv6. Most of the content wasn’t new for me, but one item caught my attention. Windows Update. I haven’t heard before that Windows Update can be done over IPv6 (but this could just be me not looking hard enough ;)), so I was eager to test it out seeing if this is really the case. I was also curious why Microsoft referenced this document in the respective column. Continue reading “Microsoft Windows Update over IPv6 (or not?)”
Some of you may already know (the ones who are following Enno on Twitter) that Enno and I had our lab day in preparation for the IPv6 Security Summit at Troopers. We had a brand new and shiny Cat4948E as our lab device to do some testing of the current generation of Cisco’s IPv6 First Hop Security (FHS) mechanisms. The Catalyst was running the latest image available (15.1(2)SG3).
In this small blog post, we will take a look at the configuration and behavior of IPv6 Snooping and DHCPv6 Guard. So let’s start with IPv6 Snooping:
I am little bit late to the party, but I had the pleasure to present a talk about VoIP based toll fraud incidents (more on this in a following blogpost, for the moment my slides can be found here) at the annual t2 security conference in Helsinki. The conference took place from 24th to 25th October in the Radisson Blu Royal hotel. I must say that it was a blast. Tomi (the host) took really good care of all speakers, and I really liked the spirit of the conference, very similar to Troopers. It is not an commercial event, seats are limited to 100 and it is all about delivering a great set of talks to the audience and having a good time during and after the conference. Sure the conference has some sponsors and tickets are sold, but Tomi doesn’t do it to earn money. His only intention is to cover the cost for setting up this great event.
Last week I read about the new networking features of the integrated vSwitch of Hyper-V 3.0. I was quite surprised that RA Guard will be natively supported and was curious about implementation and functionality. If you don’t know how RA Guard works, I recommend reading our previous blog posts here, here, here, here and here, or have a look at our workshop at Troopers12.
I downloaded Windows Server 2012 RC to do some practical testing. Since my girlfriend was working the whole weekend, I had plenty of time to play around with all that stuff without risking trouble 😉
I installed the RC and Hyper-V role on one of our lab systems.
The basic setup looks as follows:
Two VMs were installed (one Windows 7 box and one Backtrack 5) and connected to the same vSwitch and were part of the same layer-2 domain (as RAs only matter in a layer-2 domain). A layer-3 switch was the legitimate router sending RAs so that the Windows 7 box configured appropriately. All tests were done with the THC-IPv6 suite from Marc Heuse. The IPv6 configuration of the Windows 7 box looks as follows:
Before starting my tests, I wanted to verify that all is working correctly. So I began sending rogue RAs from a Backtrack VM.
Nothing special here, the Windows 7 box received the RA and configured an IPv6 address from the supplied prefix.
After the verification succeeded I enabled RA Guard in the settings of the network adapter on the Backtrack VM.
To structure the tests a little bit, I created a small test plan on what and how I wanted to test:
Test 1 – Sending Rogue RA’s with RA Guard enabled
The first test included sending of (non-fragmented) RAs with RA Guard enabled on the VM.
Test 1 – Results
Again I started to send Rogue RAs like I already did before.
As I expected, the RAs got filtered by the vSwitch and are not seen on the Windows 7 box (only the legitimate RAs from fe80::1).
Consequently, the IPv6 configuration did not change.
Test 1.1 – Sending Fragmented Rogue RAs with RA Guard enabled
In the next test I wanted to figure out how the vSwitch handles fragmented RAs with a ICMPv6 header being only present in the second fragment.
Test 1.1 – Results
I started to send fragmented RAs with an additional Destination Header to force fragmentation.
Interestingly, the first fragment with the destination header passed the vSwitch and arrived at the Windows 7 box. However, the second fragment with the ICMPv6 header got filtered by the vSwitch. You can see the legitimate RAs in between the fragments.
Like in test before, the IPv6 configuration of the client did not change and the received fragment did not have any impact on the CPU load of the box.
Interim Conclusion
The RA Guard implementation of the vSwitch in Hyper-V reliably filters router advertisements from a VM whether or not they are fragmented. This is good thing. Unfortunately I couldn’t find any entries in the event viewer (or elsewhere) indicating that a VM started to send RAs. This just hides the problem without really addressing it. As an administrator of a virtualized environment I would want to know when a VM starts to send RAs, in order to take appropriate measures.
Test 2 – Flooding RA’s with RA Guard Enabled
In the next test, I wanted to find out how the vSwitch (or better the physical Hypervisor) behaves when a lot of RAs are generated by a VM. For this test, the flood_router 6 module was used.
I started to flood RAs from the Backtrack VM.
The RAs were filtered by the vSwitch and didn’t reach the Windows 7 Box. I got the same result in Wireshark as in the test before. Only the legitimate RAs were seen by the VMs.
The more interesting thing I noticed, was the CPU load of the physical Hypervisor jumping to around 25% (occupying a whole CPU ) as soon as I started flooding.
btw: That’s how the new task manager looks like in Windows Server 2012
I started wondering: If one VM flooding RAs can produces 25% CPU load, what happens if more than one VM starts to flood RAs? Let’s find out…
… with three additional instances of the Backtrack VM for a total of four.
I had some trouble getting all 4 working simultaneously because the flood_router6 module exited quite a few times (which is fixed in version 1.9). That’s why you see those variations in the CPU load. If looking at the point where the CPU load was nearly zero, you can see how it increases in 25% steps, based on how many VMs started to flood RAs. However, even with four VMs flooding in parallel the CPU load stalled at around 75%. But this may be only a matter of how many VMs are flooding RAs.
Test 2.1 – Flooding RA’s with fragmented RA Guard Enabled
As we now knew about the impact on the Hypervisor when one or more VMs start to flood RAs, I wanted to know if anything changes for fragmented RAs.
I started to flood fragmented RAs with an additional Destination Header to see what happens.
The result equals as for Rogue RAs. The first fragment hit the Windows 7 box, but the second fragment got filtered by the vSwitch.
The received fragments did not have any impact on the CPU load of the Windows 7 box.
As before, I was mostly interested in the Hypervisor’s behavior.
Interestingly, the CPU load doubles as soon as fragmented RAs are sent by one VM.
Increasing the VM count to four again at this point appears to be obvious.
The CPU load increases to 90% on average, which is quite huge for just four VMs sending a lot of (fragmented) RAs. Other VMs are not directly affected from the RAs. However, a Hypervisor running out of resources is never a good thing.
Final Conclusion –Well said Mister but does this really matter?
I think it does, because Windows Server 2012 is positioned as “cloud operating system”. As a cloud provider, you typically want to give your customers the ability to spin up as many virtual machines as they like. When a malicious customer can exhaust the Hypervisor’s resources, normal customers will be affected as well. In particular if exhausting resources can be done so easily.
Be aware that these tests were done with the Release Candidate, and it might be possible that the final version of Windows Server 2012 behaves differently. We have to wait and see ;). I will definitely run the tests again when Windows Server 2012 ships.
So what can we do to address this issue? Unfortunately I don’t have a definite answer as of right now. Without knowing the codebase it’s hard to say how this behavior could be prevented or improved. So far it’s up the vendor to address this issue and customers (or pentester’s) duty to point at it…
One thing Microsoft could do is just to disconnect the virtual NIC if too many RAs were send from a VM in order to protect the resources of the Hypervisor.
TROOPERS12 came to an end last week on Friday; needless to say it was an awesome event. 😉
The first two days offered workshops on various topics. On Monday Enno, Marc “Van Hauser” Heuse and I gave a one day workshop on “Advanced IPv6 Security”. I think attendees as well as trainers had a real good time during and after the workshop fiddling around with IPv6. Especially Marc had quite some fun as he discovered that we provided “global” IPv6 Connectivity for the conference network, and according to one of his tweets, TROOPERS12 was the first security conference he visited, offering this kind of connectivity.
So back to the topic
Our last posts on IPv6 Security go back to the first half of 2011. If you haven’t read them already, now it’s a good time to do so. You can find them here, here, here and here.
In the last post of the series Enno discussed how RA-Guard can be circumvented with clever use of extension headers. As a short reminder, the packet dump looks like this.
The Information of the upper-layer protocol is only present in the second fragment, so RA Guard does not kick in.
As we found out on the Heise IPv6 Kongress last year, this issue can be mitigated with the following parameter in an IPv6 ACL.
deny ipv6 any any undetermined-transport
As a reminder, this parameter drops all IPv6 packets where the upper-layer protocol information cannot be determined.
After the workshop was officially over, Marc and I played a little bit with this ACL Parameter to see if it is working as intended. So I configured the following IPv6 ACL on our beautiful Cisco 4948E:
4948E(config)#ipv6 access-list IPv6
4948E(config-ipv6-acl)#deny ipv6 any any undetermined-transport
4948E(config-ipv6-acl)#permit ipv6 any any
4948E(config)#interface g1/19
4948E(config-if)#ipv6 traffic-filter IPv6 in
We started the attack again with the following parameter:
Apparently nothing happened with my (IPv6 enabled) laptop (which is a good thing ;))
The corresponding packet dump looked quite unspectacular:
Only the STP packets could be seen, and the flooded router advertisements were dropped by the Switch.
So could this parameter solve the issue with the whole RA mess?
Unfortunately the answer is no. The ACL parameter does mitigate the issue with the fragmented router advertisement. However, the ACL parameter can be circumvented by using overlapping fragments. Unfortunately we couldn’t test this scenario because this wasn’t yet implemented in the THC Tool Suite, but this is just a matter of time…
The IPv6 Packet basically looks like this:
Fragment 1:
IPv6 Header
Fragmentation Header
Destination Header (8 bytes)
ICMPv6 with Echo Request
Fragment 2:
IPv6 Header
Fragmentation Header with offset == 1 (equals position of 8th byte ==
start of Echo Request in first fragment)
ICMPv6 with RA
In this case it depends on the operating system whether or not the packet is discarded when overlapping fragments are detected. RFC 5722 is very specific on how these should be handled:
“When reassembling an IPv6 datagram, if one or more its
constituent fragments is determined to be an overlapping
fragment,the entire datagram (and any constituent fragments,
including those not yet received) MUST be silently discarded.”
So it is up to the operating system to implement this behavior. We’ll see how things work out 😉
If you’re interested in more IPv6 issues, or simply wanna chat about this topic, meet Enno and me again at the Heise IPv6 Kongress this year in Frankfurt, where we will give a talk on IPv6 as well.
