Python has reached a defacto standard in exploit development lifecycles and most of the proof of concept tools you’ll find out there are written in Python (besides the metasploit framework, which is written in Ruby). Python allows to write scripts handling with remote services, fiddling with binary data and interacting with C libraries (or Java in case of Jython/.Net in IronPython) in a fast and easy way. The huge standard library with it’s “battery included” principle removes some of the dependency hell known from other frameworks/languages. I want to share some of my python coding experiences with you, and maybe this could give some helpful tips for your future work, to make the world a bit safer 🙂 (PS: most of the examples are written in Python 3.x or compatible to both Python branches).
Setup your environment
For most of the projects/scripts you’ll write, it would be quite useful to have all dependencies in one place (and only those dependencies you’ll need for a particular project). To fulfill these requirements, a tool/library exists, called virtualenv (It’s already included in Python since version 3.3). This tool has the neat feature to generate isolated environments for your python projects, without cluttering your global environment. The generation of a new environment is as simple as
$ virtualenv <path to new environment>
or in Python >= 3.3:
$ python3 -mvenv <path to new environment>
To use this environment, you’ll have to activate it:
$ source <path to new environment>/bin/activate
Deactivating/leaving the environment is also easy:
$ deactivate
Install the dependencies
Most of the time you’ll find some libraries from the huge python community, which helps you to get fast results with your tools. You could install them by using your distributions package manager or by using one of the available pythone package managers. Perhaps the most powerful one is pip. With pip you can install your dependencies global (# pip install <package>), per user ($ pip install --user <package>) or inside a virtual environment ((venv) $ pip install <package>). You can install pip either by using your distributions package manager, manually or by using the library provided by Python 3.4 onwards ($ python3 -mensurepip).
One of the essential packages I usually install when I’m not 100% sure how I would solve the current task and want to experiment a bit is IPython. IPython is an impressive Python shell programmed in Python. To name a few features:
- Dynamic object introspection.
- Completion in the local namespace, by typing TAB at the prompt.
- Persistent command history.
- Session logging.
- Path completions.
- JIT debugger.
- Auto-indentation.
Installing via pip is easy as usual: $ pip install ipython
If you want to create tutorials or other documenting files, the notebook feature from ipython (now provided by jupyter) allows you to interact with a IPython shell via your browser including markdown, mathjax and inline matplotlib support.
(Use them by installing jupyter (pip install jupyter) and start the notebook server (jupyter notebook)).
If you want to interact with HTTP services including JSON/XML-APIs, I recommend the awesome requests library. This library handles the most encoding/decoding/parameters/cookie/redirect stuff you’ll face with during interacting with a website. For example requesting and parsing a JSON resource is done by:
r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}
HTML parsing and interaction could be done most of the time with the BeautifulSoup library. This library is able to handle HTML input equally to modern browsers, including trying to fix broken code.
Interacting with the network
Most of your targets may be reachable over the network, therefore I’ll give a short introduction to common/helpful Python libraries already included in your standard Python installation.
For the lower network level interaction the library gives you the socket and ssl modules. The socket module is a thin wrapper around the BSD socket API available on all common operating systems. So if you already have experience in socket programming in C, you could translate your code easily into Python. Some convinience functions exist, for example the create_connection function which creates a TCP socket, resolves hostnames and connects to the given host/port. Another wrapper is the sendall method, which tries to retransmit data which wasn’t able to go over the wire until all given data was sent or an error occurs.
from __future__ import unicode_literals
import socket
s = socket.create_connection(('www.ernw.de', 80))
s.sendall(b'GET / HTTP/1.1\r\nHost: www.ernw.de\r\n\r\n')
print(s.recv(1024))
Adding a TLS-tunnel to the connection is also very simple:
from __future__ import unicode_literals
import socket
import ssl
s = socket.create_connection(('www.ernw.de', 443))
s = ssl.wrap_socket(s)
s.sendall(b'GET / HTTP/1.1\r\nHost: www.ernw.de\r\n\r\n')
print(s.recv(1024))
This is even possible during a already used connection:
from __future__ import unicode_literals
import socket
import ssl
s = socket.create_connection(('smtp.example.com', 25))
s.sendall(b'HELO smtp.example.com\r\nSTARTTLS\r\n')
print(s.recv(1024))
s = ssl.wrap_socket(s)
s.sendall(b'MAIL FROM:<foo@example.com>\r\n')
print(s.recv(1024))
If you don’t need such a low level interaction with your service, several modules are available to interact at a much higher level:
- smtplib
- ftplib
- poplib
- imaplib
- httplib (http.client in Python 3.x)
- nntplib
- telnetlib (useful if you have to exploit a service and want to have an interactive shell session afterwards)
- xmlrpclib (xmlrpc.client in Python 3.x)
Binary handling/Encodings
When developing scripts which have to interact with services or files, you’ll often find yourself in the need to convert the data between different formats/encodings. In Python 2.x this was most of the time as easy as executing encode or decode on your string to convert it between different formats:
"Hello World".encode("hex")
"AAA=".decode("base64")
Sadly, this shortcut was removed in Python 3.x and the encode and decode methods now only convert between character encodings like utf-8, cp1250, iso8859, big5, …
Instead, you now have to use two methods on the bytes type for hex encodings:
bytes.fromhex('414141')
b'AAA'.hex() # starting with Python 3.5
For Base64 encodings you’ll have to use additional modules (which also exist in Python 2.x btw.):
import base64
base64.b64encode(b'Hello World')
import codecs
codecs.encode(b'Hello World', 'base64')
import binascii
binascii.b2a_base64(b'Hello World')
Encoding/parsing urls can be done with the urllib.parse module (or urllib in Python 2.x):
from urllib.parse import quote_plus, unquote_plus
quote_plus('Hello World+1=1337') # Hello+World%2B1%3D1337
unquote_plus('Hello+World') # Hello World
General conversions between Python simple types (int, float, str) and binary could be done with the struct module:
import struct
struct.pack('<I', 1337) # convert the integer 1337 into its little endian, 32 bit representation
struct.unpack('<I', b'\x10\x00\x00\x00')[0] # returns tuple of results -> get only the first result
struct.unpack('<I4s', b'\x10\x00\x00\x00Test') # returns (16, b'Test')
Since Python 3.2 you could also use the int type directly to get its binary representation:
a = 1337
a.to_bytes(4, 'little') # 32 bit little endian
a.to_bytes(2, 'big') # 16 bit big endian
int.from_bytes(b'\x10\x00', 'little') # 16
Another great feature are the Structures from the ctypes module. If you are using cpython as your interpreter (most of the time you will) you can use the ctypes.Structure type to describe C like structures and get their binary representation as if they were dumped from a C application:
from ctypes import *
import io
class TestStructure(Structure):
_fields_ = (('foo', c_int), ('bar', c_char))
t = Test()
t.foo = 1337
t.bar = b'A'
b = io.BytesIO()
b.write(t)
b.seek(0)
print(b.getvalue()) # \x39\x05\x00\x00\x41
t2 = Test()
b = io.BytesIO(b'\x10\x00\x00\x00\x42')
b.readinto(t2)
print(t2.foo) # 16
print(t2.bar) # B
The ctypes module is normally used as a clue between Python programs and C libraries for which no Python wrapper was written. With the ctypes module you can access any C library and their exported functions:
from ctypes import *
libc = ctypes.CDLL('libc.so.6')
libc.printf(b'Hello World\n')
The Structure type noted above is used to interact with C libraries which pass/expect structures during function calls.
Exploitation helpers
Many CTF-teams provide their own framework for exploitation and CTF solving. I found the pwntools framework from Gallopsled really useful, especially in terms of exploiting remote elf binaries. It contains functions for offset calculation (via cyclic patterns), format string exploitation (simple feed in the data you want to write and at which offset and it generates the format string for you), rop chaining (parses elf binaries with ropgadget and provides wrappers which results in a simple rop chain calls) and a throughout API for different transport channels (called tubes). This enables you to develop the exploit for example with a gdb backend and simply change one line to transmit the exploit to the target service:
from pwn import *
r = gdb.debug('./level3')
#r = remote(IP, PORT) # simply uncomment to interact with a remote service
r.recvuntil(': ')
r.sendline(EXPLOIT)
r.interactive() # start interactive session
Best regards and happy hacking,
Timo
PS: Some examples of a IPython notebook export:
Seems like a great quick guide on Python, not just for hacking. Thanks!