One of our customers called us recently and asked for some support in investigating a toll fraud issue they encountered in one of their sites. Their telecommunications provider had contacted them informing them that they had accumulated a bill of 30.000€ over the last ten days.
Without knowing anything more specific, I drove to the affected site to get the whole picture.
They have a VoIP deployment based on Cisco Unified Communications Manager (CUCM, aka Call Manager) as Call Agent. The CUCM is connected via a H.323 trunk to a Cisco 2911 ISR G2 which is acting as a voice gateway. The ISR has a primary rate ISDN (PRI) Interface which is connected to the PBX of the telco. Furthermore they use a feature called Direct-inward Dial (DID) or Direct Dial-in (DDI) which is offered by Telco’s to enable calling parties to dial directly to an extension on a PBX or voice gateway.
Basically one then has a so called head number (in networking terms a prefix), together with some phone extensions. When someone from outside wants to call, he dials the head number + phone extension. Before the telco forwards the call to the ISR, the head number is stripped and only the phone extension number is forwarded to the voice gateway. E.g. when calling 12345-678, the local voice gateway will only see the 678 as called number.
After having a good overview of the design, I started to dig around in the log and configuration files to understand what exactly happened and why.
So here is what happened:
Apparently someone from some East European country had called the head number of our customer and prepended a “malicious number” (in some country in Africa) to which the ISR should setup a call. The ISR only sees the malicious (African) number because, as said before, the head number was stripped by the telco. The malicious number was of course some $EXPENSIVE_LONG_DISTANCE_CALL ;). So the voice gateway received a call from the PBX and forwarded it back to setup the call with that number.
Before we proceed, a little bit of theory how a Cisco router decides how to forward a call, might be helpful:
In Cisco IOS, the call-routing table is configured based on so called dial-peers. These dial-peers specify how a call with a specific destination number should be forwarded.
As an example:
dial-peer voice 1234 pots description ===incoming_calls=== incoming called number ^[2-7]..$ port 0/3/0
This configuration tells the router that calls to a number which matches the regular expression, should be forwarded to port 0/3/0.
As it turns out our customer uses the following dial-peer which is used for outbound calls.
dial-peer voice 5678 pots description ===outgoing calls=== destination pattern 8T port 1/1/1:15 -> The ISDN Interface
The T is a placeholder value which means that any amount of digits can follow the 8. The reason the pattern matches the digit 8 is that this digit must be dialed before the actual number.
Do I have to mention that the malicious number also starts with an 8? 😉
So back to the presumed course of action:
The call with the malicious number hits the router. The router tries to match a configured dial-peer to forward the call. I think you can guess which dial peer matched for the malicious number 😉
So the router sends the call back to the PBX to setup a call to the malicious number. Which is billed to our customer…
We then monitored the situation and applied a workaround (more on this in a minute) and observed what happened. As it turned out, unfortunately the attacker was able to circumvent our workaround. We discovered that is was possible to “dial-in” to the router directly by just calling the head number (as the PBX leaves the called number field empty). E.g. the called number field in the log files looks like this:
“Called Number=”
The router subsequently provided a line and it was possible to call the number again. Our workaround did only affect incoming calls with the number prepended, but not those where the router is the actual origin of the establishment of the call.
So how can we resolve this issue and stop the toll fraud?
As a long-term solution the configured dial patterns should be reviewed and modified to prevent such things in the future, but – given the overall complexity of the setup – this could not be done overnight.
I am currently working with the customer to develop more suitable dial patterns. I will write a follow up post with the final results when we are finished.
In the mean-time, we developed a temporary workaround to prevent this from happening again:
In Cisco IOS you can manipulate the calling or called-number with so called translation rules and you are also able to reject calls based on the called number. Our customer does not use any extension beginning with 8, so we can drop all calls on the gateway which starts with 8 as called number. So we developed the following translation-rule:
voice translation Rule 11 rule 1 reject /^8+/ rule 2 reject /^$/
voice translation-profile reject_calls translate called 11
Rule number 2 addresses the case when the called number field is empty. We mapped this profile to the dial-peers responsible for the incoming calls and specified that calls with the numbers in the translation rule must be rejected.
dial-peer voice 3456 pots description ===Incoming_Calls=== call-block translation-profile incoming reject_calls call-block disconnect-cause incoming call-reject incoming called-number
Lessons learned:
Be careful when you develop and implement your dial patterns, as errors in this space can cost you quite a lot of money 😉
VoIP is a complex technology and this complexity can lead to all types of vulnerabilities, as Daniel and Enno are going to show in their talk at Troopers 2012. Toll fraud is still quite common and happens all the time, as you can see in an ERNW newsletter from 2009 covering a similar story from another environment.
On a side note:
The telco told us that our customer is the 8th customer affected by a toll fraud issue in the last two months. According to the telco all eight companies are in the same city, and the initial VoIP deployment at our customer was performed by an external service provider.
Maybe the same service provider has done the deployment in the other companies too…
Have a great day,
Chris