Binhqx's Blog

That in the database looks like:

2005-12-19 19:25:50.284

Hurrah I win! With both "Acct-Session-Id" and "h323-setup-time" I now have all the information I need to make accurate CDRs. To bad I also now have a database full of bad call information. It is a good thing I keep a backup of all radius transactions that can be used to reload the database.

Posted by binhqx at 9:41 PM | Comments (0) | TrackBack

November 16, 2005

Telephone Engineers make the internet cry.

If there is one thing that is holding back VoIP, its telephone engineers. Telephone engineers see calls as having ANI, DNIS, and call route trunks. Information in these can only be sent as strings of digites. But these are outdated notions. VoIP calls have RequestURIs, To, and From field. These are highly flexible ways storing call information. With them one can create flexible and robust methods to route call signaling.

So many VoIP vender's hardware only sees the old phone system type of information. To do any advanced call routing with these, one must pack more information into the ANI and DNIS. What a incredibly backwards way of doing it. We have a whole flexible protocol(SIP) to setup calls and vender only allow the ANI and DNIS to route calls!

Here is my most outrageous example:
One of our customers gets quite a few payphone calls. These are charged with a slightly different rare then regular calls. In the past, a two digit "Information Digits" was prepended to the ANI that could be used to Identify these calls. However, this particular can only bill by the route the call takes, and the only way to do that is to move the "Information Digits" from the ANI to the DNIS! Now we have crazy looking DNISs that that are difficult to route because they are no longer valid phone numbers!

This make me want to cry for the SIP protocol. This is not the way it was intended to be used. Special origination or termination info should be added to the parameters of the RequestURI, To, or From field; or the information could be added to a custom fields. Then any SIP device should be able to recognize the parameters and deal with them appropriately. Just please do not mess with the user portion of any URI.

And do not think Sansay is off the hook here. They are among the telephone dinosaurs that only route using the ANI or DNIS. There are so many parameter in a SIP INVITE, let me choose which ones to route with.

Posted by binhqx at 1:47 PM | Comments (0) | TrackBack

November 10, 2005

Fragmented UDP SIP packets, probibly a bad idea.

Today I found a SIP invite message so big that it would not fit in the MTU of a ethernet packet. So, of course, the message was split in to to two packets. Then I heard the wise idea of "It should be able to handle fragmented packets." I nearly killed my self. The SIP protocol has no internal management of fragmented packets.

Actually, the RFC 3261 has a provision to handle just this eventuality:
In section "18.1.1 Sending Requests"

So in the event of the message become too big, the conversation should switch over to a protocol that can handle it. I know the SER can hand this, but I do not know what other hardware does. There might also be other complications if NAT traversal is involved. I really hop that oversized SIP messages do not become the Achilles heel of SIP.

Posted by binhqx at 9:12 PM | Comments (0) | TrackBack

October 22, 2005

Sansay Review (part 2)

I did a quick review of the Sansay SIP and H.323 voce switch at the beginning of the month. Now that I have worked with it for a while, the honeymoon is over and there are some missing features that are annoying me.

Missing feature #1: number matching with wildcard characters. The Sansay uses a system called "best match" to recognize phone numbers for routing and digit manipulation. This means that it will match the search string that is most-like the number.

For example, if you had two search strings used to route a call:
route 1-> 5556
route 2-> 5556892
And the number that is being processed is 5556892621, even though the number matches both search strings, the call will be sent using route 2. This is because "5556892" is a *better* match, it has more digits in common. The system is easy to understand and implement by any layman.

Now lets try to do something a little more complicated. Here is a digit manipulation that I need implement on our Sansay. One of our telephone number providers is sending all ANI(Automatic number identification) a.k.a Caller ID, with a two digit "Information Digits" prepended to it. These Information Digits identify the type of phone that is making the call. A 00 indicates a standard non-coin telephone. A 25 indicates a call from a coin or prison telephone to a toll-free number. These are just two examples, but there are many posable Information Digits. The problem was the Caller ID on the other end did not understand how to deal with the Information Digits and could not recognize any income called numbers.

So I needed a digit manipulation table that would detect the Information Digits on the ANI and strip them off. But each posable Information Digit needed its own rule to match property. So I ended up with a digit manipulation table that looked like this:

If the number is 8005551111 and the ANI matches 00 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 02 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 06 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 07 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 20 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 23 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 24 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 25 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 27 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 29 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 30 then manipulate the ANI.
If the number is 8005551111 and the ANI matches 31 then manipulate the ANI.
< and so on for all the other posable Information Digits >

This is currently how the Sansay works and this is just for one number. The rule set will need to be duplicated for all the other toll free numbers I get from this provider. I see this as being totally unmanageable. There are 24 common Information Digits patterns, if I need to route 100 toll free numbers, that means that I will need to add 2,400 rules to the digit manipulation table. That is a difficult volume of information to maintain.

So what is the easy solution for this? Use wildcard characters to match sets of numbers. Lets say that the '\d' character matches any digit. Now lets see how many rules we need:

If the number is "8005551111" and the ANI matches "\d\d\d\d\d\d\d\d\d\d\d\d" then strip off the first two digits of the ANI.

This says that for all calls to 8005551111 where the ANI is 12 digits long, remove the first two Information digits. This replaces 24 rules with 1 rule. If you have a block of similar called number, it can make thing even easier. So if you needed to manipulate calls from 8005551110, 8005551111, 8005551112, 8005551113, through 8005551119 the rule would look like the following:

If the number is "800555111\d" and the ANI matches "\d\d\d\d\d\d\d\d\d\d\d\d" then strip off the first two digits of the ANI.

Now I have replaced 240 rules with 1 rule. I do not see why this would not be implemented by the Sansay developers. It makes life so much easer for the system administrators.

What really tick me off is I made this suggestion to Sansay and their reply was that it was too complicated and no one would use it. I call bullshit on that. I have worked with several VoIP systems and they all use wildcard number matching. Sansay is making it harder to use by not having them.

Missing feature #2: Configuration rollback.

On more then one occasions I have been working with other VoIP engineers who are making changes to their soft-switch when I hear "Oh, !#$%^!". They just made a change to the configuration that broke everything. On the Sansay it is really easy to do this because changes are made immediately upon hitting the submit button. If one does make a mistake, they need to backtrack through all the changes they made to find the one that is causing the problem.

So why is there on way to rollback the changes made to a previous configuration. Each time a setting is changed, it is time stamped and recorded. If a problem is found, just Identify when it started occurring and role the setting back to before then. it would also be nice to get a list of all the changes that there made and who made them. This would allow a system with multiple administrators to keep track of who changed what.

i do not know if any other VoIP soft-switch vender put this feature in, but they should. It would add greatly to the overall reliability and robustness of the system even when users try to bungle it up. I also mentioned this feature to Sansay but they said that it would be too resource intensive and slow down the call processing speed. I call bullshit on this too. The recored of the change would only be recorded after a setting has been modified. In term of commuting performance this is a rare occurrence.

(The rest of these features I have not yet talked to Sansay about, but I probably will in the future.)
Missing feature #3: Settings overwrite prevention.

The Sansay allows for multiple administrators to make changes to the at the same time. But what if two people change the same setting to two different things at the same time? The answer is the one who hit the submit button last. The other on continues on not knowing there changes did not commit. This can cause big problems later on.

This could be fixed with resources locking. When one admin opens a page to make some changes, those settings are locked out to all other admins till the setting have been submitted or the session times out. There would also be a overide mechanism that would allow one admin to knowingly overwrite someone else's settings. These events would be recorded and both parties would be informed of the infringement.

Basically, discourage admins from stepping on each others toes, but when they do it make sure that everyone involved knows.

Missing feature #4: Configuration Commenting.

An important part of complicated configurations is letting other people know why changes where made. That is why it is important to comment one's configuration. These could be in the form of logs or change logs or notes that are part of the configuration. It can contain reminders of why thing are the way they are for future decision makers.

Each setting field or editor page will have a place to comment or leave notes. As a bonus make these searchable so particular bits of information can be found later.

Missing feature #5: Call simulation and processing trace.
It would be useful fro call testing if the admin could run an internal simulation where a number is sent from a simulated host. It would then generate a report of as all the routing rules it and digit manipulating the call encounters. This would save a lot of time with call route testing.

Missing feature #6: Sed or Perl like regex for rule matching and digit manipulation .

Regular Expressions(regex) are an incredibly powerful method of detecting patterns in strings and manipulating them. The syntax is far more complicated then just wildcard characters but the level of flexibility it provides is astounding. There are many software languages that use regex. The most common ones are sed and Perl. Adopting this type of syntax would not be too difficult for end users. And any difficulty would be greatly offset by the power it provides.

Missing feature #7: Rule matching and digit manipulation based on more then just source or destination number.

The Sansay can only apply routing based on source host, destination host, ANI or DNIS. It would be useful to have boolean operators (AND, OR, NOT) to make very specific rules. Additionally SIP messages contain a lot of useful information that could be used in call routing. These could be used to route hardware from specific venders to different locations. Or sending particular codecs to different host.

That is all I can think about right now. I will probably have some more later.

Posted by binhqx at 12:33 PM | Comments (0)

October 13, 2005

Cool toy: Gizmo Project

I found a cool little toy called the Gizmo Project. It is a service very much like the Skype. It supports IM with voice chat ability including VoIP to PSTN bridging and incoming number service. Most importantly they both have easy to use interfaces that can be managed by average PC users.

However the big difference is Gizmo use the SIP as its voice session control protocol call. This allows any Gizmo user to interface with any other SIP service. For example, last night I was playing some WoW from home with SB, archcaptain and Jack. The headset on my ATA died so I had no way of talking with them. I fired up Gizmo and inputted the SIP address of the the conference server. It hooked up flawlessly and the audio quality was some of the best I have heard on a PC client.

Gizmo also has a fun little ability that allows you to play WAV files into the call. This can add a laugh track to any call you make to amuse or annoy the person at the other end.

Posted by binhqx at 10:41 PM | Comments (0)

October 3, 2005

Sansay VoIP soft switches rule!

Today I got my hands on one of Sansay's Session Border Controller. This thing frik'en rules! It only took me a few hours to configure it to manage all my voice traffic going to and from my PSTN gateways. This is a god-sent because of the large number of gateway providers I have been connecting to. The configuration of the SER to reliably connect to and from many gateways with full security was becoming unmanageable.

The Sansay is also a state-full "Back-to-Back User Agent" so I can now have full confidence that no one is getting free(unrecorded) calls by hacking the SER (I do not think anyone was, but now I can make sure of it).

The next step will be to connect the Sansay's CDR system to some type of billing system. It will be nice to get rid of the current hack job we are running right now. But that is what you get when you have a only a few days to pull something together with perl scripts and excel worksheets.

I give the Sansay a grade of "frik'en rules" for being the right tool at the right time.

Posted by binhqx at 6:19 PM | Comments (0)

September 29, 2005

VoIP wiretapping in practice?

I was browsing around the web looking for methods to do packet sniffing to debug some SIP configs when I found this article.
Things To Do in Cisco Land When You're Dead

its a cool hack that redirects traffic from a Cisco router through a GRE tunnel to any other host. It is posable for that host to sniff out the RTP packets and play them using tools like rtptools.

But before everyone flies off the handlebars about the insecurity of VoIP you have to rember that it would require a full compromised Cisco VOIP Gateway router. So as long as those Gateways stay secure, the world is still safe.

I wish I had the time to try this out.

Posted by binhqx at 3:33 PM | Comments (0)

September 27, 2005

Secure VoIP and Wiretaps

A article showed up on news.com discussing FCC mandates on internet phone services to allow for law enforcement to conduct wiretaps on VoIP calls. The fist argument everyone makes when they hear this is "the Internet is not under FCC jurisdiction, they can not mandate anything there." But I am going to put this aside and talk about the technical feasibility of a VoIP tap.

There are two types of VoIP calls, those that occur between IP clients and those that bridge between IP and the public telephone network. A example of the first type would be any of the Instant Messenger services that support voice chat. AIM, Yahoo Messenger, MSN Messenger, and Skype are examples of these. With the exception of Skype, none of these communications never leave the internet. The calls the bridge between IP and the public telephone network must uses a gateway service like USA Datanet or Level 3. This allows for IP phones to call the public telephone network and vice versa.

When it comes to putting in a wiretap, it is easy to intercept a IP to public telephone network. All calls must pass their voice traffic through a gateway. This makes a central trusted location from which all calls can be monitored. Once that is done, standard wiretap procedures can be applied.

However call passing over the internet are completely different. First, there is no central route through which all the voice traffic passes. The only place where one can be assured the voice traffic will be are the ISPs at both ends of the call. So the cooperation off all ISPs will be necessary to tap a VoIP call.

Next is the protocol. Each of the private voice networks has their own proprietary codec. This problem can be easily dealt with if the networks cooperate with the ones doing the wiretapping. However there are protocols that their creators have made to be as secure as posable. The open VoIP protocols have a security wrappers that will make call interception within a reasonable time span nearly impossible.

So in the short run, VoIP wiretapping will only be posable of call that are passed to the insecure public telephone network. Calls that never leave the internet can be captured, but extracting their contents could take years.

So as we move away from the old telephone system, how will the feds be able to monitor criminal activity? Even if there was federal legislation to put interception ability into every piece of hardware and software, independent VoIP providers in foreign nation will provide "secure" service that can not be hacked. It would be a unenforceable law.

Posted by binhqx at 2:48 PM | Comments (0)

September 23, 2005

Hostile network detection.

One thing that need to be built into ever VoIP device is some system to detect if the network it is being hosted on is hostile to voice traffic. All voice over internet technologies work essentially the same way. The sound from the microphone is chopped into tiny increments. This packets of voice data are stuffed into a RTP package that acts like a first class mail label of the internet. The packet is then transmitted as fast as technically possible over the open internet to it destination. Once there, the process is reverts and the sound is played out the speaker.

For the system to make acceptable calls, enough RTP packets must actually arrive at the destination. Additionally, they must arrive in the same order they left in. If a packet arrives after its subsequent packets arrive, it is usually thorn out because otherwise bits and pieces of a conversation would arrive late and be inserted into the current conversation making everything confusing.

So all the parts of the network must be working harmoniously together to handle these RTP streams. But is is almost never the case. The primary backbone of the internet between major ISPs can handle this traffic just fine. However the last mile between the ISP and the enduser is a battle field between conflicting interests. For example, the upstream bandwidth is always tightly caped per user. So even though you my have 5Mbs download speed, you may only have 256kbs of upload. Even of there is 256kbs of upload, you may find that it gets smaller during peak hours. On average, a normal VoIP call will require 64kbs. But that is a constant stream that is competing for bandwidth with higher capacity data transfer from file downloads, or god forbid P2P applications. Because of its stringent network requirements, when bandwidth becomes scarce, VoIP is the first to suffer.

I also heard today from a reputable source that consumer ISPs are purposely attacking VoIP calls buy giving them lowest priority, or just occasionally dropping the packets. THis is so the customers get feed up with there VoIP service, and are forced to by phone serves directly from their DSL or cable provider.

So my question is, why is there no mechanism in any VoIP hardware or software to alert the end user to the network becoming hostile to a voice call? When the call packets start arriving late or out of order or missing, why is there not some whey to indicate to the user that something is amiss. With hard data about the quality of the network connection, users could confront their ISP and demand better/more reliable service.

If I am ever in charge of a VoIP product, "hostile network detection" would be a key feature. With it, user and VoIP provider would have the tools to diagnose bandwidth problems and confront the people causing it.

Posted by binhqx at 4:31 PM | Comments (0)

September 7, 2005

SIP Faxing

I feel like I have let everyone down continued.

On Tuesday the integration lab received another Access 211 ATA to test with. I finally figured out why they would not receive faxes.

It was actually a two fold problem. The Access 211 has T.38 support that will activate upon detecting fax signaling in the stream. When the call was made outgoing, the Access 211 would negotiate the initial connection with the T.38 support. But when the call was received it did not include T.38. The Access 211 would detect the signaling and attempt to re-negotiate the call. The PBX the Access 211 was hooked to did not like the re-negotiation and would kill the call.

The second part of the problem came from our gateway providers not properly supporting all out of band signaling protocols as defined in RFC 2833. Specifically section 3.11 "Data Modem and Fax Events." I had setup the Access 211 to use RFC 2833 signaling and squelch all in-band signaling. However, the gateway providers simply ignored the out of band signaling and the fax setup signals came through as silence.

The solution was to disable T.38 and put the in-band signaling back into the voice stream. It is not perfect, the SIP line will now operate like dirty/laggy copper line. But it works! I will need to get on my gateway provider's cases about providing the advanced faxing features our customer's need.

Posted by binhqx at 10:47 PM | Comments (0)

September 2, 2005

Handling SIP Request-URIs that have been converted from tel URIs

We found a interesting problem with a new Sonus switch today. Apparently, Sonus converts SS7 signals into tel URI before finally converting them into SIP URI. The result is something ugly that breaks the scheme of the Sip Express Router (SER).

Here is a very friendly SIP Request-URI that the SER can handle perfectly:
sip:user@hostname
On my SER, numbers comming in from our PSTN gateway providers it looks something like this:
sip:5555555555@192.168.0.1

However, the new Sonus gateway sends Request-URI that look like this:
sip:5555555555;npdi=yes@192.168.0.1

The first time I saw that, I was like "WTF! Why is there a parameter in the username field of the Request-URI!" So when the SER parses the URI, the username comes through as "5555555555;npdi=yes". The SER has no idea who that is. The reason this happens is due to a lesser know part of RFC3261 section 19.1.6. It deals with mapping tel URI into sip URI. When this occurs, the entire tel URI is copied verbatim into the SIP username field. I find this strange because SIP already has a location to put URI permitters at the end of the URI after the host name.

Luckily, the SER is very flexible then it comes to message and URI manipulation. By using the subst_uri method. In this case, 192.168.1.10 is the gateway host and 192.168.0.1 is the SER proxy.

Calls come in with us URI of sip:5555555555;npdi=yes@192.168.0.1 and is converted into sip:5555555555@192.168.0.1. My example will only work when the user field is a number. however, the matching regex can be adjusted to include any valid characters.

I think this is a kinda ugly way to deal with the problem, but it works.

Posted by binhqx at 9:51 PM | Comments (0)

August 5, 2005

Mutiple VoIP devices behind the same NAT, oh the agony!

VoIP and Network address translation (NAT) are two technologies that are at odds with each other. Any VoIP devices that wants to receive incoming calls must be able to receive signals originating form the outside. While NAT is designed make the best of the limited availability if IP numbers by hiding network devices from the out side.

The way the world has found to get around this conundrum is using protocols like STUN that allow a network devices to know the public IP address of its NATing router. Then it opens a UDP port to the VoIP host and notifies it of how the VoIP client can be reached. By occasionally "tickling" the UDP port, the NATing router will leave the port open for the VoIP host to send signals to the client. This, of course, only applied to NATing router that actually supports STUN.

The system works well and is deployed very wildly. But what if two VoIP clients are behind NATing router? Remember that all UDP/IP ports have a originating port and a destination port. The originating port is what is opened on the NATing router. The standard port for SIP signaling (the protocol I use) is 5060. So when two SIP VoIP clients attempt to open a UDP port to the host using the default port, only one of the clients will be allowed to keep the port open. So in the end, one phone will work, and the other one will not.

The easy solution for this is to configure each SIP client to use a different originating port. But that would involved walking a customer through a procedure where they are most likely break everything. An alternate solution would be for the SIP client to randomly chose a hight number UDP port to originate from. The ATA that we had previously used supported port randomization. But that ATA became deprecate and unsupported by the vender. When I finally got a hold of the new ATA we will be using, I found out that it did not have the ability to port randomize, and the development team had never even heard of that feature.

So I made a strong argument to one of the dev team members detailing the feature and the scenarios it comes into play. They said they would add it to the feature request list. I will report back when they add it. This will be their support test.

In their defense, the port randomize was the only missing feature I found. Everything else is just hunky-dory.

Posted by binhqx at 2:21 PM | Comments (0)

June 30, 2005

SIP: the unstanderd standerd

My current job (as in what I get paid to do) is to connect the malfitting components of voice over IP into a robust and usable network. The core of this network is based on a protocol call Session Intitation Protocol or just SIP. This protocol was created by the good people at IETF and put into a document called RFC3261. RFC's are met to be the standards of internet standards, the codex of codec, the bible of protocol. Unfortunately, like all sacred documents, it still must be interpreted by human beings and each one has their own style to add to it.
RFC3261, and the other volumes that explain its details, only really describe the vocabulary two SIP devises use to talk to each other without clearly defining what should be said or how any complex conversations are conducted. It is like having vocabulary and grammar without etiquette to fill in the procedural holes. So we now have devises that can talk to each other with complete understanding but no idea how to act upon it to accomplish anything.

Case in point: Allworx vs. Epygi
Allworx and Epygi both make voice products the can communicate over IP using the SIP protocol. They both make PBX systems that allow private phone networks to communicate over the internet and PSTN. But the two companies have very different ideas about how exactly this should work. Allworx tends toward the traditional PBX system in which all the phones are hidden by the central exchange from outside world. It also only allows calls from other servers it already knows preventing it from being cold contacted my other SIP hosts. It acts as sort of a burley gate guard that must be talked to before entry is allowed. The Epygi on the other hand used a new style network where each phone acts as a independent entity that uses the central exchange only to moderate discussion. Each phone can be directly contacted from anyone on the internet. The system is similar to email in this way; to contact a user you just dial the user@domain.com and the call is connected. There is nothing inherently good or bad about the two systems, but it becomes tricky when you want the two talk to each-other.

Note: It is my personal feeling that Allworx is the way of the past and Epygi is the way of the future. The Allworx PBX concept is outdated and will make it difficult for them to enter the age of pure VOIP networks. Of course I have expressed this concern to the engineers but they are slow to except. I'm sure at some point in the future the VOIP environment will force them to change. Either that or they will perish. That being said, I think the Allworx is a damn fine product and only needs a little fixing to become the best.

We use a Allworx system as the core of our customers VOIP system. One feature it is lacking is the ability to have any more then a three participants in a phone conference. Epygi has created a product to fill the role exactly. The Epygi QuadroCS is a fully featured conference bridge that supports up to 25 simultaneous participants. The theroy of operation is simple:

1. The caller dials the number of the conference server and establishes a SIP call.
2. The attendant plays back a recording asking for the identification number of the conference the caller wishes to join.
3. The original call ends and a new call is placed from the server back to the callers phone.
4. The caller is now connected to the conference and can conduct his bisness.

When the originating caller is coming from a Alworx, the system breaks down at step 3. The conference server establishes a new call back, but the Alworx will not allow direct establishment of communication with its phones and the call goes dead...

Now for the work around: Fortunately for us, the QuadroCS has some flexibility when it comes to new conference establishment. By reconfiguring the server to allow new participants directly into the conference and bypassing the attendant, Alworx users can get into hosted conferences. This setup requires a SIP proxy to map a normal phone number to redirect to the conference ID. The users then has to dial the map phone number over the SIP trunk to the proxy. Ouch! its a ugly solution. No flexibility to make new conferences without reconfiguration of the proxy, no attendant based features. These two philosophy clash head on making everyone's life more difficult.

There is one additional feature that Epygi's design allows for. I saw this at the bottom of a email I received from them (paraphrasing to protect the innocent):

SIP: john.doe@epygi.com