Unlocking The Secrets Of SMS (Summer, 2008)

Hello, and greetings from the Central Office! After an unusually cold and rainy winter here in the Pacific Northwest, summer is in full swing. With so little good weather in this part of the world, people head outdoors and make the most of it—even with gasoline hovering near $5 per gallon.

For many young people, this means it’s time for noisy outdoor concerts, which I’m told are even louder than our diesel backup generator here at the Central Office. At a huge music festival with sound systems approaching the decibel level of a 737 taking off, how do you find your friends? Increasingly, text messages are the solution.

You may not think about it much when you’re sending “HEY CRACK DAWG WHERE U @” to your friend, but sending and receiving small text messages is incredibly complex—in fact, much more complicated than e-mail. Making matters worse, there are multiple versions of SMS, and multiple technologies involved in mobile phone systems (for example, CDMA IS-95, CDMA2000, GSM CSD, and GSM GPRS). For this article, I’ll focus on GSM networks, which are operated by AT&T and T-Mobile (along with some smaller regional carriers such as Edge Wireless) in the US.

Text messages are governed by the Short Message Service (SMS) standard. This is currently defined as part of the European Telecommunications Standards Institute (ETSI) GSM 03.38 standard. It incorporates, by reference, the MAP part of the Signaling System 7 (SS7) protocol. The specification allows for 140 byte messages. In North America, this translates to 160 characters because the character set used is limited to 7-bit ASCII characters. In Unicode alphabets (such as Arabic, Chinese or Cyrillic), where characters are 2 bytes apiece, SMS messages can only be 70 characters in length. Whichever alphabet you use, larger messages are generally split apart to be delivered (and billed) as multiple text messages. However, because additional metadata is required to accomplish this, the size of each message is reduced by 6 bytes (7 ASCII characters).

To understand how a SMS message is delivered, it’s important to first understand a little about how GSM switching works. So, here’s a crash course.

HLR 

When you sign up for service, your phone number, the IMSI from your SIM card, and information about the capabilities of your account are input into the Home Location Register (HLR). This is a database operated by your wireless carrier, and it largely controls what your handset is both allowed and configured to do on the network (e.g. place and receive calls, send and receive text messages, forward calls to voicemail, use data services, and so forth). The HLR also keeps (approximate) track of your location on the network, in order to deliver calls and messages appropriately. In general, each wireless carrier operates one HLR topology, and large carriers split up subscribers between HLR nodes. The HLR is the nerve center of a wireless carrier, and if it fails, a very bad day is guaranteed for the person who administers it. At a minimum, nobody will be able to receive incoming phone calls, text messages will be delayed, calls will not forward to voicemail, and self-important people in SUVs everywhere will be unable to use their BlackBerrys while running over old ladies in crosswalks. So, as you might imagine, an HLR outage means the carrier may lose thousands of dollars per minute. Fortunately, redundancy and failover capability are fairly sophisticated. For example, Nortel’s NSS19 platform allows for both local and geographical redundancy. HLR databases themselves are also designed with a high degree of redundancy and fault tolerance, allowing rapid recovery in the event of failure.

MSC

An MSC is a Mobile Switching Center. In effect, this is a Central Office for mobile phones. However, unlike traditional wireline Central Offices, which generally cover only one city (or in large cities, as little as one neighborhood), MSCs generally cover an entire region. These incorporate all of the functionality you would expect from a modern Central Office, along with a lot of whiz-bang features specific to mobile phone applications (such as the VLR described below).

MSCs can be either local or gateway MSCs. A gateway MSC is analogous to a tandem switch, and can communicate fully with other wireless and wireline networks. A local MSC is analogous to a local switch, although these switches can often route directly to the PSTN (and increasingly, VoIP networks) for voice calls. 

VLR

Your mobile phone will generally be registered in the Visitor Location Register (VLR) of the Mobile Switching Center (MSC) serving the area in which it is located (although the HLR does not necessarily have to be decoupled, so in smaller GSM systems the VLR may be the same as the HLR). The VLR retrieves a local copy of your subscriber profile from the HLR, so most routine queries can be processed against the VLR rather than the HLR. This minimizes load on slow and expensive inter-carrier SS7 (and sometimes even X.25) links and the HLR servers. These systems are also designed with a high degree of fault tolerance, because it’s also bad if they fail. However, the failure of a VLR will cause only a localized outage. Failed calls will generally be forwarded to voicemail in the interim, and SMS messages will be held for delivery until the VLR is again operational.

MXE / MC

The MXC (also referred to as MC) handles messaging. On GSM systems, this includes voicemail, SMS, and fax features (yes, the GSM standard includes sending and receiving faxes for some reason).

SMSC

Hey, we finally got to the piece that really matters. The SMSC is the component of the MXE which handles SMS origination and termination. SMS messages sent or received generally pass from your handset to the MSC to the MXE to the SMSC, and then either in the reverse direction (for on-network SMS) or to the gateway MSC for inter-carrier delivery.

Message flow

I’m a visual person, so here’s a visual depiction of how an SMS is sent. Read it from left to right:

 

Diagram showing how SMS originates

Figure 1: Mobile SMS Origination (Diagram drawn by Carre)

Note that the SMS protocol accounts for the unreliability of wireless networks by using an acknowledgement sequence.

Next, here’s a visual depiction of how your phone receives SMS messages from the network. Read it from right to left:

Diagram of how GSM phones receive SMS

Figure 2: Mobile SMS Termination (Diagram drawn by Carre)

Note that the acknowledgement sequence is also end-to-end, as in Figure 1.

Billing

While the GSM standard defines how the SMS protocol works and the data structures associated with it, billing is left up to the carriers. This is a contentious issue, particularly overseas where carriers do not charge for receiving SMS messages. Unlike e-mail, SMS is billed per message, and carriers will generally not deliver messages unless they have a billing arrangement with the originating carrier. This has given rise to inter-carrier SMS providers, such as VeriSign, who negotiate wholesale billing arrangements on behalf of carriers. Generally, in the absence of a billing arrangement, carriers will refuse delivery of SMS messages. This is a particularly glaring issue when using SMS short codes. For example, the popular 8762 (UPOC) short code is not available to Sprint subscribers, because Sprint lacks a billing arrangement with Dada (the owner of Upoc).

Well, it’s the end of my shift here in the Central Office, so enjoy the rest of your summer and please wear ear plugs if you dance near the big speakers. Instead, save your hearing for The Last HOPE in New York, where I’ll be speaking this year!

References 

http://www.nowsms.com/discus/messages/1/1103.html – This message board thread provides a detailed description and listing of the SMS character set.

http://www.nortel.com/solutions/wireless/collateral/nn117101.pdfNortel whitepaper for the NSS19 HLR platform.

http://www.eventhelix.com/RealtimeMantra/Telecom/ – Detailed flowcharts of common GSM call flows and sequences.

http://en.wikipedia.org/wiki/GSM_services – Well-written Wikipedia article outlining consumer services available on GSM networks.