Electronics project notes/Hilo Arduino GPRS shield notes

From Helpful
Jump to navigation Jump to search
This is for beginners and very much by a beginner / hobbyist.

It's intended to get an intuitive overview for hobbyist needs. It may get you started, but to be able to do anything remotely clever, follow a proper course or read a good book.


Some basics and reference: Volts, amps, energy, power · batteries · resistors · transistors · fuses · diodes · capacitors · inductors and transformers · ground

Slightly less basic: amplifier notes · varistors · changing voltage · baluns · frequency generation · Transmission lines · skin effect


And some more applied stuff:

IO: Input and output pins · wired local IO · wired local-ish IO · ·  Various wireless · 802.11 (WiFi) · cell phone

Sensors: General sensor notes, voltage and current sensing · Knobs and dials · Pressure sensing · Temperature sensing · humidity sensing · Light sensing · Movement sensing · Capacitive sensing · Touch screen notes

Actuators: General actuator notes, circuit protection · Motors and servos · Solenoids

Noise stuff: Stray signals and noise · sound-related noise names · electronic non-coupled noise names · electronic coupled noise · ground loop · strategies to avoid coupled noise · Sampling, reproduction, and transmission distortions

Audio notes: See avnotes


Platform specific

Arduino and AVR notes · (Ethernet)
Microcontroller and computer platforms ··· ESP series notes · STM32 series notes


Less sorted: Ground · device voltage and impedance (+ audio-specific) · electricity and humans · power supply considerations · Common terms, useful basics, soldering · landline phones · pulse modulation · signal reflection · Project boxes · resource metering · SDR · PLL · vacuum tubes · Multimeter notes Unsorted stuff

Some stuff I've messed with: Avrusb500v2 · GPS · Hilo GPRS · JY-MCU · DMX · Thermal printer ·

See also Category:Electronics.

These are primarily notes
It won't be complete in any sense.
It exists to contain fragments of useful information.


Shield / module intro

The module itself doesn't seem to have a very formal name, but is mostly known as the Sagem HiLo module ('M2M' also pops up regularly). See e.g. [1]


The module is basically the core of a quad band, GPRS-enabled phone, so it does SMS, IP (TCP sockets and some higher-level helpers, e.g. for FTP).

The shield that the module is on is fairly simple, mostly just regulating 3.3V, wiring in the SIM socket, serial communication, and some further connections to the HiLo module.


Note that the shield only uses five of the twenty pins: ground and power (from the ICSP header), power-on trigger (shield pin 2), and two for serial communication (shield pins 0+1).

...so if the shield physically blocks too many of the Arduino pins for your project, you can just put it alongside with five wires going to it - though this may make delivering power a little more complex, so you may prefer to stack a project shield under this GPRS shield.

On the hardware side

On power draw and supply

The documentation mentions the use as:

  • draws ~56µA (negligible) while off
    • the module's own time alarms can be used to power on from this state?(verify)
  • draws ~2 mA in standby (can receive calls) (verify)
    • ...except I can't find any mention how to do this (CFUN doesn't seem to be it)
The wiring around the power options
(not mentioning ground. It's on the other pin of Vin, and the bottom right pin of ICSP)
  • average draw in calls (and data exchange?) is 220mA for 850/900MHz, 160mA for 1800/1900MHz? (verify)
  • ...in short transmission bursts of 2200mA (half a millisecond at worst, with what interval?) (all networks?(verify))


Power on the shield:

The '5V' / 'Ex' jumper chooses the source of current (to feed into the 3.3V ([http://www.google.com/search?q=LD1085 LD1085 series 3A LDO regulator]) that powers the module:

  • 5V means "shield takes power from the ICSP header" (which is 5V)
  • Ex means "have to plug something into the shield's Vin", and is useful to avoid depending on the Arduino's regulator.

The three rows of holes on the shield have no direct function, but seem to be useful to fiddle with power supply, for example for the capacitor mentioned later (if you power via ICSP/5V), or even to bypass the shield's regulator.


Because of the large power draw, it takes some consideration to get the module working stably.

In may be simplest (but not so power-efficient) to use a 7.5V adapter that is wired to both the Arduino's Vin and the GPRS shield's Vin. And you'll still need a bypass capacitor unless the adapter is ~3A. See below.



On the 2.2A, input voltage, and (not) powering through the Arduino

On the 2.2A

The module needs 2.2A when actively communicating, which worst case is for 1.15ms, (two adjacent 0.577ms GSM timeslots)(verify) and with a ~20%(verify) duty cycle.


When there's not enough current, communication breaks down and the module is likely unhappy.

So if your module reacts to AT commands after powerup but becomes unresponsive perhaps 10-20 seconds later, then it probably browned out when it tried to do basic registration on the GSM network.

If it lives that long but consistently stops responding around a CGATT=1, or a little later when it sends a bunch of data, it's probably also a power problem (you're doing better in terms of current supply, but not good enough - data transfers are more work in quick succession).


If you have trouble you suspect is related to an underpowered supply (e.g. a battery with high internal resistance), you can try to add a largeish bypass capacitor. (If you use the 5V line then putting it on the shield (5V and Gnd rows) is easy. If you use Ex/Vin you may want to put it on your supply wire. See the diagram above)

Libelium suggest 220µF [2]. More could be useful, but it really makes more sense to first look at the supply all the way from source to shield, before trying to hack around the issue with a larger capacitor. Not because you couldn't fix it with a capacitor, but because the other fix is usually more robust and possibly more efficient.

If you can supply a sturdy 2.2A directly to the shield's regulator (typically via Vin/Ex) then you don't need the capacitor, but 2.2A is not necessarily an easy guarantee when you have to consider all of the current path.


On input voltage

The Hilo module itself seems designed to draw directly from a LiIon battery - it's comfortable with 3.2V..4.5V.

The shield has a 3.3V regulator to give something in that range, and that regulator itself seems most comfortable with input in the 6V..9V range. It'll work with a little less than that - specs suggest that 2A could work with ~4.9v input, so using a 5V line could work in theory, but in practice is way too close for robustness.I've had it work as a simple shield powered via the Arduino's 5V and using the capacitor. I've had that same setup fail after adding a hundred mA draw nearby. Also, basic 5V regulators are 1A so are borderline at best.

I now have it working robustly with a 6.5V switching regulator. Plus capacitor, because it too is a 1A regulator. (The switching regulator is used in this specific case to lessen power loss regulating down from a 12V battery)


The shield's regulator could be powered from 12V - it's within specs, but quite near the maximum, and the datasheet mentions it will start to throttle its output if the dropped voltage becomes large, so may not deliver all of the current it could with a 12V input. It may be fine to deliver 2.2A. I don't read data sheets well enough to be sure.

Powering on and off, power management

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)
Powering on

Done by holding POK_IN high for "at least 629ms at 25°C". Various demo code holds it high for two seconds.

POK_IN is connected to the physical button on the shield (handy for debugging in gateway mode), and the Arduino's digital pin 2 (handy for autonomy, though not ideal if you use the shield as a shield and wanted to use the AVR external interrupt on Arduino pin 2).


From experience, it seems that:

  • it may take about 6 seconds from powerup to be fairly sure that the SIM stuff is initialized (before this, SIM-related commands may give errors)
  • it may take about 17 seconds from powerup for the module to have registered itself on the GSM network. You can check that (and probably should) using AT+CREG?.

Some simple examples just wait a while after power-on to avoid the bother of doing checks.



Powering off

Can be done with AT*PSCPOF command - it will return OK and power the module off.



Sleep

Current summary: I don't know

Controlling sleep: (verify)

  • AT+KSLEEP=1 hands sleep timing to the module (meaning?(verify))
  • AT+KSLEEP=0 won't sleep while DTR is high(verify) -- but the shield does not expose DTR.

The docs suggest that you can set the thing to receive only, but also that AT+CFUN can't do that:

0: minimum functionality(not support);
1: full functionality;
2: disable phone transmit RF circuits only (not support);
3: disable phone receive RF circuits only (not support);
4: disable phone both transmit and receive RF circuits;

Communication

The shield wires the Hilo module's serial port to Arduino digital pins 0 and 1, so that it can talk to the AVR's UART and to the USB-to-serial's UART.

Whether the Hilo talks to the or the AVR depends on the way the shield is wired (switching the transmit and receive lines) - which why there are jumpers for this on the shield.


You want only two of the three possible UARTs to talk at a time, so:

  • When programming the AVR, consider:
    • If the shield is jumpered to USB gateway, you can get away with programming the AVR with the module powered off (verify) (you'll get an stk500 error if you don't)
    • If the shield is jumpered to Arduino, you'll have to take off the shield to program the AVR (or take off the jumpers if that's easier)
  • If you want to control the Hilo from the PC via the FTDI/8U2 (while it's being powered by an Arduino it's plugged into) you could use an Arduino board with the AVR pulled out (You could leave it in, but then you have to make sure the AVR doesn't use its serial port and you know about the ways in which the bootloader can mess things up)


Debugging

Debugging Arduino code is a little bothersome in general, and since the serial port is in use it's harder to watch things happen.

Options include:

  • snooping off the serial line with a wire from another serial port (e.g. another arduino's FTDI or another sort of USB-serial device). You can do this with two wires: one to share ground and one wedged in the shield's USB/gateway jumper. Libelium do this in some of their examples. You only get to snoop on one direction, though.
  • using SoftwareSerial output on different pins (2400 baud seems to work best) to output your own debug information. This way you can program the sketch to write whatever readable debug you want (e.g. human-formatted notes, all communication to and from the module, timing information, and whatever else). (Note that bit-banged serial is relatively fragile when you use interrupts)
  • Using an Arduino Mega, as it has more than one UART

Communication - commands & behaviour stuff

Fairly standard modemy stuff

You can disable character echo with ATE0.

When typing commands using your PC it may be handy to get character echo, but from the Arduino (or testing scripts from your PC) it's mostly just extra data you have to read and throw away.


It seems that

  • all responses are terminated with CRLF (0x0D 0x0A in hex )
  • you can send AT commands with CR (and CRLF?(verify)) (though remember that HTTP calls specifically for use of CRLF)


On flow control

Libelium mention using AT&K3 (seems to be a copy-paste from the Sagem docs) for hardware flow control when exchanging data - but this seems strange, as the shield doesn't pass through any serial lines other than RX and TX.

You could use software flow control (XOFF/XON) by using AT&K4, but you'ld still have to implement it in your arduino sketch.

You can try to get away with not using flow control - I'm guessing most or all responses are shorter than the hardware serial's receive buffer, and that you can tell most data-mode receives to send a specified number of bytes, so that that won't fill that buffer either. I have such an implementation and it seems to work fine.



Some tutorials mention that PSSTK happens after startup (this is the SIM menu, and one of the unsollicited notifications). Whether this happens seems to vary with firmware(verify), so don't use it as a check whether the module is working. A simple AT (and check for an OK response) will work fine for that.


On robust control

To avoid your Arduino code hanging when the module becomes non-responsive, or doesn't do exactly what you were thinking, you'll want a read function with a timeout.


As for the model/structure of the code that interacts with the module, it depends. Largely on what you want to do, and how. A simple SMS alerter, receiver and/or sender can be very simple code-wise (because most of the details are handled for you), whereas TCP/IP exchange is rather more work.


A complete implementation (like an actual phone would have) will require you to keep state about the module, and react in a state machine sort of way, which is non-trivial to write. (TCP transfers are among the most complex cases as there are perhaps a dozen different paths of responses. A description of all the possible responses, their timing, or sometimes even order is not simple. (And, in fact, some error paths blur the line between intermediate and final responses))

When writing code for a microcontroller to interact with the module, you can get away with doing one thing at a time, and you may want to, because it simplifies your code. Don't bank on this approach when doing anything fancy, but if you are using the thing for the occasional data exchange, it may be what you'll end up doing.


Responses can be categorized into: (these terms are the documentation's)

  • Final responses are those that are the last in a series, often implying "your turn now". Most final responses are also fairly immediately answered to a command, but not all (Consider e.g. NO ANSWER). (Note that in some cases (e.g. error paths) you might get a a few lines, including final responses and unsolicited messages, that you could consider part the same piece of feedback)
  • intermediate responses (though the doc doesn't seem to explain the term) are (usually fairly immediate) responses that are not final in a series. Usually means a final response can be expected soon after it, but it may not happen until a little later. For example, CONNECT in a TCP request can signal a longish pause before transfer success or failure.
  • Unsolicited responses are those that may come at (almost) any time, because they are not sent in direct response to the most recent command (but rather some event, like completion of a longer-term request, things like incoming calls/messages/data, some errors, etc.).
    • Examples include cell tower changes (off by default), incoming SMS messages (off by default, and you can poll instead), and things like battery charge changes (off by default).
    • Note that most have to be enabled before they're sent at all (so if you can do without them, code can be simpler)
    • you have no control over where in a sequence of responses these come, nor do you have a means of anticipating them
    • ...so you basically have to deal with them as special cases (at the very least ignoring them while knowing they are not part of a sequence you're handling - probably something along the lines of "was the just-read line a special case? Then handle it and do another read" meaning it'll be neutral in terms of response consumption).


Also interesting to note:

  • For many commands, the response will be a fairly immediate definitive response: OK, meaning 'command accepted'.
    • Examples: AT, ATE
  • Some will immediately return some data, then an OK
    • Examples: AT+CGSN (to get the IMEI), ATI variants
  • Some will answer OK as a "request received", then do some work and later send the information -- usually in a notification, and probably simply to avoid a holdup.
    • Examples: AT+CSQ, AT+CREG?
    • In a number of cases the informative notification comes fast enough (perhaps few dozen ms later) that your code could choose to wait for that notification



On resetting the module

There is the potential problem of the module hanging on some state you didn't considered getting out of.

The Arduino will happily go on, but you won't necessarily be able to use the GPRS module again until it is reset (I certainly managed that). While it's probably possible to polish any given code to avoid non-responsive states, the possibility looms at every code change.

Often you can drop the module to command mode (+++) and power it off via the appropriate command, which will give you a clean enough slate. But it seems possible to fudge it so that there's no command mode. And there seems to be no software way of doing a reset.


If you use the module in a leave-behind sort of situation, you can consider putting the power supply to the module under control of the Arduino (e.g. via a darlington) so that if all else fails, the Arduino can cut power to restart it.

Hardware/SIM identification (IMEI, IMSI)

In IoT-like applications it can be very useful to have both modules identify themselves (To match sent-in data to the station it came from, to know which SIM you placed in what station, and, perhaps to figure out managing SIMs), and to keep track of SIM cards, so the following may be useful:

  • AT+CGSN (or AT+KGSN, or AT+GSN) to get the phone identity (IMEI [3])
  • AT+CIMI to get the SIM identity (IMSI [4])
    • If you do this before the SIM is ready you'll get some CME/CMS errors instead. Wait for ~6 seconds(verify) after startup.


Your phone number is not defined by the SIM. Phone numbers are assigned by the network, based on IMSI(verify), which is why you can keep your number when you get different deals. However, the number is occasionally stored on the SIM, which is not guaranteed to be up to date, but often good enough.

If you want to keep track of an IMEI-to-phone number list, you could e.g. SMS the IMEI to some management phone number.


For more on IMEI and IMSI, see the #Glossary


Basic useful state: SIM stuff and security

If the SIM is protected by a PIN, you'll get +CME ERROR: 11 responses to most commands.

So you have two basic options: remove PINs from SIMs before using them in the module, or make the SIM-check-and-input part of your initialisation.

The first is a bad idea against more brute force thieves who want a useful SIM, the second is still a bad idea against hackers because you can dump an AVR's code


AT+KSREP

State related to SIM and its access:

AT+KSREP?

Responds with something like:

+KSREP: 0,0

The second of these two resulting numbers means:

  • 0: Ready (no access code is required)
  • 1: Waiting for some access code (Use AT+CPIN? to determine which).
  • 2: No SIM card
  • 3: SIM locked
  • 4: unrecoverable (...error?)
  • 5: unknown


AT+CPIN

To check whether you need to unlock something, you're probably looking for AT_CPIN (and possibly for AT+CLCK):

AT+CPIN

If AT+CPIN? reports +CPIN: READY, you do not need to enter a PIN. If it's something else, it will be asking for something like SIM PIN, SIM PUK, SIM PIN2, SIM PUK2, other.


Entering a SIM:

AT+CPIN="1234"

Note that if you store a SIM to send to the module, you would have to store it in the Arduino, which is not horribly secure, but better against non-technical thieves than a PIN-less SIM.

On registration and attachment, and checking various things related to connectivity

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

GSM network registration refers to the basic service that lets you call and exchange SMSes. Generally happens automatically, 10-20 seconds after module startup, assuming you have a (valid) SIM card, a good signal, and a cellular subscription that is served by a tower/network near you.

GPRS is an additional and optional service, and so is distinct from basic GSM service. Even if you have a working SIM and are registered on the cellcular network, use of GPRS may fail when you don't have a valid GPRS subscription(verify).

There is also a difference between GPRS registration and GPRS attachment.


AT+CREG

To check the state of GSM network registration:

AT+CREG?

Which will give you a response like:

+CREG: 0,2

The second number reflects registration state. Values include:

  • 0: not registered, not searching for a new operator to register to
  • 1: registered, home network
  • 2: still searching for an operator to register to
  • 3: registration denied
  • 4: unknown (apparently 'still working on module startup', directly after power-on, though I've also seen it stick around)
  • 5: registered, roaming

Chances that if you see +CREG: 0,2 (or 0,4), you just want to wait a little longer, until it responds with 0,1 (or a more definitive no, like 0,0 or 0,3).

Registration may take 5-20sec after module startup.


The first number reflects previously set configuration about unsollicited messages (specifically only those in response to registration/cell changes). The write command, AT+CREG=num, sets this.

  • 0: no unsollicited messages (default)
  • 1: unsolicited messages from registration changes
  • 2: unsolicited messages from registration changes and cell area changes

AT+CGATT

Attempt to attach (will return OK (or possibly ERROR (verify)))

AT+CGATT=1

Detach (will return OK (only?(verify)))

AT+CGATT=0


Check whether you are attached:

AT+CGATT?

This will almost immediately respond with one of:

+CGATT: 1
+CGATT: 0


Notes:

  • Data commands that need GPRS will implicitly attach if you were not attached (and take extra time because of it)
  • Before an explicit or implicit GPRS attachment you will want to set your GPRS network settings - see AT+KCNXCFG
  • I have seen AT+CGATT=1 react with ERROR, when the module wasn't even on the basic GSM network yet.
  • I have seen AT+CGATT=1 take ~130ms, take ~600ms, and I've seen it time out after ~900ms (possibly for other reasons), and cases where I'm not sure why it doesn't respond (which might mean it takes even longer)


  • AT+CGATT=1 doesn't block until it's done(verify). You probably want to poll for a little while to see whether it gets on there, and give up if it doesn't do so within reasonable time.
  • Attachment can take longer to fail (timeout after perhaps 6 sec?(verify)) than to succeed

AT+CSQ

You can check the signal strength with:

AT+CSQ

Which responds with something like:

+CSQ: 15,99


The first value is an integer indicating the RSSI, received signal strength

  • 0..31 is -113dBm..-51dBm in steps of 2dBm
where 31 actually means -51dBm or better
For example, the 15 above means -83dBm
  • 99 means unknown (and probably that it's not on the network yet(verify))


From some very preliminary experiments, it seems like GPRS

  • isn't too reliable under -91dBm (~11)
  • starts being useful around -85dBm (14)
  • while -81dBm to -73dBm (16-20) seem decent.

But don't trust those figures, do your own tests.


(The second value is the BER (Bit Error Rate), which would be a better indication of signal/connection quality than RSSI is, except it doesn't seem to be used by the Hilo.)



SMS

On text mode versus PDU mode

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

AT+CMGF=1 to select text mode, AT+CGMF=0 to select PDU mode.

PDU mode refers to structured packets, more or less bytestrings, that go in and out via their hexadecimal-as-ascii representation.

Most devices support PDU mode, fewer also support a text-based mode for data exchange. PDU mode is better standardized so more portable between devices than text mode is. PDU mode may also expose more features than text mode does.

The Hilo module supports both.

If you're mostly dealing with the occasional bit of GPRS data exchange, this isn't a very important detail, and leaving it in text mode can potentially be a little simpler.


The mode a device is in affects many of the SMS functions (listing, reading, writing) in that arguments are different (giving commands for the other mode yields errors), as is the format of various responses.


See also:

  • GSM 03.40

On message stores

Message stores you can specify: (yeah, I'm not sure what half of these actually mean either)

  • SM: SIM/USIM message storage
  • ME: message storage
  • MT: any store associated with ME
  • TA: TA message storage
  • BM: broadcast message storage
  • SR: status report storage



There are three different settings that different commands use when they want to use storage:

  • mem1
    • default: ME
    • relevant commands: CMGL,CMGR, CMGD
  • mem2
    • relevant commands: CMSS, CMGW (prepared messages)
    • default: ME
  • mem3
    • default: MT

You often want to make sure you're doing everything from the same store.

SMS messages can be stored in various distinct places.



Using AT+CPMS you can:

  • change the store used for each
  • get a report on the amount of messages stored in each, and the space left.

Simple SMS send

Example session of sending a simple SMS, in text mode and using ASCII (the default character set):

  • Written: AT+CMGF=1\r (set text mode)
  • Read: OK\r\n
  • Written: AT+CMGS="0612345678"\r (phone number to send to)
  • Written: Hi there
  • Written: \x1a (one character, 0x1A hex)
  • Read: > (I'm not sure what that is)
  • Read: +CMGS: 12\r\n (a message reference number, don't know what it's good for)
  • Read: OK\r\n

Reading, writing, deleting, listing

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

(Remember message store details. Most work in mem1, CMSS and CMGW in mem2)

  • AT+CMGS - Send SMS message
  • +CMGL - List SMS messages (filtered by status)
    • (sort of a read-all - if you just want to know the amount, look at CPMS)
    • in text mode, this is two lines for each message: its metadata and its contents.
  • +CMGR - Read SMS message (by index)
  • +CMSS - Send SMS message from storage (prepared messages)
  • +CMGW - Write SMS message to memory (prepared messages)
  • +CMGD - Delete SMS



Semi-sorted SMS stuff

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)
  • +CNMI - New SMS message indication


  • +CSDH - Show text mode parameters
  • +CSMP - Set SMS text mode parameters
  • +CSAS - Save settings (mostly CSMP parameters?)
  • +CRES - Restore settings (mostly CSMP parameters?)
  • +CSCA - SMS service center address
  • +CSCB - cell broadcast message
  • +CSMS - Select Message service


Also, there is AT+CSCS for character set used, which apparently applies to more functionality than just SMS(verify).


See also:

GPRS & data communication

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

For GPRS attachment, see AT+CGATT

Data transfers

AT+KCNXCFG

Necessary

Configuration for GRPS - the simplest form is:

AT+KCNXCFG=0,"GPRS","apn","username","password"
  • connection ID, chosen by you - just be consistent, and keep track whenever you use more than one connection (examples tend to use 0 or 1)
  • apn: Access Point Name[5]
  • username, password - standard, public things listed by your provider
  • Can be followed by three IPs: our own address, and two DNS server IPs. These are all 0.0.0.0 by default, meaning 'to be dynamically assigned', which is often what you want

The website of your mobile provider will often provide these details somewhere, though you may have to dig around a little. Historically phones will have it configured automatically via an SMS(verify).

Note that one provider may have two or three APNs for different types of clients.


(AT+KCNXCFG also allows you to use WAP-style communication instead of GPRS communication, in which case the command is a little different)

AT+KCNXTIMER

Optional

You'll probably only care about these if you want to lower them to try to save a little power.

It can be important to your logic to set your code's timeouts to something slightly higher than the configured timers (see mention of defaults below).

You can set attempt/timeout configuration, for example:

AT+KCNXTIMER=0,60,2,70
  • 0 (verify)
  • connection timeout in seconds (default:30)
  • network connection attempts (default: 2)
  • linger time in seconds (default: 60) Note that GPRS communication will stay active for this long unless you do something about that (verify), so you may wish to

TCP communication

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)


Note that the first argument on many data commands is the session number.

The module can deal with more than one connection at a time, and the session number is the way to have notifications and commands be specific to connections.

If you have the documentation that doesn't mention these, that's old documentation. Get the newer one, it'll be a lot less confusing.

The examples on this page tend to have 1 in them - mostly because the code this is copied from is hardcoded that way (uses just one connection).


AT+KTCPCFG

Necessary

Specify host (name or IP address) and TCP port to connect to:

AT+KTCPCFG=0,0,"www.google.com",80
  • GPRS-connection-id (as configured in AT+KCNXCFG)
  • mode: 0 for client, 1 for server, 2 for child (generated by server sockets; not yet sure what exactly that means(verify))
  • hostname
  • port

You can save a DNS lookup (and that possible point of failure) by using the IP directly, bit it frankly makes little difference, and using names is that much more flexible (it's probably easier to change DNS details than reprogram the AVR)

AT+KTCPCNX

AT+KTCPCNX=n means "try to make the configured connection for session n". A necessary step in doing a TCP data exchange.

AT+KTCPCNX=1

May easily take a few seconds on success, longer on failure (I'm guessing the 30 seconds in AT+KCNXTIMER's default applies to this). You probably want a slightly longer timeout waiting for this to respond.


Possible responses: (verify)

  • OK means the connection was established. And that you have the responsibility to close the connection for the session. That is, you should eventually AT+KTCPCLOSE this before a new AT+KTCPCNX.
  • If the GPRS config is bad (or you have no GPRS service for your SIM?)
    • you get NO CARRIER (only)
  • If underlying network communication fails...
    • you likely get NO CARRIER followed by +KTCP_NOTIF of 0. This can happen at the first data communication attempt as well as later.
    • I'm not sure yet what specific causes are behind this, but bad APN settings seems to be one of them (and one of the most relevant in practice).
  • If the network connection is okay but one of the steps to building a TCP connection fails
    • You'll get +KTCP_NOTIF (with codes in 1-6 range. 7 is for servers, 8 is a warning, 9 and 10 would have come earlier(verify))
AT+KTCPSND

Sending data on an open connection:

AT+KTCPSND=1,16

That second number is the amount of data we want to send, excluding the pattern to terminate that data (which by default is --EOF--Pattern--. For example, the length for GET / HTTP/1.0\r\n would be 16.

After you give this command, check that it responds with CONNECT before sending the your data.

If an error occurs instead, you'll get a +CME ERROR, followed by a KTCP_NOTIF(verify).


After you've sent enough bytes and/or sent the terminator pattern, the module will drop back into command mode and respond with OK. (If you didn't send the exact amount of data you said, the module will also send a notification like: +KTCP_NOTIF: 1,8. It'll work with what you gave it, but it just wants to say your code is probably counting wrong)


If nothing fails on the way then data exchange will happen and you'll see a notification of the first return packet before long (probably +KTCP_DATA: 1,1360)


Receiving data
This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

A notification like +KTCP_DATA: 1,1360 signals that (in this case) there are 1360 bytes of data in the Hilo's TCP receive buffer for session 1.


To read that data, you would say something like:

AT+KTCPRCV=1,1360

...which states that you're about to receive min(available_data, that_size) from the buffer associated with the socket. When you give this command:

  • the module sends the response line CONNECT\r\n, and go to data mode
  • the module sends the amount of bytes of packet data that you asked for
  • the module sends the termination pattern
  • the module returns to command state and sends the response line OK\r\n.


Stopping after reading too few bytes (e.g. forgetting the pattern) is a recipe for trouble, because you'll probably leave some bytes in your serial buffer when you read in the next command response or the next AT+KTCPRCV.

When controlling from an Arduino, you probably want to read out chunks smaller than Serial's 128-byte receive buffer, to avoid flow control (I'm not sure you can even get flow control without a lot of added bother). You'ld keep track of how many of the bytes are still unread and always say you're e.g. reading min(110,bytes_left):

AT+KTCPRCV=1,110

Note: When you do this, the module seems to sometimes give +KTCP_DATA notifications when we've read only part of the packet, to tell us how much data is left. Documented or not, you probably want to be robust to the possibility. If your code is counting down itself anyway, you could just ingore it.


Once you have read a whole packet (as many real-packet-data bytes as +KTCP_DATA mentioned):

  • ...if there was more data to receive, you'll get another +KTCP_DATA: notification.
  • if that was all
    • and the remote end closed the connection (which e.g. a HTTP connection will do unless you ask it not to), you'll get a +KTCP_NOTIF: 1,4, meaning remote disconnection.
    • and the connection is not closed remotely - ?(verify)


Note that you can close the connection yourself before you've read all data.(verify)


I've had a lot of trouble getting receiving code right, so expect it to be fragile and don't try to cheat-n-hack. Do it to the specs, consider error paths, and do a bunch of testing.


There are a bunch of different possible error paths in the above.

AT+KTCPCLOSE
AT+KTCPCLOSE=1,1

The second value is the type of close; 1 is a proper acknowledged close, 0 is mentioned as 'abort', which seems to mean the module just forgets the socket (and the other end's network stack will figure it out some time later).

If there is no connection to close, it will return an error.

Reponses:

  • NO CARRIER (in this case meaning success)
  • +CME ERROR (probably 3) when there is no connection to close
  • OK


Notifications:

  • +KTCP_NOTIF (in reaction to KTCPCNX, KTCPRCV, KTCPSND)
  • +KTCP_DATA
  • +KTCP_DOWN
  • NO CARRIER (in reaction to KTCPSND)
  • <ETX> character (0x03)

ETX signals early abort (us-to-module or module-to-us). ETX in the data is escaped as DTE ETC, DLE itself as DLE DLE

+KTCP_NOTIF responses (and other errors)

KTCP_NOTIF messages look like:

+KTCP_NOTIF: 1,0

The second value is the error code, and can be one of the following:

  • 0 - network error (...including bad APN settings, and probably a dozen other causes)
  • 1 - no more socket available (probably means you forgot to AT+KTCPCLOSE earlier connections)
  • 2 - Memory problem
  • 3 - DNS error
  • 4 - TCP disconnection by the server (or remote client) - which can effectively be a message rather than error
  • 5 - TCP connection error
  • 6 - generic error
  • 7 - failed to accept remote client request's
  • 8 - "okay, I sent that data, but that wasn't the amount of bytes I was expecting"
  • 9 - Bad connection/session number (verify)
  • 10 - That connection/session was already running (probably previously left in a bad state)
  • others (in updated firmware)


These are mostly returned in response to AT+KTCPCNX and AT+KTCPSND.

On errors

CME errors mostly relate to the device itself, and cases where the device knows it won't work.

CMS errors relate more to the network


However, the sets have some overlap in purpose -- and even in errors with more or less identical function. As an example, there are SIM-related errors in both CME and CMS sets.

Some errors I've caused, approximately in the order I've caused them in:

  • +CME ERROR: 3 - Operation not allowed (which can be a fairly generic error)
    • (in my case meant that I was giving a non-existing command variant - AT+CREG instead of AT+CREG?)
  • +CME ERROR: 4 - Operation not supported
  • +CME ERROR: 11 - you probably assumed there was no PIN where there was
  • +CME ERROR: 10 - 'SIM not inserted' (may be iffy contacts, or maybe trying to access too quickly after startup?(verify))
  • +CMS ERROR: 310 - SIM not inserted
  • +CMS ERROR: 500 - unknown error
  • +CMS ERROR: 47 - Recources unavailable (whatever that means)
  • +CME ERROR: 14 - SIM busy (may mean you try to access the module too quickly after startup?)


See also:

See also


Glossary

  • IMEI (International Mobile Equipment Identity) - basically the phone hardware's identity [6]
    • A 15-digit IMEI consists of:
      • TAC (Type Approval Code) - 6 digits
      • FAC (Final Assembly Code) - 2 digtis
      • SNR (Serial Number) - 6 digits
      • CD (Check digit) - 1 digit
    • A 16-digit IMEI is an IMEI-SV, which omits the check digit and adds two digits indicating the firmware being run:
      • TAC (Type Approval Code) - 6 digits
      • FAC (Final Assembly Code) - 2 digtis
      • SNR (Serial Number) - 6 digits
      • software version - 2 digits


  • IMSI (International Mobile Subscriber Identity) - basically the SIM's identity. Phone number is assigned by the network based on this(verify) [7]. Consists of:
    • country code (MCC)
    • network code (MNC)
    • subscriber identity (MSIN)


  • IRA - International Reference Alphabet, equivalent to ASCII
  • ME, Mobile Equipment - approximately 'the thing that is talking wirelessly'.
    • Example use: "ME is currently searching for an operator"
  • MS,
  • TA, Terminal Adapter - approximately 'the things that can do the dialing'; what you may think of as a modem, but a more generalized term than that. [8]
    • Example use: "The TA always responds to an AT command, except in the special case of ATQ"
  • TE, Terminal Equipment - approximately 'something with some communication intent'. In this context, it mostly means 'the thing that uses the TA, gives it commands', and for the shield it means the Arduino, or your PC via the arduino's FTDI [9]
    • Example use: "ATE controls whether the TA echoes back characters received from the TE"
  • DTE, Data Terminal Equipment - in this case, read TA (verify)
  • DCE, Data circuit-terminating equipment - in this case, read TE (verify)

Some code

I have working code that does HTTP requests, response parsing, and such, but am not posting the whole yet, because I can't explain why some things work (or break where I think they shouldn't).


Notes:

  • Much of this code counts on you having used ATE0 first
  • This code is not strictly robust to unsolicited notifications (though could be if you handled the truly unsolicited in read_line)
  • a number of things use read_line() and startswith() as mentioned
  • the STR_something things are some globally defined strings, there because some are reusable.
  • module_write is a proxy function that probably just does a Serial.write(). I also use it to have it write a copy to SoftwareSerial for debug reasons.


Some helpers

Read-with-timeout function

#define LINEBUF_SIZE 129
char databuffer[LINEBUF_SIZE];
byte last_read_timed_out=0;

unsigned int read_line(unsigned long timeout_ms=2000) {
    //Reads a line of text from the 'duino Serial library's buffer (128 bytes unless changed). 
    //Reads until 
    // - a newline   (which then makes it to the buffer - can make some string tests easier)
    // - the buffer is full (hardcoded length)
    // - reading a next character times out
    // ...whichever comes first.
    // 
    // Writes the a null-terminated string into databuffer, 
    //   ...and will read at most (LINEBUF_SIZE-1) useful characters to stay safe.
    // Returns the amount of bytes we read (==strlen(databuffer))
    //
    // On timeout:
    // - Waits no longer than 'timeout_ms' milliseconds for each next character.
    // - On timeout, sets last_read_timed_out=1
    // - Note that currently function only returns 0 when it timed out reading 
    //   anything at all, so that can also be a simple test for timeout
    // - Timeout values of <30ms can be too short
    // 
    // On newlines:
    // - was written with the idea that you may get only CR or LF
    //    (not sure if that's ever true) 
    // - will not emit empty lines
    // - which combines to mean that it will leave \n in the Serial buffer
    //   and only ignores it on the next call of read_line(). This is significant
    //   when you're doing data exchange, because that byte will mess things up.
    char *tptr=databuffer;
    unsigned long last_byte_time;

    last_read_timed_out=0;   
    databuffer[0]=0x00;      // (make sure empty string on timeout is null terminated)
    last_byte_time=millis(); // used in a 'last byte was read this long ago' test
    
    while (true) { //(not and endless loop, broken off by timeout if not by end-of-line)
      //If the buffer is full, break off.
      if ((tptr-databuffer) > (LINEBUF_SIZE-2)) { //VERIFY: for off-by-one error. 
        // -2 so that the last position can safely be set to the null terminator.
        tptr[1]=0x00;
        break;
      }
      
      // Read a byte, if it's there
      if (Serial.available()>0) { // If there's a byte to be read,
        tptr[0]=Serial.read();    // read it in
        tptr[1]=0x00; //(could be postponed until we know we're done, actually)
        tptr++;
        last_byte_time=millis();  // update 'last byte read came at' time
        
        // Terminate reading on a newline, deal with empty lines by ignoring them
        if (   ((tptr[-1]=='\n') || (tptr[-1]=='\r'))  &&  ( ((int)(tptr-databuffer)) == 1 )   )
        { //empty line or crlf? Pretend this line didn't exist, start over with next character.
          tptr=databuffer;
          databuffer[0]=0x00;
          continue;
        }
        if (tptr[-1]=='\r') { // CR, \r, but not at start of line - assume this is a complete response, return it
          break;
        } //note that this leaves the \n from a CRLF in the Serial buffer. This function deals with it, your manual Serial code may not.

        continue; //we read a byte, skip timeout check this round
      } else { //no byte available yet
         //delay(1); //probably unnecessary. 
      }

      // Time out? 
      if (millis()-last_byte_time > timeout_ms) { //TODO: check behaviour at millis overflow
        tptr[1]='\0';
        last_read_timed_out=1; //signal that we timed out
        break;
      }
    }
    
    // I use software-serial for debug to print out the time of receive,
    // the just-read line's length, string representation, and hex representation
    // (omitted here for relative brevity)

    return (int)(tptr-databuffer); // amount of bytes we read into databuffer (TODO: check)
}

Example use:

void disable_echo() {
    //Note: if this is the first read we do, 
    //  this should also swallow the bit of nonsense data that likely appears (autobauding?)
    module_write("ATE0\r"); // disable echoback of commands
    read_line(200); //If it was still enabled, the responses will be "ATE0" and "OK",
    read_line(80);  //if it was already disabled, the first will be OK and the second will time out (so do that quickly).    
}

Some fetchers

(No, I didn't write proper state machine type control, I kept things serial for simplicity)

inline byte startswith(char *str, char *start, int n=-1) {
    // Helper function.
    //Returns whether str stats with start
    //Example use:    if (startswith(read_buffer, "+CSQ"))
    //The n parameter lets you check only the first n characters 
    //  instead of all of the length of start 
    //(and, in theory, lets you hardcode to avoid a strlen 
    //  I haven't used thatbut I figured it could be useful) 
    if (n==-1)
      n=strlen(start);
    return ((strncmp(str,start,n))==0);
}


int atok_test() {
    // writes AT, waits for OK. Returns 1 if it got OK, -1 if it timed out.
    // Usable as a simple 'is the module alive' test:
    int ret=-1;
    module_write("AT\r");
    if (read_line(300)>0) { //in practice probably takes ~100ms at worst, typically less
      if (startswith(databuffer,STR_OK)) {
        ret=1;
      }
    }
    return ret;
}

void get_imei(char *target) { // (device identity)
    // Queries the IMEI, writes it to a given string via a pointer. 
    // The buffer you write to *must* have space for 17 characters (counting the null terminator)
    Serial.print("AT+CGSN\r");
    read_line(200); //this value is alone on its line
    int len = min(16, strcspn(databuffer,"\r\n"));
    memcpy(target,databuffer,len);
    target[len]='\0';
    read_line(200); //OK
}

int get_signal_strength() {  
    //returns current signal strength, in dBm 
    // Returns a signed int, valid range is -113..-51.  Returns 0 on error/unknown.
    module_write("AT+CSQ\r");
    char ret=0;
    for (int i=0;i<2;i++) { //should be +CSQ and OK
      read_line(300); //Probably takes 200ms or less, but hey
      if (startswith(databuffer,"+CSQ")) {  //should look like "+CSQ: 15,99"
        ret = atol(&databuffer[5]);
        if (ret==99) 
          ret=0;
        else
          ret = -113+2*ret;
      }
    }
    return ret;
}
  

int get_registration_state() {  //Are we on the basic GSM network?
    int ret=-1;
    //returns the integer state from the +CREG response, an integer with one of six values:
    //   0 not on network, not looking
    //   1 on home network
    //   2 not on network, looking
    //   3 denied
    //   4 unknown (seems to happen at startup, before 2)
    //   5 on network, roaming
    //  -1 couldn't get state (module not responing?)
    module_write("AT+CREG?\r");
    for (int i=0;i<2;i++) { // should get two lines, the +CREG and and OK
      read_line(300); //Probably takes 150ms or less (if successful)
      if (startswith(databuffer,"+")) {  //should look like "+CREG: 0,2"
        char *ptr = strchr( databuffer, ','); // could be done via position assumptions, but this is almost as short.
        if (ptr!=NULL)
          ret = atol(&ptr[1]);
      }
    }
    return ret;
}

int get_gprs_attach() {
    // Get GPRS attachment state. 
    // (e.g. usable after a AT+CGATT=1 to wait until we're attached,
    //       to allow code to actually notice that we're not)
    // returns 0 or 1,  or -1 if we didn't get a response.
    int ret=-1;
    module_write("AT+CGATT?\r");
    for (int i=0;i<2;i++) { // should get two lines, +CGATT and OK
      read_line(300);
      if (startswith(databuffer,"+")) {
        char *ptr = strchr( databuffer, ':');
        if (ptr!=NULL) {
          ret=atol(&ptr[1]); //the -'0' trick would be smaller code, but probably more fragile
        }
      }
    }
    return ret;
}

Semi-sorted

Some responses

The following is a list of some common responses and common (primarily unsolicited) notifications, but not in any way a complete list. (it's a mix of the list in the appendix and of the things that looked relevant to my direct needs)


response/message type description, notes controllable?
OK final acknowledges command
ERROR final command rejected (probably typo)


CONNECT(...) intermediate used both in calls and in data transfers
+CREG unsolicited By default only sent (fairly immediately) in response to AT+CREG?, can also be set to be sent in response to changes changes (verify) ...using AT+CREG=


+CME ERROR: ... final
+CMS ERROR: ... final or unsollicited


*PSSTK:... unsolicited(verify) SIM menu. May show once(verify) not too long after powerup


+CMTI SMS
+CBM new SMS(-CB) indication? controlled with AT+CNMI (verify)
+CDS new SMS indication? controlled with AT+CNMI (verify)


+COLP: ... line identification
+CLIP: ...


+KTCP_DATA: ... Signals a packet of incoming TCP data


+CCCM: ... unsolicited update of battery charge state enable/disable via AT+CAOC
+CCWA: ... unsolicited something related to call waiting enable/disable via AT+CCWA


RING unsolicited signals incoming call
+CRING unsolicited signals incoming call enable/disable via AT+CRC


NO DIALTONE final no dialtone detected on line
BUSY final 'busy' tone detected on line (verify)
NO ANSWER final No answer (after timeout)
NO CARRIER final


+CSSI 'Sumplemental Service indication' Intermediate (verify) enable/disable via AT+CSSN
+CSSU Sumplemental Service indication, Unsolicited (verify) enable/disable via AT+CSSN
+CUSD unsolicited Unstructured Supplementary Service Data (verify)


+CR 'service reporting control' (meaning?(verify))



PDP profiles

Some related standards

Standards related to communication, roughly from serial/modem related to more mobile-device related.

  • V24-style serial interface
    • http://en.wikipedia.org/wiki/V.24
    • HiLo documentation mentions that the module should auto-baud up to 115200, but how well-behaved that is seems to vary per firmware. You may wish to test various baud rates to see what works well for you. Going down to 9600 seems to always work, and is fast enough for most simple uses anyway.

You can implement GSM 07.10 multiplexing -- whatever that means exactly.

See also:


Two-letter revice references in modem/mobile contexts

  • MS: Mobile Station
    • The device used by the user to access the network. As a whole, really.
    • Divided into SIM (subscriber identity) and the Mobile Equipment (ME)
  • ME: Mobile Equipment - basically the mobile device itself, without a given network identity
    • MT: Mobile Termination - basically the physical and logical part of the actual communication. Can be divided into:
      • NT: Network Termination: network protocols, call protocols, etc.
      • RT: Radio Terminaton: the physical level of local communication
    • TE: Terminal Equipment - basically the management and command parts (display, microphone, speaker)
  • TA: Terminal Adaptation (TA) - talks to the MT for things like call management



Unsorted