Key Ideas

• Reliable communication.
• Unicast communication
• Distributed time synchronization 
• Power management

Description

This app note collects data from remote motes. There is one base mote that does the collection and any number of sensing motes that collect the data. The times of the received data are adjusted to be consistent on the base.

Compatability

eMote.NOW 1.0, eMote version 4.3.14.

Complexity Level

Intermediate. There is complexity with respect to time sync and fault handling.

Setup

Start two instances of Visual Studio. Open the base node solution into one instance and the sensing node solution into the other. Connect two motes via USB. Build and deploy the solutions to each mote, respectively. You can monitor the motes using two instances of MF Deploy.

If you want to deploy another sensing node, connect another mote via USB and deploy the sensing node solution to it. You can monitor this new mote with another instance of MF Deploy.

Protocol

The Wireless Data Collector assumes one base node and any number of sensing nodes. It is tolerant to transient link failure; any number of messages can be lost, for whatever reason: a node is down, there's radio interference, the nodes are too far apart, etc. So long as eventually the nodes connect and stay connected long enough, the program will run correctly. Note that this does not mean that no data is lost. When messages are lost, any data in them is lost. But it does mean that eventually, if links are re-established, the collection will proceed correctly. If the base node for some reason is unexpectedly slow in responding, it will be treated as a link failure.

We assume that each message sent is either received intact or is not received at all. We do not handle message corruption.

We describe the program in four parts. In the first part, we describe how the program operates when all the nodes are connected. In the second part we describe the faults that can occur (link failure, in this case). In the third part we describe how link failure is detected. In the fourth part we describe how the program makes a correction.

There are two programs: one for the base station and one for the sensing nodes.

Fault-Free Program

Base Node

• Receives "hello" messages from sensing nodes.
  • Records sensing node's local time plus base node's local time as an initial time pair for the sensing node.
  • Unicasts confirmation message to sensing node.
• Receives data message from sensing nodes.
  • Uses received node's current time and sensing node's initial time pair to calculate offset and clock skew for the sensing node.
  • Converts sensing node's data timestamps to local time using offset and skew.
  • Processes received data.

Sensing Node

• Broadcasts "hello" message that includes local time.
• Waits for base node's reply.
  • Stores base node's address.
• After reply, repeatedly collects data items and time stamps them.
  • Unicasts data and timestamp along with current local time.

Faults

The faults handled are actual link failure and apparent link failure when there is slow base node response.

Fault Detection Criteria (Detector)

• Base node
  • Base node acknowledges each received data message.
• Sensing node
  • Sensing node detects a fault when acknowledgement of sent message is not received in a timely fashion. This applies to all messages sent, including "hello" and data messages.

Fault Correction Criteria (Corrector)

• Base note
  • No action needed.
• Sensing node
  • When a fault is detected, sensing node restarts program.

Discussion

General

The programming for both the base and the sensing node is event oriented. When there is no event, each sensor can go into a low-power waiting state. However, see the comnments on power management below.

There are separate Visual Studio solutions for the sensing node program and for the base node program. However, there are classes and definitions that the two programs share. Hence there is a folder called "Wireless Data Collector - Common" that contains a number of files that are shared by each solution. If you look at one of the solutions, you'll see that there's a folder called "Utility". In that folder are links to the files. In the screenshot below, you see that BitConverter.cs, for example, is identified as a link, compared with Program.cs which is a file in the solution itself.

You can modify the files as necessary. But when you've done so, you'll need to rebuild both solutions.

Message Format

Hello

App Identifier (2)

Payload Type (1) : Hello

Payload Sequence Number (4)

Payload Send Time (8)

Data

App Identifier (2)

Payload Type (1) : Data

Payload Sequence Number (4)

Payload Send Time (8)

Sensing Time 1 (8)

Sensed Data 1 (4)

    ...

Sensing Time N (8)

Sensed Data N (4)

Reply

App Identifier (2)

Payload Type (1) : Reply

Payload Sequence Number (4)

Sensing Node Program

The sensing node program runs as a state machine in two states. In the Hello state, it periodically broadcasts hello messages. In the Sensing state, it senses data and unicasts it to the base.

Sensing nodes include in each message a unique sequence number.When the base node replies, it includes this number. For simplicity in this app note, we assume that the sensing node does not send a new message before the previous one arrives. However, one of the suggestions below is to modify the program to overcome this restriction.

The message format lets the sensing program bundle multiple sensing (time, data) pairs but  for simplicity in this app note, only one pair is sent with each message.

Base Node Program

The base node  receives messages from sensing nodes and sends a reply to each.

When sensed data messages arrive, the base node adjusts the time of each sample to correspond to the coresponding time for the base node. See Time Sync below.

For each sensing node that the base has heard from, it makes an entry in at Hashtable for that node. It keeps the time at the Base that the most recent hello message was received and the time at the sensing node when the message was sent. This is used in time sync, below.

Time Sync

The base mote converts the time data samples were taken on each sensing mote to a corresponding time at the base. The problem with doing this is that each mote has its own time frame.

On a mote you can get a fairly high-precision time value by using Datetime.Now.Ticks. This gives a 64 bit signed integer. If two motes are running, the value given will depend on how fast the clock is running and when the mote was powered up. 

The clock speed on the .NOW board can be controlled to run faster or slower to balance performance vs. energy usage and the Ticks value will vary accordingly. In addition the clock speed on the .NOW board can vary with time even if it is fixed at a particular clock frquency. Factors that influence this are such things as ambient temperature. 

To handle this, the base mote stores, for each sensing mote, a pair of initial time values: (a) the time on the base when the last hello message was received for the sensing mote and (b) the time on the sensing mote when the message was sent. It uses these values to convert sensing mote time to base mote time, as follows:

• \(\mathit{Offset}\). This is the initial base time. 
• \(\mathit{Skew}\). This is gives the proportional difference between the base and the sensing nodes' times. When a data message is received, it is timestamped with time on the sensing mote when the message was sent. Hence for a given data message,

\[\mathit{Skew} = {(\mathit{DataMessageSendTime} - \mathit{SensorInitialTime}) \over  (\mathit{DataMessageReceivedTime} - \mathit{BaseInitialTime})}.\]

Note that skew can change from one message to another. There are various ways of calculating skew. For simplicity, we've chosen to base it on total elapsed time for sensor and for base respectively. With these definitions of \(\mathit{Offset}\) and \(\mathit{Skew}\), we can calculate the time at the base when a sample was read. 

\[\mathit{BaseSampleTime} = (\mathit{SampleTime} - \mathit{SensorInitialTime}) * \mathit{Skew} + \mathit{Offset}.\]

How accurate is this time sync? While it takes into account clock skew, it does not take into account jitter: uncontrollable variations in time that occur on the sensing and on the base motes. Here are the jitter factors:

• Garbage collection. As necessary, eMote will stop to recover space from unused objects. This source of jitter can be eliminated by changing the programming; see suggestions below.
• Message send time. When messages are sent via CSMA, they are put in a buffer and are sent when CSMA is ready to do so. This wait time is bounded but unpredictable and can't be controlled by programming.
• Message receive time. When messages are received via CSMA, an interrupt occurs and the interrupt handler reference is put into a queue. When its turn comes, the handler is called. This delay is also unpredictable and can't be controlled by programming.
• Hardware clock variability. The mote's clock speed can change over time because of environmental and other factors.

Hence jitter will affect the accuracy of the time sync conversion.

Power Management

The radio is the biggest user of power in this app note. When the radio is not needed, it can be turned off to save power. 

The base station must always be ready to receive messages and so it must always be on. However, the sensing node can turn off the radio when it is not necessary.

The radio must of course be on to send a message. When a sensing node has sent a message, the radio stays on until the reply is received. At that point, the radio is turned off until it is time again to send another message.

Using Real Sensing

Incorporate a sensor and use it to take real measurements. You can combine this app note with the Data Collector, which collects radar data and uses DataStore and a microSD card to exfiltrate to a PC. Follow the Data Collector instructions to connect a BumbleBee radar to a .NOW. In this app note, you'll need to modify both the Sensing and the Base programs. Instead of radar, you can sense data any other way that suits. The discussion below assumes radar data collection.

Modifying the Sensing Program

The present structure uses a sensing timer with a callback to simulate data collection. Eliminate the timer and the callback (but keep a copy of the callback code as a pattern). Setup the ADC AnalogInput.ConfigureContinuousModeDualChannel as shown in Data Collector. For the ADC callback, adapt the timer callback code to send the collected buffer. Note that to avoid race conditions, you may need to copy and queue buffers as they come as in Data Collector. 

Modifying the Base Program

RadioReceive is the callback method for reception of a packet. After verifying that the packet is correct, you can save the data. You can use a microSD card as shown in Data Collector; since writing to the microSD is slow, you might have to use DataStore to buffer it, depending on how fast data comes in. Or, you can pass the data to a connected PC, following the pattern in On-Off Switch with Serial Link to PC. This would give you a third program, Host, that would save the data to a file. 

Suggestions

Message Bursts

As mentioned above, this app note assumes that no sensing node sends a message before the previous one is acknowledged. Modify the sensing node program so that there can be a burst of transmissions that occur close together, before the first acknowledgement has arrived. Here's a suggestion. 

1. Define an Arraylist of (integer, timer) pairs that associates a message sequence number with a timeout timer; this is a dynamic structure so it can grow as necessary.

2. When a sensing node sends a message, choose an (integer, timer) entry whose timer is stopped; if there is none, create a new entry. This keeps the Arraylist from growing without bound.

3. For the entry, set the integer to the message sequence number and start the timer to run once for the timeout interval and then stop. Use the same callback routine for all timers in the arraylist.

4. When an acknowledgement for a message is received from the base, stop the timer.

5. When any timer times-out, stop all timers and return the program to Hello state.

6. Since acknowledgements can still be coming in after a timeout, ignore any messages whose sequence number is not found associated with a running timer in the list.

Bundling Messages

Modify the sensing program to take several sensor readings and send them as a single message.

Eliminating Garbage Collection Jitter (somewhat advanced)

Modify the sensing program to eliminate the possibility that garbage collection might happen when sensing and when sending a message. Garbage collection occurs when the program needs to create a new object but there is no free space left for it. At that point, eMote pauses to recover the necessary space. However, if all running threads are using only variables whose associated objects are created just once,  garbage collection will not occur during that thread. Note, for example, the following statement in the sensing program's RadioSend method:

// Copy payload time

var payloadTime = BitConverter.GetBytes(DateTime.Now.Ticks);

This statement calls the method BitConverter.GetBytes, which creates a byte array, and at that point, garbage collection could occur, causing jitter. To eliminate the jitter, modify BitConverter.GetBytes so that allocation of the byte array is done just once. One way to do that is to pass in a statically-defined byte-array from RadioSend and return that array with the values.