Introduction
The Internet of Things, with its vision of linking all aspects to a global network, is currently in the process of radically reshaping our everyday experiences across various domains. From consumer-oriented wearable gadgets to the integration of smart energy management systems in commercial and infrastructure sectors, and culminating in the emergence of Industry 4.0, the transformation is noticeable. To realize this vision, millions of devices, services, and applications must seamlessly interconnect over the network in an efficient manner.
To address the anticipated challenges for resource-constrained devices, the 3rd Generation Partnership Project (3GPP) created Narrowband-IoT (NB-IoT), a wireless mobile network with cost-efficient access for a massive number of resource-constrained devices. To grasp the full potential of NB-IoT, IoT engineers must understand the strengths and limitations of communication over NB-IoT.
The majority of IoT engineers already have experience in the development of web and mobile solutions. Unlike desktop and web applications, the development of mobile applications relies heavily on asynchronous application callbacks driven by a mobile operating system. This is the direct result of strict control of application execution flow by the mobile operating system in order to optimize mobile battery usage. Similarly, creating optimized IoT applications using NB-IoT requires using an IoT-centric communication stack and proper application-level data protocols. A simple porting of web and mobile solutions in NB-IoT would yield suboptimal solutions for resource-constrained devices.
Message Queuing Telemetry Transport (MQTT) is currently the default choice as the IoT communication protocol. Despite the great operational success and broad industrial community acceptance of MQTT, its reliance on TCP/IP stack is suboptimal in some narrowband wireless mobile network use cases. Internet Engineering Task Force (IETF) created Constrained Application Protocol (CoAP) as a lightweight communication protocol for resource-constrained devices. It follows the same message semantics as HTTP, thus enabling seamless interoperability with web applications. OMA SpecWorks created Lightweight M2M (LwM2M) on top of CoAP to enable secure monitoring and management of operations based on openly defined objects describing the device state.
Narrowband IoT applications
NB-IoT is designed to cover use cases that require sending only small portions of data but over long distances. Therefore, it is forecasted to lead the Internet of Things connection growth, especially in areas where high connection density, extended battery life, low cost, and enhanced indoor coverage are of critical importance. In fact, NB-IoT enables the introduction of Internet of Things solutions in areas where this was not previously possible due to technical limitations and low cost-efficiency. It operates within an ecosystem that is well-recognized and consistent throughout the globe. What is more, thanks to a high level of standardization, NB-IoT proves to be interoperable across locations, vendors, and network operators. By offering an end-to-end quality of service assurance and utilizing licensed spectrum, it provides greater reliability and performance.
As such, NB-IoT is very well suited for applications requiring only low bandwidth, such as intelligent parking systems, smart meters, or waste management. NB-IoT has also been widely employed in industry, agriculture, healthcare, logistics, as well as in smart cities and buildings. Its purpose is to facilitate the work of technicians, retailers, and operators while handling machines by providing insightful data and supporting remote monitoring systems. For example, Narrowband-IoT can be used to give more precision and reliability in agricultural techniques, condition measurement IoT sensors, livestock tracking (with movement alerts), monitoring of soil parameters, pollution rates, intelligent rainfall prediction systems, and many other areas of agriculture.
It also opens up possibilities for extensive monitoring and alerting for those who suffer from chronic or age-related diseases, thanks to which doctors will monitor their patients remotely and in real-time.
What is more, NB-IoT-enabled wearable devices can be used to control the exact location of the wearer. Specially designed for elderly people or children, they can also be used for animals (pets, cattle). It proves particularly useful when it comes to tracking and identifying infected individual’s movements and points of contact with other people or animals.
Narrowband IoT has also found its uses in manufacturing, where industries can geolocate moving assets to optimize logistics, check their status, and collect valuable information about them. Also, NB-IoT-based solutions will enable tracking, monitoring, and diagnosing production objects in real-time to intervene before a failure occurs (predictive maintenance).
Lastly, the use of NB-IoT translates to reduced costs and lower environmental impact in smart city and smart buildings, especially in managing emergency or anti-theft alarms in homes and businesses; smart consumption measurements (electricity, gas, and water); intelligent lighting and air conditioning; air quality, and more.
Low-power wide-area networks (LPWANs)
There are many existing wireless communication technologies ranging from short distance technologies like Near-field Communication (NFC) or Bluetooth, through medium-range technology like WiFi, to long-distance technologies like cellular networks or satellites. The graphics below shows popular wireless technologies and where they are positioned in the context of radio range and data rate. Cellular systems provide the best radio coverage after satellite systems. However, in the latter case, the cost of their maintenance and the energy needed to operate are high and are usually used in remote places. Thus, the dominant technologies are solutions based on cellular connectivity, such as LTE/5G. A similar radio range is offered by the so-called low-power WAN [RFC8376] standards dedicated to battery-powered devices. These include technologies requiring dedicated communication infrastructure like LoRaWAN, the one based on existing cellular networks. The 3GPP organization, responsible for the standardization of mobile technologies, proposed two LPWAN technologies – LTE-M and NB-IoT. They are built on top of existing LTE solutions, reducing the production costs of communication modules, and enabling seamless adoption by M2M applications. The table below contains the main features of popular LPWAN technologies.
The LTE-M standard offers a peak data rate of 1Mbit/s and short communication latency of 100-150ms, making it suitable for mobile applications such as patient monitoring, wearables, or asset management systems where real-time communication and energy efficiency are essential. The other technology, NB-IoT, is optimized for simple sensor applications requiring low data rates that should work on batteries for an extended period of time. The NB-IoT deals very well with high connection density and focuses on providing good indoor radio coverage, long battery life, as well as low cost of communication modules. NB-IoT uses a subset of the LTE standard. It limits the spectrum bands to 200kHz, achieving a peak data rate of 159kbit/s and communication latency between 1.5s and 10s. Typical applications of NB-IoT include telemetry solutions, with many nodes in the field not requiring large amounts of data to send and for which transmission speed is not critical and communication is rare, such as smart metering, pipeline management, and power grid monitoring.
The NB-IoT offers mechanisms for minimizing battery usage. In a typical application of NB-IoT, the device is disconnected from the network for the majority of its operation time until it needs to send data, e.g., water meter readings. The graphic above shows the transmission cycle in NB-IoT and its energy-saving capabilities. When the device has data to send, it wakes up and transmits the data. After the transmission is completed, the device enters idle mode and regularly checks whether any new data has been sent to it. After a certain period of time, the device switches to the power-saving mode, in which it may stay for up to 14 days. When the device enters the power-saving mode, it reduces the amount of energy consumed to a minimum. The device is still registered with the network in this mode, so there is no need to re-establish the connection when resending data. However, during the power-saving mode, the device is not reachable by the network and does not receive data. Messages for the device from the network are queued in the network until the device leaves the power-saving mode.
Communication in IoT
In the IoT, open standards are of crucial importance, allowing for the cooperation of hardware and software from different manufacturers and exchanging information with other systems.
Transferring information between devices consists of several stages. In the first stage, the information to be sent from the device should be presented in data objects representing their semantics. Then the data objects should be converted into a sequence of bytes to be sent over the computer network. The next step is to select the application-level and transport protocols that provide the desired data delivery guarantees. The final step is to send the data via an appropriate communication medium. This section provides information on the communication technologies and protocols used in IoT and the data serialization methods.
Data serialization
The data needs to be serialized before it can be sent over a network or transferred between devices. In the case of IoT, there are several methods to serialize data that differ in their properties.
Type-Length-Value (TLV)
A method of data serialization where the encoded information consists of three elements – a code that represents the type of data, the length in bytes, and the actual values. Serialized data is in a binary form, efficient for transmission but difficult for human interpretation.
JSON
The method of encoding information in the form of attribute-value pairs and arrays containing many elements. It uses human-readable text to transmit data objects. While JSON is a universal and widely used information exchange format, its textual description of the data makes it not very efficient in data transfer. JSON is described in [RFC7159].
Sensor Measurement Lists (SenML)
A format used to represent sensor measurement results. The standard defines a meta-model of data describing both the measurement itself and the measuring device. The standard supports data description in the form of time series and measurement units. The SenML standard is based on the JSON format for data description. SenML is described in [RFC8428].
CBOR
It is a binary data serialization format based on JSON. Similarly, as JSON, data objects are represented as name-value pairs. The binary representation results in the smaller size of the serialized data than the JSON representation. It can also be applied to SenML. The binary representation increases the processing speed. It is defined in the [RFC8949].
Transport protocols
In IP networks, the transport layer is responsible for controlling the transmission of data between endpoints. The dominant transport protocols on the Internet are TCP and UDP. They differ in the provided functionalities. TCP is designed for connection-oriented transmission and provides reliable transmission. Data is transmitted in a stream-oriented way, also providing the congestion control mechanisms. On the other hand, UDP provides much simpler mechanisms for data transmission – data is sent as datagrams and does not provide reliability that has to be implemented by the application if needed.
The characteristics of these protocols affect the communication overhead they introduce. The graphic above shows the comparison of the size of the data packet headers for TCP (20 bytes) and UDP (8 bytes). Assuming that the IoT device sends several bytes of data containing telemetry data, headers make up a significant part of the transmission. The more detailed comparison of these protocols and the functionalities they provide is presented below.
The communication channel between devices needs to be secure (i.e., authenticated and encrypted). Security on the transport layer is guaranteed by TLS and DTLS protocols.TLS is designed to be used with the TCP protocol, while DTLS is used with UDP.
TCP is a complex communication protocol containing several mechanisms to reliably transfer data and deal with exceptional situations that may occur during data transmission over the network. Its full implementation requires memory and computing resources, which are usually larger than typically available in IoT devices. Therefore, in the constrained devices, some simplifications are introduced. A detailed analysis of TCP’s minimum set of functionalities for such devices is carried out in the document [RFC9006]. Below, we explain important characteristics of TCP that impact its performance in NB-IoT.
Congestion Control
The TCP protocol during data transmission aims to use the available network bandwidth efficiently and fairly. Since the network is a shared transmission medium with finite bandwidth, the sender does not know what the possible bandwidth will be achieved at the beginning of the transmission. TCP uses a congestion control mechanism to find the network’s maximum fair throughput. The protocol uses a sliding window for flow control, which determines the number of unacknowledged bytes the sender is able to send to the receiver. The slow-start mechanism increases data transmission speed until the upper limit appears, usually manifested by packet dropping. It makes the sender slow down after which the sender tries to increase throughput again (congestion avoidance mechanism).
Fast Retransmit
Reliable data transmission in TCP is ensured by acknowledging each transmitted data segment by the receiver. If the sender does not receive the acknowledgment within the expected time, the unacknowledged data segment is retransmitted. At the same time, it functions as a cue for the congestion control algorithm to slow down the data transfer because the maximum possible throughput has been reached.
Not every loss of a packet and the lack of confirmation of its acknowledgment is a sign of congestion – it might as well be a result of temporal radio signal loss or sporadic communication error. When the receiver notices the skipping of a data segment, it can inform the sender to do fast retransmission of the missing data segment without drastically slowing down the data transfer.
SACK
The Selective Acknowledgment (SACK) mechanism allows the receiver to send noncontiguous acknowledgments of the received data. As a result, retransmissions include only those segments that were lost during transmission.
Delayed Acknowledgment
The delayed acknowledgment technique [RFC1112] combines several acknowledgments together into a single response, thus reducing protocol overhead. The host may delay sending acknowledgment up to 500ms.
Application-level protocols
Application-level protocols are responsible for the communication between software applications on the network-connected devices. There are two commonly used communication models in the IoT – publish/subscribe and client/server. In the case of IoT systems, the application protocols adapted to the limited resources of devices are MQTT and CoAP.
MQTT
A publish/subscribe communication protocol for resource-constrained devices. The MQTT specification specifies two types of objects – a message broker and a protocol client. The MQTT broker is a server that receives all messages from clients and then routes them to the appropriate target clients. An MQTT client can be any device ranging from a microcontroller to a full-featured computer that runs the MQTT library and connects to the MQTT broker through a network. MQTT uses TCP for data transmission and the traffic can be secured using TLS.
AMQP
A publish/subscribe communication protocol. Originally, it was designed for the financial sector. The protocol provides message-oriented communication patterns with message delivery quarantines. AMQP also comes with more complex and comprehensive mechanisms for queue management, flow control, and error handling than MQTT. It is also based on TCP and offers secure communication over TLS.
HTTP
A client-server application protocol used mainly for accessing web pages and other resources on the Internet. HTTP introduces the concept of Uniform Resource Identifier (URI) to identify resources on the network and define the relationships between them. It finds its uses in the Internet of Things thanks to the similarities to the Representational State Transfer (REST) architectural style commonly used in IoT systems. HTTP protocol is defined in [RFC7230].
CoAP
A client-server protocol described in [RFC7252]. Its concept is based on the REST architectural style used in distributed computer systems. There are two types of nodes in the protocol – servers that make resources available and clients that use them.
Constrained Application Protocol (CoAP) has several mechanisms that are specific to IoT systems. It can use TCP or UDP as a transport protocol. Usually, UDP is used as the default transport protocol instead of TCP since the resources needed to implement UDP are much smaller than TCP. Since UDP is a datagram-based protocol, sending data larger than the maximum IP packet size causes IP packet fragmentation, which should be avoided. To address this problem, CoAP provides the Block transfer mechanism [RFC7959]. The data is split into smaller chunks, sent sequentially to the receiver, and reassembled. Thanks to the simplified confirmation mechanisms of transmitted fragments, the implementation of this mechanism on the device side does not require significant resources. However, in some cases where UDP traffic is not allowed on the network, TCP can be used.
CoAP also supports a group communication mechanism that allows clients to obtain resource status information from a group of nodes by sending one UDP packet to the entire local network instead of polling each node separately. Analogically as in HTTP protocol, XML and JSON formats for describing the transferred data are supported, and the binary formats such as CBOR and TLV.
LwM2M
Developed by the OMA SpecWorks, the Lightweight M2M (LwM2M) protocol is used to manage IoT devices and provides machine-to-machine communication. The protocol defines device management mechanisms, over-the-air updates (FOTA, SOTA), service enablement and is well suited for resource-constrained devices. The LwM2M architecture provides four logical interfaces:
- Bootstrapping interface – enables device management, which means that the device is only preconfigured during production and the actual configuration takes place in the field. This significantly reduces costs and shortens the time-to-market of the product and ensures secure and scalable lifecycle device management.
- Client registration interface – provides information about the availability and activity of the device and the supported functionalities.
- Device management and service enablement interface – enables the service provider to access the device resources, allowing the possibility of remote reconfiguration.
- Information reporting interface – enables observation of changes of monitored operating parameters of the device.
IPSO
The LwM2M protocol takes advantage of IPSO Smart Objects providing a data model that can be used for describing device state and its components. IPSO Smart Objects are well specified and peer-reviewed by the industrial community allowing device state to be exchanged between different systems without the risk of incompatibility in their interpretation. The objects describing the devices are synchronized with the LwM2M server creating its digital representation. Their status can be updated via any of the supported communication protocols by LwM2M, such as CoAP, HTTP, or MQTT.
Non-IP Data Delivery (NIDD)
The LwM2M protocol also supports device management in non-IP networks. Thanks to this, it is possible to transmit small amounts of information without the overhead of IP communication as well as the TCP and UDP transport layer protocols. In the case of NIDD, the CoAP protocol handles payload fragmentation, in-order data delivery and flow control. This type of data transfer makes it possible to limit the amount of data sent from IoT devices but must be supported by the cellular network provider (e.g., using SCEF).
Communication protocols measurement in NB-IoT
The section presents the results of experiments on the use of IoT application protocols in the NB-IoT network. Communication measurement of NB-IoT networks was performed for two different scenarios typically found in IoT:
- Transmission of telemetry data, where devices periodically transmit measurement data.
- Firmware update, where there is a need to transfer a binary file to the device.
The tests were carried out in real-world conditions using Raspberry Pi 3 (RPi3) and a Quectel BG96 communication modem connected to the NB-IoT (Cat-NB1) network. The communication of the RPi3 with the modem was ensured over the PPP connection through the USB-RS232 bridge. Energy measurements refer to energy used by the communication modem and do not include the energy needed to operate the single board computer. The experiments used the implementation of the TCP/UDP protocol stack available in the Linux kernel. The analysis of TCP options is presented later in this section.
Network packets were analyzed using a protocol analyzer installed on the RPi3 microcomputer. Server-side communication was not analyzed as data plans are usually associated with the amount of data sent by the device itself.
The experiments aimed to analyze how popular application IoT protocols perform when transferring data on NB-IoT networks. Methods of optimizing energy consumption by putting the radio module into power-saving modes, turning off the device, or selecting energy-efficient microcontrollers have not been analyzed.
Telemetry data transmission
Experiments regarding the transmission of telemetry data were carried out on a device used to measure air quality. To illustrate, the graphic below depicts two different approaches to implementing an air quality monitoring system. The first method assumes usage of the LwM2M protocol together with the appropriate IPSO objects, their serialization, and support for the application protocols such as CoAP over UDP. The second approach assumes the use of custom data description in JSON and then sending the data using the MQTT over TCP. Therefore, in the experiments, we compare communication based on the MQTT/TCP and CoAP/UDP protocols.
Data model and data serialization
An exemplary IoT device monitors air quality by providing information on PM2.5 and PM10 dust particles and air temperature. Measurement results are represented as device properties. Please note that for real-world installations, the information about the firmware version and the device's serial number is usually important for device management.
In IoT systems, the JSON format is often used to describe devices and measurement data. However, this requires agreeing on the meaning of individual measurement values and device description data. The illustration above depicts a description of the measurement data from the device in the JSON format. However, to ensure device interoperability, the appropriate IPSO objects must be chosen. IPSO object types are described in a numerical form suited to transfer over the network for IoT devices. For example, the temperature value is reported under the /3303/1/5700 identifier.
Regardless of the chosen data model, serialization is necessary before sending the data over the network. To compare commonly used serialization methods presented in the previous section, the JSON representation of the air quality measurements was compared to the serialized IPSO 3428 object using TLV, SenML+JSON, and SenML+CBOR formats. The chart above shows the amount of data required to be transferred over a network after data serialization.
The lowest communication overhead was achieved for the TLV serialization. This method serializes only telemetry data values, and both the sender and the receiver must know how to interpret the transmitted data. When serializing data using SenML, such information is sent along with the data, which improves the data interpretation process but increases the size of the data transferred. The solution, in this case, is to use CBOR for binary serialization of data stored in SenML. As a result, when choosing between these data formats, the SenML+CBOR format is a compromise between the amount of data transferred and the complexity of the data processing implementation.
Communication analysis
The result of data serialization is a series of bytes that are further sent over the IP network. We conducted tests to estimate the communication overhead introduced by the communication protocols.
We have analyzed the results for two popular protocols used in the IoT – CoAP/UDP and MQTT/TCP. Both protocols provide data delivery guarantees. In CoAP/UDP, the application protocol guarantees data delivery and their retransmissions in the case of losses in the network. In the MQTT/TCP, the data is transferred over the TCP protocol, which provides reliable data delivery, but the application protocol may additionally acknowledge the transmission.
The chart above shows the total number of transmitted bytes necessary for sending telemetry data. Assuming that a typical telemetry message is in the order of tens of bytes, all variants of MQTT/TCP require a significantly higher number of bytes to be transmitted in comparison with CoAP/UDP variants. This is a direct consequence of TCP overhead, as well as MQTT overheads resulting from application-level acknowledgment messages.
For the devices that send a small amount of information, such as an air quality sensor, the selection of effective communication protocols and data serialization significantly impacts the monthly data transmission.
The choice of the data serialization method and the communication protocol for transmitting telemetry data is crucial from the point of view of system operational costs. This applies in particular to the costs related to the data consumption for the transmission of telemetry data.
The table above shows the data usage analysis for CoAP/UDP and MQTT/TCP with the TLV serialization of telemetry data. For comparison, the data consumption for the MQTT protocol and textual data description in the JSON format are also presented. Assuming the device sends air quality data every 5 minutes, the device will use 25.01MB for MQTT and 12.85MB for CoAP each year for telemetry data. For comparison, the results for the frequently used JSON data transfer over MQTT are also included – in this case, the device will send 32.7MB annually. A good compromise is to send data in the SenML+CBOR format because it slightly affects the amount of data transferred in relation to CoAP+TLV, but it enables a more flexible way of data processing.
The charts above show energy consumed by the BG96 modem while transmitting air quality telemetry data. The example a) shows the transfer of data using the CoAP protocol that required confirmation – the information exchange involved two network packets. Examples b) and c) depict data transfer using MQTT protocol carrying TLV and JSON data. In MQTT, delivery acknowledgment takes place at the level of TCP and the MQTT itself, which results in the additional network packets sent from the device to the server – in the experiments 4 network packets were sent. What is more, the BG96 modem consumed 0.288mWh, 0.302mWh, and 0.339mWh of energy accordingly. The energy needed for the operation of the device itself has not been taken into account as it depends on the design of the device itself.
To sum up, in telemetry data transmission, the amount of data transferred becomes very important. This affects both the monthly usage of the data plan and the energy used for communication. The results of the experiments indicate that the CoAP protocol over UDP is better suited than the MQTT over TCP for transferring telemetry data using NB-IoT.
Firmware update
The software image transfer tests for the IoT device were performed for a 128kB file. The tests focused on the HTTP, CoAP, and MQTT protocols. In HTTP and CoAP, the firmware download process was initiated from the LwM2M server. The measurement results cover only the firmware download process, without the LwM2M management data. In the case of the CoAP protocol, the transfer was performed using the Block Transfer method. In the case of MQTT protocol, the file was divided into parts sent in a separate MQTT message.
NB-IoT networks are characterized by a relatively high delay introduced to communication. A test of pinging Google's DNS server was performed, and an average latency of 550ms was obtained. The link bandwidth measurement test was also carried out, and the average download speed was 17kbit/s.
Data transmission
The experiments were conducted using the RPi3, which works under Linux OS and has relatively large computing resources. Therefore, the first group of tests was led with the default settings of the TCP protocol – with Selective ACK enabled and a large receiving buffer allowing for window scaling above 64kB. The second group of tests was aimed to analyze the influence of TCP protocol settings on communication. This is especially important for lightweight TCP/IP stack implementations for resource-constrained devices.
We have tested the performance of HTTP for different maximum transmission units (MTUs), which constrains the packet size that can be transferred between the sender and receiver over the network. MTU value was set on the tested device to represent common configurations of real-world IoT systems. For the MQTT protocol, tests were performed for the default MTU equal to 1500B, but the file was split into chunks of 512B and 1024B and then sent as messages on the MQTT topics.
The chart above depicts the number of transferred network packets. In the case of HTTP, TCP is used as a transport protocol. The number of packets transmitted is inversely proportional to the MTU size. Similarly to HTTP, in MQTT, TCP is used as the transport protocol with MTU of 1500B. As a result, individual MQTT messages are aggregated into larger TCP segments not exceeding the MTU size on the network path. Nevertheless, the total number of network packets for MQTT is larger than for HTTP with the same MTU because every MQTT message published on the topic contains the appropriate MQTT header. Slight differences in the number of packets between QoS 0/1/2 result from additional MQTT messages guaranteeing appropriate message delivery semantics. The third protocol tested is CoAP, which uses UDP. This means that the data is sent as independent datagrams – the CoAP protocol splits the transferred files into smaller fragments using the Block Transfer method. The block size was set to 1024B, which is the maximum value specified in [RFC7959]. In Block Transfer, subsequent fragments are sent sequentially, i.e. the next fragment is sent only after acknowledging the previous one. Because of limitations set by MTU, the total number of transmitted packets for CoAP is similar to the number for HTTP with 1024B MTU.
The chart above shows the number of transmitted bytes. A significant transfer increase is when the data file is split into many network packets determined by MTU. This is especially evident for HTTP with MTUs of 256B and 512B. In the case of MTU 1500B, the number of transmitted bytes in HTTP is lower than the number of transmitted bytes in the MQTT protocol due to an overhead introduced by MQTT message headers. Similarly, the slight increase for MQTT with 512B message length is due to the additional message headers required to publish messages to appropriate topics. The result for CoAP is comparable to data transfer using HTTP with MTU 1024B, which corresponds to an assumed UDP datagram length of 1024B.
Communication time and energy consumption
The second part of the tests concerned the communication time and energy needed to transfer the firmware. The same protocols were used in the experiments as before.
To illustrate it, the chart shows the firmware transmission time. For HTTP and CoAP, the main activity is on the device side – the file is hosted on the server, and the device decides when to download it. In the case of the MQTT protocol, subsequent fragments of the file are pushed by the server to the device and then combined in a single file.
The test results shown in the chart above show that when downloading a data file, the energy consumed by the communication modem is proportional to the transfer time, see the previous chart for comparison. The long file transfer time in the CoAP protocol is a consequence of the fact that the datagrams are sent sequentially in the Block Transfer method. Only after receiving the acknowledgment of the previous datagram, the next one is transmitted. Consequently, this extends the data transfer time, but, on the other hand, it reduces the requirements for data buffering on the device – the firmware can be sequentially saved in the device memory. It is important for constrained devices where the implementation of the communication stack should be memory- and computationally efficient. The most energy-efficient transmission is using HTTP with a high MTU value – 1500B in the experiments. A similar result was obtained for the file transfer using MQTT with QoS 0. In this case, however, the data is not acknowledged at the MQTT protocol level. Both HTTP and MQTT use TCP.
Tweaking TCP parameters to improve overall performance
The last group of tests was aimed at analyzing the performance of TCP during firmware download using HTTP. In a simplified way, after establishing a TCP connection, the device indicates which file it would like to download. The server sends the contents of the file using the already established TCP connection and the download process is driven by the TCP congestion control. In the experiments, the whole transmission was recorded using a Wireshark sniffer installed on the device, and then the TCP flow was analyzed.
In the first experiment, the TCP protocol stack on the RPi3 device has enabled the SACK, Fast Retransmit, Delayed Acknowledgements, and Sliding Window. When the device observes out-of-order packets during data transmission, indicating that data segments were omitted, the fast retransmission mechanism causes a duplicate ACK to be sent to the sender. If the sender receives three times acknowledgment of the same segment, it retransmits the next segment after it, before the segment acknowledgment timeout expires. Thanks to the SACK mechanism, if the device received successive data segments while sending duplicate ACKs, the server is informed about it so that it does not send the already transmitted data again. We observed that retransmitted packets accounted for 11.6% of the overall transmitted bytes during the whole transmission. These packet losses result from the fact that modern TCP implementations are not well suited for NB-IoT networks. They were optimized for long fat pipe scenarios, usually observed in high-speed networks, for which it is expected to efficiently achieve high throughput. However, NB-IoT with orders of delays in seconds and throughput in tens of kilobits per second resembles rather long thin pipes.
In the second experiment, only the Selective ACK mechanism was turned off, leaving other TCP mechanisms active. During data transmission, 26% of the overall transmitted bytes were retransmitted by the server. Because of the disabled SACK, 7.8% of the overall bytes were received twice by the device due to the fact that the server was not notified that they were already received.
In the third experiment, it was decided to change the configuration of TCP to emulate the devices with scarce resources as discussed in [RFC9006]. The Selective ACK mechanism has been disabled, the scalable window and the size of the receive buffers have been set to force single-MSS (only one segment to be unacknowledged in the network). During transmission, no duplicated ACK or packet retransmissions were observed. However, the time of downloading the firmware has increased from 107s to 119s.
In summary, in resource-rich environments with the full implementation of the TCP protocol stack, the transfer of large files using TCP consumes less energy and takes less time than the transfer using the UDP protocol. However, the use of TCP for high latency and low bandwidth (long thin pipes) networks involves frequent retransmission handling, which consumes device resources. It is a result of the congestion control mechanisms, which are not adapted to such network environments and lead to the recurrent loss of data packets.
One solution to circumvent the frequent packet retransmissions in IoT devices is to limit the receiver's sliding window and enforce the single-MSS stack. However, in this case, the device cannot take advantage of such mechanisms as Fast Recovery, Selective Acknowledgments, and Window Scaling. Furthermore, when it is necessary to control each datagram separately because of, e.g., transmitting data via satellite links or when the implementation of TCP protocol is unavailable on the device, CoAP over UDP should be selected. In addition, in the case of NIDD, the CoAP protocol will also manage the communication and the transfer of data from the IoT devices.
Summary of experiments
The conducted tests show that in the case of two typical IoT scenarios, i.e., telemetry data transmission and FOTA, it is worth considering the use of different transport layer protocols. In this context, using the LwM2M application layer protocol, which covers communication nuances, is highly desirable. It also provides a data model and efficient data serialization methods ensuring high-security standards, making it a valuable protocol for IoT.
Conclusions
NB-IoT technology is designed for resource-constrained IoT devices that transmit a small amount of information and sleep most of the time, saving energy. Using the potential of this communication technology requires an appropriate selection of communication protocols and the method of telemetry data description.
In transmitting telemetry data from IoT devices using NB-IoT technology, we strongly suggest using CoAP/UDP for data transfer. Thanks to this, the communication overhead introduced to communication is reduced compared to the popular MQTT/TCP protocol. At the same time, it has a positive effect on reducing the energy consumption needed for communication.
Sending firmware updates for a device is a more complex issue. For devices with limited memory and computing resources, the use of the CoAP/UDP protocol for data transfer is recommended as their implementation is characterized by a small memory footprint. If the device fully implements the TCP/IP stack, then the firmware transfer using HTTP/TCP may shorten the transfer time due to the implemented congestion control mechanisms. In this case, however, advanced customization of the TCP/IP stack is necessary to reduce the number of packet retransmissions resulting from the characteristics of the NB-IoT network.
Finally, to ensure device interoperability between hardware and software vendors, we suggest using a well-known data model such as IPSO objects. As a result, data from devices is enriched with a semantic layer that determines their meaning. At the same time, the transmission of such objects over the network as a series of bytes is optimized so that their transmission lasts as short as possible. This significantly simplifies the design of the IoT solution and reduces the cost of its maintenance and monthly data plans.
In summary, the analyzed technologies and communication protocols were developed considering devices with limited resources, but their selection requires experience in their usage and adaptation to a specific IoT system. Therefore, we suggest using the LwM2M protocol, which is used to securely manage the life cycle of IoT devices and transfer measurement data from devices using various discussed energy-saving solutions.