JSON Decoder for PIC® and AVR® Devices

TB3239

JSON Decoder for PIC? and AVR? Devices

Introduction

Author: Alexandru Niculae, Microchip Technology Inc.

This document describes the use and implementation of a JSON subset decoder, aimed for embedded devices such as PIC? and AVR? microcontrollers. The decoder translates the string format of a JSON object into a C data structure representation. This way programmers can access the key-value pairs. Using JSON objects makes it easy to interconnect applications. The JSON Decoder code is available on GitHub.

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 1

TB3239

Table of Contents

Introduction.....................................................................................................................................................1 1. Overview................................................................................................................................................. 3 2. API.......................................................................................................................................................... 4 3. Implementation........................................................................................................................................7

3.1. Input Buffer...................................................................................................................................7 3.2. Decoding...................................................................................................................................... 7 3.3. Internal Representation................................................................................................................ 8 3.4. Encoding...................................................................................................................................... 9 4. Demo: Controlling the Four LEDs on AVR-IoT from the PC..................................................................10 4.1. Instructions................................................................................................................................. 10 5. Demo: Generic Cloud Communication with an IoT Node..................................................................... 15 6. Appendix............................................................................................................................................... 16 The Microchip Website.................................................................................................................................18 Product Change Notification Service............................................................................................................18 Customer Support........................................................................................................................................ 18 Microchip Devices Code Protection Feature................................................................................................ 18 Legal Notice................................................................................................................................................. 18 Trademarks.................................................................................................................................................. 19 Quality Management System....................................................................................................................... 19 Worldwide Sales and Service.......................................................................................................................20

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 2

TB3239

Overview

1. Overview

JSON is the most used data serialization format on the web and on desktop applications. Since it has an implementation in virtually all modern programming language, it is the perfect tool for data interchange between applications. It has a text format that is easy for humans to read and write and easy for machines to parse and generate. A JSON object is a set of key-value pairs. While keys can only be strings, values can be a lot of things. This decoder implementation supports string, number, and object values. Note: To keep this library suited for low-memory devices, only a subset of features was selected to be implemented, therefore, array, boolean, and null values are not supported. The following block of code shows an example usage of this decoder. Section 2. API describes the API functions in detail.

#define TEST_JSON "{\"main\":{\"key\" : 10,\"foo\":\"bar\"}, \"alt\":2}" ... jsonNode_t *root, *objmain; char str[64], foo[10]; int alt; memcpy(str, TEST_JSON, sizeof(TEST_JSON)); JSON_DECODER_fromString(str) JSON_DECODER_getRoot(&root); JSON_DECODER_getObject(root, "main", &objmain); JSON_DECODER_getNumber(root, "alt", &alt) JSON_DECODER_getString(objmain, "foo", sizeof(foo), foo)

Note: The outermost object in a JSON is called the root.

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 3

TB3239

API

2. API

Refer to this section for detailed explanation of the public functions in the JSON Decoder.

jsonDecoderStatus_t JSON_DECODER_fromString(char *str)

Description Parses a JSON string into a C representation. It supports string, number, and object values.

Parameters Name str

Description

Pointer to an existing string representation of a JSON object. After parsing, the string will have been altered.

Return Value Name JSON_DECODER_OK JSON_DECODER_BAD_FORMAT

Description Decoding was successful The input is invalid

JSON_DECODER_getRoot(jsonNode_t **pNode)

Description Finds the outermost object in the JSON, called the root. This function must not be called before JSON_DECODER_fromString.

Parameters Name pNode

Description Pointer to a jsonNode_t

Return Value Name JSON_DECODER_OK

Description pNode now points to the root

jsonDecoderStatus_t JSON_DECODER_getObject(jsonNode_t *current, char *key, jsonNode_t **pNode)

Description Finds a JSON object by key in another JSON object. This function must not be called before JSON_DECODER_fromString.

Parameters Name current

Description Pointer to a jsonNode_t

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 4

...........continued Name key pNode

Return Value Name JSON_DECODER_OK JSON_DECODER_KEY_NOT_FOUND

Description The key of the object to find Pointer to the object, if found

TB3239

API

Description Object was found and pNode now points to it The specified key does not exist

jsonDecoderStatus_t JSON_DECODER_getString(jsonNode_t *current, char *key, uint8_t size, char *pVal)

Description Finds a string by key in a JSON object. This function must not be called before JSON_DECODER_fromString.

Parameters Name current key size

pVal

Description The JSON object to search into The key of the string to find Maximum size of the string. Should not be bigger than the length of the pVal buffer Pointer to the string, if found

Return Value Name JSON_DECODER_OK JSON_DECODER_KEY_NOT_FOUND

Description String was found and pVal now points to it The specified key does not exist

jsonDecoderStatus_t JSON_DECODER_getNumber(jsonNode_t *current, char *key, int *pVal)

Description Finds a number by key in a JSON object. This function must not be called before JSON_DECODER_fromString.

Parameters Name current key pVal

Description The JSON object to search into The key of the string to find Pointer to the number, if found

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 5

Return Value Name JSON_DECODER_OK JSON_DECODER_KEY_NOT_FOUND

TB3239

API

Description Number was found and pVal now points to it The specified key does not exist

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 6

TB3239

Implementation

3. Implementation

This section describes the implementation details with the purpose of facilitating the in-depth understanding of the code. Moreover, it describes some decisions made to optimize the memory usage and implementation safety. For example, because using recursion is not recommended, a part of the algorithm was refactored.

3.1 Input Buffer

The input buffer is the buffer in which the JSON string is received, by an MQTT library, a UART driver or by other communication method. This buffer will be passed to the JSON Decoder using the JSON_DECODER_parseString function. When this function returns, the buffer's content will have been modified. To use as little memory as possible, the internal JSON object representation does not hold any strings, but holds pointers to where the strings were originally located in the input buffer. Therefore, null terminators are inserted into that buffer to isolate the strings. All the keys and the string values are inside the input buffer. Figure 3-1.String Buffer

{ " f oo " : " bar " }

JSON_DECODER_parseString

{ " f o o \0 : " b a r \0 }

jsonNode->key

jsonNode->value

An important consequence is that care must be taken not to rewrite the buffer before the JSON data is consumed. If the buffer is rewritten in between the JSON_DECODE_parseString and accessing the JSON structure, this will corrupt the data.

3.2 Decoding

Decoding means transforming the string format of the JSON object into a C data structure representation. This is a grammar parsing problem and it is usually solved in a two-step process: lexing and parsing.

Lexing splits the input text into tokens, as well as stripping out spare characters, such as white spaces. Tokens are both single charter (curly braces, commas, colons) and multicharacter (strings, numbers). This implementation uses a "lazy lexer", meaning that instead of creating the list of all the tokens, they are retrieved one by one as needed by the parser.

The parser verifies that the sequence of tokens makes sense in the context of the JSON grammar. It constructs the internal representation into a C data structure. The parser is inherently recursive in nature because it deals with nested JSON objects. To avoid recursion (and possible stack overflow errors), the algorithm was refactored to use a heap stack and a while loop, instead of the actual stack.

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 7

Figure 3-2.JSON Decoder HLD JSON string

JSON Decoder

Nonrecursive Parser

Request token Return next token

TB3239

Implementation

Lazy Lexer

JSON object in C representation

3.3 Internal Representation

After parsing the string format, the JSON object is stored in the memory as a tree-like structure. Using the API functions, the tree nodes are traversed to retrieve values by key name. Each node corresponds to a key and can either be a leaf node (it has a string or number value) or an object node.

Figure 3-3 shows a JSON object and its representation as a tree of jsonNode_t structures. The node structure has a type field, a key field, a value field, and two pointer fields to adjacent nodes. The h pointer points to the next key in the same JSON object and the v pointer points to the first own key, if the current node is of type object.

Figure 3-3.Internal Representation

{ "myObject": { "myNum": 10, "myString": "hi", }, "myNum": 0, "myString": "hello",

}

Type: OBJECT Key: "myObject"

h

Type: LEAF

Key: "myNum"

h

Value: 0

Type: LEAF Key: "myString"

Value: "hello"

v

Type: LEAF

Type: LEAF

Key: "myNum"

h

Key: "myString"

Value: 10

Value: "hi"

Note: The maximum number of keys a JSON object may have in order to be decoded is limited by the length of an array of jsonNode_t statically defined. Dynamic memory allocation is to be avoided in embedded systems.

Note: The maximum number of nested objects is determined by the size of the heap stack used by the parser.

? 2020 Microchip Technology Inc.

Technical Brief

DS90003239A-page 8

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download