ThingsBoard: Do not split up incoming JSON array?

Are you tired of dealing with fragmented JSON arrays in ThingsBoard? Do you struggle to make sense of the data that’s supposed to be neatly packaged in a single array? Well, you’re in luck! In this article, we’ll dive into the world of ThingsBoard and explore the solution to this common problem. So, buckle up and let’s get started!

The Problem: Splitting Up Incoming JSON Arrays

When working with IoT devices and sensors, it’s not uncommon to receive large chunks of data in the form of JSON arrays. In an ideal world, these arrays would arrive intact, ready for processing and analysis. However, in reality, ThingsBoard might split up these arrays into smaller, more manageable chunks, making it difficult to work with the data.

This issue can arise due to various reasons, such as:

• Network constraints: Large packets of data might be too big to transmit over the network, leading to fragmentation.
• Device limitations: IoT devices might not be capable of handling large datasets, resulting in split arrays.
• ThingsBoard configuration: Misconfigured settings in ThingsBoard can cause incoming JSON arrays to be broken down into smaller pieces.

The Solution: Configuring ThingsBoard to Handle JSON Arrays

Fortunately, ThingsBoard provides a simple solution to this problem. By configuring the platform to handle JSON arrays correctly, you can ensure that your data arrives intact and ready for processing.

Step 1: Understand the JSON Array Structure

Before we dive into the configuration, it’s essential to understand the structure of your JSON array. Take a closer look at the data you’re receiving and identify the following:

• The root element: Is it an array or an object?
• The array elements: Are they objects, strings, or numbers?
• The array size: How many elements are in the array?

Having a clear understanding of your JSON array structure will help you configure ThingsBoard correctly.

Step 2: Configure the ThingsBoard Device Profile

In ThingsBoard, navigate to the Device Profiles section and select the profile associated with your IoT device. Click on the “Edit” button to access the profile settings.

{
  "deviceProfile": {
    "name": "My Device Profile",
    "deviceType": "MY_DEVICE_TYPE",
    "transportType": "JSON",
    "deviceAttributes": [...],
    "deviceTelemetry": [...],
    "attributeMappings": [...],
    "telemetryMappings": [
      {
        "type": "JSON_ARRAY",
        "jsonArrayParser": {
          "arrayElementParser": {
            "type": "JSON_OBJECT"
          }
        }
      }
    ]
  }
}

In the above example, we’ve added a telemetry mapping with a type of “JSON_ARRAY”. This tells ThingsBoard to expect an array of JSON objects. The “arrayElementParser” specifies that each element in the array is a JSON object.

Step 3: Configure the ThingsBoard Data Converter

In some cases, you might need to convert the incoming JSON array into a more suitable format for processing. This is where the Data Converter comes in.

{
  "converter": {
    "type": "JSON_ARRAY_TO_SINGLE_MESSAGE",
    "jsonArrayConverter": {
      "groupId": "${groupId}",
      "deviceName": "${deviceName}",
      "ts": "${ts}"
    }
  }
}

In this example, we’ve configured the Data Converter to convert the incoming JSON array into a single message. The “groupId”, “deviceName”, and “ts” variables are placeholders that will be replaced with the actual values from the incoming data.

Common Scenarios and Solutions

In this section, we’ll explore some common scenarios where JSON arrays are split up in ThingsBoard and provide solutions to handle them.

Scenario 1: Handling Large JSON Arrays

If you’re receiving large JSON arrays, you might need to adjust the ThingsBoard configuration to handle them correctly.

Problem	Solution
Large JSON arrays are split up into smaller chunks	Increase the “maxStringLength” property in the ThingsBoard device profile to accommodate larger arrays
Memory issues with large JSON arrays	Consider using a more efficient data storage solution, such as a time-series database, to reduce memory usage

Scenario 2: Handling Nested JSON Arrays

Nested JSON arrays can be tricky to handle, but ThingsBoard provides a solution.

{
  "telemetryMappings": [
    {
      "type": "JSON_ARRAY",
      "jsonArrayParser": {
        "arrayElementParser": {
          "type": "JSON_OBJECT",
          "objectParser": {
            "type": "JSON_ARRAY",
            "jsonArrayParser": {
              "arrayElementParser": {
                "type": "JSON_OBJECT"
              }
            }
          }
        }
      }
    }
  ]
}

In this example, we’ve configured the telemetry mapping to handle a nested JSON array. The “objectParser” is used to parse the inner array, and the “arrayElementParser” is used to parse each element in the inner array.

Conclusion

In this article, we’ve explored the issue of ThingsBoard splitting up incoming JSON arrays and provided a comprehensive solution to handle this problem. By understanding the JSON array structure, configuring the ThingsBoard device profile and data converter correctly, and addressing common scenarios, you can ensure that your data arrives intact and ready for processing.

Remember, ThingsBoard is a powerful platform that offers a wide range of features and customization options. With a little creativity and problem-solving, you can overcome even the most complex data processing challenges.

Have you struggled with JSON arrays in ThingsBoard? Share your experiences and solutions in the comments below!

Frequently Asked Question

Get clarification on ThingsBoard’s JSON array handling with these frequently asked questions!