IoT Testing From The Ground Up – Why “Ground Truth” Matters

To launch, or not to launch? Anyone working in IoT product development will face the dilemma at some point. Imagine this typical scenario — After years of hard work, your product is finally almost ready for market. The device has undergone product reliability testing, the software has been put through its paces, and your customers, partners and other stakeholders are all eagerly anticipating launch day.

What happens if, let’s just say, your algorithm calculations are just a little bit off? Is this the time to let down the expectant stakeholders, call off the launch party and address those issues? You may be tempted to press on. After all, the expected outcomes were just marginally different. No big deal, right? Especially when you weigh the competitive advantages of launching as planned…

But how can you be sure that your new solution will perform as intended in its true environment?

Due to the complex nature of IoT use cases, these connected products and solutions require rigorous and reproducible development and testing of all the components under real-life conditions before going to market. This ‘ground truth’ validation is especially important as the interplay between the software, hardware and environmental conditions multiplies the number of unknown variables. Ground truth validation helps uncover the small anomalies that can snowball into big problems for an end-user device.

Let’s go back to our product scenario, which is almost ready to be launched. After you have introduced your end-user device into the new ecosystem is not the time to discover interoperability glitches, a buggy UI or any defective algorithms. At this point, you need to begin the entire product development process all over again, which costs time, money, and may well cost you in terms of competitive advantage.

What’s involved in ground truth validation?

Lab-driven approaches are proving increasingly successful at delivering ground truth validation in IoT product builds. There are two very good reasons for this. Firstly, a lab-driven approach to IoT product development and testing enables all the product’s interactions within its ecosystem to be tested together synchronously. Secondly, the major ‘wow’ factor that lab testing delivers is when all the different test capabilities are brought together to reproduce almost any scenario you can imagine. This means not only can you finesse the algorithms, you also have at your fingertips a much more holistic understanding of the business case, allowing you to reproduce multiple scenarios to refine the entire end-to-end platform for maximum competitive advantage.

There are three main steps in this type of testing:

1. Sensor validation – Is the sensor data accurate? Are the algorithms responding appropriately?
2. Interoperability issues – Are you experiencing seamless interoperability between the components, software and the broader ecosystem?
3. Functional testing – Is the product performing as intended?

It’s undeniable that IoT testing tends to be more complex than other areas of development. Entrants to this market need to be prepared for considerable investments in time and money to successfully roll out products. However, a lab-driven approach can also help reduce the amount of time required for testing. Those familiar with QE, and its approach to continuous integration (CI), will find this method of working easily transferable to an IoT lab environment. While some aspects need to be tested manually to ensure that components, sensors and software all function as they should, there is still considerable scope for automation and CI to speed up delivery, particularly when introducing new features to an existing solution.

There’s a real sense of land-grab in the IoT market currently – a sense that big opportunities can be won, or lost, and that these outcomes will define the market in coming years. It’s impossible to tell which technologies, protocols or systems will come to dominate. What’s certain is that the winning IoT solutions will be those that deliver outstanding performance and seamless compatibility within the ecosystem. And ground truth validation will have a decisive role to play in separating winners from losers. If you want to find out more information about lab-driven IoT development and testing, contact us.