Live Network Testing

Design validation

We test in line with the user’s guide (as the end-user would experience the product), against a development specification, or consult with our customer to jointly develop a field test programme using best practices. Field testing during the design process ensures the product works correctly on live networks and is interoperable with a selection of network infrastructure, network operators, service plans and other variables that may affect performance in the field.

Regression testing

As new versions of firmware or application software are released, during pre-production or mid-life commercial deployment, it is imperative that the performance of the product is measured, to ensure no regression has occurred as a result of the change. The impact of poorly tested software updates can include:

• poor user experience
• reduced customer satisfaction
• increased customer support costs
• reduced brand loyalty.

The team at IoTAS can report on device integrity following the introduction of new features or functionality. By testing against an agreed specification on live networks, we help to maintain integrity at all stages of the product lifecycle as well as triage with development teams to review any performance degradation. We are used to integrating into the customer’s development team including the use of custom test, logging, debug and reporting tools as appropriate.

Regional test resource

IoTAS can help address the dynamic requirements for qualified and experienced engineering test resource. We are experienced in managing individual projects that are wholly outsourced to IOTAS, as well as larger projects during which we integrate into the customer’s development and verification team as required during product development. We provide the opportunity for companies to scale their validation efforts at critical times during product release, as well as the means to manage the costs associated with regional validation and product localisation.

Minor or major OS testing solution (Android)

IoTAS have been testing Android builds since 2009, starting with Android 1.6 (Donut), and have developed a robust testing solution aimed at giving confidence not only for validating new features in minor or major software updates, but also ensuring there has been no regression on existing features. While this plan is aimed squarely at companies using Android, the plan has been cleverly created so it can be applied to any device, even those using a different OS to Android, which is receiving a minor or major software update where continued confidence in the device behaviour is needed.

The IoTAS test solution covers two main aspects; Delta Testing and Regression Testing.

• Delta Testing; This not only covers testing any new features which are new to the baseline code, but can also include any bespoke changes the customer had newly added to their update, the plan is flexible.
• Regression Testing; This ensures that existing functionality and features have not been broken by the update.  The Regression aspect also checks that there are no issues with the actual update process (we can check all options a customer would potentially use) and that settings or files in the device storage, which are not expected to change, have been unaffected by the update.

IoTAS are currently offering detailed test plans for verification of:

• Android 4.4 (KitKat)
• Android 5 (Lollipop)
• Android 6 (Marshmallow)
• Android 7 (Nougat)
• Android 8 (Oreo)
• Android 9 (Pie)
• Android 10
• Android 11

Performance measurement

Reliability is a key metric for manufacturers and consumers. The reliability of the device and its associated software in the real world can be affected by many variables. Adequate field testing ensures the stability of the product under test by measuring performance of key parameters:

• in urban and rural environments, to maintain performance benchmarks
• in stationary locations, or mobile operation including drive test, that considers network handover and mixed infrastructure technologies
• in challenging situations such as high load, crowded population and limited network coverage.

Functional testing

Functionality is one the most important traits an end users wants from a device, there is little admiration for an all singing all dancing smartphone which crashes, cannot load applications correctly, has MMI issues or any number of problems an end user is expected to “live with”.

IoTAS has executed numerous test plans to cover Functionality and have found unique ways to report on issues found which makes sharing the issue with the customer much easier. IoTAS also keep a number of  devices from numerous manufacturers, and a number of “golden sample” devices, which allows IoTAS to explain how other device behave under the same

IoTAS have ran Functional test plans on phones (smart, feature and basic types), tablets, dongles (MMI install and use), smart TVs, set top boxes and an array of other consumer devices.

IoTAS are able to work with customers who have specific requirements or customers who need advice on where to start.

MVNO testing

IoTAS has experience with helping MVNOs test and benchmark devices sold on their networks.

MVNO testing differs slightly from standard live field testing as some results not only be compared to those of a reference also registered on the MVNO network, but should be also compared to a control device branded for the MVNOs host network.

Some areas IoTAS can help check are;

• Mobility
• Throughput
• MoS scoring for call audio

Please see our Open Market And MVNO Testing page.