Testing and Results#

Test Applications#

Full details about the applications used during testing can be found in Detailed Test Setup.

Throughput and Latency#

The throughput and latency is tested in a controlled network (wired configuration) to test each hop against different packet payloads.

The normal configuration is to test to 7 hops. Testing is done with one source node and a series of routing nodes to allow the number of hops to be varied.

This testing is done using the following configuration:

1. Each device is contained in a Ramsey isolation box for a controlled scenario where each device can only hear its neighbor

2. Zigbee application layer messages

3. Packet payload from 8 bytes to up to 300 bytes in 8 byte increments for the latency test

4. Testing is done with security on

5. From 1 to 7 hops

6. Using 1 packet in flight

7. Sending as fast as possible given ack timing after roundtrip payload returned

8. Measuring round trip latency (source to destination to source) in milliseconds

Zigbee Multi-hop Latency#

The graph belows shows Zigbee average round trip latency with APS ACK.

The graph belows shows another test run with APS acknowledgement disabled.

As this testbed is conducted in an isolated environment where only each neighbor can hear each other, we decided to run the same test but with all devices configured as a High RAM concentrator vs a typical network configuration with just the ZC as the concentrator. There is no noticeable difference when comparing the results, as shown below.

Network Tests versus Network Size#

Open-air large network testing is required to validate stack performance under less controlled conditions. These networks are config- ured within normal Silicon Labs office space with normal Wi-Fi interference, other network operations, and building control systems. No attempt is made to isolate these network RF conditions.

The networks to be tested for each stack include:

• Small network: 25 devices

• Medium network: 1 – 50 devices

• Medium network: 2 – 100 devices

• Large network: 1 – 150 devices

• Large network: 2 – 200 devices

Note: For any of these tests, the specific number of devices is acceptable within +/- 10% of these test network targets for a given set of testing. Testing in this large network is done in SoC mode for devices.

These networks are all configured as powered devices unless there is specific testing for sleeping end devices.

For each of these networks, the testing will validate reliability and latency for a set of traffic conditions. Testing is intended to be done over 1000 messages. The same devices were used across the tests to keep the topology and density of the different test runs similar. The actual over-the-air conditions will vary and cannot be controlled in these tests.

Zigbee Large Network Testing Results#

Zigbee testing was done with the latest version of the Zigbee stack. Zigbee network testing was done with 8, 16, 32, and 64 byte payloads with a normal broadcast transmission interval of 1 seconds. The histograms of results are shown below.

Zigbee Network Testing with 25 devices

Zigbee Network Testing with 50 devices

Zigbee Network Testing with 100 devices

Zigbee Network Testing with 150 devices

Zigbee Network Testing with 200 devices

The following should be noted from this testing:

• As packet payload increases, latency to the devices increases. This is expected and normal behavior as it takes more time to transmit larger packets.

• We see 100% reliability in all of these tests. Note these are 1000 broadcasts tests conducted at a 1 second transmit delay for ~ 17 minutes of active traffic testing per payload size scenario.

• As the network size grows, we see an increase and spreading out of latency as it takes multiple hops to deliver all the messages. Larger networks also have more contention over the air as all devices are trying to repeat the message.