Z-Wave Gateway Advanced Configurations, Log Files and Cloud Connectivity#
This lab exercise will explore how to change the default configuration of Z/IP Gateway, and study the log file. This exercise is the second exercise in the ‘Z-Wave controller training course’ series.
Finding version information
Finding network information about Z-Wave and IP networks
Extracting network keys
Analyzing runtime communications
Change configuration
Exploring deep packet inspection
Introduction#
In this exercise, the node information frame of Z/IP Gateway will be modified, and the consequences that this has, on "deep packet inspection" and filtering will be explored. Furthermore, Z/IP Gateway will be connected to a cloud hosted instance of Z-Ware and the zipgateway.log file will be studied to identify crucial debugging information.
Hardware Requirements#
2 UZB Controllers
1 Raspberry 3B (+)
1 SD card
1 IP Router with built in DHCP
1 WSTK Main Development Board
1 Z-Wave Radio Development Board: ZGM130S SiP Module
1 USB Zniffer
Software Requirements#
VNC and optionally SSH client https://www.realvnc.com/en/connect/download/viewer/ https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
Simplicity Studio v4
Z-Wave 7 SDK
Z-Wave PC Controller
Z-Wave Zniffer
Prerequisites#
Familiarize yourself with the Z-Wave protocol by referring to the following materials:
https://www.silabs.com/support/training/z-wave-700-series
Videos 1-3 and
exercise 6,7 and 10 will be relevant.
Further the participant should have completed
Controller training session 1
Controller training lab exercise 1
Controller training session 2
You should be familiar with connecting to the Raspberry Pi.
Finding Essential Debug Information in zipgateway.log#
Z/IP Gateways log file, zipgateway.log is one of the main sources for debugging information on all layers of the controller stack. Some of the most useful information available will be explored. The log file is by default cleared each time Z/IP Gateway is re-started.
Finding Version Information#
Open a prompt on the Raspberry Pi and open zipgateway.log using the following command:
$ less /tmp/zipgateway.log
q to quit.
Here the following entries regarding versions can be found:
First the version of Z/IP Gateway itself:
1m16888 Starting zipgateway ver7_13.01 build ver7_13.01And then information about the versions on the connected USB Z-Wave module:
32;1m16937 Version: Z-Wave 7.13, type 7
32;1m20309 700 series chip version 0 serial api version 8These numbers can be used to validate that the protocol version and Z/IP Gateway version are compatible. The release note for Z/IP Gateway always lists which protocol version it was qualified and tested with.
Finding Information about the Z-Wave Network#
The most essential information about the Z-Wave network is the home ID, furthermore it is practical to know Z/IP Gateways own node ID, along with the node IDs used for “virtual” nodes by Z/IP Gateway.
ESC[0mESC[34;1m20565 ZIP_Router_Reset: pan_lladdr: b5:b0:ea:f9:00:01 Home ID = 0xf9eab0b5
ESC[0mESC[34;1m1751818 nodeid=1 0x00
ESC[0mESC[32;1m1753617 temp_assoc_virtual_nodeids (count=4): 2 3 4 ... 5At this stage, extract the home ID from your own Gateway. If working in an environment with multiple Z-Wave networks, the home ID can be copied to the Zniffer to prevent seeing traffic from other networks:
Click the funnel icon in the “home” column, type in the home ID and click OK to enable filtering.
Extracting Network Keys#
The network keys are shown at the beginning of the log file:
ESC[0mESC[34;1m1751707 Key class 80
ESC[0m793D832D9525A1BF21703DDC7019B36A
ESC[33;1m1751713 sec0_set_key
ESC[0mESC[34;1m1751734 Key class 1
ESC[0mF5D22D437C62FAA3B05A3EC1C79EDB9D
ESC[34;1m1751749 Key class 2
ESC[0m19329CAF850FAAF74166B7A2528B8C31
ESC[34;1m1751768 Key class 4
ESC[0mA792EBDCBD5E0F231B2BAB4F6143D9A3They will change whenever Z/IP Gateway is reset or included in another network. In this case, a newer key may be listed later in the file. The easiest way to extract the network key is to use the grep utility:
$ grep -A 2 'Key' /tmp/zipgateway.log
1751707 Key class 80
793D832D9525A1BF21703DDC7019B36A
1751713 sec0_set_key
1751734 Key class 1
F5D22D437C62FAA3B05A3EC1C79EDB9D
1751749 Key class 2
19329CAF850FAAF74166B7A2528B8C31
1751768 Key class 4
A792EBDCBD5E0F231B2BAB4F6143D9A3
1751775 Network scheme is:1751775 S2 ACCESS
pi@raspberrypi:~ $The key set listed last is the most recent if multiple keysets are listed.
Include the switch on/off sample app in the network, if not already included, and start the Zniffer. To decrypt the trace, ensure that there is a nonce exchange in the trace, which is achieved by pressing the reset button on the developer board and then transmitting a frame to the Z-Wave node.
Finding Information about the IP Network#
Z/IP Gateway writes information to the log file about the current setup of the IP network.
IPv6#
While the IPv6 addresses are in many cases fixed through the setup in zipgateway.cfg, this is not always the case. If leaving them unset, in zipgateway.cfg, they will be automatically generated. Try pinging Z/IP Gateway:
$ ping fd00:aaaa::3This should result in an answer from Z/IP Gateway.
Next let us edit zipgateway.cfg:
$ sudo nano /usr/local/etc/zipgateway.cfgFind the entries for ZipLanIp6 and ZipPanIp6, and comment them by inserting a # in front of them, or change them to 0::0. Exit by pressing "ctrl+X" and chose "yes" to save.
Restart the gateway:
$ sudo /etc/init.d/zipgateway restartAnd try to ping it again on fd00:aaaa::3 Now this does not work, as Z/IP Gateway no longer has a fixed address.
To find the new address, inspect the log file:
$ less /tmp/zipgateway.logScrolling down a bit, reveals an entry looking like this:
ESC[32;1m750873 Lan address ESC[0mfdd1:3ca1:d800:01:21e:32ff:fe1e:eec
ESC[32;1m750873 Han address ESC[0mfdcf:d2d9:1152:02::
ESC[32;1m750873 Gateway address ESC[0mfd00:aaaa::1234
ESC[32;1m750873 Unsolicited address ESC[0mfd00:aaaa::b156:ad51:8fb:2032Try pinging the “Lan address”, in this case fdd1:3ca1:d800:01:21e:32ff:fe1e:eec. Be aware that this is an auto generated address that is individual for each gateway.
$ ping fdd1:3ca1:d800:01:21e:32ff:fe1e:eecZ/IP Gateway is now replying at this address.
IPv4#
For IPv4, the IP configuration is always dynamic. To check the IP configuration, check the log file. Open the logfile. Look for entries of the form:
ESC[0mESC[34;1m759092 got OFFER
ESC[0mESC[34;1m759092 send REQUEST
ESC[0mESC[34;1m759094 got ACK
ESC[0mESC[32;1m759094 Node 1 has IPv4 address 192.168.80.233
ESC[0mESC[34;1m759094 Checking for new sessions
ESC[0mESC[34;1m759094 We should send a discover
ESC[0mESC[32;1m759094 New IPv4 assigned for node 1There will be an entry for each node in the network. As seen earlier, Z/IP Gateways node ID is 1, thus this is the IP address of the Z/IP Gateway.
Try to ping it and see if it responds.
Seeing Runtime Communication#
In the previous exercise a Switch on/off sample app was included in the network, as node ID 6, here the end device will be exercised further.
From libzwaveip, the IP addresses of the Z-Wave devices can be found using the list command.
(ZIP) list
List of discovered Z/IP services:
"Static Controller [c8cc0ca8-001-000]" IP:fd00:aaaa::3
"Switch Binary [c8cc0ca8-006-000]" IP:fd00:bbbb::6
(ZIP)First, open a live updating instance of the gateway log file:
$ tail -f /tmp/zipgateway.log
Next, try pinging the IP address of node ID 6, fd00:bbbb::6 When looking at the log, entries like below can be seen:
251800 Sending long attempt
2251800 process_node_queues: uip_len:104
2251800 ClassicZIPNode_input: uip_len:104
2251800 Echo request for classic node 6
2251800 Sending 1->6, class: 0x0, cmd: 0x0, len: 1
2251800 Sending with scheme 255Traffic is being sent to node ID 6, which means that an ICMP ping is translated by Z/IP Gateway to a Z-Wave “nop” frame.
Send a command that controls the end device. To do so, use the send command in libzwaveip.
(ZIP) send "Switch Binary [c8cc0ca8-006-000]" COMMAND_CLASS_BASIC BASIC_SET FFWhile libzwaveip does not do tab completion on command classes that are not listed in either the NIF, or secure command supported report, it can still be typed in manually.
libzwaveip tells whether the command was delivered:
(ZIP) Transmit OK
(ZIP)And this output is shown in the log file:
2566216 Sending long attempt
2566216 process_node_queues: uip_len:61
2566216 ClassicZIPNode_input: uip_len:61
2566216 ClassicZIPUDP_input len: 13 secure: 1
fdd1:3ca1:d800:01:7415:87ff:fe38:24c6
2566217 temp_assoc_lookup_by_zip_client
2566217 temp_assoc_lookup_by_zip_client() found: 0x7d716c
2566217 Sending 2->6, class: 0x20, cmd: 0x1, len: 3
2566217 Sending with scheme 2
2566217 received get inside multi cmd encap
2566217 S2_fsm_post_event event: SEND_MSG, state IDLE
2566217 Sending S2_send_frame 15
2566241 S2_fsm_post_event event: SEND_DONE, state SENDING_MSG
2566242 s2incl process_event event: S2_INCLUSION_SEND_DONE state: S2_INC_IDLE
2566242 send_using_temp_assoc_callback_ex for node 6 status 0
2566242 queue_send_done to node 6 queue 2 status: 0Here both the “class”, 0x20, and the “cmd”, 0x01 are seen, and if checked against the command class specification or classcmd.h, they match the command typed in in libzwaveip! The entry “2->6” can also be seen, because Z/IP Gateway is using one of its virtual nodes, node 2, to send to node 6.
Next step is to send something that triggers incoming traffic as well. Send a “basic get”. This triggers this response:
2739937 ClassicZIPNode_dec_input: pkt: 23 2 len: 12
2739938 Sending long attempt
2739938 process_node_queues: uip_len:60
2739938 ClassicZIPNode_input: uip_len:60
2739938 ClassicZIPUDP_input len: 12 secure: 1
fdd1:3ca1:d800:01:7415:87ff:fe38:24c6
2739938 temp_assoc_lookup_by_zip_client
2739938 temp_assoc_lookup_by_zip_client() found: 0x7d716c
2739938 Sending 2->6, class: 0x20, cmd: 0x2, len: 2
2739938 Sending with scheme 2
2739938 S2_fsm_post_event event: SEND_MSG, state IDLE
2739938 S2_set_timeout interval =65000 ms
2739939 Sending S2_send_frame 14
2739963 S2_fsm_post_event event: SEND_DONE, state VERIFYING_DELIVERY
2739963 S2_set_timeout interval =274 ms
2739973 ApplicationCommandHandler 6->2 class 0x9f cmd 0x03 size 17
2739973 S2_fsm_post_event event: GOT_ENC_MSG, state VERIFYING_DELIVERY
2739973 s2incl process_event event: S2_INCLUSION_SEND_DONE state: S2_INC_IDLE
2739973 send_using_temp_assoc_callback_ex for node 6 status 0
2739974 queue_send_done to node 6 queue 2 status: 0
2739974 ApplicationCommandHandler 6->2 class 0x20 cmd 0x03 size 5
2739975 bridge_virtual_node_commandhandler Handled
2739978 Sending long attempt
2739978 process_node_queues: uip_len:65
2739978 ClassicZIPNode_input: uip_len:65
2739978 DeMangled HAN node 6 Virtual Node 2
2739978 Packet from Z-Wave side (nodeid: 6) to port: 51607 IP addr: fdd1:3ca1:d800:01:7415:87ff:fe38:24c6
2739979 queue_send_done to node 6 queue 2 status: 0The initial transmission proceeds as in the previous example, however from the entry “ApplicationCommandHandler” a frame is returned from node ID 6 to node ID 2. This frame is seen in multiple stages as the encapsulation layers are handled: First as class 0x9f cmd 0x03 size 17, an encrypted S2 frame. After decryption, as class 0x20 cmd 0x03 size 5, a basic report. This frame is forwarded over IP to libzwaveip, and shown on the screen:
cmd_class: COMMAND_CLASS_BASIC v2
cmd: BASIC_REPORT
param: Current Value 0xFF
param: Target Value 0xFF
param: Duration 0x00
bytestream: 20 03 ff ff 00
(ZIP)This allows for the following frames going in and out of Z/IP Gateway to Z-Wave nodes and see the encapsulation layers used.
Modifying the Node Information Frame#
The node information frame is essential as it is used to expose the command classes supported by the controller. Without adding a command class to the node information frame, the other nodes in the network will not know that the controller offers support of a command classes.
Z/IP Clients will typically modify the node information frame and secure command supported report remotely through the Z/IP Gateway command class, while it can also be modified locally by editing the file zipgateway.cfg. Open zipgateway.cfg to stydy its default contents:
$ sudo nano /usr/local/etc/zipgateway.cfgBy default, “ExtraClasses” is commented, hence nothing is added to the node information frame. This can be changed using a Z/IP client, such as libzwaveip. In libzwaveip, use the send command to transmit to Z/IP Gateway itself, COMMAND_CLASS_ZIP_GATEWAY, COMMAND_APPLICATION_NODE_INFO_SET, payload 32
(ZIP) send "Static Controller [c8cc0ca8-001-000]" COMMAND_CLASS_ZIP_GATEWAY COMMAND_APPLICATION_NODE_INFO_SET 32
(ZIP) Transmit OK
(ZIP)Reopen zipgateway.cfg and find the extra classes entry. Z/IP Gateway has created a new entry at the end of the file, which looks like this:
ExtraClasses = 0x32 0x00 0xF1000x32 is now listed as supported.
To test this, include Z/IP Gateway into an existing network, a “PC Controller”, to see the command classes listed.
When including Z/IP Gateway in an existing network, it is important that there is a SIS present. To set the PC Controller up as SIS, go to the “Network management“ tab. Select the PC Controller on the lefthand pane, and click the “Set as SIS” button. Finally, verify that the PC Controller is now “SIS”:
On the Raspberry Pi, have the Z/IP Gateway log running, as PyZIP does not show the S2 DSK by default:
$ tail -f /tmp/zipgateway.logOn the PC Controller click “Add” and in libzwaveip enter “learn mode”:
(ZIP) learnmode
(ZIP) Transmit OK
(ZIP)Once prompted, grant Z/IP Gateway all keys:
From the logfile take the first 5 digits of the DSK and enter them on the PC controller when prompted.
ECDH Public key is
25875-64173-06769-30600-
03879-01638-56419-31396-
50609-25355-54523-00870-
38142-11347-16713-16247-After inclusion, select Z/IP Gateway, node 2, on the top right pane of the PC Controller, and inspect the supported command classes on the bottom right pane:
Here the "meter command class" is now listed as supported.
Working with Deep Packet Inspection#
Z/IP Gateway performs deep packet on incoming frames to see if they are valid and should be accepted before forwarding them to the Z/IP Client.
To understand this, try to send some different commands from the PC Controller to Z/IP Gateway. In the PC Controller, select node 2 and double click “meter command class” The window is by default populated with the “meter get” selected:
Click “Send” and observe the gateway log:
4899142 ApplicationCommandHandler 1->2 class 0x9f cmd 0x03 size 16
4899142 S2_fsm_post_event event: GOT_ENC_MSG, state IDLE
4899143 ApplicationCommandHandler 1->2 class 0x32 cmd 0x01 size 4
4899143 nm_fsm_post_event event: NM_EV_FRAME_RECEIVED state: NM_IDLE
4899143 Cmd class 0x32 command 0x01 identified as version 5
4899143 Command class 0x32 : 0x01 not supported
4899143 Unhandled command 0x32:0x01 from fda3:9838:93fe:02::01
4899143
4899143 Check security for rnode:1 scheme 03 class 32 command 01
4899143 Sending Unsolicited to IP app...In this case, the command ends up being forwarded to the unsolicited destination.
Try to send a command that does not require “support” by the destination, but “control”, select meter report from the “command” drop down, and click send:
The log output is mostly identical:
5514344 ApplicationCommandHandler 1->2 class 0x9f cmd 0x03 size 19
5514345 S2_fsm_post_event event: GOT_ENC_MSG, state IDLE
5514345 ApplicationCommandHandler 1->2 class 0x32 cmd 0x02 size 7
5514345 nm_fsm_post_event event: NM_EV_FRAME_RECEIVED state: NM_IDLE
5514345 Cmd class 0x32 command 0x02 identified as version 5
5514345 Command class 0x32 : 0x02 not supported
5514345 Unhandled command 0x32:0x02 from fda3:9838:93fe:02::01
5514345
5514345 Check security for rnode:1 scheme 03 class 32 command 02
5514345 Sending Unsolicited to IP app...Try to lower the security level by selecting the radio button “non-secure”:
5558013 ApplicationCommandHandler 1->2 class 0x32 cmd 0x02 size 7
5558013 nm_fsm_post_event event: NM_EV_FRAME_RECEIVED state: NM_IDLE
5558013 Cmd class 0x32 command 0x02 identified as version 5
5558013 Command class 0x32 : 0x02 not supported
5558013 Unhandled command 0x32:0x02 from fda3:9838:93fe:02::01
5558014
5558014 Check security for rnode:1 scheme ff class 32 command 02
5558014 Frame not forwarded to unsolicited because it was not sent on right scheme or not supported.Because the frame was sent at a lower security level than the source nodes highest key class, the frame is discarded.
Conclusion#
At this point, you should be familiarized with the Raspberry Pi platform for running Z/IP Gateway, Z/IP Gateway itself, and the two Z/IP Clients, PyZIP and PC Controller. The Raspberry Pi is the reference platform for the controller stack, and Z/IP Gateway the core of the controller stack. This setup will be used in the coming exercises.