Basic Z/IP Gateway#

What is Z/IP Gateway#

The Z-Wave over IP Gateway (Z/IP Gateway) is a device providing a layer above the serial API, and the Z-Wave protocol running on the Z-Wave module. It allows internet "Z/IP clients" connected over the Internet Protocol (IP) to contact and control nodes in a Z-Wave network.

Z/IP Gateway bridges IP UDP frames to Z-Wave, enabling communication between these two technologies. The number of Z-Wave node IDs in a network is a limited resource, with only 232 available. Z/IP Gateway is usually used to represent Z-Wave nodes as IP hosts, and because the IPv4/6 address space is larger than the Z-Wave Node ID space, it can represent all Z-Wave nodes as IP hosts. Additionally, the Gateway also represents IP hosts as Z-Wave nodes but due to ID limitations, it cannot represent every IP host as a Z-Wave node.

Z/IP Gateway key features:

Enables communication between the two technologies by
- Representing Z-Wave nodes as IP hosts
- Representing IP hosts as Z-Wave nodes

IPv6 Environment#

Z/IP Gateway works in both IPv6 and IPv4 environments. The recommendation is to use IPv6, as IPv4 is subject to certain limitations. In an IPv6 environment, the Z/IP Gateway has a preset IPv6 address and creates a separate IPv6 subnet for the Z-Wave nodes with a preconfigured IPv6 prefix.

IPv6 PAN and LAN Connected through Z/IP Gateway

On the left-hand side, a LAN with a Z/IP Gateway-shared prefix is shown. This allows all IP hosts on the LAN to reach Z/IP Gateway, which acts as an IP router for the Z-Wave nodes on the PAN side.

To configure the IP hosts on the network, Z/IP Gateway will, in an IPv6 environment, transmit router advertisements for itself as a router telling other hosts on the IP network within the same broadcast domain, that it provides access to the Z-Wave subnet. If these hosts accept these router advertisements, they will autoconfigure an IPv6 address that has the same prefix as Z/IP Gateway, allowing them to communicate directly with Z/IP Gateway itself.

The router advertisements will also configure a route on the hosts, telling that the PAN subnet, typically fd00:bbbb::/64, is reachable through Z/IP Gateway, allowing the hosts to communicate with the Z-Wave nodes through Z/IP Gateway.

Z/IP Gateway
- Has a preset IPv6 address
- Creates a separate IPv6 subnet for the Z-Wave nodes
- Does router advertisements for itself as a router, providing access to the Z-Wave subnet
Other IP hosts in the same layer 2 broadcast domain
- Receive router advertisements
  - If accepting router advertisements
  - Configure themselves with an IPv6 address having the same prefix as Z/IP Gateways preset address
    - Configure themselves with a route to the Z-Wave subnet through Z/IP Gateway

IPv4 Environment#

In comparison to IPv6, IPv4 does not have router advertisements. This prevents Z/IP Gateway from automatically notifying an IPv4 the Z-Wave subnet is available through a the Z/IP Gateway. Instead, Z/IP Gateway requests a DHCP lease from the DHCP server on the network on behalf of each Z-Wave node in the Z-Wave network. This places all nodes on the Z-Wave network in the same IPv4 broadcast domain and subnet as the gateway itself and any other host on the same IP network. The problem with this approach is that most consumer routers are configured with a small pool of DHCP addresses. Typically, they are configured to less than 100 hosts, and some may even not allow the user to reconfigure them with a larger pool. In a normal home network, 20-40 hosts may already be present so adding the Z-Wave nodes on top of this, may cause the local DHCP server to run out of IPv4 addresses, causing Z-Wave nodes or IP hosts to go offline. For this reason, IPv6 is recommended, especially when bridging to larger networks or network segments beyond the physical gateway device.

Announcements of routers are not possible
IPv4 addresses for the Gateway and every Z-Wave node is requested from the DHCP server on the network
- The DHCP server must have a sufficiently large pool of addresses (consumer routers typical default configuration is less than 100 hosts)
The entire network (IP hosts and Z-Wave nodes) are in the same IPv4 subnet

Z/IP Encapsulation#

When Z-Wave traffic is relayed over IP, some concepts that are essential to Z-Wave, do not have an equivalent on IP:

Multichannel
Security
Handling of sleeping Z-Wave devices

To convey this information, the Z/IP encapsulation layer is used.

Z/IP encapsulation is a command class like everything else in Z-Wave. It starts with a command class identifier, 0x23, and a command identifier, 0x02; ZIP_PACKET. It has 2 leading bytes carrying “options”:

“ACK Request”, indicates to the destination that it needs to acknowledge the frame.
“ACK response” indicates that a frame is an ACK to another frame. If a Z/IP client sends a frame to a Z-Wave node, it might set the ACK request” flag, and include a Z-Wave payload. Z/IP Gateway will forward the frame to the Z-Wave destination. If the destination ACKs on the RF interface, the Z/IP Gateway will return a Z/IP packet with the “ACK response” flag set.
“NACK response” indicates that the frame was not delivered by the Z/IP Gateway. Additional flags that may appear in a NACK response are described later.
“Header extension” indicates that a header extension is included. The use of header extensions is described later.
“Z-Wave payload included” is used on most commands sent. It denotes that Z-Wave non-ACK frame payloads are included in the packet.
“More information flag” relates to sleeping nodes and is described later.
“Secure origin” indicates if the source of the frame can be considered secure, and if it was a Z-Wave node, whether it was using encryption.

The option bytes are followed by 3 byte fields:

“Sequence number” - which relates incoming ACKs to previously sent frames with ACK requests” so the Z/IP Client knows which frame an ACK response belongs to.
"Multichannel" - where source and destination endpoints can be specified. If either endpoint is nonzero, the Z/IP Gateway automatically employs multichannel encapsulation to the PAN destination. Likewise, if Z/IP Gateway receives a frame from the Z-Wave PAN using multichannel encapsulation, it will strip the Multichannel encapsulation layer and set the multichannel information in the source and endpoint fields of the Multichannel byte. There is optionally space for a header extension, and finally optionally space for a Z-Wave payload.

Resource Directory#

Z/IP Gateway maintains a resource directory, which lists the resources on the Z-Wave network. When a Z-Wave node is included into the network, the Z/IP Gateway will probe (or 'interview') the node for information about its capabilities. This process is similar to the interview process of the PC Controller but Z/IP Gateway probes more, and deeper. It probes for the following data types:

Role type
Device type
Multichannel endpoints.
Command classes supported by the root device, and all endpoints.
The versions for these command classes.
The security levels supported by the device.
If the device is a sleeping device, what the supported wakeup interval is.

This information is made available through mDNS and network management proxy command class. However, there are slight differences in which information is available where.

Node Discovery from Z/IP Client#

A Z/IP client can discover the gateway either through multicasting a COMMAND_NODE_INFO_CACHED_GET, as An IPv4 broadcast, or through an IPv6 'all routers multicast' packet. In either case, this command is sent without any encryption. All Z/IP Gateway(s) receiving a 'Node Info Cached Get' will respond with COMMAND_NODE_INFO_CACHED_REPORT enabling the Z/IP client to enumerate the gateways available on the IP network.

An alternative method is listening for mDNS, which allows Z/IP Gateways to broadcast their presence to the network.

After the gateway has been discovered, all the Z-Wave nodes connected to Z/IP Gateway can be discovered. As with gateway discovery, nodes on a network can be discovered through command classes or mDNS. A Z/IP client issuing a COMMAND_NODE_LIST_GET to the Z/IP Gateway causes it to respond with a COMMAND_NODE_LIST_REPORT, which infora the client of all nodes in the Z-Wave network. For each node, the Z/IP client can then request the command classes discovered and cached by Z/IP Gateway during the interview process. If the Z-Wave node supports multichannel, the Z/IP client can also request the properties of each multichannel endpoint.

As an alternative to command class node discovery, a Z/IP client can use a mDNS listener. Z/IP Gateway will announce all nodes and multichannel endpoints in the network on mDNS which a client can capture using this listener.

All information is cached by Z/IP Gateway in the resource directory, so even if a node is offline or sleeping, the Z/IP Gateway can still provide cached information to any Z/IP Client sending a COMMAND_NODE_INFO_CACHED_GET.

Gateway discovery
- Multicast
  - COMMAND_NODE_INFO_CACHED_GET IPv4 broadcast or an IPv6 ‘all routers’ multicast packet.
- MDNS advertisements for the Z/IP Gateway
Node discovery
- Command class based, using network management proxy command class
  - Request node list from gateway COMMAND_NODE_LIST_GET
  - Request node info cached from each Z-Wave node COMMAND_NODE_INFO_CACHED_GET
  - Request capabilities of each endpoint NM_MULTI_CHANNEL_END_POINT_GET
- MDNS based
  - Using an MDNS listener

Node Discovery Using mDNS#

Avahi browse can be used to display mDNS traffic. In the example below, it is used to browse for _zwave._udp, adding the r flag to resolve addresses, and the v for verbose. In this example, a number of Z-Wave devices are available on the network.

$ avahi-browse _z-wave._udp –rv
Server version: avahi 0.6.32; Host name: raspberrypiW.local
E Ifce Prot Name                                          Type                 Domain
+ br-lan IPv6 Static Controller [d24ac4700200]              _z-wave._udp         local
+ br-lan IPv6 Static Controller [d24ac4700100]              _z-wave._udp         local
+ br-lan IPv4 Static Controller [d24ac4700200]              _z-wave._udp         local
+ br-lan IPv4 Static Controller [d24ac4700100]              _z-wave._udp         local
= br-lan IPv6 Static Controller [d24ac4700200]              _z-wave._udp         local
  hostname = [zwD24AC47002.local]
  address = [192.168.0.3]   port = [41230]
  txt = ["icon=" "txtvers=1" "securityClasses=�" "productID=" "mode=" "epid=" "info="#VX^lt��"]
= br-lan IPv6 Static Controller [d24ac4700100]              _z-wave._udp         local
  hostname = [zwD24AC47001.local]
  address = [192.168.0.4]
  port = [41230]
  txt = ["icon=" "txtvers=1" "securityClasses=�" "productID=" "mode=" "epid=" "info=^Vt�h#"]
= br-lan IPv4 Static Controller [d24ac4700200]              _z-wave._udp         local
  hostname = [zwD24AC47002.local]
  address = [192.168.0.3]
  port = [41230]
  txt = ["icon=" "txtvers=1" "securityClasses=�" "productID=" "mode=" "epid=" "info="#VX^lt��"]
= br-lan IPv4 Static Controller [d24ac4700100]              _z-wave._udp         local
  hostname = [zwD24AC47001.local]
  address = [192.168.0.4]
  port = [41230]
  txt = ["icon=" "txtvers=1" "securityClasses=�" "productID=" "mode=" "epid=" "info=^Vt�h#"]
: Cache exhausted
: All for now

The Z-Wave devices are announced on the IP network using both IPv6 and IPv4. The host name, a hex string in brackets, is composed of the home ID, the node ID and the multichannel endpoint. From this example, it can be seen that:

All Z-Wave nodes are on the same home ID, d24ac470.
There are two nodes, node 1 and node 2.
No nodes are multichannel, because the only endpoints seen are zero.
Both nodes are controllers. Z/IP Gateway also announces the command class list and security classes. Because this data is provided as hex values, output will appear garbled under avahi browse, which prints this information as ascii characters.

Session Handling (Temporary IP Associations)#

Z/IP Gateway allows for handling multiple sessions using what is called temporary IP associations. In this figure, a setup is seen with multiple IP hosts communicating with the same Z-Wave node at the same time. In the absence of temporary IP associations, the Z/IP Gateway could not determine which IP to which each response should be forwarded

Temporary IP Association works as-follows:

Starting from the top-left device; a smartphone or tablet transmits a frame destined for the Z-wave device on the right, through Z/IP Gateway, using the read arrow. Z/IP Gateway then allocates a virtual node to the source IP host. In this case virtual node ID 2. This virtual node is used as a source address for the Z/IP client on the Z-Wave network, when it communicates with the Z-Wave device on the right.
A second IP host, the middle device, with the green arrow, is also communicating with the same Z-Wave device. Z/IP Gateway allocates the second device another new virtual node, using node ID 3 to represent this IP host in the Z-Wave network.
Finally, a third device, using the blue arrow, transmits. Again, a new temporary IP association is allocated, using node ID 4.

This allows Z/IP Gateway to separate traffic returned from the Z-Wave node, and forward it to the correct IP destination based on the virtual node to which return traffic is addressed. The temporary IP association is bound to the source socket, and the destination node ID. This means that the virtual node IDs change when the IP socket used by the Z/IP client changes. Depending on the client, this may occasionally occur. After an amount of time with no communication, temporary address associations time out and are freed for reuse.

(Persistent) IP Associations#

The other type of IP associations are persistent IP associations. The common example of this is the “unsolicited destination”. Most Z-Wave nodes transmit frames, without them having been explicitly requested. An example of this is a motion sensor that transmits when it detects motion. It will send a notification report on the lifeline associations group without the controller having explicitly requested this report. A client that desires to monitor unsolicited reports must first configure the Z/IP Gateway with an unsolicited destination address, which is used as the destination address for these frames.

Persistent IP associations can also be configured for specific Z-Wave devices to specific IP destinations through the concept of IP associations, that makes a permanent mapping of a virtual node to a specific IP host. If configuring an IP association, Z/IP Gateway will allocate an additional virtual node, and use the virtual node specifically to associate a specific IP destination. If the right hand Z-Wave devices sends a report to the bottom virtual node, it will always be forwarded to the specific IP destination.

All frames addressed to the gateway's own node ID are forwarded to the ”unsolicited IP destination”. This will typically be ”lifeline reporting”.
IP associations are used to make permanent mapping of virtual nodes to specific IP hosts that are different from the unsolicited destination

IMA (Installation and Maintenance)#

Z/IP Gateway maintains data about network performance. In Z-Wave terminology, this is typically referred to as “IMA” for Installation and MAintenance. Z/IP Gateway allows for collecting and capturing:

RSSI levels for any in- and outgoing frames.
Routes used in the network; which repeaters was used to reach a specific destination.
Transmission speed.
Route changes.
Background RSSI. This data can be used by a Z/IP client, for example, for jamming detection, or with Z-Ware to monitor the health of the Z-Wave network.

Security – Z-Wave S2#

Z/IP Gateway supports encryption on the Z-Wave RF interface through the command classes security 0 and security 2.

When a frame arrives at Z/IP Gateway, before it is forwarded to its final destination, Z/IP Gateway will, if the destination supports either S0 or S2, apply encryption automatically. If the destination supports S2, the Z/IP Gateway automatically applies the highest key class supported by the destination node. This can be overridden by the Z/IP client for specific purposes.

In the same fashion, when Z/IP Gateway receives frames from the Z-Wave network, it strips off the S2 or S0 encapsulation, and provides this information to Z/IP clients via a header extension that describes which key class was used. The Z/IP client can then tell whether the frame was sent on the correct key class for that device.

Z/IP Gateway implements S2.
- For inclusion Z/IP Gateway generates onetime use DSKs
- For learn mode, Z/IP Gateway has a persistent DSK
- Z/IP Gateway uses highest key class supported by destination node
  - Unless otherwise specified
- Z/IP Gateway reports key class used when receiving frames

Security – IP on with Local Connectivity#

On the IP side, frames are encrypted with DTLS rather than S0 or S2 encapsulation present on the Z-Wave network. The traffic is secured using a preshared key, which is made known to Z/IP Gateway and the Z/IP client out of band.

The option of using no security on the IP layer also exists. This is typically intended for applications like embedded Linux controllers, where the Z/IP traffic is generated locally (for example, through a webserver running on the gateway) on the internal logical network interface such that no Z/IP traffic exits the device. For obvious reasons, using an unsecured IP layer is not recommended for cases were the Z/IP Client exists off of the Z/IP Gateway's internal network.

Non-secure:
- Z/IP frames are sent without any encryption at all. Not recommended on any kind of public networks.
Secure:
- Relies on DTLS with a pre shared key, made known to Z/IP Gateway and clients out of band.

Security – IP Globally#

Z/IP Gateway supports connections to a cloud hosted Z/IP client. In this case, IPv6 is sent TLS encapsulated through a tunnel to the cloud instance. The connection uses mutual authentication through RSA certificates allowing the cloud hosted Z/IP client to verify that it is an authentic Z/IP Gateway controller and vice-versa, allowing Z/IP Gateway to authenticate that it is a trusted cloud service.

IPv6 sent through TLS tunnel
Mutual authentication through RSA certificates

Network Configurations#

Network Configuration 1 – bridge - wired#

The outline box that contains the other components represents the 'host' hardware platform, for example, a Raspberry Pi. This setup is the default configuration for Z/IP Gateway, which allows the host itself, as well as any devices connected to the LAN to communicate with Z/IP Gateway. This is achieved by bridging the Ethernet interface with the logical TAP interface created by the Z/IP Gateway. In this configuration, the Ethernet interface has no IP addresses, while the br-lan bridge interface has both an IPv4 address and an IPv6. The bridge interface serves as a layer 2 device like an Ethernet switch. This configuration is supported by the provided Debian package installer.

This installation the most widely used in development because it is incredibly useful for debugging purposes. It allows for Z/IP clients running inside or outside the host platform as needed, and it provides good options for capturing IP traffic using TCPdump or Wireshark.

ETH0 has no IP addresses
TAP0 has no IP addresses
BR LAN
- Relays layer 2 traffic between ETH0 and TAP0, effectively connecting Z/IP Gateway directly to the LAN, and and exposing it directly to external hosts
- IPv6 with same prefix as Z/IP Gateway
- IPv4 DHCP assigned from external DHCP server
Supported by released Debian package installer

Network Configuration 2 – udp relay - wireless#

The second network configuration allows for wireless networking. Additional configuration is required as a wireless network interfaces cannot be added to a network bridge. A wireless network interface provides connection to the IP network, and the logical TAP interface provides an interface to the Z-Wave mesh through the Z/IP Gateway. These two interfaces are assigned the same IPv4 and IPv6 addresses by the Z/IP Gateway tunnel script. The system relies on a daemon, called UDP relay, that carries traffic back and forth between the wireless network interface and the TAP interface, bridging layer 4 traffic between these two interfaces. This configuration is supported by the provided Debian package installer.

Wireless network interface and TAP has same IPv4 and IPv6 addresses.
Traffic is transferred between the two interfaces based on source or destination ports, only transferring Z/IP and DHCP traffic between the interfaces.
Supported by released Debian package installer

Network Configuration 3 – local host only#

Although it is not supported by the Debian package installer, the third configuration is the most common in finalized products. In this configuration, the Z/IP Gateway has no connectivity to the outside world. Instead, it connects to the logical TAP interface, which has an IPv6 with the same prefix as Z/IP Gateway. Optionally, there may be a DHCP server on this host that provides IPv4 addresses for Z/IP Gateway and the Z-Wave nodes. The main application for this configuration is a system where the Z/IP client is running on the same host as Z/IP Gateway. An example is Z-Ware, or another webserver, running on the same host as Z/IP Gateway. In this case, outside communication passes through the web UI and Z/IP traffic to other (remote) clients is not present.

The Gateway has no external connectivity
- TAP0
  - Has an IPv6 address with same prefix as Z/IP Gateway
  - Is optionally connected to an internal DHCP service if relying on IPv4.
- Not supported by released Debian package installer

Documentation - Doxygen#

Z/IP Gateway ships with comprehensive documentation written in Doxygen. When opening the documentation, note the left-hand pane where there is a Z/IP Gateway user guide that has information on compiling, installing, and troubleshooting the Z/IP Gateway; specific details and procedures regarding firmware update, migration from previous versions, RF settings for the Z-Wave module; and more.

Included with Z/IP Gateway
- Details:
  - Building
  - Installation
  - usage

Installation#

Specify Z-Wave Controller#

Prior to installing the Z/IP Gateway, the location of the Z-Wave module on the host is required. To find the Z-Wave device, run the following command after connecting the module to the host:

$ dmesg | grep tty

The Z-Wave controller will typically be the last entry, often /dev/ttyUSB0 or /dev/ttyACM0

Installation – Specify IPv6 Address#

The installer will prompt the user for the IPv6 addresses of Z/IP Gateway. The recommendation is to use default addresses unless on a network with multiple Z/IP Gateways, leading to multiple hosts with the same address.

The same goes for the Z-Wave PAN prefix; run with the default unless multiple Z/IP Gateways on the same network. Setting the IPv6 addresses 0::0, will cause them to be autogenerated, preventing collisions but making it harder to identify the Z/IP Gateway.

There are no specific limitations on the IPv6
If running multiple instances on Z/IP Gateway on the same network segment make sure the addresses are unique!

The same limitations apply in that the PAN subnet must be unique on the LAN segment, as Z/IP Gateway will do router advertisement.

Network Interface Type Selection#

Select whether a wired or wireless connection is used. - If using a wired, the connection will be bridged, providing full layer 2 connectivity. - If using wireless, layer 4 frames between the interfaces will be relayed by UDP relay.

Network Interface Selection#

When selecting the network interface, ensure that it matches the type selected in the previous step.

Z/IP Clients#

PyZIP#

PyZIP is a lightweight multiplatform Z/IP client written in Python. It is fairly limited in terms of command classes supported and is manually driven with no automation. PyZIP is also not Z-Wave certified. However, due to its lack of automation, PyZIP is a very good tool for manual testing.

Provided with Z/IP Gateway
Multiplatform, runs on the Linux OS
Lightweight, only does exactly what instructed by the user
Does not support all features
Not Z-Wave certified

PC Controller#

The PC Controller is a Windows OS application is more fully-featured than PyZIP. It provides much of the mandatory functionality that required to have a working Z-Wave network:

Configuration of:
- wakeup destinations,
- lifeline associations
- etc.

This means that it is far more advanced than PyZIP, but it only runs on Windows. The PC Controller is also intended as a test tool, so it allows for actions that are not allowed within the Z-Wave spec, and is thus not certifiable.

Windows OS-only application
Handles mandatory configuration like configuring wakeup destinations, lifeline associations etc.
Not Z-Wave certified
Offers more control and support than PyZIP

Z-Ware#

Z-Ware is both an SDK and a finalized controller, with many APIs at different levels. Two of these should be emphasized:

The Web portal, which is a fully certified controller, that does everything required by a Z-Wave controller. This also means that it is fairly heavyweight and does a lot of things behind the users back. For testing purposes, when careful manual control is desired, Z-Ware may unwieldy.
The C-API, and the related sample applications, which are intended to demonstrate the C-API. The Z-Wave Controller sample demonstrates the controller API along with a range of sample applications. Each of these sample apps provides a working example of the C-API and enables a developer to eventually write a fully certifiable controller.
Web portal
- Fully Z-Wave certified
- Heavy weight implementation, does everything required by a controller
Sample apps
- Does very little
- inflexible