Creating and Accessing Tokens#

Now that you understand what tokens are and a few ways they may be categorized depending on how you view them, you can learn how to use tokens. This includes knowing where to find default tokens, how to create new tokens, and how to read and potentially modify tokens. Keep in mind that methods may vary depending on the type of tokens involved.

Token Header Files#

A token header file is simply a .h file that contains token definitions. Manufacturing tokens and dynamic tokens have separate token header files. There may be more than one header file in an application for dynamic tokens: one for stack tokens, variable number of application framework tokens, and possibly one for custom application tokens.

Creating Dynamic Tokens#

Adding a dynamic token to the header file involves three steps:

Define the token name.
Add any typedef needed for the token, if it is using an application-defined type.
Define the token storage.

The rest of this section looks at each of these steps one at a time.

Define the Token Name#

When defining the name, do not prepend the word TOKEN. For SimEEv1/v2 dynamic tokens, use the word CREATOR:

/**
* Custom Application Tokens
*/
// Define token names here
#define CREATOR_DEVICE_INSTALL_DATA (0x000A)
#define CREATOR_HOURLY_TEMPERATURES (0x000B)
#define CREATOR_LIFETIME_HEAT_CYCLES (0x000C)

For NVM3 dynamic tokens, use the word NVM3KEY. Note that the example below assumes a Zigbee application. For a different stack, the NVM3 domain would be different. Note also that the NVM3KEY value for HOURLY_TEMPERATURES is set to a value where the subsequent 0x7F values are unused because this is an indexed token. Refer to Using Third Generation Non-Volatile Memory (NVM3) Data Storage for more information on NVM3 default instance key spaces and restrictions on selecting NVM3KEY values for indexed tokens.

/**
* Custom Zigbee Application Tokens
*/
// Define token names here
#define NVM3KEY_DEVICE_INSTALL_DATA  (NVM3KEY_DOMAIN_USER | 0x000A)
// This key is used for an indexed token and the subsequent 0x7F keys are also reserved
#define NVM3KEY_HOURLY_TEMPERATURES  (NVM3KEY_DOMAIN_USER | 0x1000)
#define NVM3KEY_LIFETIME_HEAT_CYCLES (NVM3KEY_DOMAIN_USER | 0x000C)

These examples define the token key and link it to a programmatic variable. The token names are actually DEVICE_INSTALL_DATA, HOURLY_TEMPERATURES and LIFETIME_HEAT_CYCLES, with different tags prepended to the beginning depending on the usage. Thus, they are referred to in the example code as TOKEN_DEVICE_INSTALL_DATA, and so on.

The token key values must be unique within this device. The token key is critical to linking application usage with the proper data. As such, a unique key should always be used when defining a new token or even changing the structure of an existing token. Always using a unique key guarantees a proper link between application and data. CREATOR code values are 16-bit and NVM3KEY code values are 20-bit. For SimEEv1/v2, the first bit is reserved for manufacturing tokens, stack tokens, and those application tokens defined by the application framework, so all custom tokens should have a token key less than 0x8000.

For NVM3, custom application tokens should use the NVM3KEY_DOMAIN_USER range so they do not collide with the stack tokens in the NVM3KEY_DOMAIN_ZIGBEE range.

Define the Token Type#

The token type can be either a built-in C data type or defined as a custom data structure using typedef. Note that the token type must be defined only in one place, as the compiler will complain if the same data structure is defined twice.

Each token in the code examples in Section Creating Dynamic Tokens is a different type; the HOURLY_TEMPERATURES and LIFETIME_HEAT_CYCLES types are built-in types in C, while the DEVICE_INSTALL_DATA type is a custom data structure:

#ifdef DEFINETYPES
// Include or define any typedef for tokens here
typedef struct {
int8u install_date[11] /** YYYY-mm-dd + NULL */
int8u room_number; /** The room where this device is installed */
} InstallationData_t;
#endif //DEFINETYPES

Define the Token Storage#

After any custom types are defined, the token storage is defined. This informs the token management software about the tokens being defined. Each token, whether custom or default, gets its own entry in this part:

#ifdef DEFINETOKENS
// Define the actual token storage information here
DEFINE_BASIC_TOKEN(DEVICE_INSTALL_DATA,
InstallationData_t,
{0, {0,...}})
DEFINE_INDEXED_TOKEN(HOURLY_TEMPERATURES, int16u, HOURS_IN_DAY, {0,...})
DEFINE_COUNTER_TOKEN(LIFETIME_HEAT_CYCLES, int32u, 0}
#endif //DEFINETOKENS

The following expands on each step in this process.

DEFINE_BASIC_TOKEN(DEVICE_INSTALL_DATA,
InstallationData_t,
{0, {0,...}})

DEFINE_BASIC_TOKEN takes three arguments: the name (DEVICE_INSTALL_DATA), the data type (InstallationData_t), and the default value of the token if it has never been written by the application ({0, {0,...}}).

The default value takes the same syntax as C default initializers. In this case, the first value (room_number) is initialized to 0, and the next value (install_date) is set to all 0s because the {0,...} syntax fills the remainder of the array with 0.

The syntax of DEFINE_COUNTER_TOKEN is identical to DEFINE_BASIC_TOKEN.

DEFINE_INDEXED_TOKEN requires a length of the array—in this case, HOURS_IN_DAY or 24. Its final argument is the default value of every element in the array. Again, in this case it is initialized to all 0s.

Accessing Dynamic Tokens With HAL APIs#

Refer to this information if you are developing with any EmberZNet or Connect in GSDK v2.7x, and with EmberZNet in GSDK v3.0.

The networking stack provides a simple set of APIs for accessing token data. The APIs differ slightly depending on the type of the tokens. The rest of this section discusses how to access tokens depending on the types. You can find the full documentation in the stack API Reference.

Accessing Basic (Non-indexed) Tokens#

The non-indexed/basic token API functions include:

void halCommonGetToken(data, token)
void halCommonSetToken(token, data)

In this case, 'token' is the token key, and 'data' is the token data. Note that halCommonGetToken() and halCommonSetToken() are general token APIs that can be used for both basic dynamic tokens and manufacturing tokens. See Accessing Manufacturing Tokens.

The following example explains the usage of these APIs. Assume an application needs to store configuration data at installation time. A basic token has been defined to use the token key DEVICE_INSTALL_DATA and the data structure looks like this:

typedef struct {
int8u install_date[11] /** YYYY-mm-dd + NULL */
int8u room_number; /** The room where this device is installed */
} InstallationData_t;

Then you can access it with a code snippet like this:

InstallationData_t data;
// Read the stored token data
halCommonGetToken(&data, TOKEN_DEVICE_INSTALL_DATA);
// Set the local copy of the data to new values
data.room_number = < user input data >
MEMCOPY(data.install_date, < user input data>, 0, sizeof(data.install_date));
// Update the stored token data with the new values
halCommonSetToken(TOKEN_DEVICE_INSTALL_DATA, &data);

Accessing Counter Tokens#

There is a special API to increment the counter token (which is a special type of non-indexed tokens):

void halCommonIncrementCounterToken(token)

Note that although you can write the counter token with the common halCommonSetToken() call, doing so is inefficient and defeats the purpose of using a counter token.

The following example explains the usage of a counter token and the special API to increment the counter token. Counting the number of heating cycles a thermostat has initiated is a perfect use for a counter token. Assume it is named LIFETIME_HEAT_CYCLES, and it is an int32u data type.

void requestHeatCycle(void) {
/// < application logic to initiate heat cycle >
halCommonIncrementCounterToken(TOKEN_LIFETIME_HEAT_CYCLES);
}
int32u totalHeatCycles(void) {
int32u heatCycles;
halCommonGetToken(&heatCycles, TOKEN_LIFETIME_HEAT_CYCLES);
return heatCycles;
}

Note that to read a counter token, use halCommonGetToken() just as you would read a general basic token.

Accessing Indexed Tokens#

The indexed token API functions include:

void halCommonGetIndexedToken(data, token, index)
void halCommonSetIndexedToken(token, index, data)

The following example explains the usage of these APIs for an indexed token. To store a set of similar values, such as an array of preferred temperature settings throughout the day, use the default data type int16s to store the desired temperatures and define an indexed token called HOURLY_TEMPERATURES.

A local copy of the entire data set would look like this:

int16s hourlyTemperatures[HOURS_IN_DAY]; /** 24 hours per day */

In the application code, you can access or update just one of the values in the day using the indexed token functions:

int16s getCurrentTargetTemperature(int8u hour) {
int16s temperatureThisHour = 0; /** Stores the temperature for return */
if (hour < HOURS_IN_DAY) {
halCommonGetIndexedToken(&temperatureThisHour,
TOKEN_HOURLY_TEMPERATURES, hour);
}
return temperatureThisHour;
}
void setTargetTemperature(int8u hour, int16s targetTemperature) {
if (hour < HOURS_IN_DAY) {
halCommonSetIndexedToken(TOKEN_HOURLY_TEMPERATURE, hour,
&temperatureThisHour);
}
}

Accessing Dynamic Tokens with Token Manager APIs#

Refer to this information if you are developing with Connect in GSDK v3.0.

Accessing Basic (Non-Indexed) Tokens#

The non-indexed/basic token API functions include:

Ecode_t sl_token_get_data(uint32_t token,
uint32_t index,
void *data,
uint32_t length);
Ecode_t sl_token_set_data(uint32_t token,
uint32_t index,
void *data,
uint32_t length);

In this case, 'token' is the token key, ‘index’ is 1 for non-indexed/basic tokens, 'data' is the token data, and ‘length’ is the size in bytes of the data. Note that sl_token_get_data() and sl_token_set_data() are specific for basic dynamic tokens. For manufacturing tokens see Accessing Manufacturing Tokens.

typedef struct {
int8u install_date[11] /** YYYY-mm-dd + NULL */
int8u room_number; /** The room where this device is installed */
} InstallationData_t;

Then you can access it with a code snippet like this:

InstallationData_t data;
// Read the stored token data
sl_token_get_data(TOKEN_DEVICE_INSTALL_DATA, 1, &data, sizeof(data));
// Set the local copy of the data to new values data.room_number = < user input data >
MEMCOPY(data.install_date, < user input data>, 0, sizeof(data.install_date));
// Update the stored token data with the new values
sl_token_set_data(TOKEN_DEVICE_INSTALL_DATA, 1, &data, sizeof(data));

Accessing Counter Tokens#

There is a special API to increment the counter token (which is a special type of non-indexed tokens):

Ecode_t sl_token_increment_counter(uint32_t token);

Note that although you can write the counter token with the common sl_token_set_data() call, doing so is inefficient and defeats the purpose of using a counter token.

void requestHeatCycle(void) {
/// < application logic to initiate heat cycle >
sl_token_increment_counter(TOKEN_LIFETIME_HEAT_CYCLES);
}
int32u totalHeatCycles(void) {int32u heatCycles;
sl_token_get_data(TOKEN_LIFETIME_HEAT_CYCLES, 1, &heatCycles, sizeof(heatCycles));
return heatCycles;
}

Note that to read a counter token, use sl_token_get_data() just as you would read a general basic token.

Accessing Indexed Tokens#

The indexed token API functions are the same as the non-indexed/basic token API functions. The only difference is the ‘index’ parameter can be something other than 1.

A local copy of the entire data set would look like this:

int16s hourlyTemperatures[HOURS_IN_DAY]; /** 24 hours per day */

In the application code, you can access or update just one of the values in the day using the indexed token functions:

int16s getCurrentTargetTemperature(int8u hour) {
int16s temperatureThisHour = 0; /** Stores the temperature for return */
if (hour < HOURS_IN_DAY) {
 sl_token_get_data(TOKEN_HOURLY_TEMPERATURES, hour, &temperatureThisHour, sizeof(temperatureThisHour));
}
return temperatureThisHour;
}
void setTargetTemperature(int8u hour, int16s targetTemperature) {
if (hour < HOURS_IN_DAY) {
 sl_token_set_data(TOKEN_HOURLY_TEMPERATURE, hour, &temperatureThisHour, sizeof(temperatureThisHour));
}
}

Manufacturing Tokens#

Manufacturing tokens are defined in a similar way as basic (non-indexed) dynamic tokens, but use the DEFINE_MFG_TOKEN in the manufacturing token header, instead of the other DEFINE_*_TOKEN macros. Note that the CREATOR_* prefix – not NVM3KEY_* - should always be used with manufacturing tokens, even when the NVM plugin is being used for non-volatile storage. Refer to Creating Dynamic Tokens and to the API documentation on the DEFINE_MFG_TOKEN macro for more information.

There is a major difference, however, on where manufacturing tokens are stored and how they are accessed. Manufacturing tokens reside in the dedicated flash page for manufacturing tokens (with fixed absolute addresses). The rest of this section describes the necessary considerations for accessing manufacturing tokens.

Accessing Manufacturing Tokens#

Manufacturing tokens, as the name suggests, are usually written once at manufacturing time into fixed locations in a dedicated flash page. Because their addresses are fixed, they can be easily read from external programming tools. Note, however, when the Read Protection feature has been enabled on the chip or on the dedicated flash page, manufacturing tokens can only be read from on-chip code.

Manufacturing tokens are not meant to be written often or from on-chip code, because they are at fixed locations. The same flash cell cannot be written repeatedly without erase operations in between. Writing a manufacturing token from on-chip code works only if the token is currently in an erased state. This means that manufacturing tokens can only be overwritten with external programming tools and not with on-chip code. Overwriting any manufacturing token that has been already written requires erasing the dedicated flash page for the manufacturing tokens. If Read Protection is enabled, it must be disabled first, which will erase the contents of the chip as a side effect. Manufacturing tokens are not wear-leveled so they should be overwritten sparingly.

Access manufacturing tokens with their own dedicated APIs, which take the same parameters as the basic token APIs:

HAL: halCommonGetMfgToken() and halCommonSetMfgToken()
Token Manager: sl_token_get_manufacturing_data() and sl_token_set_manufacturing_data()

The two primary purposes for using the dedicated manufacturing token access APIs are:

For slightly faster access;
For access early in the boot process before emberInit() or sl_token_init() is called.

If you are using the HAL APIs, manufacturing tokens can also be accessed through the basic token APIs: halCommonGetToken() and halCommonSetToken().

Accessing Default and Custom Tokens#

Where to Find Default Token Definitions#

To view the stack tokens, refer to the file:

`<install-dir>/stack/config/token-stack.h`

To view the Application Framework tokens, refer to the file <project_name>_tokens.h and the protocol-specific token file (such as znet-token.h) under your project directory after the project has been generated in AppBuilder in Connect v2.7.x and EmberZNet SDK v6.7.x/v6.8.x SDKs. <project_name>_tokens.h contains tokens for ZCL attributes that the application has selected to be stored in non-volatile memory. The protocol-specific token file includes plugin token headers and the custom application token header.

To view the manufacturing tokens for the EFR32 series of chips refer to the following files, respectively:

<install-dir>/hal/micro/cortexm3/efm32/token-manufacturing.h

Search for CREATOR to see the defined names. If the entire file seems overwhelming, focus only on the section describing the tokens. Some of the fixed manufacturing tokens may be set by the manufacturer when the board is created. For example, a custom EUI-64 address may be set by the vendor to override the internal EUI-64 address provided by Silicon Labs. Other tokens, such as the internal EUI-64, cannot be overwritten.

Add Custom Tokens#

Refer to Section Creating Dynamic Tokens on the methods and APIs for creating custom tokens. You can also refer to the stack token definition file as mentioned near the top of Section Where to Find Default Token Definitions as a guide for creating custom application tokens.

After creating the tokens in a custom token header file, you need one more step: add the header file to the application by using the Includes tab in the .isc file in Simplicity Studio under the "Token Configuration" section. If you define both custom manufacturing tokens and custom application tokens, Silicon Labs recommends that you separate them into two different header files.

Note: For custom token files, the inclusion guards (#ifndef) seen in token-stack.h should not be used.