NVM3 - NVM Data Manager#

NVM3 Non-Volatile Memory Data Management driver.

Introduction#

The NVM3 driver provides a way for an application to safely store and retrieve variable-size objects in a page-based non-volatile memory (NVM). Objects are identified with 20-bit object identifiers denoted as keys.

The driver is designed to use pages in a sequential order to provide equal usage and wear. The driver is resilient to power loss or reset events, ensuring that objects retrieved from the driver are in a valid state. A valid object will always be the last successfully stored object. NVM3 can detect NVM defects and mark pages as unusable. NVM3 will continue to operate on good pages after defect pages are detected.

Objects#

An NVM3 object is data that can be stored in NVM. The object is handled as an array of bytes up to NVM3_MAX_OBJECT_SIZE in size. NVM3 can handle two types of objects.

1. Regular data objects. Data objects can store information of any size up to maximum NVM3_MAX_OBJECT_SIZE bytes.

2. 32-bit counter objects. Counter objects can store 32-bit counters that are accessed with a separate set of API functions. The counter object is designed to be compact while minimizing memory wear in applications that require frequent persistent counter increments.

See The API for more details on the API.

Repacking#

As the NVM fills up, it reaches a point where it can no longer store additional objects, and a repacking operation is required to release out-of-date objects to free up NVM. Because both writing data and erasing pages take a long time, the NVM3 driver does not trigger the process by itself unless free memory reaches a critical low level. As an alternative, the application can trigger the repacking process by calling the nvm3_repack() function. During the call, NVM3 will either move data to a new page or erase pages that can be reused. At most, the call will block for a period equal to a page erasure time or the time to write the largest size object (whatever is largest) plus a small execution overhead. Page erasure and flash write timing for the EFM32 or EFR32 parts can be found in the datasheet.

NVM3 uses two thresholds for repacking:

1. Forced threshold. This is the threshold used to force automatic repacking when free memory reaches a critical low level.

2. User threshold. This is the threshold used by nvm3_repackNeeded(). nvm3_repack() will not perform repacking unless free memory is below this threshold.

The user can define the user threshold by entering a value in the repackHeadroom member of the nvm3_Init_t structure used by the nvm3_open() function. The repackHeadroom value defines the difference between the user and forced threshold. The forced threshold is the minimum low memory threshold defined by the page size and maximum object size and can't be changed by the user. The default value for the repack headroom is 0, meaning that the forced and user thresholds are equal.

An NVM3 function that deletes or modifies data or counter object will trigger an automatic repack operation when free memory is below the forced threshold. The check is done before the object is modified, not after.

The application can use nvm3_repackNeeded() to determine if repacking is needed. To initiate repacks, call nvm3_repack(). Note that this function will perform repacks only if they are needed.

Note

• The repack threshold can be changed to prevent multiple modifications of objects between user called repacks from causing forced repacks. Note that "high" values of the repack headroom may cause increased NVM wear from increased number of repacks.

See Execution Timing section for more details on repack timing.

Caching#

NVM3 includes an object location lookup cache to speed up object access, as searching through the entire NVM3 contents for an object could otherwise be slow. It is important to note that this cache only stores the location of the object and not the object data itself. To ensure that cache can hold all necessary information, it must be configured to a size equivalent to or larger than the number of objects stored in NVM, including those deleted, as long as they are not discarded by the repack function. If the cache is available, the driver will first look in the cache to find the position of the object in NVM. If the object position is not found in the cache, the object position will be found by searching the NVM. The search will start at the last stored object and search all the way to the oldest object. If the object is found, the cache is updated accordingly.

The application must allocate and support data for the cache. See the nvm3_open function for more details. The size of each cache element is one uint32_t and one pointer giving a total of 8 bytes (2 words) pr. entry for EFM32 and EFR32 devices.

Note

• The cache is fully initialized by nvm3_open() and automatically updated by any subsequent write, read, or delete function call.

Global Data (variables)#

The NVM3 service uses global variables to store intermediate data during open, read, write, increment, and delete calls. The application must set the value of the nvm3_maxFragmentCount at run-time before any NVM3 functions are called.

NVM3 does not support overlapped calls. If there is any chance that the application can issue overlapped calls, the NVM3 locking mechanism must be present and protected from that.

Note

• If the application uses more than one NVM3 instance, the variables will be shared between the instances. Be sure to allocate data that have a size that is large enough for the largest usage.

Stack Usage#

NVM3 function calls are nested several levels deep. The stack usage has been measured on some EFM32, and EFR32 targets with NVM3 source for IAR and ARM GCC. The maximum stack usage measured was 420 bytes for IAR, and 472 bytes for ARM GCC builds. The unit test used to validate the stack usage has a 10% margin and uses a stack limit of 462 bytes for IAR and 520 for ARM GCC. The maximum stack usage measured was 715 bytes for ARM GCC builds with security. The unit test used to validate the stack usage has a 10% margin and uses a stack limit of 794 bytes for ARM GCC with security. Note that the actual stack usage is a little different on the Cortex M0 Plus, M3, M4, and M33 with NVM3 source.

The API#

The NVM3 API is defined in the nvm3.h file. The application code must include the nvm3.h header file to get access to all definitions, datatypes, and function prototypes defined by NVM3.

This section contains brief descriptions of NVM3 functions. For more information about parameters and return values, see the Function documentation section. Most functions return an sl_status_t that has the value SL_STATUS_OK on success, or see nvm3.h for other values.

nvm3_open() and nvm3_close().
These functions open and close an NVM3 instance. nvm3_open() takes a handle of type nvm3_Handle_t and initialization data of type nvm3_Init_t. The helper macro pair NVM3_DEFINE_SECTION_STATIC_DATA() and NVM3_DEFINE_SECTION_INIT_DATA() are provided to simplify initialization data definition. For usage examples, see the Examples section.

nvm3_getObjectInfo(), nvm3_enumObjects(), nvm3_deleteObject() and nvm3_countObjects()
These functions work on all objects. nvm3_enumObjects() gets a list of keys to valid objects in the NVM. The search can also be constrained by the function parameters. nvm3_countObjects() can be useful at startup to distinguish between a first startup without any valid objects present and later reboots with valid objects persistently stored in NVM.

nvm3_writeData() and nvm3_readData()
Write and read data objects.

nvm3_writeCounter(), nvm3_readCounter() and nvm3_incrementCounter()
Write, read, and increment 32-bit counter objects.

nvm3_eraseAll()
Erase all objects in NVM.

nvm3_getEraseCount()
Return the erasure count for the most erased page in NVM.

nvm3_repack() and nvm3_repackNeeded()
Manage NVM3 repacking operations.

nvm3_resize()
Resize the NVM area used by an open NVM3 instance.

API Locking and Interrupt handling#

Common for all NVM3 API calls is that they are not re-entrant. By default, all functions are protected with protection functions that disable interrupts.

Note

• The default NVM3 protection functions can be substituted by the application if other synchronization functions are available and disabling interrupts for extended periods is not desired.

If the application does all the nvm3-calls from the same thread and guarantees no overlapping calls, the lock functions don't have to do anything.

Memory Placement#

The application is responsible for placing the NVM area correctly. The minimum requirements for memory placement are as follows:

1. NVM area start address must be aligned with a page of the underlying memory system.

2. NVM area size must be a multiple of the page size.

The minimum required NVM size is dependent on both the NVM page size and the NVM3_MAX_OBJECT_SIZE value. For a device with 2 kB page size and typical values for NVM3_MAX_OBJECT_SIZE, the following is the minimum required number of pages:

• For NVM3_MAX_OBJECT_SIZE=208: 3 pages

• For NVM3_MAX_OBJECT_SIZE=1900: 4 pages

• For NVM3_MAX_OBJECT_SIZE=4096: 5 pages

NVM3_DEFINE_SECTION_STATIC_DATA() and NVM3_DEFINE_SECTION_INIT_DATA() macros are provided to support the creation of the NVM area and initialization data. A linker section called 'name'_section is defined by NVM3_DEFINE_SECTION_STATIC_DATA(). The NVM area is placed within the linker section. The application linker script must place the section according to the requirements above. An error is returned by nvm3_open() on alignment or size violation.

Note

• When the start address and size of the data area are defined and used by an application, make sure you use the same values at every program startup and also reuse by new versions of the software after an upgrade. If an application tries to open an instance with a start address or size that does not match the previous use, it can result in permanent data loss and failure.

Configuration Options#

There are no compile-time configuration options for NVM3. All configuration parameters are contained in nvm3_Init_t.

Note

• The Global Data (variables) must however be configured for correct size and have correct values for NVM3 to behave correctly.

Bad NVM Page Handling#

NVM3 has been designed to detect page erase and write errors during normal operation and mark failing pages as BAD. If a write operation fails, all objects that have been written to the page prior to the write error are copied to the next free page before the page is marked as BAD and the write operation resumes. If the recovery operation is successful, the operation is regarded as complete, and the function will return SL_STATUS_OK status.

Note

• Erase and write errors may not be detected by NVM3 if the device is used until End-of-Life (EOL), where the failure mode can be that the NVM content is changing during a power cycle.

Error Handling#

The NVM3 error handling involves most functions returning an error code. The nvm3_countObjects is different because it returns the actual number of objects found.

The behavior and return values for most functions, such as nvm3_readData, nvm3_writeData, and so on should be self explanatory, while the nvm3_open is slightly different. nvm3_open will always try to recover from the previous state and continue without an error, if possible. In other words, if a valid NVM3 instance is established, nvm3_open will recover from brownouts and power cycles at any time in any operation and bring the system to a valid state where all pages and objects are in a known state and return success whenever possible. From this state, normal operation can resume. If nvm3_open returns an error, it's an indication of either a design or coding error or that many of the NVM pages have been marked as BAD, leaving insufficient space in the NVM to progress. Operation may not resume if nvm3_open returns an error.

Note

• Because the nvm3_open may need to do recovery operations, the execution time will occasionally vary.

Storing Objects in Internal Flash#

NVM3 has support for writing and reading objects in internal, i.e., memory-mapped Flash memory through the nvm3_hal_flash.c "driver". nvm3_hal_flash.c is using EMLIB functions to write and erase data while using regular memory functions to read data from Flash.

The "driver" for internal Flash is selected by setting the halHandle in the nvm3_open initialization structure to point to nvm3_halFlashHandle.

Storing Objects in External Flash#

NVM3 has support for writing and reading objects in external Flash by using the nvm3_hal_flash_ext.c "driver". The nvm3_hal_flash_ext.c is using the SE manager functions to write and erase data while using regular memory functions to read data from external Flash.

The "driver" for external Flash is selected by setting the halHandle in the nvm3_open initialization structure to point to nvm3_halFlashHandle.

All data objects are authenticated + encrypted using AES-GCM mode and counter objects are stored in plain. NVM3 encryption key is stored in SE MTP region. Reading the external memory alone will not give access to the unencrypted data objects.

HAL for crypto is added to have the flexibility to use available crypto which can be selected by setting the halCryptoHandle in the nvm3_open initialization structure to point to desired crypto handle.

NVM3 Source code#

The NVM3 source code can be compiled for Cortex M0, M3, M4, and M33 with either Arm GCC or IAR toolchains. In addition, the NVM3 driver provides a means to securely store and retrieve the objects in external Flash using authenticated encryption. Objects in internal Flash need not be encrypted, while objects in external Flash must always be encrypted.

Storage Capacity#

Basic storage is defined as the size of on instance of all objects, including any overhead stored with the data. For NVM3 the maximum amount of data you can store is dependent on the number of flash pages used for storage and the max object size used for NVM3. The following table shows the maximum allowed basic storage for a varying number of 4 kB or 8 kB flash pages and the minimum (204 bytes), default (254 bytes), high (1900 bytes) and maximum (4096 bytes) max object size. Note that this is a theoretical limit, and if the basic storage is at this limit, no space is left for wear-levelling, and page erases will be forced for every object written. The NVM3 instance should therefore be configured with enough flash pages to put the maximum allowed basic storage significantly higher than the actual basic storage.

Max Allowed Basic Storage with 8 kB page size#

Max Allowed Basic Storage with 4 kB page size#

Default Instance#

Several NVM3 instances can be created on a device and live independently of each other, but to save memory, it is usually desirable to use only one NVM3 instance as each instance adds some overhead. For this reason, a default instance exists that is used by all Silicon Labs wireless stacks. The API to initialize the default instance and the handles to use with the regular NVM3 API are described in NVM3 Default Instance.

NVM3 in Simplicity Commander#

Simplicity Commander is a single, all-purpose tool to be used in a production environment. It is invoked using a simple Command Line Interface (CLI) that can also be scripted. Simplicity Commander supports reading out the NVM3 data area from a device and parsing the NVM3 data to extract stored values. This can be useful in a debugging scenario where you may need to find out the stored state of an application that has been running for some time.

For more information about using the Simplicity Commander with NVM3, see UG162: Simplicity Commander Reference Guide.

Execution Timing#

There are several factors that affect the execution time for NVM3 calls that can update the NVM, described below.

The primary factor when doing updates is that data must be written to flash. Writing to flash is relatively slow, and timing information for the particular device is available in the datasheet and can be used to calculate the approximately minimum execution time. Note that NVM3 will, in addition to the user data, write object headers, and the software will add some overhead. The relative overhead will be larger for smaller compared to larger objects.

When updating the flash store, repacking must be done from time to time. See the Repacking section for more details about why repacking is needed and how it works.

To minimize the time when repacks are executed, there are a few configurations that affect how NVM3 works. The most important configurations are listed below:

1. The repackHeadroom parameter in the nvm3_Init_t structure can be used to set the number of bytes that can be written before a forced repack is triggered. To make this work, the nvm3_repack() function must be called until the nvm3_repackNeeded() returns false before the actual write.

2. Use as small objects as possible and define the NVM3_MAX_OBJECT_SIZE accordingly. Writing and repacking large objects is time-consuming. Limiting the maximum object size will limit the time spent in both write and repack functions.

When triggered, the repack function will either copy data or erase a page. To limit the time spent when copying, the repack function will return when the NVM3_MAX_OBJECT_SIZE number of bytes have been copied. The copy operation will resume on the next call to repack, and the application may have to call the repack function several times to complete a full repack operation.

Note

• Performing the /@refnvm3_repackNeeded() loop is highly recommended before any timing-sensitive procedure.

Examples#

Example 1 shows initialization, usage of data objects, and repacking.

#include "nvm3.h"
#include "nvm3_hal_flash.h"

// Create a NVM area of 24kB (size must equal N * FLASH_PAGE_SIZE, N is integer). Create a cache of 10 entries.
NVM3_DEFINE_SECTION_STATIC_DATA(nvm3Data1, 24576, 10);

// This macro creates the following:
// 1. An array to hold NVM data named nvm3Data1_nvm
// 2. A section called nvm3Data1_section containing nvm3Data1_nvm. The application linker script must place this section correctly in memory.
// 3. A cache array: nvm3Data1_cache

void nvm3_example_1(void)
{
  // Declare a nvm3_Init_t struct of name nvm3Data1 with initialization data. This is passed to nvm3_open() below.
  NVM3_DEFINE_SECTION_INIT_DATA(nvm3Data1, &nvm3_halFlashHandle);

  nvm3_Handle_t handle;

  sl_status_t status;
  size_t numberOfObjects;
  unsigned char data1[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
  unsigned char data2[] = { 11, 12, 13, 14, 15 };
  uint32_t objectType;
  size_t dataLen1;
  size_t dataLen2;

  // Initialize the handle to zero
  (void)memset(&handle, 0, sizeof(nvm3_Handle_t));
  status = nvm3_open(&handle, &nvm3Data1);
  if (status != SL_STATUS_OK) {
    // Handle error
  }

  // Get the number of valid keys already in NVM3
  numberOfObjects = nvm3_countObjects(&handle);

  // Skip if we have initial keys. If not, generate objects and store
  // persistently in NVM3 before proceeding.
  if (numberOfObjects < 2) {
    // Erase all objects and write initial data to NVM3
    nvm3_eraseAll(&handle);
    nvm3_writeData(&handle, 1, data1, sizeof(data1));
    nvm3_writeData(&handle, 2, data2, sizeof(data2));
  }

  // Find size of data for object with key identifier 1 and 2 and read out
  nvm3_getObjectInfo(&handle, 1, &objectType, &dataLen1);
  if (objectType == NVM3_OBJECTTYPE_DATA) {
    nvm3_readData(&handle, 1, data1, dataLen1);
  }
  nvm3_getObjectInfo(&handle, 2, &objectType, &dataLen2);
  if (objectType == NVM3_OBJECTTYPE_DATA) {
    nvm3_readData(&handle, 2, data2, dataLen2);
  }

  // Update and write back data
  data1[0]++;
  data2[0]++;
  nvm3_writeData(&handle, 1, data1, dataLen1);
  nvm3_writeData(&handle, 2, data2, dataLen2);

  // Do repacking if needed
  if (nvm3_repackNeeded(&handle)) {
    status = nvm3_repack(&handle);
    if (status != SL_STATUS_OK) {
      // Handle error
    }
  }
}

Example 2 shows initialization and usage of counter objects. The counter object uses a compact way of storing a 32-bit counter value while minimizing NVM wear.

#include "nvm3.h"
#include "nvm3_hal_flash.h"

// Create a NVM area of 24kB (size must equal N * FLASH_PAGE_SIZE, N is integer). Create a cache of 10 entries.
NVM3_DEFINE_SECTION_STATIC_DATA(nvm3Data2, 24576, 10);

#define USER_KEY        1

// This macro creates the following:
// 1. An array to hold NVM data named nvm3Data2_nvm
// 2. A section called nvm3Data2_section containing nvm3Data2_nvm. The application linker script must place this section correctly in memory.
// 3. A cache array: nvm3Data2_cache

void nvm3_example_2(void)
{
  // Declare a nvm3_Init_t struct of name nvm3Data2 with initialization data. This is passed to nvm3_open() below.
  NVM3_DEFINE_SECTION_INIT_DATA(nvm3Data2, &nvm3_halFlashHandle);

  nvm3_Handle_t handle;
  sl_status_t status;
  uint32_t counter = 1;

  // Initialize the handle to zero
  (void)memset(&handle, 0, sizeof(nvm3_Handle_t));
  status = nvm3_open(&handle, &nvm3Data2);
  if (status != SL_STATUS_OK) {
    // Handle error
  }

  // Erase all objects
  nvm3_eraseAll(&handle);

  // Write first counter value with key 1
  nvm3_writeCounter(&handle, USER_KEY, counter);

  // Increment the counter by 1 without reading out the updated value
  nvm3_incrementCounter(&handle, USER_KEY, NULL);

  // Read the counter value
  nvm3_readCounter(&handle, USER_KEY, &counter);
}

Modules#

nvm3_CacheEntry_t

nvm3_Init_t

nvm3_ObjFrag_t

nvm3_Obj_t

NVM3 Default Instance

NVM3 HAL

NVM3 Lock

0

1

2

NVM3_TRACE_PORT_NONE

0

1

2

3

NVM3_TRACE_LEVEL_WARNING

false

(SL_STATUS_OK)

(SL_STATUS_NVM3_ALIGNMENT_INVALID)

(SL_STATUS_NVM3_SIZE_TOO_SMALL)

(SL_STATUS_NVM3_NO_VALID_PAGES)

(SL_STATUS_NVM3_PAGE_SIZE_NOT_SUPPORTED)

(SL_STATUS_NVM3_OBJECT_SIZE_NOT_SUPPORTED)

(SL_STATUS_FULL)

(SL_STATUS_NOT_INITIALIZED)

(SL_STATUS_NVM3_OPENED_WITH_OTHER_PARAMETERS)

(SL_STATUS_INVALID_PARAMETER)

(SL_STATUS_INVALID_KEY)

(SL_STATUS_NOT_FOUND)

(SL_STATUS_NVM3_OBJECT_IS_NOT_DATA)

(SL_STATUS_NVM3_OBJECT_IS_NOT_A_COUNTER)

(SL_STATUS_FLASH_ERASE_FAILED)

(SL_STATUS_NVM3_WRITE_DATA_SIZE)

(SL_STATUS_FLASH_PROGRAM_FAILED)

(SL_STATUS_NVM3_READ_DATA_SIZE)

(SL_STATUS_OBJECT_READ)

(SL_STATUS_NVM3_INIT_WITH_FULL_NVM)

(SL_STATUS_NVM3_RESIZE_PARAMETER)

(SL_STATUS_NVM3_RESIZE_NOT_ENOUGH_SPACE)

(SL_STATUS_NVM3_ERASE_COUNT_ERROR)

(SL_STATUS_NVM3_WRITE_TO_NOT_ERASED)

(SL_STATUS_NVM3_INVALID_ADDR)

(SL_STATUS_NVM3_KEY_MISMATCH)

(SL_STATUS_NVM3_SIZE_ERROR)

(SL_STATUS_NVM3_EMULATOR)

(SL_STATUS_FAIL)

(SL_STATUS_NVM3_NVM_ACCESS)

(SL_STATUS_NVM3_INVALID_ADDR)

512U

204U

4096U

1900U

NVM3_MAX_OBJECT_SIZE_DEFAULT

0xFFFFFFFFU

20U

((1U << NVM3_KEY_SIZE) - 1U)

0U

NVM3_KEY_MASK

0U

1U

(2U)