RecordStorage

RecordStorage is a small file system for flash storage. It is used for storing persistent data, for example the enrollment data of a node or the module settings. Internally, it uses FlashStorage to access the flash and queue its operations. RecordStorage is fully failsafe against power loss and corruption. It will intelligently write all records to a single flash page until it is full and will swap the valid data to another page (defragmentation) once space is needed, thereby minimizing erase operations on the flash.

The most important features of RecordStorage are:

A record is identified by its recordId, thus every record need to have unique recordId. Records store some metadata like: ID, length, CRC, and the record version in addition to the data.

The CRC of the record is checked every time a record is accessed. Records with invalid CRC are ignored and removed during next defragmentation or swap.

The versionCounter is used determine the version of the record. When a record is updated, it is not deleted, but a new record with an incremented versionCounter is created. When reading, only the record with newest versionCounter is returned, which means that a record becomes inactive as soon as a newer version of this record is stored. Inactive records are removed during defragmentation.

Record storage needs to be assigned a number of pages in flash memory that are not used by the application. The minimium number of pages is 2 (one data and one swap page). The swap page is the page that currently doesn’t contain any data. When all other pages are full, the page which can be defragmented the most is defragmented and copied to the swap page. After validation of the records, the old page is erased and becomes the swap page. During defragmentation, all active records will be moved but inactive records will be omitted.

There are two ways to create an immortal, either pass isMortal=0 to the SaveRecord method or grant immortality to an existing record using the ImmortalizeRecord Method.

Immortal records are just like regular/mortal records but they have the mortal bit set to zero. This makes it possible for them to survive a factory reset, an event that typically whipes out all records. During factory reset, the devices enters a lockdown mode to clear the settings before a reboot. Internally there are different lockdown stages. NO_LOCKDOWN is the default operation mode.

After the LockDownAndClearAllSettings method is called (during factory reset) the lockdown stage changes to LOCKDOWN_SCHEDULED if a repair or defragmentation is currently in progress. Otherwise, or if the ongoing progress finishes the lockdown stage is set to LOCKING_DOWN. In this stage a defragmentation is triggered on every page, one after the other.

The defragmentation method will usually relocate every active record from its current page to the swap page and clear the target page afterwards. During a factory reset however, only active immortal records are relocated, all mortals are not relocated and therefore cleared.

Once every page has been defragmented, the stage is set to LOCKED_DOWN and all mortals have been erased. New records can only be saved in the NO_LOCKDOWN stage (after a reboot) or by the module that initiated the lockdown. A reboot is necessary to unlock the storage . If a flash operation such as a page erase fails in the LOCKDOWN_SCHEDULED stage, it is retried a couple of times by setting the defragmentation stage. If it still does not work the device reboots.

Saving or updating records and deleting them are all non-blocking operations which are cached and executed asynchronously. Users can register a listener when scheduling an operation to get notified once the operation was executed. In the handler, the user receives information about the result of the operation. A userType and user context data can be given to identify the operation.

Reading from RecordStorage is done synchronously as a simple access to flash memory.