It is my understanding that the indexes/keys for tables are stored in the database. What is the logic behind this behavior? My logic would be that the table does not exist, have any entries in it, until the table is polled from the actual device and saving the indexes/keys in the database can cause problems, as we have seen, because the protocol code can reference a table row that does not exist because it has not been retrieved/polled from the device yet. The table, or any access reference to it, does not exist until it is polled from the device and having any history of indexes for the state of a table prior to actually getting the table from the device is irrelevant. When an element is restarted then all the tables will be empty until they are retrieved from the actual device and the table's structure may have changed from what was last stored in the database, but since the table indexes are retrieved from the database upon startup prior to device polling the protocol code could think the table has entries when it really doesn't.
Thanks for any insight into this in advance.
Data can take many forms and each have their own purpose.
Some tables can contain configuration data which is send to the device. It makes sense to save these so they are persisted between restarts.
Other tables only contain monitoring data, which is polled from the device as you described. This kind of data should indeed not be persisted in the database.
To accommodate this second kind of behaviour you can add the "volatile"-tag to these tables. This will tell the software that the data in the tables will be polled and the primary keys will not be saved in the database.