Eram bit flipping at random

snow · Posted: Mon Mar 15, 2021 8:41 pm Post subject: Eram bit flipping at random

Hi All.
I've encountered an EEPROM issue. I've had a project running for a few years. Occasionally I get a product return and after a bit of investigation I've found some EEPROM bits have been randomly flipped. This causes issues with the operation of the product.
The address looked to be flipped seem to be at random in the EEPROM map. This product is used out doors in agricultural sector. A lot of the memory addresses affected have not been declared in Bascom . any thoughts?? could this be atmospheric conditions or something?.

The image is the EEPROM dump.

Address H320 - H326 has been declared. Everything that outside that and is not FF is bit flipping..
When the device came back. The address H320 and H321 came back with the value of 01 witch caused the device to malfunction as they are parameter settings.

Any Ideas?
Thanks
Snow

snow · Posted: Tue Mar 16, 2021 3:11 am Post subject:

Hello all.
Just a bit of an update. Look's like issue could be a noisy power supply. Found a scorched nearly dis-connected wire on power supply. Testing further

Thanks
Snow

albertsm · Joined: 09 Apr 2004 Posts: 5920 Location: Holland

you should use/enable the brown out detector.
it prevents eeprom corruption when the voltage goes down the selected level.
another way is to use external i2c eeprom.

a bad contact in the power will make the brown out kick in and reset the device.

snow · Posted: Tue Mar 16, 2021 9:30 am Post subject:

Thanks Mark. Duly noted.

Whats the best practice for reading EEPROM data??
1. Read all EEPROM data and store in local variables at the start of the program and use the local variable in your code.
2. Just read EEPROM data on the fly when required

Thanks
Snow

O-Family · Joined: 23 May 2010 Posts: 320 Location: Japan

Similarly, I have been consulted that the data in the built-in EEPROM will be corrupted.
Probably due to the abnormality of the power supply, I enabled BOD and took measures against noise in the power supply circuit.

In addition, the total of the entire EEPROM data is calculated and a checksum is written so that an EEPROM error is detected at startup.
Furthermore, the flags that are important for operation are written to the same data in three places of EEPROM at the same time so that they operate normally when all are the same.

albertsm · Joined: 09 Apr 2004 Posts: 5920 Location: Holland

I think O-Family has some excellent points.

In fact i have been using EEPROM for over 25 years and since the start it could be problematic in some cases. Due to problems in the hardware (chip or else).
So in the software i added some safe mechanism to check a register when a write is done. For products i often use an external eeprom.
But that is not always convenient for security for example.
So using eeprom can use some more thoughts and work.
I think using crc is a good way to at least check on start if the data is ok.
Maybe i can add such an option to bascom.

normally one would read the eeprom data on start up. then copy it to a local variable.
when ever it changes, write it back to eeprom.
but you can also do that later, for example when you have a menu system with multiple variables to write.
you can even go further by using multiple locations.
you can also keep track of how many times data is written. and then use different locations. i think i have seen code for such in the shared code section.

SZTRAD · Joined: 30 Dec 2019 Posts: 165

Well,
I would add that I personally have proven myself to have init settings in Flash, and when the EEPROM content is corrupted, I load the data from the flash so that the device does not lose the settings and does not go at all.
In this case, the EEPROM is erased and the blanc settings are retrieved from memory. It has not yet happened to me that EEPROM should be rewritten without writing in it.
I mean, it did, but we tested a device designed for "a different kind of operation", and there is damage to memory contents by high-speed particles.
I don't suppose this phenomenon is occurring in this case.
Otherwise, the Q-Family procedure is correct, only I use a 2 out of 3 selection. I personally load parameters only when the device restarts and when writing when they are automatically checked. It is still a good habit to move addresses after a certain time.
Doesn't the questioner write too often in memory of the same place?

RS

hgrueneis · Joined: 04 Apr 2009 Posts: 902 Location: A-4786 Brunnenthal

Snow,

the limited number of write cycles can be overcome by the use of an external Fram.
Has also the advantage of no wait cycles when writing.
You will most likely never reach the write limit of a Fram chip in your lifetime.
Those chips are very cheap and available for I2C and SPI operation.
Formerly Ramtron, now Cypress.

Regards
Hubert

snow · Posted: Wed Mar 17, 2021 1:09 am Post subject:

Hi all
Thanks for all your feedback. You have all made some very interesting comments and that has given me more questions.
The write cycle count into the EEPROM for my device is reasonably short for the life on the device. Maybe 500 for the most accessed address. The most critical address is only written maybe 10 times over the life of the device. I will

I like writing the same variable 4 time and recheck all for validation. Can you specify where in the EEprom map the variable will be stored? I would like to give room between variables as most of the corruption is confined to one area of the map. I also like the CRC error check idea.

A lot of the Bascom documation states 'An external EEPROM is a safe alternative to the internal EEPROM.'... Why is the onboard EEPROM unreliable vs external? I know there is good speed and cycle write improvements with external EEPROM but this is not an unreliability issue with a short cycle write count.

The datasheet on the ATMEGA328P states 100k write/erase cycles. Is there a limit of the read cycle count. I've seen some documents as 100000 Read/Write cycles. is this just a typo?

Does the EEPROM only get corrupted if there is a power issue only during a Read/Write action?? or can corruption on EEPROM happen with general power issues?

Thanks everyone again for your replies

Snow.

O-Family · Joined: 23 May 2010 Posts: 320 Location: Japan

The number one cause of EEPROM data corruption is the runaway AVR near 0.7V when the power supply rises and falls slowly.
At this time, if the user's program has an instruction to access the EEPROM, the data in the EEPROM may be corrupted.
This is resolved by enabling BOD.

Second, if the power supply has a lot of pulsed noise, the AVR can run out of control as well.
In such a case, I insert an inductor of about 10uH into VCC of AVR.
Resolving these two can usually prevent EEPROM data corruption.

In the case of external EEPROM, I2C or SPI interface is used, so error checking works to some extent at the time of communication.
However, I don't think the conditions of damage caused by the runaway of AVR will change significantly whether it is built-in or external.

As for the number of EEPROM rewrites, 100,000 is the minimum guaranteed number, and I don't think that damage will progress suddenly even if it exceeds that.

Although the absolute address of EEPROM can be specified with the [WRITE EEPROM] command, it is not recommended because it complicates the program.
If you want to space the write areas, you can declare a dummy ERAM variable (not used) to spread the addresses widely.

albertsm · Joined: 09 Apr 2004 Posts: 5920 Location: Holland

the first AVR chips had a problem at the first memory location. That was simple to work around by not using this location.
i think the power supply is the biggest cause for failure. but since no details were shared about the cases in which failure occurs i can not guarantee that using brown out fix it all.
since the begin i used the internal eeprom in devices i made for customers. i took all precautions but still once in a while there was an odd case of bad eeprom.
since using I2C or SPI requires specific commands to be executed the changes that something goes wrong when the chip goes crazy is way smaller.
in fact i never had a problem again after i switched to use an external eeprom. So that is why you find these remarks.
when you make a lot of pcb/devices or you need the additional security of the internal eeprom it makes more sense to use the internal eeprom. (hackers can not read the internal eeprom).

but what ever you use it is good idea to have defaults in flash code. this way you can reset things when needed.
to use multiple locations is an excellent idea. i never thought about it myself.
adding crc will make it secure and better.

O-Family · Joined: 23 May 2010 Posts: 320 Location: Japan

Hello, Mark

A long time ago, I also encountered a case where the address 0 of the built-in EEPROM was corrupted.
Since then, I still try not to use EEPROM addresses 0 to 3 as dummies.

I have sold hundreds of AVR chips with many customer products and self-made programs written on them, but there are only a few failures due to EEPROM corruption.
Probably, it is an unavoidable error that the power supply error occurs while accessing the built-in EEPROM when the power is turned on/off frequently or the power supply has a lot of pulse noise.

For products where EEPROM data is important, it is best to take a majority vote by distributed writing of data or to alert an abnormality with a checksum.

If you want to secure security with external storage, you can encrypt the data and prevent copying of the device by using a chip that has an ID for each device such as DS2505.