Added option to check for possible overlapping during write_flash o…

[slaff]

…peration.

[projectgus]

Hi @slaff,

Thanks for submitting this. This seems like a good thing to check for.

Is there a use case for writing overlapping data in a single write_flash command? I'd be happy to have this check always fail write_flash, no additional command line option required. If people really need to flash overlapping data, they can do it by running esptool.py write_flash multiple times.

[slaff]

Is there a use case for writing overlapping data in a single write_flash command?

Probably not. I added it as an option to not break immediately the build of projects using esptool. If you want I can change the PR to enforce overlap detection?

[slaff]

@projectgus Check the latest version of this PR.

[projectgus]

Because esptool erases entire sectors, the address & size can be aligned to 4096-byte boundaries when checking for overlaps - even regions which don't obviously overlap can cause data loss if they land on the same sector.

Something like:

next_end = (address + size + FLASH_SECTOR_SIZE-1) & ~(FLASH_SECTOR_SIZE-1)
address = address & ~(FLASH_SECTOR_SIZE-1)
... # if end > address, etc.
end = next_end

[slaff]

Sorry, my bad. The calculation should be better now and there are two small unit tests related to overlapping.

[projectgus]

Looks good to me. Just have the one addition above about sector alignment.

Also, could you please write a couple of tests for these in tests/test_esptool.py? A test case for the correct error directly overlapping images, and maybe one where they don't overlap but land in the same sector, would be fantastic.

[slaff]

Just have the one addition above about sector alignment.

I am not sure that this should be the case. Look at the following example: AI-Thinker are suggesting in their AT software (http://bbs.espressif.com/download/file.php?id=1470) to initialize the flash memory with the following setup for 4MB flash

0x3fc000 esp_init_data_default.bin
0x3fe000 blank.bin

The second address is in the same sector as the first address.
So I guess that the sector is erased once per session and esptool.py allows writes in that sector. As long as a BIN file does not overlap another bin file, that should be fine.
I do not know if that is plain wrong, but maybe you can throw some light :)

hexdump -C esp_init_data_default.bin
00000000  05 00 04 02 05 05 05 02  05 00 04 05 05 04 05 05  |................|
00000010  04 fe fd ff f0 f0 f0 e0  e0 e0 e1 0a ff ff f8 00  |................|
00000020  f8 f8 52 4e 4a 44 40 38  00 00 01 01 02 03 04 05  |..RNJD@8........|
00000030  01 00 00 00 00 00 02 00  00 00 00 00 00 00 00 00  |................|
00000040  e1 0a 00 00 00 00 00 00  00 00 01 93 43 00 00 00  |............C...|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000070  03 00 01 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080

and

hexdump -C blank.bin 
00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00001000

[projectgus]

Sector size is 4096 bytes (hex 0x1000), so 0x3fe000 and 0x3fc000 are two sectors apart.

(There is also a larger block erase command, but the minimum erase size is one sector ie 0x1000 bytes.)

[slaff]

@projectgus Angus, did you have to look over the changes? It there something more that is needed for the PR to be merged?

[projectgus]

Sorry for the delay @slaff. Need some small changes (run_esptool_error) before one of the tests passed, but squashed and merged as 3fb8eb7 (that commit mentions the wrong PR number by mistake).

Thanks!

[slaff]

@projectgus Great, thanks!