Updated spiffs

[raburton]

@ALL I've tried moving to the latest upstream spiffs. You can check it out in the feature/newspiffs branch. You'll need to use an updated spiffy as well, which you can find in my own repo (feature/newspiffs branch again). I've improved the integration somewhat so it will be easier to update again in the future. Also trying to push a couple of changes upstream. If people who use spiffs could run with this branch a bit and see if it works and if there are any problems I'd appreciate it. Please report any issues (or success) here...

[alonewolfx2]

finally someone make new and good spiffy and lastest spiffs. thank you man :)

[raburton]

I have now added spiffy to the branch so it is part of Sming. It may have been based on a third party tool, but it's had so much rework now that it can't really be an external component now. So we'll take the new version in to the project and maintain it ourselves. It uses the same copy of spiffs that Sming uses so it should always be the right version for the user. It should build as standard and work out of the box for apps that need it. Not tested on windows, please try out this branch and provide feedback.

[hreintke]

@raburton
I did not follow the spiffs/spiffy discussion in detail but at first sight have the following questions.

• We add a new dependency to sming (gcc compiler which builds spiffy).

Is there any specific requirement on versions for that (mingw/mingw64/other ??)

• We have a new version of spiffs in sming.

Will current spiffs filesystems on esp8266 still work correct or need they a fresh flash ?

[hreintke]

Update (early morning here..)

We can add another repository to the sminghub organization where spiffy is located.
Would that give a clearer view on building "host software" necessary for sming ?

[raburton]

Yes you need gcc, but then you do if you want to build spiffy outside the Sming tree. I suppose if people don't want to use spiffs at all at the moment they wouldn't need gcc. I can easily make it optional to build it.

There shouldn't be any specific requirements, but I've only tested it on Linux, so who knows about windows (hence why I asked people to test it). As recently discussed Windows is a mess - so many versions of compilers, different environments and people generally not knowing what they are doing with them!

The version of spiffs we have in master has a bug in it's packing of some structures, which meant that spiffs images produces by different builds of spiffy and/or libsming might not be compatible already. The updated version of spiffs fixes that, but might make the spiffs image incompatible with the old (incorrectly) created previous images. Adding our own version of spiffy to Sming means that the user will always have the right version of code in spiffy to match the version in libsming.

[raburton]

We can add another repository to the sminghub organization where spiffy is located.
Would that give a clearer view on building "host software" necessary for aming ?

It would be a repository with 2 files in it and would be awkward to link to the spiffs files in Sming. Or you could duplicate those files and be left in the same situation as present, where the two versions are separate, need to be separately maintained and users will get them out of sync. That would defeat the object.

[hreintke]

Agree with that.
The solution you mentioned to only include "spiffy build" when needed by application would solve the dependency of gcc for "non spiffs" applications.

[raburton]

Ok, made building spiffy optional.

[raburton]

After a number of people have provided feedback on the new spiffs and spiffy (and a few little windows compiler related issues have been fixed) I have merged the feature branch into develop for wider testing. Spiffy is now included and can be built by calling make spiffy in the Sming/Sming, or just make directly in the spiffy dir. Going forward people should use this copy, which is built from the same spiffs source as the libsming, to ensure compatibility. There should be a comment added to the release notes to this effect.

[flexiti]

tests continued.
@raburton
Sorry for this basic question, but
spiffy_rom - It should always be the same size as spiffs mount size (like full disc mirror),
or less sized spiffy.rom is copied in a larger area of spiffs ( like just copy file to existing disc) ?

[raburton]

@flexiti a good question, but I'm afraid I don't know the answer. I think if the extra space you are trying to grow the fs in to is erased it will probably work fine, if it isn't them I'm not sure. I think you'd either need to test it out or ask the spiffs author upstream. I've had a quick look through and it's not obvious to me if it will assume unknown blocks are already erased or if they need to be.

[flexiti]

@raburton
OK, we have to recognize
Here a small example: if spiffs is 64 kb or 16 kb after that :)

Second:
For a user who enters here from the time the settings that you have prepared (they are ok :)) can be difficult.
Maybe you can do something prepared, which will not require knew the flash addresses, etc.

for example.
Basic_rboot with spiffs the way you you offered
and/or
Spiffs_mount_default (type)

eg. type1 = 512K, 100K fs
       type2 = 4M, fs 512K
       type3 = 4M, 3M fs

[raburton]

That's definitely a bug, but not one related to the change in spiffs/spiffy (nor is it specific to the rBoot build). When you don't create a spiffs rom from actual files (i.e. a blank one should be created) the makefile just copies a blank 16k rom, instead of creating a new empty one of the correct size. If you'd had a files directory with stuff in it you'd have got a correctly sized filesystem created. I'll have a look at that when I get chance...

I disagree with the second suggestion though. We need to get away from the previous bad system and move to a simple one where the user is in more direct control of the where the filesystem is written to the flash and how big it is. Especially when using rBoot where the user has a huge amount of flexibility on how to lay out the flash.

[flexiti]

@raburton
so, your rboot is very good, but only a small percentage of people will benefit.
Most (90% ??? )would like to use your one rom rboot, and mount the file system without going into details about the organization of memory - rboot is the only addition to the actual program :)

For a large number of programmers memory organization, type of flash is a subject which never encountered - because there was no need, moreover, a large part of them do not have any (or have basic) electronically knowledge. If you want to rboot was used more widely, however, try to understand my advice.
Anyway, my proposal does not limit your rboot,I sugest just such a "fast defaults" function - who wants to play can take advantage of rboot offered .

just at the begining of user program I can call Spiffs_mount_defalut(type1) or spiffs_mount_manual(...) as most adwatage version, Basic Reboot can show these two methods.

Or Spiffs_mount_default() where what exactly does this function is defined in the make and Spiffs_mount_manual(...) as so far.
For example.
#define MANUALSPIFFS and all definition as so far
..
..
or if not MANUALSPIFFS
#define AUTOSPIFFS TYP1 // typ1 = 512k flash, 1 rom, 100k spiffs etc.

I am with you and not against - just look from the user side :)
The decision is yours.

ps. I still have doubts (perhaps I do not understand) for rom0.ld in new project, I have to read a little more and test.

[raburton]

You can have a set of sensible presets for different layouts based on flash size. rBoot creates a default config with the chip split in half for two roms, and you could easily use this as the basis for some sensible default positions +/- sizes, but there will still be many combinations (different flash sizes, big flash support, two rom mode, etc. and people are using combinations of these options with Sming) and it'll make a mess of the Makefile and include files to try and offer all these as options. I think a much more sensible approach would be to simply offer a set of options in a simply explained tutorial. e.g. "If you have a 4mb flash you can have 2 x 1mb roms and 2 x 2mb spiffs, to do this paste these options into your Makefile-user.mk..., and this block of code into init to mount the fs..."

Anyway this discussion is unrelated to the purpose of this issue, which is about the new spiffs/spiffy code, so if you want to make suggestions (or better yet provide PRs for how they these suggestions might be well implemented!) please open a separate issue.

[flexiti]

I finish the rboot test because it misses the point - all is ok

[raburton]

included in v1.4.0