After Reboot openwisp keep crashing

[momothefox]

I have this issue i could not find a solution for,
My controller is openwisp2 installed by this week with SSL from letsencrypt.
I have my org. added
and i am building custom firmware image for each org. contains the pkg and it's configuration.
including Shared secret and UUID while the option key left empty.

After 1st boot device get registered on the fly with no problems at all..
i can update device configuration as long as it is still powered on..
i am using file method to update Chilli configuration file and file also for changeing SSID to avoid much work - it is easier for me that way.

if the device gets rebooted there is no way to connect again to controller. i have to reset it.

[nemesifier]

@momothefox please show the logread | grep openwisp output.

[momothefox]

@nemesisdesign This is the output after 1st boot
daemon.info openwisp: OpenWISP config agent started
daemon.notice procd: /etc/rc.d/S99openwisp_config: openwisp: OpenWISP config agent started
daemon.info openwisp: Registering device...
daemon.info openwisp: New device registered successfully as Sameh-Makram, id: a1f26d7afdba405c85570ad3d1ee6c02
daemon.info openwisp: Local configuration outdated
daemon.info openwisp: Downloading configuration from controller...
daemon.info openwisp: Configuration downloaded, now applying it...
daemon.info openwisp: Testing configuration...
daemon.info openwisp: Configuration test succeeded
daemon.info openwisp: Configuration applied successfully

And this is the output after a reboot
daemon.info openwisp: OpenWISP config agent started
daemon.notice procd: /etc/rc.d/S99openwisp_config: openwisp: OpenWISP config agent started
daemon.info procd: Instance openwisp_config::instance1 s in a crash loop 6 crashes, 0 seconds since last crash

Note that if i force the device to reregister it self again it will work.

I tried manualy excuting command like /usr/sbin/openwisp_config --url https://sub.domain.com --interval 120 --verify-ssl 1 --uuid a1f26d7afdba405c85570ad3d1ee6c02 --key f43ea66672c7c34c4452b6c1ee8b660c --consistent-key 1 --merge-config 1 --test-config 1 --mac-interface eth0

i get this output
daemon.info openwisp: Device info updated on the controller

and if changed the config for the device on the controller and re run the command
this is the output
daemon.info openwisp: Local configuration outdated
daemon.info openwisp: Downloading configuration from controller...
daemon.info openwisp: Configuration downloaded, now applying it...
daemon.info openwisp: Testing configuration...
daemon.info openwisp: Configuration test succeeded
daemon.info openwisp: Configuration applied successfully

my openwisp version is 0.6.0a

[nemesifier]

Please can you try to run the command manually before this line:

daemon.info procd: Instance openwisp_config::instance1 s in a crash loop 6 crashes, 0 seconds since last crash

This should give us a hint about what is going on.

[momothefox]

@nemesisdesign Ok i had to disable the service 1st.
after boot up i started it with /etc/init.d/openwisp_config start
and then excuted that manual command. it did not crash. and when i changed config on the controller. device updated it self withing 120 sec.
output of logread | grep openwisp
service start >
daemon.info openwisp: OpenWISP config agent started
after command >
daemon.info openwisp: Device info updated on the controller
After change of config on the controller>
daemon.info openwisp: Local configuration outdated
daemon.info openwisp: Downloading configuration from controller...
daemon.info openwisp: Configuration downloaded, now applying it...
daemon.info openwisp: Service chilli has been reloaded via procd/ubus
daemon.info openwisp: Testing configuration...
daemon.info openwisp: Configuration test succeeded
daemon.info openwisp: Configuration applied successfully

and this is the output of excuting the command.
sh: out of range
sh: /tmp/openwisp/update_info: unknown operand
sh: /tmp/openwisp/checksum: unknown operand
sh: /tmp/openwisp/checksum: unknown operand
uci: Entry not found
uci: Entry not found
tar: removing leading '/' from member names
uci: Entry not found
uci: Entry not found
sh: /tmp/openwisp/status_report: unknown operand
sh: /tmp/openwisp/checksum: unknown operand
sh: /tmp/openwisp/checksum: unknown operand

[nemesifier]

Can you try the same with the latest build to see if it happens there too?

http://downloads.openwisp.io/?prefix=openwisp-config/latest/

I am not sure what is going on though.

[momothefox]

@nemesisdesign
it says Not downgrading package openwisp-config on root from 2021-12-07-08df34db-1 to 0.6.0a-1.
note that this build from latest snapshot of openwrt 2 days ago.
but it also happen with 19.07.9 with another device.

[nemesifier]

Try removing it first, then install that build, the openwrt feeds doesn't have the latest master.

[momothefox]

@nemesisdesign well after checking my config and comparing it to manual command the only thing in differance is
option bootup_delay '10'
so after removing it from configuration
agent did not crash

[momothefox]

@nemesisdesign and thats it !!!!

[nemesifier]

Glad you found out what went wrong, I opened a bug issue for the bootup delay feature which seems broken: #167.