For modem 2, yes I can get it to work… Although only manually… i.e. not by re-flashing the modem.
Modem 1 has not been fixable through the re-flashing process. But I’ve just had a session with the customer and after researching the “Union” style filesystem way of managing deleted files etc through whiteout files, I was able to fix the problem… So in case it is useful for others, I detailed a bit of it below.
Basically the problem was that the filesystem which is AUFS and UFS based got itself out of sync. These filesystems allow multiple mapping for the same directories and use .wh file to indicate a file is deleted if one of the two “branches” or source locations are read-only (as they are on the FX30).
However on my modem, it got the two branches out of synch… i.e. the one branch suggested the file (“startlegato.sh” in my case) was deleted (whiteout in the error message) and the other suggested is was present (or “real” in the error message).
This seems to have happened because I was replacing the file on each boot (bad implementation and now fixed). And something went wrong… maybe a power failure?
When I restarted the modem I found this error message in the logs (au_lkup_dentry:223:chmod: I/O Error, both of real entry and whiteout found, startlegato.sh, err -5)
I could not replace, delete, chmod the file… The filesystem would complain as the current state of the file was unresolvable.
The solution was to (spend many hours understanding how the filesystem works ) and then on the two branches find the .wh (the whiteout file) related to my startlegato.sh and remove it, so that both branches thought the file existed. In my case the .wh was located in /mnt/flash/ufs/etc/init.d. It is hidden so ls -la or similar…
Now that the two branches where in synch again, I could replace/delete/chmod the file again and restore my system by re-flashing it…
Conclusion: I could not find a way to fix the filesystem or general linux problems on the modem using SW provided re-flash, reset.cwe, factory reset methods. None would re create/re install the linux system and the filesystems.
So the only way to resolve the problem was to fix the problem in linux directly.
Lesson: Don’t overwrite files if not required… Other than the flash write cycle limit, the OS and filesystem can get corrupted if the process is (I assume) interrupted…
I hope that saves someone a lot of pain down the line…