We may also having a very similar issue with the R15 on FX30S. We are seeing AT+WDSC? report an error in both microcom /dev/ttyAT as well as in the logread logs from the at service startup.
Even worse, In our case, the avcService is reporting this error and then, within 5 minutes, causing a watchdog reboot of the entire modem. Stopping the app doesnt prevent the reboot so we are at a standstill (for now) moving to R15. I suspect it related to the WDSC issue, but couldnt be certain.
We loaded the R15 legato via leaf and we got the R15 legato cwe firmware file from Sierra Wireless in Australia (courtesy of Bill Z) with the RS232 DCD control signal memory leak fix for ioctl() included.
Would you be able to provide a summary or link to the details of this R15 bug/issue for either avcService or AT+WDSC? so we can apply the same fix to the R15 code base we are using?
Mar 17 13:18:38 fx30s user.notice root: config_avms_at: AT+WDSC? returned ERROR; retrying…
Mar 17 13:18:38 fx30s user.debug Legato: DBUG | atQmiLinker/atQmiLinker T=unknown | atForward.c QmiAtCmdCb() 161 | Received:at+WDSC?^M ' Mar 17 13:18:38 fx30s user.debug Legato: DBUG | atQmiLinker/atQmiLinker T=Writer | atLinker.c Writer() 334 | cmd: at+WDSC?^M ’
Mar 17 13:18:38 fx30s user.err Legato: =ERR= | atServerDaemon/atServer T=main | le_atServer.c ParseAtCmd() 2439 | Error in parsing AT command, lastState 0, current state 2
Mar 17 13:18:38 fx30s user.debug Legato: DBUG | atQmiLinker/atQmiLinker T=Reader | atForward.c SendIntermediateResponse() 416 | ##^M ERROR^M ##
Mar 17 13:18:39 fx30s user.warn Legato: -WRN- | modemDaemon/le_pa T=main | pa_sim_qmi.c MapSimState() 1012 | More than one application 2
The Watchdog issue after 5 minutes (My real issue)
Mar 17 13:23:25 fx30s user.crit Legato: CRT | watchdog/watchdogDaemon T=main | watchdog.c WatchdogHandleExpiry() 487 | Mandatory watchdog double fault on process [avcService][avcDaemon]
Mar 17 13:23:25 fx30s user.emerg Legato: EMR | watchdog/le_pa_wdog T=main | pa_wdog.c pa_wdog_Shutdown() 72 | Watchdog expired. Restart device.
Mar 17 13:23:25 fx30s user.emerg Legato: EMR | xtensorMain/framework T=main | messagingSession.c ClientSocketHangUp() 870 | Session closed by server (xtensorMain.xtensorMain.le_wdog:572ba176b790548eed89b7ba6f9f4b14).
Mar 17 13:23:25 fx30s user.emerg Legato: EMR | dcsDaemon/framework T=main | messagingSession.c ClientSocketHangUp() 870 | Session closed by server (le_wdog:572ba176b790548eed89b7ba6f9f4b14).
Mar 17 13:23:25 fx30s user.emerg Legato: EMR | cellNetService/framework T=main | messagingSession.c ClientSocketHangUp() 870 | Session closed by server (cellNetService.watchdogChain.le_wdog:572ba176b790548eed89b7ba6f9f4b14).
Not quite the same version. The build we are using came from Bill Z at Sierra Wireless in Australia. He added a patch for us for the serial DCD memory leak issue and he is also not seeing the reboot. Looking at the versions (below), it looks like you may have a point release better version of the Yocto and RootFS. I will send Bill the link to this chat. my WDSC? command simply fails with ERROR. If I have an older code base, it may also explain the rebooting Legato. The firmware came from Bill’s build and the Dev environment came from Leaf within the last week or two.
│ Workspace: /home/dtluser/myWorkspace │
│ Profile: R15fx30catmstable [current] (sync) │
│ Included │ swi-fx30-catm_4.1.0 │ SDK for FX30-CATM (Release 15.0.5.002 + Legato 21.05.0) │
A quick thought from Bill on this topic, potentially relevant. We are testing this upgrade on an ex Octave FX30S. (Only stock available was Octave) which we have repurposed for non Octave use - we dont use Octave.
I am not sure a firmware upgrade can fully convert an Octave FX30S to standard FX30S.
There is a discussion here which explains at least it would have issue with connecting to AV. Possible this is the cause of triggering avcService watchdog.
Bill said: AT+WDSC? works on my FX30S.
Brett aid (me): We didnt have any issues replacing Octave using R14.
The radio firmware version should not be the issue here. Brett is using the radio version certified to Telstra which is a little different than the one in your WP7702.
I run the same firmware as Brett’s on a standard FX30S(non-Octave) and I cannot replicate this issue. So far I can see the difference between Brett and my side is that Brett is using an Octave FX30S. I tend to think it’s Octave version of FX30S related, but Brett also mentioned no issue to run R14 on the same device. Would be useful to get some input from Octave team on how to apply standard firmware to Octave FX30 I suppose? Otherwise, best chance is to go with standard FX30S I’d think.
Full app status and app info list below if it helps. the udpsend, xtensorAutoExec and xtensorMain ones are ours and I have reproduced this issue with them removed. Happy to delete again and retry if this is important.
I was using the Verizon release directly off leaf and I could never get it to work. What was interesting is that when I built the Yocto image off the source code and updated my FX30s it worked properly and I could make an AV connection but the ones packaged with leaf did not work for me.
The issue was that our Octave modems, once upgraded to R15 started rebooting after 5-8 minutes. It seems that this is because AT+WDSC? and related commands started failing (ERROR), which in turn resulted in AVC/QMI failing to initialise and causing a modem wide watchdog reboot.
The issue doesn’t occur when you take a non-Octave generic modem and then apply R15. Ie. AT+WDSC? works and no reboot occurs. We have a number of Octave modems that were provided due to low stock levels for FX30S.
The good news is that we have found that downgrading an Octave modem to R14 and then upgrading it to R15 solves the problem and the AT+WDSC? command starts working and the reboots go away. I haven’t completed testing so while I cant be 100% certain it is stable – it seems ok.
This suggests that the R15 upgrade doesn’t replace all of the components in the modem, and R14 may replace the missing Octave specific settings. This appears to be an Octave related issue and we now have a workaround.
A painful week, but with a positive end for the weekend.