We have had several instances of our product get stuck in in a reboot loop but have never been able to diagnose the issue because we didn’t have UART console access enabled when the issue occured, until now that is.
I managed to capture the follow trace which appears to denote the I2C is busy but yet both SDA and SCL are high according to the log message.
Console Log Stuck I2C on Legato Boot.log (89.4 KB)
Hey Darren,
Sorry to hear this bug is still around. I’ve always suspected this is a bug somewhere in either:
a) the low power MCU inside the WP85 module (I believe it’s on the stuck address 0x3a but scan the bus to be sure)
b) the driver for the Qualcomm Universal Peripheral Engine (QUP) i2c adapter
Any comment regarding the MCU would have to come from Sierra Wireless since that code is closed source. Perhaps @dfrey could point you in the right direction.
The QUP i2c driver was introduced to the Linux kernel in this patch (i2c: QUP based bus driver for Qualcomm MSM chipsets [LWN.net]). You may be able to find some more discussion in the kernel mailing list for this driver. I would also check to see which version is rolled into the WP85 firmware by downloading the release 16 tarball. I suspect it’s a custom/back-ported version of the driver since it’s not maintained for Linux 3. You may be able to back-port some of the newer patches yourself (have a look at the history here History for drivers/i2c/busses/i2c-qup.c - torvalds/linux · GitHub).
Best of luck with this bug.
Happy hacking,
Nick
Any thoughts on this issue? I was pointed to this post as the possible cause WP8548 I2C bus lock up when read ADC2 and ADC3 - #9 by plu but I’m not reading the WP85 ADC.
Hey @dbeckwith,
This patch makes some critical additions to the QUP i2c adapter driver, specifically around the bus busy issue, so I would be very surprised if this did not help. I compiled it for my mangOH Red and I’ve only seen a positive result so far. I believe the ADC on the mangOH is implemented over i2c but I could be incorrect here.
Happy hacking
hi @dbeckwith,
can you provide logs when you face the issue, please?
BR
Sorry for the delay here are the logs with the extra debugging you asked me to run. I have removed our our app logs for the moment.
Please let me know if you need additional details.
I2C Lockup Debug Oct 24.log.zip (280.3 KB)
The source I2C bus lock up turned out to be the LTC2942 battery gas gauge. The SDA pin is stuck low on boot. I have contacted Analog Devices regarding this issue but there suggestions didn’t correct the issue. What’s interesting is the LTC2942 SDA line comes high after some clock cycles on the bus.
Has anyone else encountered this problem?
Darren
Fantastic work @dbeckwith, thanks for sharing.
Were you ever able to get a reading from the LTC2942 sensor?
Yes, when the unit booted properly I was able to get a readings from the LTC2942.
Did you solve the problem of LTC2942 above?
Hi, No I did not. I just stopped using it.
