How to record large amount of data when device is offline

Legato Application Framework
1

Hello all,

Using a MangOH Red board with WP7607 module, I tried to develop an app very similar to the RedSensorToCloud example from the MangOH github. The main difference is that I need to record a lot of sensor data even when my device is offline. For instance I need to record the accelerometer and gyro 3-axis every 100ms + GPS every 1sec during several hours.

I used the le_avdata_CreateRecord API to create my record. But after few seconds I can not add more data into it (le_avdata_Recordxxxx API returns LE_NO_MEMORY).
I tried to create a list of records, but again after few minutes or hours my application stops (getting “le_avdata_client.c SessionCloseHandler() 444 | ======= ‘recorder.avPublisherComponent.le_avdata’ service spontaneously disconnected ========” )

What is the best way to deal with this?

Based on the available RAM on the WP7607 I thought that I could keep more sensor data in RAM.

Or should I store the records in non-volatile memory? Is there a sample code that shows how to do so?

Thanks,
Julien.

3

Thanks Julien!

I tried the 2nd solution (patching Legato), but after few minutes each call to any le_avdata_Recordxxxx API takes longer and longer. After 2 minutes it takes more than 200ms.
Is that expected?

If it is, I will try to store my data outside of the avdata record.

Thanks,

4

I stored my data outside of avdata record.
I initially tried to push the data using le_avdata_PushRecord API (by filling one or several small records just when I needed to push the data). But filling the records took too much time (~5sec for 60sec of recording).

So I am now trying to use the le_avdata_PushStream to push directly the binary data to AirVantage server. However I am not sure if I can then easily retrieve the binary data from the server.
From the timeline page, only some of the transmissions appears.
Using Postman I get this kind of output:

{
    "d421f50f99a04d6daecb3796cc7f036a": {
        "Recorder.sensors": [
            {
                "ts": 1539012235240,
                "v": "ޫETf\u0001"
            }
        ]
    }
}

Is it possible to retrieve the binary data using AirVantage API?

Thanks,

5

@Julien_B

I have a similar situation and I’m curious if you managed to solve the issue?

Thanks,
Darren

6

@dbeckwith

Sorry for the late reply.
I ended up storing my sensors data into files and I send those files periodically using le_avdata_PushStream API.
I also had to convert the sensors data into its ASCII representation in order to “see” it correctly on the AirVantage interface (and through its API)

Hope it helps.

Best Regards,
Julien.

7

@Julien_B

Thanks for getting back to me and describing your solution. If you don’t mind me asking did you create your own file structure or did you use JSON or something like it?

8

I created my own file structure.
Basically I stored the sensor data in raw format, without headers and so on, to optimize the file size as much as possible. On the server side, the decoding is based on the assumption that data won’t be corrupted and will match the expected scheme without any check apart from the file size itself.

9

@Julien_B Thanks for the details!

10

Hello @Julien_B, How did you manage to extract the data from the timeline page? Also does the data from sensors need to be converted to ASCII, if it needs to be written in a local text file?