Sandboxed app does not start

Legato Application Framework
1

I can’t start my sandboxed application unless I use dev studio. I run legato 14.10. Below you can see the log when I try to start my application. It does not matter if autostart or if I use target tools like “app start …” or host tools like startapp. It seems like the /tmo/ds/ folder is never created.

Jan 22 14:22:14 swi-mdm9x15 user.warn Legato: -WRN- | appCtrl[1776]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'ClientThreadData' overflowed. Expanded to 1 blocks. Jan 22 14:22:14 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/bin' into sandbox '/tmp/legato/sandboxes/cloud/bin'. Jan 22 14:22:14 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/dev/log' into sandbox '/tmp/legato/sandboxes/cloud/dev/log'. Jan 22 14:22:14 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/dev/null' into sandbox '/tmp/legato/sandboxes/cloud/dev/null'. Jan 22 14:22:14 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/dev/zero' into sandbox '/tmp/legato/sandboxes/cloud/dev/zero'. Jan 22 14:22:14 swi-mdm9x15 user.err Legato: =ERR= | supervisor[572]/security T=main | sandbox.c ImportFile() 626 | Could not import '/tmp/ds/cloud/core' into sandbox destination '/tmp/legato/sandboxes/cloud/home/appcloud/core'. No such file or directory Jan 22 14:22:14 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c sandbox_Remove() 1257 | 'cloud' sandbox removed. Jan 22 14:22:14 swi-mdm9x15 user.err Legato: =ERR= | supervisor[572]/security T=main | app.c app_Start() 582 | Could not create sandbox for application 'cloud'. This application cannot be started.

If I use dev studio to start my app it works fine and the log looks like this. I can see that the /tmp/ds/ is created.

Jan 22 14:22:45 swi-mdm9x15 user.warn Legato: -WRN- | appCtrl[1870]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'ClientThreadData' overflowed. Expanded to 1 blocks. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/bin' into sandbox '/tmp/legato/sandboxes/cloud/bin'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/dev/log' into sandbox '/tmp/legato/sandboxes/cloud/dev/log'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/dev/null' into sandbox '/tmp/legato/sandboxes/cloud/dev/null'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/dev/zero' into sandbox '/tmp/legato/sandboxes/cloud/dev/zero'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/tmp/ds/cloud/core' into sandbox '/tmp/legato/sandboxes/cloud/home/appcloud/core'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportDir() 674 | Imported directory '/opt/legato/apps/cloud/lib' into sandbox '/tmp/legato/sandboxes/cloud/lib'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/lib/ld-linux.so.3' into sandbox '/tmp/legato/sandboxes/cloud/lib/ld-linux.so.3'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/lib/libc.so.6' into sandbox '/tmp/legato/sandboxes/cloud/lib/libc.so.6'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/lib/libgcc_s.so.1' into sandbox '/tmp/legato/sandboxes/cloud/lib/libgcc_s.so.1'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/usr/local/lib/liblegato.so' into sandbox '/tmp/legato/sandboxes/cloud/lib/liblegato.so'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/lib/libm.so.6' into sandbox '/tmp/legato/sandboxes/cloud/lib/libm.so.6'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/lib/libpthread.so.0' into sandbox '/tmp/legato/sandboxes/cloud/lib/libpthread.so.0'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/lib/librt.so.1' into sandbox '/tmp/legato/sandboxes/cloud/lib/librt.so.1'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/usr/lib/libstdc++.so.6' into sandbox '/tmp/legato/sandboxes/cloud/lib/libstdc++.so.6'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/tmp/legato/serviceDirectoryClient' into sandbox '/tmp/legato/sandboxes/cloud/tmp/legato/serviceDirectoryClient'. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | sandbox.c ImportFile() 630 | Imported file '/tmp/legato/serviceDirectoryServer' into sandbox '/tmp/legato/sandboxes/cloud/tmp/legato/serviceDirectoryServer'. Jan 22 14:22:45 swi-mdm9x15 user.warn Legato: -WRN- | supervisor[572]/security T=main | cgroups.c cgrp_mem_SetLimit() 510 | The memory limit for appcloud was actually set to 40960000 instead of 40000k because of either page rounding or memory availability. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxCoreDumpFileBytes to value 524288. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxFileBytes to value 524288. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxLockedMemoryBytes to value 8192. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxFileDescriptors to value 256. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxMQueueBytes to value 512. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxThreads to value 20. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | resourceLimits.c SetRLimit() 287 | Setting resource limit maxQueuedSignals to value 100. Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[572]/security T=main | proc.c StartProc() 767 | Starting process cloud with pid 1874 Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | supervisor[1874]/security T=main | proc.c StartProc() 744 | Execing 'cloud' Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/cloud_exe T=main | _main.c main() 51 | == Log sessions registered. == Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/<invalid> T=main | cloud_server.c cloudinterface_AdvertiseService() 269 | ======= Starting Server cloudexternal ======== Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/framework T=main | messagingMessage.c msgMessage_CreatePool() 110 | Pool name truncated to 'msgs-db95c53235d995b5addabbb901' for protocol 'db95c53235d995b5addabbb9012cf3cb18a8d0207c401dda2068062d73a57dad'. Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/framework T=main | messagingMessage.c msgMessage_CreatePool() 110 | Pool name truncated to 'msgs-b9d250b855f3dff38ca6cf6fc8' for protocol 'b9d250b855f3dff38ca6cf6fc8a2f761a972379695e06b324d1d15ea84a79190'. Jan 22 14:22:45 swi-mdm9x15 user.warn Legato: -WRN- | configTree[578]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'userPool' overflowed. Expanded to 3 blocks. Jan 22 14:22:45 swi-mdm9x15 user.warn Legato: -WRN- | cloud[1874]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'ClientThreadData' overflowed. Expanded to 1 blocks. Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/<invalid> T=main | le_cfg_client.c le_cfg_ConnectService() 307 | ======= Starting client for 'cloud.cloud_component.le_cfg' service ======== Jan 22 14:22:45 swi-mdm9x15 user.warn Legato: -WRN- | cloud[1874]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'SafeRef-Map' overflowed. Expanded to 11 blocks. Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/framework T=main | messagingMessage.c msgMessage_CreatePool() 110 | Pool name truncated to 'msgs-eb053a52294ff2a141235ae33f' for protocol 'eb053a52294ff2a141235ae33faacd3e8e9813406ddc799112207095a63f6001'. Jan 22 14:22:45 swi-mdm9x15 user.warn Legato: -WRN- | cloud[1874]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'ClientThreadData' overflowed. Expanded to 1 blocks. Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/<invalid> T=main | le_sms_client.c le_sms_ConnectService() 307 | ======= Starting client for 'cloud.cloud_component.le_sms' service ======== Jan 22 14:22:45 swi-mdm9x15 user.debug Legato: DBUG | cloud[1874]/cloud_exe T=main | _main.c main() 67 | == Starting Event Processing Loop == Jan 22 14:22:45 swi-mdm9x15 user.info Legato: INFO | cloud[1874]/cloud_component T=main | cloud_component.c _cloud_component_COMPONENT_INIT() 429 | Cloud App started

This is the content of my adef file

[code]start: auto
version: 1.0.0
maxFileSystemBytes: 512K
executables:
{
cloud = ( cloud_component
$LEGATO_BUILD/airvantage/runtime/lib/libreturncodes.so
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_AirVantage.so
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_DeviceTree.so
$LEGATO_BUILD/airvantage/runtime/lib/libEmp.so
$LEGATO_BUILD/airvantage/runtime/lib/libyajl.so
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_log.so
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_DSet.so )
}

processes:
{
envVars:
{
LE_LOG_LEVEL = “DEBUG”
}
run:
{
( cloud )
}
priority: medium
faultAction: restartApp
maxCoreDumpFileBytes: 512K
maxFileBytes: 512K
}

bundles: { file: { $LEGATO_BUILD/airvantage/runtime/lib/libreturncodes.so /lib
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_AirVantage.so /lib
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_DeviceTree.so /lib
$LEGATO_BUILD/airvantage/runtime/lib/libEmp.so /lib
$LEGATO_BUILD/airvantage/runtime/lib/libyajl.so /lib
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_log.so /lib
$LEGATO_BUILD/airvantage/runtime/lib/libSwi_DSet.so /lib } } bindings:
{
cloud.cloud_component.le_sms → modemService.le_sms
cloud.cloud_component.le_cfg → modemService.le_cfg
}
requires:
{
configTree: {
[w] cloud } file: { /tmp/ds/cloud/core /home/appcloud/core }
}
provides:
{
api: { cloudexternal = cloud.cloud_component.cloudinterface }
}[/code]

2

When you create your app with DS, a binding is automatically added to this file.
The purpose of this binding is to have the potential core file (generated if the app crashes) bound out of the sandbox. Without that, the core file can’t be retrieved once the sandbox is destroyed.
The drawback is that the /tmp/ds/app/core file must exist before the app is started.
This is managed automatically by DS; but you you want to start the app from the command line (after a device reset, as /tmp storage is volatile), you have two options:
[ul]
[li] either remove the binding from the adef file (knowing that you won’t have core file persisted if the app crash)[/li]
[li] or touch the file before doing the app start[/li][/ul]

Note that this behavior is going to change in 15.01: core files will be automatically preserved without having to make an explicit binding in the adef file.

3

Hi,

I have a similar problem, I also buid my application with the developer studio.

I removed the binding in my adef file but the problem persists.

Here are my console logs:

root@swi-mdm9x15:~# app stop pocauto
Stopping app 'pocauto'...
DONE
root@swi-mdm9x15:~# app start pocauto
Starting app 'pocauto'...
There was an error.  Application 'pocauto' could not be started.
root@swi-mdm9x15:~#

And here legato logs

00:11:27 INFO | supervisor[565]/security T=main | app.c KillAppProcs() 669 | App 'pocauto' runs as root.  Can't kill all processes running as root.
00:11:27 INFO | supervisor[565]/security T=main | proc.c proc_SigChildHandler() 1253 | Process 'GPSEXE' (PID: 1199) has exited due to signal 15.
00:11:27 INFO | supervisor[565]/security T=main | proc.c proc_SigChildHandler() 1253 | Process 'AVConnectorEXE' (PID: 1200) has exited due to signal 15.
00:11:27 INFO | supervisor[565]/security T=main | app.c app_SigChildHandler() 1483 | app 'pocauto' has stopped.
00:11:27 =ERR= | supervisor[565]/security T=main | cgroups.c cgrp_Delete() 365 | Could not remove cgroup '/sys/fs/cgroup/cpu/apppocauto'.  Tasks (process) list may not be empty.  Device or resource busy.
00:11:27 =ERR= | supervisor[565]/security T=main | resourceLimits.c resLim_CleanupApp() 462 | Could not remove cpu cgroup for application 'pocauto'.
00:11:27 =ERR= | supervisor[565]/security T=main | cgroups.c cgrp_Delete() 365 | Could not remove cgroup '/sys/fs/cgroup/memory/apppocauto'.  Tasks (process) list may not be empty.  Device or resource busy.
00:11:27 =ERR= | supervisor[565]/security T=main | resourceLimits.c resLim_CleanupApp() 462 | Could not remove memory cgroup for application 'pocauto'.
00:11:27 INFO | supervisor[565]/supervisor T=main | supervisor.c DeleteAppObj() 487 | Application 'pocauto' has stopped.
13:35:29 user.err kernel: [  119.747535] NOHZ: local_softirq_pending 08
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPDISCOVER(eth0) 10.41.51.9 74:e5:0b:86:b2:a2 
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPOFFER(eth0) 10.0.0.45 74:e5:0b:86:b2:a2 
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPDISCOVER(eth0) 10.41.51.9 74:e5:0b:86:b2:a2 
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPOFFER(eth0) 10.0.0.45 74:e5:0b:86:b2:a2 
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPINFORM(eth0) 10.41.51.9 74:e5:0b:86:b2:a2 
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPACK(eth0) 10.41.51.9 74:e5:0b:86:b2:a2 asgard
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPDISCOVER(eth0) 10.41.51.9 74:e5:0b:86:b2:a2 
13:35:33 daemon.info dnsmasq-dhcp[323]: DHCPOFFER(eth0) 10.0.0.45 74:e5:0b:86:b2:a2 
13:35:33 -WRN- | appCtrl[1249]/framework T=main | mem.c le_mem_ForceAlloc() 664 | Memory pool 'ClientThreadData' overflowed. Expanded to 1 blocks.
13:35:33 =ERR= | supervisor[565]/security T=main | cgroups.c cgrp_Create() 289 | Cgroup /sys/fs/cgroup/cpu/apppocauto already exists.
13:35:33 =ERR= | supervisor[565]/security T=main | cgroups.c cgrp_Delete() 365 | Could not remove cgroup '/sys/fs/cgroup/cpu/apppocauto'.  Tasks (process) list may not be empty.  Device or resource busy.
13:35:33 =ERR= | supervisor[565]/security T=main | app.c app_Start() 600 | Could not set application resource limits.  Application pocauto cannot be started.

Thanks for your help

David

4

Just a heads-up that there may be confusion because the name “binding” is used to mean two different things in this discussion thread.

There is a “bindings” section in the .adef. This is NOT the “binding” that needs to be removed. Actually, it is a “required file” that needs to be removed from the .adef.

Maybe this was clear to you already, though.

1 Like
5

Hi David,

It looks like the problem you have with not being able to start your app, pocauto, is due to a bug with unsandboxed apps. When an unsandboxed app’s processes forks another process, the child process inherits its parent’s cgroup. However, when the app is stopped the child process is not killed and so the cgroup is not empty and cannot be deleted.

I have created a ticket for this bug and will work to get it in the next release.

In the meantime, the workaround is to kill all the processes in all of the app’s cgroups. You can do this in a few different ways:

  1. If you know what the names of the forked/exec’d processes are in the app you can just kill them with the killall utility.

  2. You can get the list of PIDs in the cgroup by reading “/sys/fs/cgroup/cpu/apppocauto/tasks” using cat. Then kill all those processes with the kill utility.

  3. Restart your target.

Sorry for the inconvenience.

Best regards,

Alex

6

Hi Alex,

Thanks your answer, it will help !

I will kill all processes manually.

David

1 Like