Adding custom root certificate authority (secure MQTT)

We are using a custom Certificate Authority (signing certificates) to make sure http (s) and mqtt (s) is secure.

I’ve been trying to add the custom root ca file to the FX-30 system, but can’t get it to work.

It seems like the root ca file (custom_CA.crt) shall be added to /usr/share/ca-certificates/, but that path is readonly and I don’t know know how to make it writable, or if there is a way to include the custom_CA.crt in the build process. If someone knows if this possible, please share.

The connection and SSL verification works properly if I specify the custom CA like this:

openssl s_client -CAfile /etc/ssl/certs/custom_CA.crt -connect <hostname>:8883 

with response:

SSL handshake has read 2410 bytes and written 335 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

But it doesn’t work if I don’t specify the -CAfile option.

Due to the problems with accepting the server certificate I get an error if I try to connect the MQTT using the secure connection with the “ssl://hostname:8883” format. Connecting to the MQTT broker using plain (1883) works fine.

If anyone has stumbled across something similar, please share you experience. The same will apply if you are trying to access a https server with a customer CA.

you can try this:


root@fx30:~# echo 123 > /usr/share/ca-certificates/123.txt
-sh: can't create /usr/share/ca-certificates/123.txt: Read-only file system
root@fx30:~# ^C
root@fx30:~# mkdir /tmp/tmp_usr_share_ca-certificates;mkdir /tmp/tmp_usr_share_c
a-certificates_wr;

root@fx30:~# mount -t overlay overlay /usr/share/ca-certificates -o lowerdir=/us
r/share/ca-certificates,upperdir=/tmp/tmp_usr_share_ca-certificates,workdir=/tmp
/tmp_usr_share_ca-certificates_wr;
root@fx30:~# echo 123 > /usr/share/ca-certificates/123.txt
root@fx30:~# cat /usr/share/ca-certificates/123.txt
123

I followed the recommendation from @jyijyi and now the openssl s_client test command works. The MQTT broker exposed certificate is accepted.

Now I’m trying to the get my MQTT client (publisher) to work, but I get an error. I’ve tried with both DNS name and IP

const char mqttBrokerURI[] = "ssl://1.2.3.4:8883";

and

const char mqttBrokerURI[] = "ssl://mqtt.mydomain.com:8883";

but I get the same error:

Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | =========================================================
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] |                    Trace Output
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | Product name: Eclipse Paho Synchronous MQTT C Client Library
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | Version: 1.3.0
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | Build level: fre 28 jul 2023 22:52:59 CEST
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | OpenSSL version: OpenSSL 1.1.1n  15 Mar 2022
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | OpenSSL flags: compiler: arm-poky-linux-gnueabi-gcc  -march=armv7-a -mfpu=neon -mfloat-abi=softfp -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-securi
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | used-debug-types -fmacro-prefix-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=  -DOPENSSL_USE_NODELETE -DOPENSS
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | OpenSSL build timestamp: built on: Tue Mar 15 14:37:47 2022 UTC
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | OpenSSL platform: platform: linux-armv4
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | OpenSSL directory: OPENSSLDIR: "/usr/lib/ssl-1.1"
Aug  8 22:28:50 fx30 user.info Legato:  INFO | mqttClientService[10782] | =========================================================
Aug  8 22:28:51 fx30 user.notice kernel: [21494.894770] audit: type=1400 audit(1691533731.792:6): lsm=SMACK fn=smack_inode_permission action=denied subject="app.mqttClient" object="admin" requested=x pid=10782 comm="mqttClientServi" name="ssl" dev="ubifs" ino=30297
Aug  8 22:28:52 fx30 user.notice kernel: [21495.900773] audit: type=1400 audit(1691533732.802:7): lsm=SMACK fn=smack_inode_permission action=denied subject="app.mqttClient" object="admin" requested=x pid=10782 comm="mqttClientServi" name="ssl" dev="ubifs" ino=30297
Aug  8 22:28:53 fx30 user.warn Legato: -WRN- | mqttClientService[10782]/mqttClientService T=main | mqttClientService.c mqtt_Connect() 405 | Socket error

Is there some sort of permission issue:

smack_inode_permission action=denied subject="app.mqttClient" object="admin" requested=x pid=10782 comm="mqttClientServi" name="ssl"

It works when I use a plain connection like this:

const char mqttBrokerURI[] = "tcp://1.2.3.4:1883";

have you tried to set “sandboxed: false” in .adef file?

I tried changing to sandboxed: false in my application.adef before but it didn’t help.

The application I’m writing is using the mqttClientService (and mqttClient). Looking at the .adef definition file here https://github.com/legatoproject/legato-af/blob/master/apps/platformServices/mqttClient/mqttClient.adef the sandboxed is set to true.

I changed the mqttClient.adef files sandboxed to false and now it works. I can also use a DNS name instead of IP address.

Should the mqttClient.adef sandboxed property be set to true, or should all platformServices be set to false by default?

I don’t know, maybe the original code is not for SSL.
If you find it works, then you can use unsandboxed app

I can actually run my application with sandboxed: true. The solution was to change the Legato mqttClient sandboxed property to false.

The code (mqttClient, part of the legato framework) is written and documented to support SSL and also using domain names.

As a new developer getting used to the legato framework it would be much better if the sandboxed attribute is set to false. Then it will also match the code and documentation.

I’m new to the Legato framework and not sure if there is anyone picking up issues like this and will feed it back to the Legato framework. Does anyone know?

I looked around in the forum and there are a couple of similar issues related to the MQTT setup. Addressing the above might help them as well.

@jyijyi , thanks for your help in this topic :pray:

You can see this document on how to use SSL within legato application:

I don’t quite see the original code of mqttClient (platformServices) is including those SSL libraries.
So i expect it is used for plain connection.

The mqttClient/mqttClientService is actually a wrapper around the Paho MQTT client, which is supporting SSL. The reference is found here https://github.com/legatoproject/legato-af/blob/master/apps/platformServices/mqttClient/mqttClientService/Component.cdef

Here is what the documentation looks like in the mqttClientService:

//--------------------------------------------------------------------------------------------------
/**
 * Creates an MQTT session object.
 *
 * @return
 *      LE_OK on success or LE_FAULT on failure
 */
//--------------------------------------------------------------------------------------------------
le_result_t mqtt_CreateSession
(
    const char* brokerURIPtr,           ///< [IN] The URI of the MQTT broker to connect to.  Should be in
                                        ///  the form protocol://host:port. eg. tcp://1.2.3.4:1883 or
                                        ///  ssl://example.com:8883

If you look at the Phao definition here https://github.com/legatoproject/legato-af/blob/master/components/3rdParty/paho.mqtt.c/Component.cdef you will find the references to ssl folders, etc.

Can you see the dependency and how the SSL support is available through the Phao library?

I also had to change the file references to /etc/ssl in the Phao definition file as they didn’t exist in the latest framework.

Not sure if it needs to import ca-certificates inside the sandboxed application:

It seems like the mqttClient/mqttClientService can access the /etc/ssl/certs/ca-certificates.crt file without my app running in a sandbox environment. Maybe it’s because the mqttClient/mqttClientService is running with sandboxed: false and hence can access the /etc/ssl/* files.

It would be great if there is a way to update the /etc/ssl/certs/ca-certificates.crt file as part of my application deployment. I can use the “mount trick” that @jyijyi showed above, but it’s not ideal.

The process to add a new intermediate/root CA certificate is to first add it to /usr/share/ca-certificates, update /etc/ca-certificates.conf and then run update-ca-certificates, but I can pre-generate the /etc/ssl/certs/ca-certificates.crt and make it available in the /etc/ssl/certs folder.

Is it possible to add files to the file system outside of the application structure?

see if these help to create file and access it in sandboxed app