Socket connection timeout (but ACK received?)

karlkuhn · July 27, 2018, 1:08pm

Hi, I am creating a standard socket connection from within legato (FX30 16.10.1).

First I resolve the host name successfully. So the bearer and data service are all connected and ready.
Then I try and connect. But it keeps timing out (errno = 110). I’ve attached a wireshark output showing it is getting an ack back… but the app does not seem to receive it?

Any ideas? My code is as follows.

What am I missing?

hSocket = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
  if (hSocket == -1)
  	continue;> 

  getnameinfo(result->ai_addr, result->ai_addrlen, hbuf, sizeof(hbuf), sbuf,
  	sizeof(sbuf), NI_NUMERICHOST | NI_NUMERICSERV);
  
  if (connect(hSocket, rp->ai_addr, rp->ai_addrlen) != -1)
  {
  	pTrace->TraceLn("Socket connected. %s resolved to, %s: %s ", sHost.c_str(), hbuf, sbuf);
  	break;                  /* Success */
  }
  
  pTrace->TraceLn("Failed to connect to %s on IP: %s: %sErr: %d\n", sHost.c_str(), hbuf, sbuf, errno);
  close(hSocket);
  hSocket = -1;

davidc · July 30, 2018, 5:08am

Hi @karlkuhn,

Have you opened the port in firewall (iptables) on the FX30?

The firewall for all interfaces other than USB/ECM is by default extremely restrictive - off the top of my head only DNS(udp:53) and ping(icmp) are opened by default - the last rule in the input chain is DROP ALL.

Oh … I’ve also started using getaddrinfo() instead of getnameinfo() - apparently getnameinfo() has been deprecated and doesn’t play well with IPv6…

ciao, Dave

karlkuhn · July 30, 2018, 7:54am

Hi @davidc, thanks for the reply. It does seem to be the firewall as suggested. I found some script for disabling it using ip-tables -F which is now allowing my connections…

I tried changing the iptables previously by editing rules.v4 in etc/iptabes on the target. Which had no impact. Running the iptables -F command does not seemed to have changes those files either? Are the firewall rules saves somewhere else on a legato target (in this case an FX30s)? What am I missing. As I do not want to disable the firewall, just add my ports…

Sorry if this is a newbie linux question…

Thanks

davidc · July 30, 2018, 8:46am

Hi @karlkuhn,

Not a newbie question at all. This stuff is both subtle and complex.

The FX30 has put the iptables rules in a different location to the ‘stock’ firmware used in the WP on the mangOH boards.

I haven’t got a FX30 with me at the moment … but I think the correct way to save the current set of iptables rules is as follows:

iptables-save > /etc/iptables/rules.v4

Note that the content of the file is NOT a list of commands to execute - it’s actually a list of rules in a specific format that iptables-restore can use to restore the rules at boot. I also don’t think you can put -F as a rule in the file.

If you can, I would avoid disabling the firewall all together - it’s better to work out the holes you need to open them. Below is an example to open traffic back into (RELATED or ESTABLISHED) the FX30 on port 443 (SSL) on the WWAN network:

iptables -I INPUT -i rmnet0 -p tcp -m tcp --sport 443 -m state --state RELATED,ESTABLISHED -j ACCEPT
(all one line)

This uses iptables to INSERT the following rule at the beginning of the INPUT rule chain:

-i rmnet0 only packets coming in interface rmnet0
-p tcp with protocol TCP
-m tcp --sport 443 matching tcp packets with a tcp source port of 80
-m state RELATED,ESTABLISHED matching connections state RELATED or ESTABLISHED
-j ACCEPT jump to the ACCEPT chain (i.e. let the packet through the firewall)

Hope this helps.

ciao, Dave

karlkuhn · July 31, 2018, 10:27am

Thanks @davidc, that did the trick.

What is the current best practice for rolling these “tweaks/config” changes out to manufacturing?

Is it

Have a script that configures each modem when installing the application?
Is it create a complete image somehow that has everything pre-configured?
Some other process.

Thanks,

Karl

davidc · July 31, 2018, 11:53pm

Hi @karlkuhn,

I’ve pretty much tried all three of your options - with varying success rates.

Option 1 works reasonably well if you only do installs in your factory. I ended up having to do this as I couldn’t build a complete sdef image for the FX30 and needed to install multiple apps at the same time. But if you need to change the rules for some reason, then you have to physically go to each device and re-run the installer. Can get ugly.

Option 2 has got a couple of issues - the first is that even if you can successfully build a sdef image for the FX30 (which I haven’t yet been able to do for a number of reasons), that’s only the Legato bundle - not the Linux operating system (where the startup scripts are). Tweaking the Yocto build is somewhat more complex than building a sdef image - and I haven’t even tried rebuilding Yocto for the FX30. And then you have to reinstall both the yocto image and the legato sdef image - again, either a road trip to the device or a multi-megabyte FOTA download.

I settled on Option 3 - for each app that requires holes opened in the firewall, I add an additional processes: stanza to the adef file, and use that to execute a shell script that adds the ip tables rules every time the app is started.

Something like this:

add to app.adef

processes:
{
    run:
    {
        ( set-iptables.sh )
    }
    faultAction: ignore
}

requires:
{
    file:
    {
        /bin/sh /bin/
        /usr/sbin/iptables /usr/sbin/
    }
}

bundles:
{
    file:
    {
        /* this is our script to config firewall */
        [x] root/bin/set-iptables.sh /bin/
    }
}

and then in the application build directory, create the following directory:

root/bin/

and then create a shell script called set-iptables.sh in root/bin/

#!/bin/sh

PATH=$PATH:/usr/sbin

/usr/sbin/iptables <blah blah blah>

where <blah blah blah> is the rule you want to apply.

Make sure that you INSERT (-I) the rules at the beginning of the chain, not APPEND (-A) them at the end or they may be matched by the DROP rule that already exists at the end of the chain.

I did it this way as then I was SURE that the rules would always be implemented every time the app started … no more head scratching why the app stopped working after a reboot

Hope this helps.

Ciao, Dave

karlkuhn · August 1, 2018, 6:37am

Thank you @davidc for all the detailed replies. It is very useful and appreciated!

karlkuhn · August 9, 2018, 2:34pm

Hi @davidc, using sbtrace I had to add a load of files to my required section which has these files in…

   /bin/sh /bin/
  /usr/sbin/iptables			/usr/sbin/
  /usr/lib/libiptc.so.0		usr/lib/
  /usr/lib/libip4tc.so.0		usr/lib/
  /usr/lib/libip6tc.so.0		usr/lib/
  /usr/lib/libxtables.so.10	usr/lib/
  /lib/libnss_db.so.2         /lib/libnss_db.so.2
    /etc/protocols              /etc/protocols
    /usr/lib/xtables/libxt_tcp.so       /usr/lib/xtables/libxt_tcp.so
    /usr/lib/xtables/libxt_state.so     /usr/lib/xtables/libxt_state.so
    /usr/lib/xtables/libxt_standard.so  /usr/lib/xtables/libxt_standard.so
    /proc/net/ip_tables_names           /proc/net/ip_tables_names

However I am now getting the following error.

can’t initialize iptable ‘filter’:Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded

I am running 16.10.1.m3 R14…

Have I got something fundamentally wrong?

lotam · August 9, 2018, 2:56pm

The result make sense to me as even on Desktop Linux we need root permission to manipulate iptables rules.

Easy way is to do this with non-sandbox app (set sandboxed:false in .adef).

But are you sure you need to modify the iptables rule during runtime via app?
Per understanding, it is one time operation/configuration, personally, I will add via Yocto for better security, or in boot script if not rebuilding Yocto.

Thx

karlkuhn · August 9, 2018, 3:01pm

Hi @lotam I am kind following @davidc suggestion. But I presume from what you are saying that is not running this in a sandbox.

I need a solution that can be put into production easily and also be update OTA.

From what I understand you are saying. When within the sandbox, it runs under a different user to when running in the sandbox? Is that correct?
Can the boot script be added through an adef?

Thanks,

Karl

lotam · August 9, 2018, 3:16pm

Hi Karl,
Per understanding, yes to both question.
For instance, will you try set “sandboxed: false” in .adef and check does it able to get the job done?
Thx

davidc · August 9, 2018, 10:27pm

Hi @karlkuhn,

It was interesting that you needed to add all the additional .so files to your build. I didn’t need to do that - but I will also admit that I haven’t been able to successfully build anything for the new FX30 R14/Legato 16.10.1.m3 release.

Yes, you need to add sandboxed: false to force the app to run as root to the top of your adef. Sorry about missing that out in my example.

I have successfully used this method to download application updates OTA from Airvantage to an FX30 in the field. In fact, putting the iptables script in the application means that you don’t have to download an entire linux image to the device.

As an aside, where did you get the Legato source for 16.10.1.m3? I haven’t been able to build anything more sophisticated than hello, world without the application dying with API version mismatches.

ciao, Dave

lotam · August 10, 2018, 5:13am

Hi Dave,

I guess this is because you marked “sandboxed: false”, as app run outside sandbox it can access all system files, thus no need to add in requires section.

Thx

karlkuhn · August 10, 2018, 5:29am

Hi both thanks for the replies.

@lotam I will give that ago later today and let you both know.

@davidc see my two posts here and here relating to R14. I am running my full (fairly complex) application on R14 using the references downloads, but still get a mismatch warning when installing, but it still runs OK. Is that what you wanted?

davidc · August 10, 2018, 5:51am

Hi @karlkuhn,

Thanks for the links to your other posts. I’m currently having the issue that you were having in the post you made to the Sierra forum (with the mismatches and app not running).

I am looking for the 16.10.1.m3 legato source bundle (NOT the toolchain - different thing) - the equivalent to this here for stock 16.10.3

ciao, Dave

karlkuhn · August 10, 2018, 5:55am

Hi @davidc, thought that might be the case. I think the source code you want is here , recently updated?. @cchenry is that correct? Or is this the one you are having problems with?

Either way, I did manage to build and run successfully using the toolchain provided by enoch, where 16.10.3 toolchain did not work…

davidc · August 10, 2018, 6:11am

Hi @karlkuhn,

Thanks, much appreciated. That link wasn’t there yesterday … and I had already pulled down the 3.7GB ‘source’ tarball hoping it would be in there (it’s not - that’s the Yocto source).

Thanks very much!

ciao, Dave

karlkuhn · August 10, 2018, 8:47am

@lotam @davidc setting the app to not run in the sandbox worked. Thanks!

cchenry · August 10, 2018, 9:56pm

Hi Karl,
That’s correct, we just posted the Legato 16.10.1.m3 source code on the FX30 firmware page, since it is not available on legato.io

Dave, my apologies, as we only just posted the Legato source yesterday.

BR,
Chris

Topic		Replies	Views
Socket connection fail Legato Linux distribution (Yocto project)	13	2335	April 6, 2019
Device keeps disconnecting	8	1338	November 19, 2018
Getaddrinfo failing after closing socket	13	1831	February 28, 2023
TCP connection error , port 443 N: Connect failed errno=13 Legato Application Framework	9	1591	July 6, 2020
Device looses the ability to resolve hostnames (Unsandboxed R14)	7	1025	February 15, 2019

Socket connection timeout (but ACK received?)

Related topics