"OS.EnableFirewall=y" is blocking DNS queries if a switch to TCP is required

[malachma]

SHORT DESCRIPTION

The current implementation of the following security rule is to restrictive

iptables -L -t security
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             168.63.129.16        owner UID match root
DROP       tcp  --  anywhere             168.63.129.16        ctstate INVALID,NEW

it does block any DNS query traffic if a switch from UDP to TCP is necessary if a large payload has to to be returned to client initiating the DNS reolving process.

HOW TO REPRODUCE

Log on to a Linux Azure VM and run the following query
--> dig aerserv-bc-us-east.bidswitch.net @168.63.129.16
which will result in the following response

;; Truncated, retrying in TCP mode.

; <<>> DiG 9.9.9-P1 <<>> aerserv-bc-us-east.bidswitch.net @168.63.129.16
;; global options: +cmd
;; connection timed out; no servers could be reached

The reason for this behaviour is the rule which allows TCP connections against 168.63.129.16
only for the root user!

Run the same query again but this time as root
#dig aerserv-bc-us-east.bidswitch.net @168.63.129.16

;; Truncated, retrying in TCP mode.

; <<>> DiG 9.9.9-P1 <<>> aerserv-bc-us-east.bidswitch.net @168.63.129.16
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51459
;; flags: qr rd ra; QUERY: 1, ANSWER: 133, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1280
;; QUESTION SECTION:
;aerserv-bc-us-east.bidswitch.net. IN   A

;; ANSWER SECTION:
aerserv-bc-us-east.bidswitch.net. 119 IN CNAME  bidcast-bcserver-gce-sc.bidswitch.net.
bidcast-bcserver-gce-sc.bidswitch.net. 119 IN CNAME bidcast-bcserver-gce-sc-multifo.bidswitch.net.
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.51.125
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.78.105
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.193.229
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.166.205
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.175.225
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.117.99
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.29.9
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.236.194
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.131.33
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.75.175
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.55.252
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.190.56
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.172.232
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.61.237
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.23.245
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.56.153
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.175.142
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.166.124
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.17.175
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.60.30
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.241.92
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.109.194
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.70.45
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.37.223
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.21.191
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.54.230
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.247.128
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.248.106
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.201.160
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.204.171
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.139.113
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.73.85
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.130.95
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.49.200
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.123.219
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.230.248
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.13.126
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.18.234
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.45.75
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.192.26
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.182.35
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.190.79
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.28.65
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.213.32
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.189.137
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.205.98
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.148.225
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.124.105
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.29.109
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.40.174
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.7.162
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.82.120
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.51.91
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.190.86
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.212.197
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.160.123
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.180.252
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.52.158
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.0.44
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.155.208
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.119.133
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.12.175
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.97.73
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.180.174
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.236.230
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.165.199
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.255.194
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.155.238
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.25.234
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.225.231
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.21.156
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.130.94
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.123.196
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.137.11
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.154.229
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.210.111
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.54.244
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.3.121
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.198.80
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.249.122
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.196.219
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.214.84
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.145.178
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.150.67
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.31.235
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.70.114
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.31.68
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.72.15
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.193.160
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.44.246
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.31.91
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.74.237
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.228.172
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.57.93
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.59.85
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.47.227
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.32.4
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.97.135
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.48.199
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.92.2
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.46.91
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.159.137
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.48.5
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.35.73
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.102.29
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.45.140
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.15.251
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.230.178
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.225.15
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.59.8
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.60.160
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.177.221
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.118.20
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.163.92
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.207.52.29
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.239.215
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.143.99
bidcast-bcserver-gce-sc-multifo.bidswitch.net. 59 IN A 35.211.119.122

The result is quit large, therfore the switch to TCP.

The current security rule needs therefore to be extended, to accept traffic against port 53 as well, with the following one

iptables -t security -I OUTPUT 1 -d 168.63.129.16/32 -p tcp --destination-port 53 -j ACCEPT

So we end up with the following rules

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             168.63.129.16        tcp dpt:domain
ACCEPT     tcp  --  anywhere             168.63.129.16        owner UID match root
DROP       tcp  --  anywhere             168.63.129.16        ctstate INVALID,NEW 

ADDITONAL INFO

The reason why traffic against 168.63.129.16, via TCP, is only allowed for processes with the UID '0' is not explained in detail, also our docu does not give further hints.
So further information are required why this rule does exists and is enabled by default.

[vrdmr]

@malachma We will improve the documentation you linked (link), but access is restricted only to the root due to security restrictions.

What is the impact? Please let us know.

[malachma]

The Impact is that any process which is not owned by root (uid=0) is not able to work on large resultsets returned by a DNS query, against the Azure DNS, as given in my sample. The communication is blocked because of this strict Security rule. One of our customers is therefore not able to communicate with its Partner. An exception for DNS queries is therefore needed

iptables -t security -I OUTPUT 1 -d 168.63.129.16/32 -p tcp --destination-port 53 -j ACCEPT

Which allows TCP for DNS queries as well.