VNC brute force false positives on MacOS

Hello,

We are seeing false positive alerts claiming VNC Brute Force Login (NVT OID: 1.3.6.1.4.1.25623.1.0.106056) on a couple of MacOS devices. The underlying reason seems to be that the built-in VNC server in MacOS violates the RFB protocol and sends the authentication result field using the wrong endianness, which is interpreted by standards-compliant clients (like Greenbone) as always succeeding. However, the server also claims to use an unknown version of the protocol, which means this condition should be possible to handle in gb_vnc_brute_force.nasl.

Tcpdump reports the server’s handshake packet as (with IP addresses censored):

0x0000:  0800 0000 0000 0010 0001 0006 0009 0f09  ................
0x0010:  0a1c 0000 4500 0040 0000 4000 3c06 b8f1  ....E..@..@.<...
0x0020:  2020 2020 2020 2020 170c 81c6 b193 a47f          ........
0x0030:  2bde 97ba 8018 0804 f136 0000 0101 080a  +........6......
0x0040:  1b7f ede7 5a8c 782e 5246 4220 3030 332e  ....Z.x.RFB.003.
0x0050:  3838 390a                                889.

That’s RFB 3.889, which is not a standard - RFC 6143 describes 3.3, 3.7, and 3.8.

The client selects 3.8 instead:

0x0000:  0800 0000 0000 0010 0001 0406 f4a8 0d4c  ...............L
0x0010:  0745 0000 4500 0040 5b01 4000 4006 59f0  .E..E..@[.@.@.Y.
0x0020:  2020 2020 2020 2020 81c6 170c 2bde 97ba          ....+...
0x0030:  b193 a48b 8018 01f6 85f9 0000 0101 080a  ................
0x0040:  5a8c 788e 1b7f ede7 5246 4220 3030 332e  Z.x.....RFB.003.
0x0050:  3030 380a                                008.

and authentication continues according to the 3.8 standard with method selection and challenge-response until the server replies with:

0x0000:  0800 0000 0000 0010 0001 0006 0009 0f09  ................
0x0010:  0a1c 0000 4500 0064 0000 4000 3c06 b8cd  ....E..d..@.<...
0x0020:  2020 2020 2020 2020 170c 81c6 b193 a4a0          ........
0x0030:  2bde 97d7 8018 0803 42d0 0000 0101 080a  +.......B.......
0x0040:  1b7f fbe8 5a8c 8485 0100 0000 0000 0027  ....Z..........'
0x0050:  4175 7468 656e 7469 6361 7469 6f6e 206f  Authentication.o
0x0060:  7220 6175 7468 6f72 697a 6174 696f 6e20  r.authorization.
0x0070:  6661 696c 7572 6500                      failure.

Note this line in the authentication response:

0x0040:  1b7f fbe8 5a8c 8485 0100 0000 0000 0027  ....Z..........'
                             ^result^^ ^length^^

gb_vnc_brute_force.nasl looks at the fourth byte of result, sees that it’s 0, and reports a successful authentication:

auth_res = ord( res[3] );

if( auth_res == 0 ) {
  report = "It was possible to connect to the VNC server with the password: " + password;

So does tshark -T json:

      "tcp.payload": "01:00:00:00:00:00:00:27:41:75:74:68:65:6e:74:69:63:61:74:69:6f:6e:20:6f:72:20:61:75:74:68:6f:72:69:7a:61:74:69:6f:6e:20:66:61:69:6c:75:72:65:00"
    },
    "vnc": {
      "vnc.auth_result": "0"
    }

This is what RFC 6143, section 7.1.3 says you should do:

The server sends a word to inform the client whether the security
handshaking was successful.

+--------------+--------------+-------------+
| No. of bytes | Type [Value] | Description |
+--------------+--------------+-------------+
| 4            | U32          | status:     |
|              | 0            | OK          |
|              | 1            | failed      |
+--------------+--------------+-------------+

If successful, the protocol passes to the initialization phase
(Section 7.3).

If unsuccessful, the server sends a string describing the reason for
the failure, and then closes the connection:

+---------------+--------------+---------------+
| No. of bytes  | Type [Value] | Description   |
+---------------+--------------+---------------+
| 4             | U32          | reason-length |
| reason-length | U8 array     | reason-string |
+---------------+--------------+---------------+

And the protocol is explicitly big-endian (section 7):

All multiple-byte integers (other than pixel values themselves) are in big endian order (most significant byte first).

But is obviously not what MacOS does. Interestingly, this only applies to the result code - the error message string is correctly encoded according to the RFC:

Some messages use arrays of the basic types, with the number of entries in the array determined from fields preceding the array.

The length field is 00 00 00 27, indicating a 39-byte string in big-endian form.

Would it be possible for gb_vnc_brute_force.nasl to check for protocol minor version 889 and, if so, byte-swap the authentication result field?

Also, someone with more direct access to a Mac than me should probably check what happens if you do manage to authenticate successfully. Or I’ll see if I can get one of my users to cooperate.

I have no idea how long MacOS has behaved this way, but I found this bug from 2011 about it claiming to be version 3.889.

I managed to dig up a Mac to take a closer look at in the lab, and it looks like I thought based on the network traffic seen in the wild, above.

The machine I have identifies itself as a MacBook Pro (13-inch, 2017, Two Thunderbolt 3 ports, Intel Core i5), running macOS Ventura Version 13.6.6. I enabled the built-in VNC server by going to Settings → General → Sharing, toggling “Screen sharing” to “On” and entering a password in the “VNC viewers may control screen with password” box. I then talked to it using TigerVNC 1.13.1 on Fedora Linux.

This Mac also claims to speak RFB 003.889:

13:57:57.968324 IP 192.168.228.2.5900 > 192.168.228.1.39686: Flags [P.], seq 1:13, ack 1, win 2058, options [nop,nop,TS val 202153586 ecr 2968794406], length 12
    0x0000:  f4a8 0d4c 0745 a0ce c86c 867c 0800 4500  ...L.E...l.|..E.
    0x0010:  0040 0000 4000 4006 f162 c0a8 e402 c0a8  .@..@.@..b......
    0x0020:  e401 170c 9b06 c96c da36 4ed7 b523 8018  .......l.6N..#..
    0x0030:  080a d1f7 0000 0101 080a 0c0c 9e72 b0f4  .............r..
    0x0040:  3526 5246 4220 3030 332e 3838 390a       5&RFB.003.889.

Which tshark happily decodes:

      "tcp.payload": "52:46:42:20:30:30:33:2e:38:38:39:0a"
    },
    "vnc": {
      "vnc.server_proto_ver": "003.889"
    }

Which my TigerVNC client doesn’t understand, so it instead offers 3.8:

13:57:57.968630 IP 192.168.228.1.39686 > 192.168.228.2.5900: Flags [P.], seq 1:13, ack 13, win 251, options [nop,nop,TS val 2968795237 ecr 202153586], length 12
    0x0000:  a0ce c86c 867c f4a8 0d4c 0745 0800 4500  ...l.|...L.E..E.
    0x0010:  0040 031b 4000 4006 ee47 c0a8 e401 c0a8  .@..@.@..G......
    0x0020:  e402 9b06 170c 4ed7 b523 c96c da42 8018  ......N..#.l.B..
    0x0030:  00fb 4988 0000 0101 080a b0f4 3865 0c0c  ..I.........8e..
    0x0040:  9e72 5246 4220 3030 332e 3030 380a       .rRFB.003.008.

tshark:

      "tcp.payload": "52:46:42:20:30:30:33:2e:30:30:38:0a"
    },
    "vnc": {
      "vnc.client_proto_ver": "003.008"
    }

When I enter the wrong password, I get the same little-endian 32-bit value “1” and a human-readable error message:

13:58:26.470109 IP 192.168.228.2.5900 > 192.168.228.1.48390: Flags [P.], seq 35:83, ack 30, win 2058, options [nop,nop,TS val 3726513705 ecr 2968823233], length 48
    0x0000:  f4a8 0d4c 0745 a0ce c86c 867c 0800 4500  ...L.E...l.|..E.
    0x0010:  0064 0000 4000 4006 f13e c0a8 e402 c0a8  .d..@.@..>......
    0x0020:  e401 170c bd06 ebfa 8596 9d25 f18e 8018  ...........%....
    0x0030:  080a 091a 0000 0101 080a de1e 1629 b0f4  .............)..
    0x0040:  a5c1 0100 0000 0000 0027 4175 7468 656e  .........'Authen
    0x0050:  7469 6361 7469 6f6e 206f 7220 6175 7468  tication.or.auth
    0x0060:  6f72 697a 6174 696f 6e20 6661 696c 7572  orization.failur
    0x0070:  6500                                     e.

Which tshark claims to decode to a successful authentication, since it’s the wrong endianness according to the standard:

      "tcp.payload": "01:00:00:00:00:00:00:27:41:75:74:68:65:6e:74:69:63:61:74:69:6f:6e:20:6f:72:20:61:75:74:68:6f:72:69:7a:61:74:69:6f:6e:20:66:61:69:6c:75:72:65:00"
    },
    "vnc": {
      "vnc.auth_result": "0"
    }

If I enter the correct password, I get 32 bits of zeros with no message, which is correct according to the standard:

13:58:01.874159 IP 192.168.228.2.5900 > 192.168.228.1.39686: Flags [P.], seq 35:39, ack 30, win 2058, options [nop,nop,TS val 202157492 ecr 2968799140], length 4
0x0000:  f4a8 0d4c 0745 a0ce c86c 867c 0800 4500  ...L.E...l.|..E.
0x0010:  0038 0000 4000 4006 f16a c0a8 e402 c0a8  .8..@.@..j......
0x0020:  e401 170c 9b06 c96c da58 4ed7 b540 8018  .......l.XN..@..
0x0030:  080a 1908 0000 0101 080a 0c0c adb4 b0f4  ................
0x0040:  47a4 0000 0000                           G.....

tshark agrees that this is successful:

      "tcp.payload": "00:00:00:00"
    },
    "vnc": {
      "vnc.auth_result": "0"
    }

And TigerVNC then gives me a working VNC session that looks reasonable in Wireshark, so apparently the rest of the protocol is standard enough.

I don’t have an ARM-based Mac to test with, but according to Apple’s porting documentation they are also little-endian, so I assume they will be weird the same way as this Intel-based one.

It looks like my original hunch was correct - if the RFB protocol version is 3.889, interpret the authentication result field as little-endian, not big-endian. Would it be possible to make this change in Greenbone?

Hello,

and welcome to this community portal.

For such special cases where a remote service is violating a specification / RFC i would suggest to create an override as described below once it has been manually confirmed that the login is not possible with the reported credentials.

https://docs.greenbone.net/GSM-Manual/gos-22.04/en/reports.html#using-overrides-and-false-positives

I don’t think that anything from Greenbone side will be done on this, it is usually better to risk some / few false positive then trying to work around such non-compliant services and risking false negatives by doing so.