Failed to update NVTs in container

Hello everyone,

I am still learning so please forgive my inexperience. I have a freshly installed Ubuntu 22.04 machine running Docker version 20.10.21. I have the latest update pulled (second pull since installing a week ago) using:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull

The containers are up and running, and I have an admin user set up. The issue I have is with the feed status. I was unable to update the NVTs. According to the documentation, the commands should be:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

I was checking the logs and noticed the updates failing, and then updating to an older version of the NVTs.

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition logs -f ospd-openvas

Side note: I believe there is a typo on the documentation, logs worked but log did not.

The output is shown below.

ospd-openvas_1 | OSPD[7] 2023-06-27 20:01:27,174: ERROR: (ospd_openvas.openvas) OpenVAS Scanner failed to load VTs. Command ‘[‘openvas’, ‘–update-vt-info’]’ returned non-zero exit status 1.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:01:27,174: ERROR: (ospd_openvas.daemon) Updating VTs failed.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:02:13,666: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait…
ospd-openvas_1 | OSPD[7] 2023-06-27 20:02:21,838: ERROR: (ospd_openvas.openvas) OpenVAS Scanner failed to load VTs. Command ‘[‘openvas’, ‘–update-vt-info’]’ returned non-zero exit status 1.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:02:21,838: ERROR: (ospd_openvas.daemon) Updating VTs failed.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:03:10,503: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait…
ospd-openvas_1 | OSPD[7] 2023-06-27 20:04:32,562: INFO: (ospd_openvas.daemon) VTs were up to date. Feed version is 202306210559.

I thought to try updating the NVTs manually, so I ran the command:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition
run --rm ospd-openvas greenbone-nvt-sync

The tail-end gave the following errors:

rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zml_cgi_traversal.nasl.GUqSXp” failed: Permission denied (13)
3,867 100% 12.46kB/s 0:00:00 (xfr#85942, to-chk=5/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zone_alarm_fw_p67.nasl.di6G4p” failed: Permission denied (13)
2,716 100% 8.75kB/s 0:00:00 (xfr#85943, to-chk=4/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zope_path_disclosure.nasl.48ByOr” failed: Permission denied (13)
3,142 100% 10.09kB/s 0:00:00 (xfr#85944, to-chk=3/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zope_zclass.nasl.6nQyzt” failed: Permission denied (13)
2,721 100% 8.74kB/s 0:00:00 (xfr#85945, to-chk=2/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zyxel_http_pwd.nasl.4PylEr” failed: Permission denied (13)
2,871 100% 9.22kB/s 0:00:00 (xfr#85946, to-chk=1/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zyxel_pwd.nasl.9OkBQt” failed: Permission denied (13)
3,128 100% 10.05kB/s 0:00:00 (xfr#85947, to-chk=0/87229)

sent 4,884,622 bytes received 18,216,874 bytes 669,608.58 bytes/sec
total size is 353,823,588 speedup is 15.32
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1819) [generator=3.2.3]
<27>Jun 27 20:38:13 greenbone-nvt-sync: rsync for nasl failed.

There is some kind of permission issue with rsync, and based on the error messages, I believe it may be due to not having permissions to make a temporary file in the plugins directory.

I got a shell into the container

sudo docker exec -it greenbone-community-edition_ospd-openvas_1 /bin/bash

and check that the folders belong to root.

I’m not sure what I did wrong with the set up. I appreciate any help on the matter. If any additional information is needed for troubleshooting, I’ll be happy to see if I can provide it.
Thank you!

Hi,

it’s very likely you did break the permissions in the volumes for the feed content. Try to remove the volumes and restart all containers again.

docker volume rm greenbone-community-edition_vt_data_vol greenbone-community-edition_scap_data_vol greenbone-community-edition_notus_data_vol greenbone-community-edition_gvmd_data_vol greenbone-community-edition_data_objects_vol greenbone-community-edition_cert_data_vol

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull
docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d

The volumes will be recreated and filled with the newest content from the feed automatically.

Hi bricks,

Thank you for your response!

I have removed the volumes as advised and did a new pull. I’ve included the outputs below.

Creating volume “greenbone-community-edition_scap_data_vol” with default driver
Creating volume “greenbone-community-edition_cert_data_vol” with default driver
Creating volume “greenbone-community-edition_data_objects_vol” with default driver
Creating volume “greenbone-community-edition_gvmd_data_vol” with default driver
Creating volume “greenbone-community-edition_vt_data_vol” with default driver
Creating volume “greenbone-community-edition_notus_data_vol” with default driver
greenbone-community-edition_pg-gvm_1 is up-to-date
greenbone-community-edition_mqtt-broker_1 is up-to-date
Creating greenbone-community-edition_data-objects_1 … done
Creating greenbone-community-edition_cert-bund-data_1 … done
Creating greenbone-community-edition_notus-data_1 … done
Creating greenbone-community-edition_vulnerability-tests_1 … done
Creating greenbone-community-edition_gpg-data_1 … done
Creating greenbone-community-edition_redis-server_1 … done
Creating greenbone-community-edition_scap-data_1 … done
Creating greenbone-community-edition_report-formats_1 … done
Creating greenbone-community-edition_dfn-cert-data_1 … done
Creating greenbone-community-edition_notus-scanner_1 … done
Creating greenbone-community-edition_gvmd_1 … done
Creating greenbone-community-edition_gsa_1 … done
Creating greenbone-community-edition_ospd-openvas_1 … done
Creating greenbone-community-edition_gvm-tools_1 … done

The logs are also included below:

Attaching to greenbone-community-edition_ospd-openvas_1
ospd-openvas_1 | OSPD[7] 2023-06-28 17:35:01,361: INFO: (ospd.main) Starting OSPd OpenVAS version 22.5.1.
ospd-openvas_1 | OSPD[7] 2023-06-28 17:35:01,557: INFO: (ospd_openvas.messaging.mqtt) Successfully connected to MQTT broker
ospd-openvas_1 | OSPD[7] 2023-06-28 17:35:12,654: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait…
ospd-openvas_1 | OSPD[7] 2023-06-28 18:17:28,036: INFO: (ospd_openvas.daemon) Finished loading VTs. The VT cache has been updated from version 0 to 202306210559.

Strange - I was expecting the feed to be up to date by its version seems to be outdated by a week.

I noticed this issue initially when I tried to scan a Windows machine and the scan completed almost instantaneously - that is when I suspected there were no tests being run, which explained why it completed so quickly.

I did not really do any tinkering with the setup from installation. Any ideas how I could have broken the permissions in the volumes?

Any further help is much appreciated.

Thanks for your time!

Hi,

actually this was an issue with the feed deployment that I’ve just fixed

4 Likes

Hi bricks,

Thank you for the workaround - I was able to to see the NVTs when pulling the latest image. I waited a day to try performing a feed synchronization to test if the issue is fully resolved, but I seem to have run into more errors.

For reference, the commands I used are listed in the documentation:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

The logs themselves are a bit vague, but here are all the contents from today’s timestamp:

ospd-openvas_1 | OSPD[7] 2023-06-30 03:22:41,028: INFO: (ospd.main) Shutting-down server …
greenbone-community-edition_ospd-openvas_1 exited with code 0

When running the up command to perform the sync, here are the errors that are raised:

vulnerability-tests_1 | For more information please contact Greenbone Networks GmbH:
vulnerability-tests_1 |
vulnerability-tests_1 | https://www.greenbone.net or info@greenbone.net
vulnerability-tests_1 |
report-formats_1 |
report-formats_1 | Copying report formats… files copied.
cert-bund-data_1 | Copying CERT-Bund data… files copied.
dfn-cert-data_1 | Copying DFN-CERT data… files copied.
Exception in thread Thread-22 (watch_events):
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 445, in _make_request
six.raise_from(e, None)
File “”, line 3, in raise_from
File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 440, in _make_request
httplib_response = conn.getresponse()
File “/usr/lib/python3.10/http/client.py”, line 1374, in getresponse
response.begin()
File “/usr/lib/python3.10/http/client.py”, line 318, in begin
version, status, reason = self._read_status()
File “/usr/lib/python3.10/http/client.py”, line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), “iso-8859-1”)
File “/usr/lib/python3.10/socket.py”, line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out

The errors form a chain, below are the other errors:

urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)

Not sure if anyone else is running into this error. Any ideas?

Thank you for your support thus far.


Edit: I tried pulling notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-object and that seems to have updated the NVTs (the docker compose pull command still does not work as expected). Interestingly, the Origin is listed as unknown. I’ve included a screenshot below.

Thanks!

Without knowing which service/container issued that errors I don’t have a clue.

Just pulled the newest images and it looks good to me:

Could you try shutting down all containers, deleting all volumes beside the postgres data volume and afterwards restart all services?

2 Likes

Hi bricks,

I did a few days of testing just to be sure. The newer images are fine now - the feeds can be updated. Thank you very much for your help!

Additional notes for anyone running into this thread in the future: You can get logs for for each docker process related to the scanner through running docker ps.

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition logs -f [NAME HERE]

1 Like