Failed to update NVTs in container

[Stavis]

Hello everyone,

I am still learning so please forgive my inexperience. I have a freshly installed Ubuntu 22.04 machine running Docker version 20.10.21. I have the latest update pulled (second pull since installing a week ago) using:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull

The containers are up and running, and I have an admin user set up. The issue I have is with the feed status. I was unable to update the NVTs. According to the documentation, the commands should be:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

I was checking the logs and noticed the updates failing, and then updating to an older version of the NVTs.

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition logs -f ospd-openvas

Side note: I believe there is a typo on the documentation, logs worked but log did not.

The output is shown below.

ospd-openvas_1 | OSPD[7] 2023-06-27 20:01:27,174: ERROR: (ospd_openvas.openvas) OpenVAS Scanner failed to load VTs. Command ‘[‘openvas’, ‘–update-vt-info’]’ returned non-zero exit status 1.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:01:27,174: ERROR: (ospd_openvas.daemon) Updating VTs failed.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:02:13,666: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait…
ospd-openvas_1 | OSPD[7] 2023-06-27 20:02:21,838: ERROR: (ospd_openvas.openvas) OpenVAS Scanner failed to load VTs. Command ‘[‘openvas’, ‘–update-vt-info’]’ returned non-zero exit status 1.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:02:21,838: ERROR: (ospd_openvas.daemon) Updating VTs failed.
ospd-openvas_1 | OSPD[7] 2023-06-27 20:03:10,503: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait…
ospd-openvas_1 | OSPD[7] 2023-06-27 20:04:32,562: INFO: (ospd_openvas.daemon) VTs were up to date. Feed version is 202306210559.

I thought to try updating the NVTs manually, so I ran the command:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition
run --rm ospd-openvas greenbone-nvt-sync

The tail-end gave the following errors:

rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zml_cgi_traversal.nasl.GUqSXp” failed: Permission denied (13)
3,867 100% 12.46kB/s 0:00:00 (xfr#85942, to-chk=5/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zone_alarm_fw_p67.nasl.di6G4p” failed: Permission denied (13)
2,716 100% 8.75kB/s 0:00:00 (xfr#85943, to-chk=4/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zope_path_disclosure.nasl.48ByOr” failed: Permission denied (13)
3,142 100% 10.09kB/s 0:00:00 (xfr#85944, to-chk=3/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zope_zclass.nasl.6nQyzt” failed: Permission denied (13)
2,721 100% 8.74kB/s 0:00:00 (xfr#85945, to-chk=2/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zyxel_http_pwd.nasl.4PylEr” failed: Permission denied (13)
2,871 100% 9.22kB/s 0:00:00 (xfr#85946, to-chk=1/87229)
rsync: [receiver] mkstemp “/var/lib/openvas/plugins/pre2008/.zyxel_pwd.nasl.9OkBQt” failed: Permission denied (13)
3,128 100% 10.05kB/s 0:00:00 (xfr#85947, to-chk=0/87229)

sent 4,884,622 bytes received 18,216,874 bytes 669,608.58 bytes/sec
total size is 353,823,588 speedup is 15.32
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1819) [generator=3.2.3]
<27>Jun 27 20:38:13 greenbone-nvt-sync: rsync for nasl failed.

There is some kind of permission issue with rsync, and based on the error messages, I believe it may be due to not having permissions to make a temporary file in the plugins directory.

I got a shell into the container

sudo docker exec -it greenbone-community-edition_ospd-openvas_1 /bin/bash

and check that the folders belong to root.

I’m not sure what I did wrong with the set up. I appreciate any help on the matter. If any additional information is needed for troubleshooting, I’ll be happy to see if I can provide it.
Thank you!

[bricks]

Hi,

it’s very likely you did break the permissions in the volumes for the feed content. Try to remove the volumes and restart all containers again.

docker volume rm greenbone-community-edition_vt_data_vol greenbone-community-edition_scap_data_vol greenbone-community-edition_notus_data_vol greenbone-community-edition_gvmd_data_vol greenbone-community-edition_data_objects_vol greenbone-community-edition_cert_data_vol

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull
docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d

The volumes will be recreated and filled with the newest content from the feed automatically.

[Stavis]

Hi bricks,

Thank you for your response!

I have removed the volumes as advised and did a new pull. I’ve included the outputs below.

Creating volume “greenbone-community-edition_scap_data_vol” with default driver
Creating volume “greenbone-community-edition_cert_data_vol” with default driver
Creating volume “greenbone-community-edition_data_objects_vol” with default driver
Creating volume “greenbone-community-edition_gvmd_data_vol” with default driver
Creating volume “greenbone-community-edition_vt_data_vol” with default driver
Creating volume “greenbone-community-edition_notus_data_vol” with default driver
greenbone-community-edition_pg-gvm_1 is up-to-date
greenbone-community-edition_mqtt-broker_1 is up-to-date
Creating greenbone-community-edition_data-objects_1 … done
Creating greenbone-community-edition_cert-bund-data_1 … done
Creating greenbone-community-edition_notus-data_1 … done
Creating greenbone-community-edition_vulnerability-tests_1 … done
Creating greenbone-community-edition_gpg-data_1 … done
Creating greenbone-community-edition_redis-server_1 … done
Creating greenbone-community-edition_scap-data_1 … done
Creating greenbone-community-edition_report-formats_1 … done
Creating greenbone-community-edition_dfn-cert-data_1 … done
Creating greenbone-community-edition_notus-scanner_1 … done
Creating greenbone-community-edition_gvmd_1 … done
Creating greenbone-community-edition_gsa_1 … done
Creating greenbone-community-edition_ospd-openvas_1 … done
Creating greenbone-community-edition_gvm-tools_1 … done

The logs are also included below:

Attaching to greenbone-community-edition_ospd-openvas_1
ospd-openvas_1 | OSPD[7] 2023-06-28 17:35:01,361: INFO: (ospd.main) Starting OSPd OpenVAS version 22.5.1.
ospd-openvas_1 | OSPD[7] 2023-06-28 17:35:01,557: INFO: (ospd_openvas.messaging.mqtt) Successfully connected to MQTT broker
ospd-openvas_1 | OSPD[7] 2023-06-28 17:35:12,654: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait…
ospd-openvas_1 | OSPD[7] 2023-06-28 18:17:28,036: INFO: (ospd_openvas.daemon) Finished loading VTs. The VT cache has been updated from version 0 to 202306210559.

Strange - I was expecting the feed to be up to date by its version seems to be outdated by a week.

I noticed this issue initially when I tried to scan a Windows machine and the scan completed almost instantaneously - that is when I suspected there were no tests being run, which explained why it completed so quickly.

I did not really do any tinkering with the setup from installation. Any ideas how I could have broken the permissions in the volumes?

Any further help is much appreciated.

Thanks for your time!

[bricks]

Hi,

actually this was an issue with the feed deployment that I’ve just fixed

NVTs not updating and showing orgin unknown

Hi, can you just pull the images again? I’ve uploaded a new image for the vulnerability tests just 5 minutes ago. There have been new images uploaded every day but they missed setting the latest container image tag. You can always check https://hub.docker.com/r/greenbone/vulnerability-tests/tags if there are newer images.

[Stavis]

Hi bricks,

Thank you for the workaround - I was able to to see the NVTs when pulling the latest image. I waited a day to try performing a feed synchronization to test if the issue is fully resolved, but I seem to have run into more errors.

For reference, the commands I used are listed in the documentation:

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects

The logs themselves are a bit vague, but here are all the contents from today’s timestamp:

ospd-openvas_1 | OSPD[7] 2023-06-30 03:22:41,028: INFO: (ospd.main) Shutting-down server …
greenbone-community-edition_ospd-openvas_1 exited with code 0

When running the up command to perform the sync, here are the errors that are raised:

vulnerability-tests_1 | For more information please contact Greenbone Networks GmbH:
vulnerability-tests_1 |
vulnerability-tests_1 | https://www.greenbone.net or info@greenbone.net
vulnerability-tests_1 |
report-formats_1 |
report-formats_1 | Copying report formats… files copied.
cert-bund-data_1 | Copying CERT-Bund data… files copied.
dfn-cert-data_1 | Copying DFN-CERT data… files copied.
Exception in thread Thread-22 (watch_events):
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 445, in _make_request
six.raise_from(e, None)
File “”, line 3, in raise_from
File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 440, in _make_request
httplib_response = conn.getresponse()
File “/usr/lib/python3.10/http/client.py”, line 1374, in getresponse
response.begin()
File “/usr/lib/python3.10/http/client.py”, line 318, in begin
version, status, reason = self._read_status()
File “/usr/lib/python3.10/http/client.py”, line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), “iso-8859-1”)
File “/usr/lib/python3.10/socket.py”, line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out

The errors form a chain, below are the other errors:

urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)

Not sure if anyone else is running into this error. Any ideas?

Thank you for your support thus far.

---

Edit: I tried pulling notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-object and that seems to have updated the NVTs (the docker compose pull command still does not work as expected). Interestingly, the Origin is listed as unknown. I’ve included a screenshot below.

Thanks!

[bricks]

@Stavis

Not sure if anyone else is running into this error. Any ideas?

Without knowing which service/container issued that errors I don’t have a clue.

Just pulled the newest images and it looks good to me:

Could you try shutting down all containers, deleting all volumes beside the postgres data volume and afterwards restart all services?

[Stavis]

Hi bricks,

I did a few days of testing just to be sure. The newer images are fine now - the feeds can be updated. Thank you very much for your help!

Additional notes for anyone running into this thread in the future: You can get logs for for each docker process related to the scanner through running docker ps.

docker-compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition logs -f [NAME HERE]