Feed Sync Still showing "Too old Please check the automatic synchronization of your system." despite following docs

mahidhar190699 · October 18, 2024, 11:40am

Hi, I have tried to do feed syn but it still shows NVT Feed too old (39 days)

I followed the reference: Workflows - Greenbone Community Documentation

as wel as I checked: How to Configure and Check Feed Sync Timing in Greenbone Docker Container?

Ran the commands:

docker compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats
docker compose -f $DOWNLOAD_DIR/docker-compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats

And then After running the sync, everything is done, But still in Openvas Frontend it shows Old version.

The terminal Output:

ubuntu@ubuntu-openvas:~/maintools/openvas$ docker-compose -f compose.yml -p greenbone-community-edition pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects
Pulling vulnerability-tests ... done
Pulling notus-data          ... done
Pulling scap-data           ... done
Pulling cert-bund-data      ... done
Pulling dfn-cert-data       ... done
Pulling data-objects        ... done
Pulling report-formats      ... done
ubuntu@ubuntu-openvas:~/maintools/openvas$ 
ubuntu@ubuntu-openvas:~/maintools/openvas$ docker-compose -f compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-object
ERROR: No such service: data-object
ubuntu@ubuntu-openvas:~/maintools/openvas$ docker-compose -f compose.yml -p greenbone-community-edition up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats
Recreating greenbone-community-edition_cert-bund-data_1      ... done
Recreating greenbone-community-edition_data-objects_1        ... done
Recreating greenbone-community-edition_notus-data_1          ... done
Recreating greenbone-community-edition_vulnerability-tests_1 ... done
Recreating greenbone-community-edition_scap-data_1           ... done
Recreating greenbone-community-edition_dfn-cert-data_1       ... done
Recreating greenbone-community-edition_report-formats_1      ... done
ubuntu@ubuntu-openvas:~/maintools/openvas$ 
ubuntu@ubuntu-openvas:~/maintools/openvas$ python3
Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux

greenbone feed sync

I Refered to: Error: exec: "greenbone-feed-sync": executable file not found in $PATH - #8 by bricks

But when running,

 docker-compose -f compose.yml -p greenbone-community-edition run --rm greenbone-feed-sync greenbone-feed-sync --type nvt

I am getting service not found error.

ERROR: No such service: greenbone-feed-sync

I also checked the GitHub - greenbone/greenbone-feed-sync: New script for downloading the Greenbone Community Feed However when running

sudo /usr/local/bin/greenbone-feed-sync

I am getting the user not found as shown below,

ubuntu@ubuntu-openvas:~$ sudo /usr/local/bin/greenbone-feed-sync
Running as root. Switching to user 'gvm' and group 'gvm'.
❌Error: Can't run as user 'gvm'. User 'gvm' is unknown.

Can someone please explain how to fix the issue and sync the feeds correctly. Also is greenbone-feed-sync is correct one or not. Does it work, if so how to use it?

apkg · October 21, 2024, 3:08pm

Can you post your compose.yml file? Have you compared this against the updated documentation?

Greenbone Community Containers - Greenbone Community Documentation

mahidhar190699 · October 21, 2024, 7:03pm

@apkg
yes i iused the same compose file from the website.

Also when i pull data-objects image and run it using docker compose up -d, it actually throwed error.

So i have removed the data objects from the docker compose command and tried.

voron · October 24, 2024, 9:36am

Have the same problem, but i have
image: greenbone/vulnerability-tests
not
image: registry.community.greenbone.net/community/vulnerability-tests

mahidhar190699 · October 31, 2024, 4:24pm

Hi @apkg Any help in fixing the issue, would be great.

rippledj · October 31, 2024, 6:06pm

The request from @apkg is to post your docker-compose.yml file:

mahidhar190699 · November 1, 2024, 5:35am

@apkg this is the docker compose file i am using,

Its the same as in the website.

services:
  vulnerability-tests:
    image: greenbone/vulnerability-tests
    environment:
      STORAGE_PATH: /var/lib/openvas/22.04/vt-data/nasl
    volumes:
      - vt_data_vol:/mnt

  notus-data:
    image: greenbone/notus-data
    volumes:
      - notus_data_vol:/mnt

  scap-data:
    image: greenbone/scap-data
    volumes:
      - scap_data_vol:/mnt

  cert-bund-data:
    image: greenbone/cert-bund-data
    volumes:
      - cert_data_vol:/mnt

  dfn-cert-data:
    image: greenbone/dfn-cert-data
    volumes:
      - cert_data_vol:/mnt
    depends_on:
      - cert-bund-data

  data-objects:
    image: greenbone/data-objects
    volumes:
      - data_objects_vol:/mnt

  report-formats:
    image: greenbone/report-formats
    volumes:
      - data_objects_vol:/mnt
    depends_on:
      - data-objects

  gpg-data:
    image: greenbone/gpg-data
    volumes:
      - gpg_data_vol:/mnt

  redis-server:
    image: greenbone/redis-server
    restart: on-failure
    volumes:
      - redis_socket_vol:/run/redis/

  pg-gvm:
    image: greenbone/pg-gvm:stable
    restart: on-failure
    volumes:
      - psql_data_vol:/var/lib/postgresql
      - psql_socket_vol:/var/run/postgresql

  gvmd:
    image: greenbone/gvmd:stable
    restart: on-failure
    volumes:
      - gvmd_data_vol:/var/lib/gvm
      - scap_data_vol:/var/lib/gvm/scap-data/
      - cert_data_vol:/var/lib/gvm/cert-data
      - data_objects_vol:/var/lib/gvm/data-objects/gvmd
      - vt_data_vol:/var/lib/openvas/plugins
      - psql_data_vol:/var/lib/postgresql
      - /home/ubuntu/.openvas/run/gvmd:/run/gvmd
      - ospd_openvas_socket_vol:/run/ospd
      - psql_socket_vol:/var/run/postgresql
    depends_on:
      pg-gvm:
        condition: service_started
      scap-data:
        condition: service_completed_successfully
      cert-bund-data:
        condition: service_completed_successfully
      dfn-cert-data:
        condition: service_completed_successfully
      data-objects:
        condition: service_completed_successfully
      report-formats:
        condition: service_completed_successfully

  gsa:
    image: greenbone/gsa:stable
    restart: on-failure
    ports:
      - 127.0.0.1:9392:80
    volumes:
      - /home/ubuntu/.openvas/run/gvmd:/run/gvmd
    depends_on:
      - gvmd
  # Sets log level of openvas to the set LOG_LEVEL within the env
  # and changes log output to /var/log/openvas instead /var/log/gvm
  # to reduce likelyhood of unwanted log interferences
  configure-openvas:
    image: greenbone/openvas-scanner:stable
    volumes:
      - openvas_data_vol:/mnt
      - openvas_log_data_vol:/var/log/openvas
    command:
      - /bin/sh
      - -c
      - |
        printf "table_driven_lsc = yes\nopenvasd_server = http://openvasd:80\n" > /mnt/openvas.conf
        sed "s/127/128/" /etc/openvas/openvas_log.conf | sed 's/gvm/openvas/' > /mnt/openvas_log.conf
        chmod 644 /mnt/openvas.conf
        chmod 644 /mnt/openvas_log.conf
        touch /var/log/openvas/openvas.log
        chmod 666 /var/log/openvas/openvas.log

  # shows logs of openvas
  openvas:
    image: greenbone/openvas-scanner:stable
    restart: on-failure
    volumes:
      - openvas_data_vol:/etc/openvas
      - openvas_log_data_vol:/var/log/openvas
    command:
      - /bin/sh
      - -c
      - |
        cat /etc/openvas/openvas.conf
        tail -f /var/log/openvas/openvas.log
    depends_on:
      configure-openvas:
        condition: service_completed_successfully

  openvasd:
    image: greenbone/openvas-scanner:stable
    restart: on-failure
    environment:
      # `service_notus` is set to disable everything but notus,
      # if you want to utilize openvasd directly removed `OPENVASD_MODE`
      OPENVASD_MODE: service_notus
      GNUPGHOME: /etc/openvas/gnupg
      LISTENING: 0.0.0.0:80
    volumes:
      - openvas_data_vol:/etc/openvas
      - openvas_log_data_vol:/var/log/openvas
      - gpg_data_vol:/etc/openvas/gnupg
      - notus_data_vol:/var/lib/notus
    # enable port forwarding when you want to use the http api from your host machine
    # ports:
    #   - 127.0.0.1:3000:80
    depends_on:
      vulnerability-tests:
        condition: service_completed_successfully
      configure-openvas:
        condition: service_completed_successfully
      gpg-data:
        condition: service_completed_successfully
    networks:
      default:
        aliases:
          - openvasd

  ospd-openvas:
    image: greenbone/ospd-openvas:stable
    restart: on-failure
    hostname: ospd-openvas.local
    cap_add:
      - NET_ADMIN # for capturing packages in promiscuous mode
      - NET_RAW # for raw sockets e.g. used for the boreas alive detection
    security_opt:
      - seccomp=unconfined
      - apparmor=unconfined
    command:
      [
        "ospd-openvas",
        "-f",
        "--config",
        "/etc/gvm/ospd-openvas.conf",
        "--notus-feed-dir",
        "/var/lib/notus/advisories",
        "-m",
        "666"
      ]
    volumes:
      - gpg_data_vol:/etc/openvas/gnupg
      - vt_data_vol:/var/lib/openvas/plugins
      - notus_data_vol:/var/lib/notus
      - ospd_openvas_socket_vol:/run/ospd
      - redis_socket_vol:/run/redis/
      - openvas_data_vol:/etc/openvas/
      - openvas_log_data_vol:/var/log/openvas
    depends_on:
      redis-server:
        condition: service_started
      gpg-data:
        condition: service_completed_successfully
      vulnerability-tests:
        condition: service_completed_successfully
      configure-openvas:
        condition: service_completed_successfully

  gvm-tools:
    image: greenbone/gvm-tools
    volumes:
      - gvmd_socket_vol:/run/gvmd
      - ospd_openvas_socket_vol:/run/ospd
    depends_on:
      - gvmd
      - ospd-openvas

volumes:
  gpg_data_vol:
  scap_data_vol:
  cert_data_vol:
  data_objects_vol:
  gvmd_data_vol:
  psql_data_vol:
  vt_data_vol:
  notus_data_vol:
  psql_socket_vol:
  gvmd_socket_vol:
  ospd_openvas_socket_vol:
  redis_socket_vol:
  openvas_data_vol:
  openvas_log_data_vol:

The only modification i have done is mounting /run/gvmd socket to the host OS so gvm-cli can be used: - /home/ubuntu/.openvas/run/gvmd:/run/gvmd

rippledj · November 1, 2024, 6:00am

The location of the container registries has changed. You need to update your docker-compose.yml file to use the current version:

services:
  vulnerability-tests:
    image: registry.community.greenbone.net/community/vulnerability-tests
    environment:
      STORAGE_PATH: /var/lib/openvas/22.04/vt-data/nasl
    volumes:
      - vt_data_vol:/mnt

  notus-data:
    image: registry.community.greenbone.net/community/notus-data
    volumes:
      - notus_data_vol:/mnt

  scap-data:
    image: registry.community.greenbone.net/community/scap-data
    volumes:
      - scap_data_vol:/mnt

  cert-bund-data:
    image: registry.community.greenbone.net/community/cert-bund-data
    volumes:
      - cert_data_vol:/mnt

  dfn-cert-data:
    image: registry.community.greenbone.net/community/dfn-cert-data
    volumes:
      - cert_data_vol:/mnt
    depends_on:
      - cert-bund-data

  data-objects:
    image: registry.community.greenbone.net/community/data-objects
    volumes:
      - data_objects_vol:/mnt

  report-formats:
    image: registry.community.greenbone.net/community/report-formats
    volumes:
      - data_objects_vol:/mnt
    depends_on:
      - data-objects

  gpg-data:
    image: registry.community.greenbone.net/community/gpg-data
    volumes:
      - gpg_data_vol:/mnt

  redis-server:
    image: registry.community.greenbone.net/community/redis-server
    restart: on-failure
    volumes:
      - redis_socket_vol:/run/redis/

  pg-gvm:
    image: registry.community.greenbone.net/community/pg-gvm:stable
    restart: on-failure
    volumes:
      - psql_data_vol:/var/lib/postgresql
      - psql_socket_vol:/var/run/postgresql

  gvmd:
    image: registry.community.greenbone.net/community/gvmd:stable
    restart: on-failure
    volumes:
      - gvmd_data_vol:/var/lib/gvm
      - scap_data_vol:/var/lib/gvm/scap-data/
      - cert_data_vol:/var/lib/gvm/cert-data
      - data_objects_vol:/var/lib/gvm/data-objects/gvmd
      - vt_data_vol:/var/lib/openvas/plugins
      - psql_data_vol:/var/lib/postgresql
      - gvmd_socket_vol:/run/gvmd
      - ospd_openvas_socket_vol:/run/ospd
      - psql_socket_vol:/var/run/postgresql
    depends_on:
      pg-gvm:
        condition: service_started
      scap-data:
        condition: service_completed_successfully
      cert-bund-data:
        condition: service_completed_successfully
      dfn-cert-data:
        condition: service_completed_successfully
      data-objects:
        condition: service_completed_successfully
      report-formats:
        condition: service_completed_successfully

  gsa:
    image: registry.community.greenbone.net/community/gsa:stable
    restart: on-failure
    ports:
      - 127.0.0.1:9392:80
    volumes:
      - gvmd_socket_vol:/run/gvmd
    depends_on:
      - gvmd
  # Sets log level of openvas to the set LOG_LEVEL within the env
  # and changes log output to /var/log/openvas instead /var/log/gvm
  # to reduce likelyhood of unwanted log interferences
  configure-openvas:
    image: registry.community.greenbone.net/community/openvas-scanner:stable
    volumes:
      - openvas_data_vol:/mnt
      - openvas_log_data_vol:/var/log/openvas
    command:
      - /bin/sh
      - -c
      - |
        printf "table_driven_lsc = yes\nopenvasd_server = http://openvasd:80\n" > /mnt/openvas.conf
        sed "s/127/128/" /etc/openvas/openvas_log.conf | sed 's/gvm/openvas/' > /mnt/openvas_log.conf
        chmod 644 /mnt/openvas.conf
        chmod 644 /mnt/openvas_log.conf
        touch /var/log/openvas/openvas.log
        chmod 666 /var/log/openvas/openvas.log

  # shows logs of openvas
  openvas:
    image: registry.community.greenbone.net/community/openvas-scanner:stable
    restart: on-failure
    volumes:
      - openvas_data_vol:/etc/openvas
      - openvas_log_data_vol:/var/log/openvas
    command:
      - /bin/sh
      - -c
      - |
        cat /etc/openvas/openvas.conf
        tail -f /var/log/openvas/openvas.log
    depends_on:
      configure-openvas:
        condition: service_completed_successfully

  openvasd:
    image: registry.community.greenbone.net/community/openvas-scanner:stable
    restart: on-failure
    environment:
      # `service_notus` is set to disable everything but notus,
      # if you want to utilize openvasd directly removed `OPENVASD_MODE`
      OPENVASD_MODE: service_notus
      GNUPGHOME: /etc/openvas/gnupg
      LISTENING: 0.0.0.0:80
    volumes:
      - openvas_data_vol:/etc/openvas
      - openvas_log_data_vol:/var/log/openvas
      - gpg_data_vol:/etc/openvas/gnupg
      - notus_data_vol:/var/lib/notus
    # enable port forwarding when you want to use the http api from your host machine
    # ports:
    #   - 127.0.0.1:3000:80
    depends_on:
      vulnerability-tests:
        condition: service_completed_successfully
      configure-openvas:
        condition: service_completed_successfully
      gpg-data:
        condition: service_completed_successfully
    networks:
      default:
        aliases:
          - openvasd

  ospd-openvas:
    image: registry.community.greenbone.net/community/ospd-openvas:stable
    restart: on-failure
    hostname: ospd-openvas.local
    cap_add:
      - NET_ADMIN # for capturing packages in promiscuous mode
      - NET_RAW # for raw sockets e.g. used for the boreas alive detection
    security_opt:
      - seccomp=unconfined
      - apparmor=unconfined
    command:
      [
        "ospd-openvas",
        "-f",
        "--config",
        "/etc/gvm/ospd-openvas.conf",
        "--notus-feed-dir",
        "/var/lib/notus/advisories",
        "-m",
        "666"
      ]
    volumes:
      - gpg_data_vol:/etc/openvas/gnupg
      - vt_data_vol:/var/lib/openvas/plugins
      - notus_data_vol:/var/lib/notus
      - ospd_openvas_socket_vol:/run/ospd
      - redis_socket_vol:/run/redis/
      - openvas_data_vol:/etc/openvas/
      - openvas_log_data_vol:/var/log/openvas
    depends_on:
      redis-server:
        condition: service_started
      gpg-data:
        condition: service_completed_successfully
      vulnerability-tests:
        condition: service_completed_successfully
      configure-openvas:
        condition: service_completed_successfully

  gvm-tools:
    image: registry.community.greenbone.net/community/gvm-tools
    volumes:
      - gvmd_socket_vol:/run/gvmd
      - ospd_openvas_socket_vol:/run/ospd
    depends_on:
      - gvmd
      - ospd-openvas

volumes:
  gpg_data_vol:
  scap_data_vol:
  cert_data_vol:
  data_objects_vol:
  gvmd_data_vol:
  psql_data_vol:
  vt_data_vol:
  notus_data_vol:
  psql_socket_vol:
  gvmd_socket_vol:
  ospd_openvas_socket_vol:
  redis_socket_vol:
  openvas_data_vol:
  openvas_log_data_vol:

apkg · November 1, 2024, 10:17am

Beat me to it

mahidhar190699 · November 1, 2024, 2:43pm

@rippledj

Thanks for the response, I will try the new compose file as suggested. Once It works i will close the issue. Thank you again.

However can you please give clarity on greenbone-feed-sync. is this really required?
Also How to manually sync the feeds like running

sudo /usr/local/bin/greenbone-feed-sync

rippledj · November 1, 2024, 6:10pm

Updating your feed regularly is high recommended for detecting all the latest vulnerabilities. Updating the feed for Community Containers is different than for other Greenbone installation methods. See the link above.

mahidhar190699 · November 20, 2024, 8:19am

Great. This isn’t working. I used exactly the same yml file.

I have done this:Workflows - Greenbone Community Documentation and /run/gvmd/gvmd.sock is never getting created.

On checking the logs,

I got lot of errors, permission erorrs like:

gvmd_1                 | md manage:WARNING:2024-11-20 07h17.59 utc:48: Failed to open lock file '/var/lib/gvm/gvmd/gvm-checking': No such file or directory
gvmd_1                 | md   main:CRITICAL:2024-11-20 07h17.59 utc:48: gvmd: Error trying to get checking lock
gvmd_1                 | md   main:WARNING:2024-11-20 07h17.59 utc:49: init_semaphore_set: error creating semaphore file /var/lib/gvm/gvmd.sem: Permission denied
gvmd_1                 | md   main:MESSAGE:2024-11-20 07h17.59 utc:49:    Greenbone Vulnerability Manager version 24.0.0 (DB revision 256)
gvmd_1                 | md manage:WARNING:2024-11-20 07h17.59 utc:49: Failed to open lock file '/var/lib/gvm/gvmd/gvm-checking': No such file or directory
gvmd_1                 | md   main:CRITICAL:2024-11-20 07h17.59 utc:49: gvmd: Error trying to get checking lock
vulnerability-tests_1  | changed user permissions

When i used the old compose file: Feed Sync Still showing "Too old Please check the automatic synchronization of your system." despite following docs - #7 by mahidhar190699 this is working and the permission denied errors did not occur.
I have deleted all old volumes and images and pulled everything from sratch using both new compose file as you suggested as well as old compose file I was using. So i dont think its somthing related to permissions issue at the host level, even the old compose file should not work if its permissions issue at the host level.
So i believe its the issue with the new compose file.

mahidhar190699 · November 20, 2024, 8:55am

Looks like permission issue on host. When i try to run on brand new system its all working and the feed status are upto date.

Can you please tell me why the permission error was occuring earlier.