Ospd-openvas died because of redis

My issue is I scan a target with openvas from: https://greenbone.github.io/docs/latest/22.4/container/index.html
I get Stopped at 99% on a task of this target. When I look the logs, ospd-openvas stopped to run. The logs has:
get_redis_ctx: redis connection error to /run/redis/redis.sock: Connection refused
The redis is still running tough.
When I restarted ospd-openvas I got libgvm util: get_redis_ctx: connected to redis:///run/redis/redis.sock/2 and not redis:///run/redis/redis.sock/
This issue only happens when I am scanning a specific target.
More logs:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/redis/connection.py", line 512, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/resp2.py", line 15, in read_response
    result = self._read_response(disable_decoding=disable_decoding)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/resp2.py", line 25, in _read_response
    raw = self._buffer.readline()
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/socket.py", line 115, in readline
    self._read_from_socket()
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/socket.py", line 65, in _read_from_socket
    data = self._sock.recv(socket_read_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/ospd-openvas", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/daemon.py", line 1264, in main
    daemon_main('OSPD - openvas', OSPDopenvas, NotusParser())
  File "/usr/local/lib/python3.11/dist-packages/ospd/main.py", line 153, in main
    daemon.run()
  File "/usr/local/lib/python3.11/dist-packages/ospd/ospd.py", line 1103, in run
    self.scheduler()
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/daemon.py", line 708, in scheduler
    self.check_feed()
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/daemon.py", line 682, in check_feed
    current_feed = self.nvti.get_feed_version()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/nvticache.py", line 70, in get_feed_version
    return OpenvasDB.get_single_item(self.ctx, NVTI_CACHE_NAME)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/db.py", line 268, in get_single_item
    return ctx.lindex(name, index)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/commands/core.py", line 2677, in lindex
    return self.execute_command("LINDEX", name, index)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 548, in execute_command
    return conn.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/retry.py", line 65, in call_with_retry
    fail(error)
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 552, in <lambda>
    lambda error: self._disconnect_raise(conn, error),
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 538, in _disconnect_raise
    raise error
  File "/usr/local/lib/python3.11/dist-packages/redis/retry.py", line 62, in call_with_retry
    return do()
           ^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 549, in <lambda>
    lambda: self._send_command_parse_response(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 525, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 565, in parse_response
    response = connection.read_response()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/connection.py", line 520, in read_response
    raise ConnectionError(
redis.exceptions.ConnectionError: Error while reading from /run/redis/redis.sock : (104, 'Connection reset by peer')
OSPD[2] 2024-10-20 19:06:33,543: ERROR: (ospd.ospd) f3df7005-0e24-4e09-810f-945fec56e161: Exception Error while reading from /run/redis/redis.sock : (104, 'Connection reset by peer') while scanning
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/redis/connection.py", line 512, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/resp2.py", line 15, in read_response
    result = self._read_response(disable_decoding=disable_decoding)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/resp2.py", line 25, in _read_response
    raw = self._buffer.readline()
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/socket.py", line 115, in readline
    self._read_from_socket()
  File "/usr/local/lib/python3.11/dist-packages/redis/_parsers/socket.py", line 65, in _read_from_socket
    data = self._sock.recv(socket_read_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/ospd/ospd.py", line 583, in start_scan
    self.exec_scan(scan_id)
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/daemon.py", line 1194, in exec_scan
    target_is_finished = kbdb.target_is_finished(scan_id)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/db.py", line 576, in target_is_finished
    status = self._get_single_item(f'internal/{scan_id}')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/db.py", line 470, in _get_single_item
    return OpenvasDB.get_single_item(self.ctx, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd_openvas/db.py", line 268, in get_single_item
    return ctx.lindex(name, index)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/commands/core.py", line 2677, in lindex
    return self.execute_command("LINDEX", name, index)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 548, in execute_command
    return conn.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/retry.py", line 65, in call_with_retry
    fail(error)
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 552, in <lambda>
    lambda error: self._disconnect_raise(conn, error),
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 538, in _disconnect_raise
    raise error
  File "/usr/local/lib/python3.11/dist-packages/redis/retry.py", line 62, in call_with_retry
    return do()
           ^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 549, in <lambda>
    lambda: self._send_command_parse_response(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 525, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/client.py", line 565, in parse_response
    response = connection.read_response()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/redis/connection.py", line 520, in read_response
    raise ConnectionError(
redis.exceptions.ConnectionError: Error while reading from /run/redis/redis.sock : (104, 'Connection reset by peer')
OSPD[2] 2024-10-20 19:06:33,591: WARNING: (ospd.ospd) Error sending data to the client while executing a scan f3df7005-0e24-4e09-810f-945fec56e161.
Exception ignored in atexit callback: <function exit_cleanup at 0x7f8039e19760>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/ospd/main.py", line 69, in exit_cleanup
    daemon.daemon_exit_cleanup()
  File "/usr/local/lib/python3.11/dist-packages/ospd/ospd.py", line 447, in daemon_exit_cleanup
    self.scan_collection.clean_up_pickled_scan_info()
  File "/usr/local/lib/python3.11/dist-packages/ospd/scan.py", line 251, in clean_up_pickled_scan_info
    if self.get_status(scan_id) == ScanStatus.QUEUED:
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/ospd/scan.py", line 352, in get_status
    status = self.scans_table.get(scan_id, {}).get('status', None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in get
  File "/usr/lib/python3.11/multiprocessing/managers.py", line 818, in _callmethod
    self._connect()
  File "/usr/lib/python3.11/multiprocessing/managers.py", line 805, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/connection.py", line 501, in Client
    c = SocketClient(address)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/connection.py", line 629, in SocketClient
    s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

You need to ensure that your redis is running correct. Please check why your redis died ?

I did check for OOM, disk space, logs, erros, socket permission but no indication of failure, here the logs:
The only log I found in ospd-openvas which indicates error is:
Ospd-openvas log:

lib  misc:   [3650] gnutls_handshake: A TLS fatal alert has been received.
lib  misc:   [3650] release_connection_fd: fd > 0 fd=19
....

 libgvm util: Key 'HostDetails' already contained value 'EXIT_CODE'
 main:  pre_fn_call: called
libgvm util:   Key 'HostDetails/NVT' already contained value '1.3.6.1.4.1.25623.1.0.801957'
libgvm util:  Key 'HostDetails/NVT/1.3.6.1.4.1.25623.1.0.801957/EXIT_CODE' already contained value 'EXIT_NOTVULN'
....
post_fn_call: called
libgvm util:   get_redis_ctx: connected to redis:///run/redis/redis.sock/3
sd   main:  procs_cleanup_children: checking 5 ipc.
sd   main:  create_ipc_process: closed 0 fd.
sd   main:  pluginlaunch_wait_for_free_process. Number of running processes >= maximum running processes (4 >= 4). Waiting for free slot for processes.
....
libgvm util:CRITICAL: No redis DB available

Redis log

 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
 # Redis version=7.0.15, bits=64, commit=00000000, modified=0, pid=7, just started
 # Configuration loaded
 * monotonic clock: POSIX clock_gettime
 * Running mode=standalone, port=0.
 # Server initialized
 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
 * The server is now ready to accept connections at /run/redis/redis.sock
Killed
 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
 # Redis version=7.0.15, bits=64, commit=00000000, modified=0, pid=8, just started
 # Configuration loaded
 * monotonic clock: POSIX clock_gettime
 * Running mode=standalone, port=0.
 # Server initialized
 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
 * The server is now ready to accept connections at /run/redis/redis.sock

I have the felling the ospd-openvas got stuck and somehow blew the redis or the connection pool.

I got more logs running locally.
PG-GVM:

 gvmd@gvmd ERROR:  canceling statement due to user request
 gvmd@gvmd STATEMENT:  SELECT EXISTS (SELECT * FROM information_schema.tables               WHERE table_catalog = 'gvmd'               AND table_schema = 'public'               AND table_name = 'meta')        ::integer;

OPENVAS:

 libgvm util: fetch_max_db_index: maximum DB number: 1025

GVMD:

 Status of task Customer scan (2eb3ee34-26ee-4b0e-be91-f560ebc7ce17) has changed to Running
 BACKTRACE: gvmd(+0x6e574) [0xaaaad667e574]
 BACKTRACE: linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffff953ad7a0]
 BACKTRACE: /lib/aarch64-linux-gnu/libc.so.6(gsignal+0xdc) [0xffff94c88e6c]
 BACKTRACE: /lib/aarch64-linux-gnu/libc.so.6(abort+0x108) [0xffff94c75a60]
 BACKTRACE: gvmd(sql_string+0) [0xaaaad6690b80]
 BACKTRACE: gvmd(manage_db_empty+0x1c) [0xaaaad670b7bc]
 BACKTRACE: gvmd(manage_db_version+0x10) [0xaaaad66a7950]
 BACKTRACE: gvmd(manage_create_sql_functions+0x18) [0xaaaad670bdfc]
 BACKTRACE: gvmd(init_manage_process+0x68) [0xaaaad66a9cfc]
 BACKTRACE: gvmd(init_gmp_process+0x4c) [0xaaaad6743afc]
 BACKTRACE: gvmd(serve_gmp+0x6c) [0xaaaad66821f0]
 BACKTRACE: gvmd(+0x6e294) [0xaaaad667e294]
 BACKTRACE: gvmd(+0x6e850) [0xaaaad667e850]
 BACKTRACE: gvmd(gvmd+0x1ae4) [0xaaaad6681874]
 BACKTRACE: /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe8) [0xffff94c75dd8]
 BACKTRACE: gvmd(+0x6deb8) [0xaaaad667deb8]
 Received Aborted signal
 Status of task Customer scan (2eb3ee34-26ee-4b0e-be91-f560ebc7ce17) has changed to Interrupted

Based on the logs, seems the scan reached a max of redis DB’s

Did you enable it ? Here is the error message from redis :wink:

I don’t think is the cause, when I ran locally, I don’t get this warning, but I still got the issue.
I also increase the redis databases 50500 and redis-server container to 12 cores. Seems before redis stop to work and restart the CPU usage get’s high(100%).