OOM Killing Redis on Large Scan with OpenVAS

Scanning / scan configuration
1

I have a large, sparsely populated scan environment. One particular scan is of a /21 range, hitting 187 hosts. The scan is running out of memory. The Linux oom killer is killing Redis.

The scan is limited to 10 simultaneous hosts, with 3 consecutive nvts per host.

Currently our OpenVAS environment is a bit old, I need to have the team upgrade the software, presently 21.4.1

The machine is presently running on 16 cores and 24GB of RAM.

How do I tune this? I would rather not have to break up the scan into smaller scans, as distribution of hosts in the subnets is not even, so I can’t just use /24s, I would have to dig and analyze and fiddle with lists of hosts, losing broad coverage of the “unknowns” on the network.

Why is Redis using so much RAM? Is there something that can be further tuned?

2

Hello mgjk, and welcome to the Greenbone community!

The problem you describe is not easy to solve as it can have several root causes, from known issues to usage behaviour. In particular, there can be problems with vHosts and CGI caching.

In general, we recommend the following:

  • Prevent overloading the system by adjusting the usage:
    • Do not start scan tasks all at once, use schedules to start them at intervals
    • Reconfigure scan targets to include less hosts, split the hosts into more targets and tasks instead
    • Do not run or schedule feed updates for times where scan tasks are running or scheduled to run
    • Do not view or download large reports while scan tasks are running
  • Disable vHost expansion for scans that cause problems:
    • Clone and edit the used scan config
    • Set the scanner preference expand_vhosts to 0 and save the change
  • Disable CGI caching for scans that cause problems:
    • Clone and edit the used scan config
    • Browse to the VT family Settings
    • Edit the VT Global Variable Settings (OID: 1.3.6.1.4.1.25623.1.0.12288)
    • Set the preference Disable caching of web pages during CGI scanning to Yes and save the change

Last but not least, if you think that you can narrow the problem down to a specific host and/or vulnerability test, please either open an issue for the scanner at https://github.com/greenbone/openvas-scanner/issues or the vulnerability test at Vulnerability Tests - Greenbone Community Forum!

2 Likes
3

Thank you so much for the tips. Is there a recommended way to increasing log verbosity? to e…g, figure out exactly which plugins launched when resource usage started to get excessive?

What in redis would be using so much memory? Is that what the vHost expansion and redis caching suggestion is trying to address?