Scanning cause OOM with 32Gb memory

Building from Source and Advanced Topics
1

I have an ubuntu sever with greenbone scanner v22 built from source for a few years.
I have created a schedule to scan my ~20 servers once a week.
Recently I found out that scan caused OOM.
I have reduced number of concurrent hosts and NVTs to 1 in scan configuration.
I have scaled my server up 32Gb memory.
Still catching OOM kill.
According to the logs:
postgres invoked oom-killer: gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE)
Here is screenshot from my grafana server:

Screenshot from 2024-12-09 14-37-52
Screenshot from 2024-12-09 14-37-52967×408 42.3 KB

Any advices?

2

Hello chushev, the problem you describe is not easy to solve as it can have several root causes, from known issues to usage behaviour. In particular, there can be problems with vHosts and CGI caching.

In general, we recommend the following:

  • Prevent overloading the system by adjusting the usage:
    • Do not start scan tasks all at once, use schedules to start them at intervals
    • Reconfigure scan targets to include less hosts, split the hosts into more targets and tasks instead
    • Do not run or schedule feed updates for times where scan tasks are running or scheduled to run
    • Do not view or download large reports while scan tasks are running

There are some known issues, these could be classified as bugs, and we are currently working on a new scanner generation to solve these. Until then, there are some workarounds:

  • Disable vHost expansion for scans that cause problems:
    • Clone and edit the used scan config
    • Set the scanner preference expand_vhosts to 0 and save the change
  • Disable CGI caching for scans that cause problems:
    • Clone and edit the used scan config
    • Browse to the VT family Settings
    • Edit the VT Global Variable Settings (OID: 1.3.6.1.4.1.25623.1.0.12288)
    • Set the preference Disable caching of web pages during CGI scanning to Yes and save the change

Oftentimes, it is a specific host being scanned that cause such problems. If you can narrow down the host(s), and if they do not contain vHosts (we are working on that problem), feel free to open an issue for our developers at Issues · greenbone/openvas-scanner · GitHub with as much information as you can gather.

4 Likes