Can scan one host fine, but not 20

sawozny · July 21, 2020, 10:09pm

Was wondering if anyone could tell me what I’m doing wrong, here. I’ve been successfully testing (GVM 8, OpenVAS 6 from Atomic repo) on individual systems so I stepped up to group scan, but things got weird and I’m not sure where I went wrong. My task was set to 20 concurrent systems so I chose 20 hosts I know are all up and responsive.

Target

Name: My servers (uncredentialed)
Hosts: Manual 192.168.1.190-209
Exclude Hosts: None
Port List: All TCP and Nmap 5.51 top 1000 UDP
Alive Test: ICMP, TCP-ACK Service & ARP
No credentials
Reverse Lookup Only: No
Reverse Lookup Unify: No

Alert

Name: Email Scott (notify only)
Event: Task run status changed to Done
Condition: Always
Delta Report: None
Method: Email
To Address: scott@internaltestdomain.com
From address: reports@openvas.internaltestdomain.com
Subject: [GVM] Task ‘$n’: $e
Email Encryption: None
Content: Simple Notice
Active: Yes

Task:

Name: All Servers Uncredentialed
Scan Targets: My servers (uncredentialed)
Alerts: Email Scott (notify only)
Schedule: None
Add results to assets: Yes
Apply Overrides: Yes
Min QoD: 70
Auto Delete Reports: Auto delete, but keep newest 5 reports
Scanner: OpenVAS Default
Scan Config: Full and Fast
Network Source Interface: Unspecified (only 1 on system, anyway)
Order for target hosts: Sequential
Max concurrent NVTs per hostL 4
Mac concurrent scanned hosts: 20

When I run this task, it appears from the scanned systems that it’s working, but the scanning eventually stops (about 40 minutes later) and the progress is stuck at 1%. I can request a stop, but it never completes and I have to eventually reboot the server, at which point the status changes to “Interrupted at 1%”.

/var/log/gvm/openvassd.log shows:
sd main:MESSAGE:2020-07-21 05h07.05 utc:18613: Starts a new scan. Target(s) : 192.168.1.190-209, with max_hosts = 20 and max_checks = 4
sd main:MESSAGE:2020-07-21 05h07.05 utc:18760: Testing 192.168.1.194 [18760]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18757: Testing 192.168.1.191 [18757]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18756: Testing 192.168.1.190 [18756]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18758: Testing 192.168.1.192 [18758]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18762: Testing 192.168.1.196 [18762]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18761: Testing 192.168.1.195 [18761]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18764: Testing 192.168.1.198 [18764]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18759: Testing 192.168.1.193 [18759]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18766: Testing 192.168.1.200 [18766]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18769: Testing 192.168.1.203 (Vhosts: repo) [18769]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18763: Testing 192.168.1.197 [18763]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18765: Testing 192.168.1.199 [18765]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18767: Testing 192.168.1.201 [18767]
sd main:MESSAGE:2020-07-21 05h07.05 utc:18768: Testing 192.168.1.202 [18768]
lib nasl:MESSAGE:2020-07-21 05h31.14 utc:32597: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h31.14 utc:32597: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h32.09 utc:2366: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h32.09 utc:2366: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h32.15 utc:2598: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h32.15 utc:2598: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h32.18 utc:2706: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h32.18 utc:2706: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h32.43 utc:3519: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h32.43 utc:3519: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h33.29 utc:4905: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h33.29 utc:4905: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h33.32 utc:4961: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h33.32 utc:4961: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h33.43 utc:5182: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h33.43 utc:5182: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h33.46 utc:5194: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h33.46 utc:5194: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h33.58 utc:5530: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h33.58 utc:5530: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h34.08 utc:5957: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h34.08 utc:5957: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h35.34 utc:8569: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h35.34 utc:8569: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h35.41 utc:8650: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h35.41 utc:8650: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
lib nasl:MESSAGE:2020-07-21 05h35.50 utc:8700: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 05h35.50 utc:8700: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
sd main:MESSAGE:2020-07-21 05h43.55 utc:18767: Finished testing 192.168.1.201. Time : 2209.83 secs
sd main:MESSAGE:2020-07-21 05h44.23 utc:18763: Finished testing 192.168.1.197. Time : 2238.70 secs
sd main:MESSAGE:2020-07-21 05h44.37 utc:18768: Finished testing 192.168.1.202. Time : 2252.54 secs
sd main:MESSAGE:2020-07-21 05h44.40 utc:18762: Finished testing 192.168.1.196. Time : 2255.42 secs
sd main:MESSAGE:2020-07-21 05h44.49 utc:18764: Finished testing 192.168.1.198. Time : 2264.41 secs
sd main:MESSAGE:2020-07-21 05h45.03 utc:18761: Finished testing 192.168.1.195. Time : 2278.65 secs
sd main:MESSAGE:2020-07-21 05h45.25 utc:18766: Finished testing 192.168.1.200. Time : 2299.73 secs
sd main:MESSAGE:2020-07-21 05h45.31 utc:18765: Finished testing 192.168.1.199. Time : 2306.48 secs
sd main:MESSAGE:2020-07-21 05h45.33 utc:18760: Finished testing 192.168.1.194. Time : 2307.77 secs
sd main:MESSAGE:2020-07-21 05h47.52 utc:18758: Finished testing 192.168.1.192. Time : 2447.72 secs
sd main:MESSAGE:2020-07-21 05h48.05 utc:18759: Finished testing 192.168.1.193. Time : 2460.54 secs
sd main:MESSAGE:2020-07-21 05h50.14 utc:18769: Finished testing 192.168.1.203. Time : 2589.22 secs
sd main:MESSAGE:2020-07-21 05h51.09 utc:18757: Finished testing 192.168.1.191. Time : 2644.36 secs
sd main:MESSAGE:2020-07-21 05h51.57 utc:18756: Finished testing 192.168.1.190. Time : 2692.14 secs

And then nothing until I reboot the server. The SSH Proto errors (2 for each actually scanned system), while something I’m going to eventually chase down, don’t appear to be the cause of this issue.

/var/log/gvm/gvmd.log shows in the same timeframe:
event task:MESSAGE:2020-07-21 05h06.42 UTC:18586: Status of task (6ab6e4b5-1197-456b-8880-fe97b683bd12) has changed to New
event task:MESSAGE:2020-07-21 05h06.42 UTC:18586: Task All Servers Uncredentialed (6ab6e4b5-1197-456b-8880-fe97b683bd12) has been created by admin
event task:MESSAGE:2020-07-21 05h06.46 UTC:18611: Status of task All Servers Uncredentialed (6ab6e4b5-1197-456b-8880-fe97b683bd12) has changed to Requested
event task:MESSAGE:2020-07-21 05h06.46 UTC:18611: Task All Servers Uncredentialed (6ab6e4b5-1197-456b-8880-fe97b683bd12) has been requested to start by admin
event task:MESSAGE:2020-07-21 05h07.00 UTC:18616: Status of task All Servers Uncredentialed (6ab6e4b5-1197-456b-8880-fe97b683bd12) has changed to Running

And then nothing until I rebooted the server.

So I tried again with the same setup but only one server (which was successful before, but I thought maybe something I thought was an unrelated parameter in my group scans was fouling me up). To do this I cloned the Target, but set the host to manual: 192.168.1.192 and I cloned the task, switching to the newly cloned target.

Here’s what I got for /var/log/gvm/openvassd.log:
sd main:MESSAGE:2020-07-21 19h11.32 utc:2078: Starts a new scan. Target(s) : 192.168.1.192, with max_hosts = 20 and max_checks = 4
sd main:MESSAGE:2020-07-21 19h11.32 utc:2218: Testing 192.168.1.192 [2218]
lib nasl:MESSAGE:2020-07-21 19h32.02 utc:9664: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-512’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-512)
lib nasl:MESSAGE:2020-07-21 19h32.02 utc:9664: Function ssh_login called from ssh_proto_version.nasl: Failed to set SSH key type ‘rsa-sha2-256’: Setting method: no algorithm for method “server host key algo” (rsa-sha2-256)
sd main:MESSAGE:2020-07-21 19h39.29 utc:2218: Finished testing 192.168.1.192. Time : 1677.44 secs
sd main:MESSAGE:2020-07-21 19h40.09 utc:2078: Test complete
sd main:MESSAGE:2020-07-21 19h40.09 utc:2078: Total time to scan all hosts : 1722 seconds

And /var/log/gvm.gvmd.log:
event target:MESSAGE:2020-07-21 19h09.39 UTC:1822: Target My servers (uncredentialed) Clone 1 (e4f421a1-4322-4ed3-9db6-8205add84b37) has been created by admin
event target:MESSAGE:2020-07-21 19h10.05 UTC:1856: Target One server (uncredentialed) (e4f421a1-4322-4ed3-9db6-8205add84b37) has been modified by admin
event task:MESSAGE:2020-07-21 19h10.29 UTC:1934: Task All Servers Uncredentialed (6ab6e4b5-1197-456b-8880-fe97b683bd12) has been modified by admin
event task:MESSAGE:2020-07-21 19h10.36 UTC:1964: Status of task (23d3d963-30cf-40c1-82de-649fed9022ea) has changed to New
event task:MESSAGE:2020-07-21 19h10.36 UTC:1964: Status of task All Servers Uncredentialed Clone 1 (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has changed to New
event task:MESSAGE:2020-07-21 19h10.36 UTC:1964: Task All Servers Uncredentialed Clone 1 (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has been created by admin
event task:MESSAGE:2020-07-21 19h11.01 UTC:2040: Task One Server Uncredentialed (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has been modified by admin
event task:MESSAGE:2020-07-21 19h11.11 UTC:2072: Status of task One Server Uncredentialed (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has changed to Requested
event task:MESSAGE:2020-07-21 19h11.11 UTC:2072: Task One Server Uncredentialed (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has been requested to start by admin
event task:MESSAGE:2020-07-21 19h11.27 UTC:2081: Status of task One Server Uncredentialed (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has changed to Running
event task:MESSAGE:2020-07-21 19h40.10 UTC:2081: Status of task One Server Uncredentialed (5c05312a-0e7d-4c04-83e5-ad8f924c2c11) has changed to Done
event alert:MESSAGE:2020-07-21 19h40.10 UTC:2081: The alert Email Scott (notify only) was triggered (Event: Task status changed to ‘Done’, Condition: Always)

So everything works, but not for a range of 20 systems. One thing I noticed when looking through the logs is that while the OpenVAS scanner was OK’d to scan 20 systems, it only started scanning 14 (which it finished) and never any more. The only unusual thing about that system is that it was tagged with “(Vhosts: repo)” in the scan which is both the name of the host and the Apache virtual host on it that servers RPMs. Not sure why that would make a difference, but it IS where it stopped progressing through the hosts list.

Any ideas on what I might have done wrong or next steps to troubleshoot this would be greatly appreciated.

Thanks,

Scott

cfi · July 22, 2020, 6:43am

Could you clarify / specify the used versions in more detail? Because there wasn’t a GVM 8 (only OpenVAS 8) and OpenVAS 6 would be end-of-life since many years.

Generally this looks like a known bug in older GVM versions:

https://community.greenbone.net/tag/1-percent

Make sure that you run the latest releases of either GVM-10 or GVM-11:

sawozny · July 22, 2020, 7:31pm

Here are my GVM and OpenVAS Scanner versions.

[sawozny@openvas ~]$ gvmd --version
Greenbone Vulnerability Manager 8.0.0
Manager DB revision 205
Copyright © 2010-2017 Greenbone Networks GmbH
License GPLv2+: GNU GPL version 2 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[sawozny@openvas ~]$ openvassd --version
OpenVAS Scanner 6.0.0
Most new code since 2005: © 2018 Greenbone Networks GmbH
Nessus origin: © 2004 Renaud Deraison deraison@nessus.org
License GPLv2: GNU GPL version 2
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

According the the GVM-10 link you provided:

Source Code Release

Main:

So, admittedly, I’m a couple minor versions back but not years out of date. Unless Atomic updates their build I’m sort of stuck there. I’m using the Atomic repo for CentOS 7 since compiling from source for CentOS 7 has a number of roadblocks and compiling against static libraries is outside my comfort zone. Also I can’t use the GCE image because I don’t use VirtualBox or VMWare and, according to the documentation, there won’t be schedules or alarms, both of which I need.

The list of potential issues that cause 1 percent hangups is long, but I thank you for the link. I’ll start looking through them to see if my issue (and a potential workaround) is in there. I’m also planning on turning up the log level. I just posted here to see if there was anything obviously wrong in my configuration like “don’t use parentheses in names” or something weird like that.

Thanks,

Scott

cfi · July 23, 2020, 12:19pm

Thanks for clarifying this.

Unfortunately this outdated version of the scanner could contain bugs like e.g. the following which could be responsible for this seen behavior.

An issue which caused a scan to hang for ever if there was no redis kb
available has been addressed.
→ openvas-scanner/CHANGES at v6.0.1 · greenbone/openvas-scanner · GitHub

You could try to review your redis setup based on Hint: Redis setup / configuration for GSE/GVM/OpenVAS - Greenbone Community Edition - Greenbone Community Forum but an upgrade to a newer release is still hightly recommended.

sawozny · July 25, 2020, 10:54pm

Thanks for the suggestions. After looking through your suggested links and turning up the logging level it turned out redis databases was set to 16 which isn’t a workable value for a 4/20 scan. Somaxconn was already up to 1024 so when I upped the databases value to 128 everything behaved. Thanks very much for the suggestions.

And yes, I would absolutely like to be on the most current build of the version I’m working with, but I’m restricted to what the package manager makes available after trying to build from source on CentOS 7 which has system libraries that can’t be updated to the needed level with the package manager (and modifying the make files to statically link, as needed, being beyond my present abilities) and being unable to use GCE due to hypervisor compatibility and application functionality restrictions. If I run into anything that can’t be fixed I’ll probably build a CentOS 8 image just for GVM and build from source, but in the meantime I appreciate your assistance in tracking down the problem.

I’m actually surprised atomic didn’t adjust the installed redis.conf knowing a default 4/20 scan would immediately overwhelm it, but this is part of the joy of open source, isn’t it?

Thanks,

Scott

cfi · July 28, 2020, 5:18am

They could be not aware of this. You could raise an issue at Issues · Atomicorp/gvm · GitHub to get such a higher limit by default.