Can scan one host fine, but not 20

Could you clarify / specify the used versions in more detail? Because there wasn’t a GVM 8 (only OpenVAS 8) and OpenVAS 6 would be end-of-life since many years.

Generally this looks like a known bug in older GVM versions:

https://community.greenbone.net/tag/1-percent

Make sure that you run the latest releases of either GVM-10 or GVM-11: