Openvasmd authentication error after starting big task

[tatooin]

Hello all,

System: Linux mint 18.3
openvas9 7.0.3-1
openvas9-cli 1.4.5-1xenial
openvas9-manager 7.0.3-1
_openvas9-scanner 5.1.3-1 _
redis-server 2:3.0.6-1ubuntu0.3 (with openvas configuration settings)

Issue:
Starting a task on a somehow big target (1000 IP addresses) will lead openvasmd stop answering requests with error:

#### Internal error: exec_omp_get:3113 (GSA 7.0.3)

An internal error occurred. Diagnostics: Could not authenticate to manager daemon.

According to openvasmd.log the scan is started and task status changed to “Running”. Right after this, the following error is logged in openvasmd.log:

md manage:WARNING:2019-01-27 16h32.40 utc:18932: sql_exec_internal: sqlite3_step failed: interrupted
md manage:WARNING:2019-01-27 16h32.40 utc:18932: sqlv: sql_exec_internal failed

However the scan continue in the background.

The only way to restore access to openvasmd is either by flushing redis cache or by restarting openvasm, which will put the task status to “Stopped”.

This is a major issue as basically openvas9 is unuseable with big tasks in this environment. I must note that starting small tasks doesn’t create any issue.

Last point, this doesn’t seem related with bug described in https://community.greenbone.net/t/sqlite- error-and-tasks-stuck/199/10 as patching the current version with the patch available doesn’t fix the problem.

Any clue ?

Thanks a lot !

[Lukas]

Did you checked that your machine had enough resources, did you turned memory over-commitment off ? I don´t think this is a GVM9 issue, more of your build, setup or machine.

[cfi]

This might be related to/the same as Gvm-cli connections fail under openvassd load where the same messages in the openvasmd.log are showing up. The “high load” could happen on “big tasks” as well.

[tatooin]

The machine has a lot of ressources and is clearly not overloaded. Memory over-commitment is set to default, and according to what I can read elsewhere, changing this setting is not recommended. But I don’t really believe in this anyway, as again my system is full of RAM/Disks space.

[tatooin]

I checked that post but unfortunately there doesn’t seem to be a fix for this available yet. Also, this problem wasn’t happening with GVM8 (same machine, same config) where I was managing big tasks without any problems.

[tatooin]

Now, assuming it’s a bug in GVM9 (which I believe it is), is there anything I can do to help developers find the root cause of this and a cure ?

I’m not familiar with how Greenbone is organized and if there is any guidelines to follow to submit a bug report for instance ?

Thanks

[cfi]

I checked that post but unfortunately there doesn’t seem to be a fix for this available yet.

Please keep in mind that there can’t be a solution for every problem, especially not within such a short time-frame the linked post was created.

Also, this problem wasn’t happening with GVM8 (same machine, same config) where I was managing big tasks without any problems.

New major releases might introduce additional capabilities / features bringing additional load and complexity where e.g. the SQLite database is hitting its limits.

Now, assuming it’s a bug in GVM9 (which I believe it is), is there anything I can do to help developers find the root cause of this and a cure ?

You could subscribe to the referenced issue tracker linked in the thread above to see if there are any questions of the development team: https://github.com/greenbone/gvmd/issues/347

I’m not familiar with how Greenbone is organized and if there is any guidelines to follow to submit a bug report for instance ?

Generally its recommended to create a new issue at one of the github issue trackers of the problematic component if you think you have found a bug within the code and not only a configuration / environmental issue.

Greenbone Networks GmbH

[tatooin]

Just an update on this; actually you can connect back to GSA after some times. You don’t need to restart / flush anything. I’ve scanned around 50 targets from different sizes and I confirm the problem happens randomely, but almost on every targets.

I also notices that the dashboards are often broken, and everytime this happens I see the sqlite3 error in my logs.

Has anyone else seen this problem ?

Thanks !

[Lukas]

With lot´s of results, you will get much better results by using PostgreSQL SQLITE does have many locking issues, so this might be worth looking into a real database.

[tatooin]

Thanks Lukas. Well, fact is my database is already quite large (almost 3GB large) so switching to a real database as you said couldn’t harm. I will try your suggestion. Is there any guide or howto to available to do this transition ?

Thanks !

[Mogimbo]

Hi tatooin, I think the problem is similar to mine (Openvas with PostgresSQL Could not authenticate to manager daemon)
I have quite 700 Tasks and 20 daily scans (during the night).
In the beginning i used SQLite3 and had the connection problems with quite 300 tasks. The problem was SQLite timeout and I solved modifying the openvas source code.
Because this is not a good idea, I tried to use PostgresSQL but today I have the same problems.
I think the problem is still about timeout connection to the DB.
I create a my own (simple) interface in PHP end connect directly to openvas PostgresSQL and it works, so the problem shouldn’t be the DB.

Hope this helps.
Giovanni

[tatooin]

Hi Giovanni,

Thanks for your answer. What version of openvas manager are you using ? Have you tried with the latest version available on git or like me with a rather old version provided by packages maintainers (in my case, ubuntu).

Thanks a lot

[Mogimbo]

Hi tatooin,
I compiled from source openvas9-manager-7.0.3 on ubuntu 18.08

here a summary about my personal guide

sudo apt install -y postgresql
sudo apt install -y postgresql-contrib postgresql-server-dev-10
cd /usr/local/src
sudo mkdir -p openvas/debs
cd openvas
sudo add-apt-repository -s ppa:mrazavi/openvas
sudo apt update
sudo apt install -y dpkg-dev debhelper cmake pkg-config libglib2.0-dev libgnutls28-dev libgcrypt11-dev libsqlite3-dev libgpgme11-dev rats libopenvas9-dev smbclient

sudo su -
apt source openvas9-manager
cd openvas9-manager-7.0.3
dpkg-checkbuilddeps
[etc etc]

[tatooin]

Would be interesting to see if latest version available on git is fixing this. I’ve installed the latest version from git on ubuntu as well and now trying to have it work. I’ll see if I get the same results there.

I think it’s a bug so hopefully it has been fixed already.

[Mogimbo]

I have a complete guide written by myself (crawling on internet )
Maybe you have to change only the source url
If u need just ask

Giovanni

[tatooin]

Hi Giovanni,

Yes please. This would certainly help me get this done

[Mogimbo]

I create a post:

Guide to install Openvas-Manager with Postgres

This a guide I have written during my installation of openvas. All commands are taken from internet, and maybe something could be useless, but for it works Please try it and give me feedback to improve and fix the guide Hope it helps sudo apt update; sudo apt -y dist-upgrade; sudo apt -y autoremove sudo apt install -y postgresql sudo systemctl status postgresql sudo -u postgres createuser root sudo -u postgres createdb -O root tasks sudo -u postgres psql tasks create role dba…

[tatooin]

Thanks Giovanni. However it seems you’re recompiling the sources used by the package maintainer, which are probably not the latest version available.

I’m going to try by git cloning the official repository and see if I can replicate the issues.

Thanks !

[Mogimbo]

Indeed
Let me know if it works

Giovanni

[tatooin]

Hi Giovanni,

Just to keep you updated;
I have recompiled the source of the latest stable version available (GVM-9) and I confirm the problem is still there. It seems it’s a bug introduced in GVM-9, as this wasn’t happening prior upgrading to this version.

There seems to be a fix in the master branch (https://github.com/greenbone/gvmd/pull/338)