Trouble getting ospd-openvas working in GVM11 [GSE]

Archive Greenbone Community Edition
1

Hi,

I compiled everything from the released tarballs.
And I get stuck at getting NVT’s into the db, I think…
I’m having big trouble to get my head around what process should do that?

And when I try to start up ospd-openvas i get this error:

2019-11-04 09:35:14,114 OSPD - openvas: DEBUG: (ospd.server) New connection from /opt/gvm/var/run/ospd.sock
2019-11-04 09:35:34,006 OSPD - openvas: ERROR: (ospd.ospd) While handling client command:
Traceback (most recent call last):
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd/ospd.py", line 762, in handle_client_stream
    response = self.handle_command(data)
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd/ospd.py", line 1624, in handle_command
    return self.handle_get_vts_command(tree)
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd/ospd.py", line 1011, in handle_get_vts_command
    vts_xml = self.get_vts_xml(vt_id, filtered_vts)
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd/ospd.py", line 1548, in get_vts_xml
    vts_xml.append(self.get_vt_xml(vt_id))
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd/ospd.py", line 1487, in get_vt_xml
    vt_id, vt.get('affected')
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd_openvas/daemon.py", line 684, in get_affected_vt_as_xml_str
    _affected.text = affected
  File "src/lxml/etree.pyx", line 1023, in lxml.etree._Element.text.__set__
  File "src/lxml/apihelpers.pxi", line 734, in lxml.etree._setNodeText
  File "src/lxml/apihelpers.pxi", line 722, in lxml.etree._createTextNode
  File "src/lxml/apihelpers.pxi", line 1527, in lxml.etree._utf8
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

The setup I use is documented here https://sadsloth.net/post/install-gvm11-src_part1/.
Is there anything I can do to help out with debugging?

–
Regards Falk

2

I think it will be fixed with https://github.com/greenbone/ospd-openvas/pull/139

2 Likes
3

Hi,

I tried to do a new build with ospd-2.0.0 git and ospd-openvas-1.0.0 git today.
and the ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters is now fixed it seems like.

But instead I got a new error, that seems redis related.

(ospd-scanner) gvm@gvm:/opt/gvm/src/ospd-openvas$ ospd-openvas --pid-file /opt/gvm/var/run/ospd-openvas.pid \
>              --unix-socket=/opt/gvm/var/run/ospd.sock \
>              --log-file /opt/gvm/var/log/gvm/ospd-scanner.log \
>              --log-level DEBUG -f
2019-11-06 19:18:11,303 OSPD - openvas: DEBUG: (ospd_openvas.daemon) Loading NVTs in Redis DB
Traceback (most recent call last):
  File "/opt/gvm/bin/ospd-scanner/bin/ospd-openvas", line 8, in <module>
    sys.exit(main())
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd_openvas/daemon.py", line 1532, in main
    daemon_main('OSPD - openvas', OSPDopenvas)
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd/main.py", line 155, in main
    daemon.init()
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd_openvas/daemon.py", line 293, in init
    self.openvas_db.set_redisctx(ctx)
  File "/opt/gvm/bin/ospd-scanner/lib/python3.7/site-packages/ospd_openvas/db.py", line 133, in set_redisctx
    raise RequiredArgument('set_redisctx', 'ctx')
ospd.errors.RequiredArgument: set_redisctx: Argument ctx is required
2019-11-06 19:18:12,920 OSPD - openvas: DEBUG: (ospd.misc) Finishing daemon process

So I went back to the original ospd-openvas-1.0.0 and as you wrote, with an update of nvt’s (today) things started working.

But, the above “problem” still exists with the git ospd-openvas-1.0.0.

–
Regards Falk

4

Hi @falk,
I have already seen this error. If I’a not mistaken, this seems to be a problem caused by mixing gvm-libs, scanner and ospd-openvas versions.
I will prepare a patch to improve the error handling in this case.
Regards.

2 Likes
5

Ah,

Are there anything I can help out to troubleshoot?
In that setup I used the release tarballs except ospd and ospd-openvas.

I didn’t know what to cherrypick from the ospd-openvas code to get it working.

–
Regards Falk

6

When you use different versions (mix modules of GVM-10, GVM-11 and master branch) it is possible to get this error. After the scanner loaded the plugins, ospd-openvas tries to connect to redis to get the plugins from the nvticache. If the nvticache has a wrong name (mixing modules version) or it just does not exist, it will get this error.
You can try with

$ redis-cli -s <path-to-redis-socket> FLUSHALL

Maybe this fixes the nvticache name issue. And be sure you have ospd-2.0 and ospd-openvas-1.0, and not the master version.

$ ospd-openvas --version
OSP Server for openvas: 1.0.0 
OSP: 1.2
OSPd: 2.0.0
1 Like
7

This PR the nvticache version will be tested. Ospd-openvas will not start if the version is not compatible.

2 Likes
8

We had this issue too, now fixed (without errors in debug though).
I found ospd was not fully active (after systemctl start ospd-openvas).
In my case, after running redis flush (gvm@OPENVAS:~$ redis-cli -s /run/redis-openvas/redis.sock FLUSHALL) like jjnicola suggested, I was able to see the socket file however process could not be used.

I saw loads of these when running an strace against the ospd process:
ioctl(5, FIONBIO, [1]) = 0
recvfrom(5, 0x2891940, 65536, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(5, FIONBIO, [1]) = 0
poll([{fd=5, events=POLLOUT}], 1, 60000) = 1 ([{fd=5, revents=POLLOUT}])
sendto(5, “*2\r\n$4\r\nKEYS\r\n$15\r\nnvticache11.0”…, 36, 0, NULL, 0) = 36
poll([{fd=5, events=POLLIN}], 1, 60000) = 1 ([{fd=5, revents=POLLIN}])
recvfrom(5, “*1\r\n$15\r\nnvticache11.0.1\r\n”, 65536, 0, NULL, NULL) = 26

I found the permissions were not correct on both the pid and sock file did the following to fix:
#chown gvm:gvm /opt/gvm/var/run/ospd.sock
#chown -R gvm:gvm /opt/gvm/var/run

All working now.

1 Like