I guess that PKCS#8 should be supported from this post. In the python-gvm docs for the create_credential function, PKCS#1 and PKCS#8 are not mentioned, but PKC#12 is (a typo in the docs, should be PKCS#12).
It also mentions PEM is supported and PKCS#8 formatted keys can be PEM encoded - perhaps you can try the PKCS#8 in PEM format (base64 encoding).
Also, perhaps your can confirm or clarify the details of what you are trying to achieve. It seems you are trying to use GMP/XML to create a SSH key credential in GVM and then configure that credential for an authenticated scan. You can create the credential, but when using it for the authenticated scan you get an Auth-SSH-Failure error in the results of the scan report.
Is that correct?