After updating to Firefox 38 (ESR) I recently came across a sec_error_bad_der error when trying to access an internal server. I couldn't figure out what was the problem (and still can't even now), but now I found a public website with the same error: https://nolb.dhl.de/
Firefox 38 throws:
An error occurred during a connection to nolb.dhl.de. security library: improperly formatted DER-encoded message. (Error code: sec_error_bad_der)
IE11 and Chrome44 don't care.
After some digging I found this similar case and noticed that it applies here as well:
Here the DPDHL TLS CA I3 intermediate certificate provided by the server contains trailing spaces for some X509v3 Name Constraints entries, i.e.:
# openssl x509 -in /tmp/DPDHLTLSCAI3.cer -text -noout | egrep '\.de $'
It appears there was some major work around X.509 certificate parsing since Firefox 37, causing it to reject certificates with erroneous entries like this.
In both, dev and prod the server gets a full score for the certificate and nothing indicates that something is wrong. The handshake simulation for reference browser Firefox 37 / OS X shows success too when it would in fact fail:
I know this is an intermediate certificate and not the end certificate, but it would be helpful if such X509/asn1 sanity checks could be implemented in the SSL Server Test as well. Especially now that major browsers are more tight around checking proper certificate formats.