Trusting self-signed SSL certificates in Cloudera Hue

Using self-signed certificates is an easy process. Its very useful in local test environments, something we often rely on at my work.

However, it has its cons when the width of the ecosystem weighs in (software written in languages other than Java, such as Python, C++, etc.), as the support and configuration for trusting self-signed certificates varies from platform to platform.

For Cloudera Hue, which runs on Python (and uses PyOpenSSL plus the Python Requests libraries), while running it over a self-signed certificate file is as easy as configuring it, making it trust other services that it talks to (such as HDFS or YARN) gets difficult when the mentioned services also utilise self-signed certificates. You may often run into the following error or similar forms, within Hue’s File Browser, and other parts, for example:

Processing exception: [Errno 1] _ssl.c:492: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed: Traceback (most recent call last):
  File "/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/db/transaction.py", line 371, in inner
    return func(*args, **kwargs)
  File "/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hue/apps/jobbrowser/src/jobbrowser/views.py", line 119, in jobs
    raise ex
RestException: [Errno 1] _ssl.c:492: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

While not something you’d want to do on a production grade cluster, you can still get Hue to work in such an environment by applying a small change in the Hue process environment. If you use Cloudera Manager, you could append the below into the CM -> Hue -> Configuration -> Hue Server Environment Advanced Configuration Snippet (Safety Valve) field, save and restart the Hue service:

REQUESTS_CA_BUNDLE=/opt/certificates/host-certs.pem

This utilises the Python Requests library feature of using a custom CA Certificate bundle instead of the default one (which PyOpenSSL appears to self-bundle, rather than reuse the system default certs). You only need to make sure that the pointed PEM file carries all the necessary host certificates (and chain certificates) to allow Hue to talk to every applicable service in the cluster.

Hat tip to my most excellent colleague, Chris Conner, for pointing me to this feature.