Recipe 14.2. Making an HTTPS Web Request

Problem

You want to connect to an HTTPS web site, one whose traffic is encrypted using SSL.

Solution

You need the OpenSSL extension to Ruby. You'll know if it's installed if you can require the net/https library without getting a LoadError.

	require 'net/https'                       # => true

You can't make HTTPS requests with the convenience methods described in Recipe 14.1, but you can use the Net::HTTP::Get and Net::HTTP::Post class described in Recipe 14.3. To make an HTTPS request, just instantiate a Net::HTTP object and set its use_ssl member to TRue.

In this example, I try to download a page from a web server that only accepts HTTPS connections. Instead of listening on port 80 like a normal web server, this server listens on port 443 and expects an encrypted request. I can only connect with a Net::HTTP instance that has the use_ssl flag set.

	require 'net/http'
	uri = URI.parse("https://www.donotcall.gov/")

	request = Net::HTTP.new(uri.host, uri.port)
	response = request.get("/")
	# Errno::ECONNRESET: Connection reset by peer

	require 'net/https'
	request.use_ssl = true
	request.verify_mode = OpenSSL::SSL::VERIFY_NONE
	response = request.get("/")
	# => #<Net::HTTPOK 200 OK readbody=true>
	response.body.size                              # => 6537

Discussion

The default Ruby installation for Windows includes the OpenSSL extension, but if you're on a Unix system, you might have to install it yourself. On Debian GNU/Linux, the package name is libopenssl-ruby[Ruby version]: for instance, libopenssl-ruby1.8. You might need to download the extension from the Ruby PKI homepage (see below), and compile and install it with Make.

Setting verify_mode to OpenSSL:SSL::VERIFY_NONE suppresses some warnings, but the warnings are kind of serious: they mean that OpenSSL won't verify the server's certificate or proof of identity. Your conversation with the server will be confidential, but you won't be able to definitively authenticate the server: it might be an imposter.

You can have OpenSSL verify server certificates if you keep a few trusted certificates on your computer. You don't need a certificate for every server you might possibly access. You just need certificates for a few "certificate authorities:" the organizations that actually sign most other certificates. Since web browsers need these certificates too, you probably already have a bunch of them installed, although maybe not in a format that Ruby can use (if you don't have them, see below).

On Debian GNU/Linux, the ca-certificates package installs a set of trusted server certificates into the directory /etc/ssl/certs. I can set my request object's ca_path to that directory, and set its verify_mode to OpenSSL::SSL::VERIFY_PEER. Now OpenSSL can verify that I'm actually talking to the web server at donotcall.gov, and not an imposter.

	request = Net::HTTP.new(uri.host, uri.port)
	request.use_ssl = true
	request.ca_path = "/etc/ssl/certs/"
	request.verify_mode = OpenSSL::SSL::VERIFY_PEER
	response = request.get("/")
	# => #<Net:: 
HTTPOK 200 OK readbody=true>

The SSL certificate for www.donotcall.gov (http://www.donotcall.gov) happens to be signed by Network Solutions. I already have Network Solutions' certificate installed on my computer, so I can verify the signature. If I trust Network Solutions, I can trust donotcall.gov.

See Also

• Recipe 14.1, "Grabbing the Contents of a Web Page"

• HTTPS is just one more thing a robust web client needs to support; Recipe 14.20, "A Real-World HTTP Client," shows how to integrate it into a general framework

• The Ruby OpenSSL project homepage (http://www.nongnu.org/rubypki/)

• The (unofficial) Mozilla Certificate FAQ provides a good introduction to SSL certificates (http://www.hecker.org/mozilla/ca-certificate-faq/background-info)

• If you don't have any certs on your system or they're not in a format you can give to Ruby, you can download a bundle of all the certs recognized by the Mozilla web browser; instead of setting ca_path to a directory, you'll set ca_file to the location of the file you download (http://curl.haxx.se/docs/caextract.html)

• You can create your own server certificates with the QuickCert program; your certificates won't be recognized by any certificate authority, but if you control the clients as well as the server, you can manually install the server certificate on every client (http://segment7.net/projects/ruby/QuickCert/)