Probably the best solution is to enable "fast-path" during compilation, which
lets OpenSSL handle that part of the process for TCP sockets.  This will
eventually be made the default, it seems to be much better.