This was discussed on the chat yesterday.  Errors in negotiation can be received
by [tls::handshake], but a bgerror is still produced.  This short example
demonstrates:

package require tls

tls::init -cadir -require true -ssl2 0 -ssl3 0 -tls1 0 -tls1.1 0

proc bgerror {err} {
    puts "bgerror: $err"
}

proc do_connect {} {
    set chan [tls::socket 9999]

    # ensure handshake completes
    try {
        while {![tls::handshake $chan]} {puts handshake}
    } on error {err opts} {
        puts "handshake error: $err"
        return false
    }

    return true
}

do_connect
puts "synchronous code done"
update

Output:

handshake error: handshake failed: certificate verify failed
synchronous code done
bgerror: SSL channel "sock556449741e30": error: certificate verify failed

Note that without the call to tls::handshake, the same error will be produced
on first attempt to read or write the socket.

There are calls to Tcl_BackgroundError() in both Tls_Error() and
VerifyCallback().  The Tls_Error() invocations all come through stacked channel
machinery, so it *may* be possible to instead pass the error back to Tcl.  Note
the twapi ticket says this isn't an option there, but refers to reflected
channels.  I'm not sure if transchans will propagate errors from callbacks
VerifyCallback() is called by openssl so probably not so simple.

If extra plumbing is needed, one suggestion was to use a hash table keyed by
the channel ID to mark the presence of a synchronous operation that can return
the error.  This may need mutex protection.

Another question when looking at the code:  it seems like tcltls requires/assumes
that the tls channel is the top of a channel stack.  I'm not sure this is
correct, since it might make sense to push another transform on top of tls.  It
may be worth investigating this.