A walk-through of an SSL handshake

In my last post, I walked through a complete TCP handshake which initiates, among other things, browser/webserver interaction. Although TCP, and the internet itself, had been around quite a while before HTTP was created, it was HTTP and the world-wide-web that made the internet a household concept in the mid 90's. As HTTP was originally specified, though, the TCP handshake should be immediately followed by an HTTP request - something of the form GET /index.html HTTP/1.0 or similar. This meant that in its first incarnation, all of the HTTP data would be visible to each hop between the client and the server. Netscape revolutionized the world in 1995 by commercializing the web browser, and one of the many innovations associated with that commercialization was the introduction of a new "layer" between TCP and HTTP which they called SSL. Before a single byte of HTTP could be transmitted, a secure cryptographic context would be negotiated between the browser and the server, and all HTTP transmission would be performed under that context. In this post, I pick up where I left off last time, walking through the beginning of an SSL handshake as it takes place after the TCP handshake but before any HTTP protocol data can be sent in an HTTPS connection. Before continuing, you may want to review my high-level overview of the TLS handshake if you're not familiar with the theory behind the TLS handshake. I'll focus in this post on the client and server hello messages that begin any TLS negotiation.

A note on terminology: SSL has undergone 5 (soon to be six) revisions and one name change to the awkward "TLS" name (which, for all practical purposes, nobody actually uses) and has been greatly expanded beyond its original purpose as a secure layer between HTTP and TCP, but the fundamentals remain the same: before any sensitive application is transmitted, SSL is responsible for performing a set of secure key exchanges and ensuring that data is transmitted both encrypted and signed. To be technically precise, I'll use the term TLS rather than the more common SSL in the description below, since what I'm describing is not actually SSL, but TLS — specifically, TLS 1.2, the latest version.

In figure 1, below, I've captured below the initial handshake between my Chrome browser and Wikipedia, which is configured to use up-to-date TLS configuration settings. I'll assume you're familiar with the TCP handshake (if not, review my previous post), and I'll jump right into an examination of the TLS handshake. Remember, all of this takes place immediately after the TCP handshake completes, and before the HTTP request for the index page is transmitted.

The first packet exchanged in any version of any SSL/TLS handshake is the client hello packet which signifies the client's wish to establish a secure context. I've omitted the IP and TCP prologue below and jumped right to the payload which is a TLS handshake message.

16:41:51.180853 IP 10.32.2.6.55956 > text-lb.ulsfo.wikimedia.org.https: Flags [P.], seq 1:206, 
                    ack 1, win 4117, options [nop,nop,TS val 570453704 ecr 420531432], length 205
	...
	0x0040:       1603 0100 c801 0000 c403 03ec 12dd
	0x0050:  1764 a439 fd7e 8c85 46b8 4d1e a06e b3d7
	0x0060:  a051 f03c b817 470d 4c54 c5df 7200 001c
	0x0070:  eaea c02b c02f c02c c030 cca9 cca8 c013
	0x0080:  c014 009c 009d 002f 0035 000a 0100 007f
	0x0090:  dada 0000 ff01 0001 0000 0000 1600 1400
	0x00a0:  0011 7777 772e 7769 6b69 7065 6469 612e
	0x00b0:  6f72 6700 1700 0000 2300 0000 0d00 1400
	0x00c0:  1204 0308 0404 0105 0308 0505 0108 0606
	0x00d0:  0102 0100 0500 0501 0000 0000 0012 0000
	0x00e0:  0010 000e 000c 0268 3208 6874 7470 2f31
	0x00f0:  2e31 7550 0000 000b 0002 0100 000a 000a
	0x0100:  0008 1a1a 001d 0017 0018 1a1a 0001 00

Figure 1: Client Hello packet

The apparently meaningless blob of bytes in figure 1 contains a complete TLS client hello message, which indicates to the server that the client wants to begin a TLS handshake. If the server responds with anything other than a legitimately formed TLS server hello message, the connection will immediately be aborted (and, in most cases, the browser will display an inexplicable error message). In figure 2, I've reformatted the client hello in pieces which I'll examine in more detail below.

  0x0042: 16        Handshake message type
  0x0043: 0301      Message version
  0x0045: 00c8      content length
    0x0047: 01      client hello message
    0x0048: 0000c4  client hello length
    0x004b: 0303    client hello protocol version
    0x004d: ec12dd1764a439fd7e8c8546b84d1ea06eb3d7a051f03cb817470d4c54c5df72  random value
    0x006d: 00      session ID length (would be followed by session ID if non-zero)
    0x006e: 001c    Cipher suites length
    0x0070: eaea    GREASE compatibility check
    0x0072: c02b    ECDHE/ECDSA/AES 128/GCM/SHA2  (RFC 5289)
    0x0074: c02f    ECDHE/RSA/AES 128/GCM/SHA2  (RFC 5289)
    0x0076: c02c    ECDHE/ECDSA/AES 128/GCM/SHA3  (RFC 5289)
    0x0078: c030    ECDHE/RSA/AES 256/GCM/SHA3  (RFC 5289)
    0x007a: cca9    ECDHE/ECDSA/CHACHA 20/POLY 1305/SHA2  (RFC 7905)
    0x007c: cca8    ECDHE/RSA/CHACHA 20/POLY 1305/SHA2 (RFC 7905)
    0x007e: c013    ECDHE/RSA/AES 128/CBC/SHA  (RFC 4492)
    0x0080: c014    ECDHE/RSA/AES 256/CBC/SHA (RFC 4492)
    0x0082: 009c    RSA/RSA/AES 128/GCM/SHA2 (RFC 5288)
    0x0084: 009d    RSA/RSA/AES 256/GCM/SHA2 (RFC 5288)
    0x0086: 002f    RSA/RSA/AES 128/CBC/SHA  (RFC 5246)
    0x0088: 0035    RSA/RSA/AES 256/CBC/SHA  (RFC 5246)
    0x008a: 000a    RSA/RSA/3DES/CBC/SHA  (RFC 5246)
    0x0090: 01      number of compression methods to follow
    0x0091: 00      the compression method "no compression"
    0x0092: 007f    length of extensions
    0x0090: dada 0000     GREASE compatibility check
    0x0094: ff01 0001 00  renegotiation info
    0x0099: 0000 0016 0014 0000117777772e77696b6970656469612e6f7267  SNI
    0x00b3: 0017 0000  Extended master secret
    0x00b7: 0023 0000  Session ticket TLS
    0x00bb: 000d 0014 0012 040308040401050308050501080606010201  Signature algorithms
    0x00d3: 0005 0005 0100000000   Status request OCSP
    0x00dc: 0012 0000    signed certificate timestamp
    0x00e0: 0010 000e 000c 02683208687474702f312e31   ALPN: HTTP/1.1
    0x00f2: 7550 0000  channel ID  
    0x00f6: 000b 0002 0100   Elliptic curve point formats uncompressed
    0x00fc: 000a 000a 0008 1a1a001d00170018   Elliptic curves
    0x010a: 1a1a 0001 00  GREASE compatibility check

Figure 2: "unrolled" TLS client hello handshake message

Below, I'll examine each line of the captured handshake message show in figure 1 and describe what each piece is used for.

  0x0040:       16 0301 00c8 01 0000c4 0303
                  |    |    |  |      |
          TLS Handshake len     length version
                   version   hello

TLS is, like both IP and TCP, a "wrapper" protocol that serves to encapsulate actual data that's of interest to the client and the server. This makes sense — TLS is responsible for encrypting actual protocol data. This is taken one step farther by the actual protocol; the handshake itself is wrapped inside the TLS record protocol. The first five bytes above are the TLS record protocol header: 16 0301 00c8. 0x16 is the marker identifying what follows - in this case, a handshake message. The TLS record protocol wraps four types of "sub- messages": handshake, alert, change cipher spec, and application data. The goal of any TLS handshake is to get to where application data (sufficiently encrypted) can be exchanged. The next two bytes 0301 represent the version of the TLS record layer protocol — in this case, 3.1.

Version numbers in TLS are a bit of a mess. The first version of SSL was version 2.0, identified as 0200, naturally. This was followed by version 3.0: 0300. So far, so good. However, the next version of SSL wasn't called SSL at all - it was renamed by the IETF (for no particular reason) to TLS, and given a version number of 1.0. However, for backwards compatibility, TLS version 1.0 is identified as 0301: SSL v3.1. This means that TLS 1.1 was actually identified as 0302, and TLS 1.2 as 0303. Notice above that the version is 0301, suggesting that Chrome is actually advertising TLS 1.0. In fact, that's not quite right - it's advertising version 1.0 of the record layer protocol but as you'll see, the handshake itself is version 1.2. This is necessary for backwards compatibility with very old servers.

The final two bytes of the record protocol are 0x00c8, which is decimal 200: the length of the remainder of the TLS segment. So now the receiver (the server) knows that what follows is 200 bytes that should be interpreted as an TLS handshake message. If you count up the bytes in the packet above, you'll see that it starts at 0x0046 (66) and ends at 0x10e (270): the rest of the packet is the payload of the wrapped TLS handshake packet. The first byte of the subsequent handshake message, then, is which type of handshake message. That's right - there are four types of TLS packets, but at least 10 types of handshake messages! ("At least 10?" you may be asking: stay tuned). This is where the client has first identified that what follows is a client hello message 0x01. This is itself followed by three bytes of length: 0000c4. It may seem strange (and I suppose it is, a little) that the wrapper protocol declares two bytes worth of length whereas the wrapped protocol declared three — you might be inclined to think that the high- order byte in this case would always be zero. Although this is, for all practical purposes, the case, it doesn't necessarily have to be. If, in the future, a very long client hello message were needed, it could be split across multiple TLS record layer messages — after all, the actual application payload will always span multiple TLS record layer messages.

Unsurprisingly, though, the length indicated here is 0xc4: 196 bytes, four less than the 200 declared by the record layer protocol (accounting for the four bytes already used up to declare the handshake type and the length). This is followed by another version declaration 0303 — TLS version 1.2. Although the record layer protocol can remain at version 1.0, the handshake must be recognized as 1.2, since the handshake itself will take advantage of TLS 1.2-specific semantics (it's worth noting that this actual client hello doesn't contain anything TLS 1.2 specific - it's valid and would be parsed correctly, other than the version number, by a TLS 1.0- only server. However, subsequent parts of the handshake do require TLS 1.2 semantics, and this version identifies the entire handshake).

	0x0040:                                  ec 12dd
	                                        | random
	0x0050:  1764 a439 fd7e 8c85 46b8 4d1e a06e b3d7
	0x0060:  a051 f03c b817 470d 4c54 c5df 72 00 
	                                         | session ID

After the type, length and version comes the actual client hello parameterization, starting with a 32-byte random value which will be used for the remainder of the handshake to establish entropy and guard against replay attacks. Note that this random value is not secure against eavesdroppers - the point of the TLS handshake itself is to establish a cryptographic context which, of course, hasn't been established yet, so this random value is sent "in the clear". The random value is followed by a session ID — if the client is trying to re-establish a previously negotiated TLS session, this will be populated. In this case, it isn't, so the session ID is declared as empty by providing the single byte '0': the length of the session ID which follows.

	0x0060:                                     001c
	                                           |len|
	0x0070:  eaea c02b c02f c02c c030 cca9 cca8 c013
	        |  cipher suites
	0x0080:  c014 009c 009d 002f 0035 000a 0100
	                                      |    |
                                         compression

Probably the most important piece of information that the client is responsible for transmitting during a client hello message is the list of supported cipher suites: specifically, which key-negotiation, encryption and MAC algorithms the client understands. The next two bytes are 001c, indicating that there are 28 bytes of cipher suites available for use. Each is two bytes long (so there are 14 cipher suites available). You might expect that the cipher suites would encode a key exchange, followed by an encryption algorithm, followed by a MAC algorithm but this isn't the case: TLS assigns a unique code to each possible triple of key exchange/encryption/MAC that can be supported. The definitions for these codes are actually scattered all over the place — although RFC 5246 which specifies TLS 1.2 defines some of them, other are defined by supplemental RFCs that were introduced after the adoption of TLS 1.2. RFC 5246 doesn't even define any elliptic-curve cipher suites! Interestingly, the first value in the list, eaea isn't a cipher suite at all, but a failsafe check that is specific to Chrome. Chrome deliberately advertises an invalid cipher suite in the first place to verify that the server ignores it correctly; if it doesn't, then Chrome has the ability to report that the server is not performing according to the specification. This is called Generate Random Extensions and Sustain Extensibility or GREASE as a sort of stretched-thin acronym.

The TLS specification requires that the client follow the list of supported cipher suites with the list of supported compression methods available. When SSL was first specified, the designers expected it to take responsibility both for encrypting as well as for compression. This capability has been virtually unused and unimplemented (in fact, given the recent CRIME and BREACH TLS vulnerabilities, has been shown to be dangerous), but must be declared for a client hello message to be correct. Therefore, the next two bytes are 01 and 00: 1 byte of compression mechanisms, and one compression mechanism: 0, meaning "no compression".

According to RFC 2246, this is the end of the client hello message, but the designers of TLS knew that no protocol is set in stone; the next two bytes indicate how many of the subsequent bytes of the message are "extensions". Extensions, in turn, are type/length/data triples (type and length being mandatory, data being optional) whose function is defined elsewhere and which may or may not be understood by the receiver. I'll walk through each of the extensions present in this example handshake below — for the most part, as you'll see, client hello extensions tend to consist of the browser advertising additional capabilities which the server may or may not take advantage of later on. This does not illustrate all available TLS client hello extensions; there are actually dozens defined in various RFCs.

  0x0090:  dada 0000 ff01 0001 00
           grease   |renegotiate |

The first extension is another GREASE failsafe check; this is followed by the TLS renegotiation extension defined in RFC 5746. This extension is effectively always present and tells the server that the client is able to defend against the renegotiation flaw in the original TLS specification which was identified by Marsh Ray in 2009. If this extension is not reflected by the server, then the client will not attempt to renegotiate a TLS connection, since it can't be done safely without this capability.

  0x0090:                         0000 0016 00 1400
                                 |        SNI
  0x00a0:  0011 7777 772e 7769 6b69 7065 6469 612e
  0x00b0:  6f72 67

The most interesting of the extensions is the third, extension 0000: server name. The need for this came up in the early days of the web. It's less common now (but not unheard of), but a lot of web servers hosted multiple web domains under a single IP address. This presented a challenge for TLS: which domain should it negotiate a connection for? The server-name extension provides a way for the client to indicate which actual server it's trying to connect to. If you look at the data included in the extension, you'll see that it includes the ASCII-encoded name of the website I'm connecting to: www.wikipedia.org.

  0x00b0:          0017 0000 0023 0000 000d 0014 00
                  | extended|session  | signature algorithms
                    master   ticket
                    secret
  0x00c0:  1204 0308 0404 0105 0308 0505 0108 0606
  0x00d0:  0102 01

Similar to the renegotiation info extension, the fourth extension, "extended master secret" guards against the "triple handshake" man-in-the-middle vulnerability that was identified in 2014, and will always be presented. The session ticket algorithm advertises that this client is capable of handling a server-side session ticket which simplifies session resumption; as you'll see below, the server doesn't support it, so this extension is ignored. This extension is followed by a list of signature algorithms: in the first version of SSL, the signature algorithms were hardcoded into the algorithm.

  0x00d0:          0005 0005 0100000000 0012 0000
                  | OCSP request       | signed certificate timestamp

The next extension is slightly different — rather than advertising a capability of the client, it's requesting (if possible) one from the server: namely, OSCP stapling. The server is in almost all cases required to present a certificate which includes a public key that will be used either to perform a key negotiation or at minimum to sign one. That certificate must in turn be signed by another key contained in another certificate which is either already trusted by the client or signed by yet another key contained in yet another certificate which is trusted ad infinitum. This series of certificates that vouch for one another is called a certificate chain. In theory, the client is responsible for keeping track of whether or not any of these certificates has been found to be compromised (Online Certificate Status Protocol), but this extension requests that the server take on this responsibility. As you'll see below, the server was in this case willing to do so.

This is followed by the signed certificate timestamp extension which is actually an extension to the previous extension - it requests that the OCSP status request returned by the server include log data that can be used to verify that the certificate has been sufficiently audited.

  0x00e0:  0010 000e 000c 0268 3208 6874 7470 2f31
          | ALPN
  0x00f0:  2e31 

The next extension is Application Layer Protocol Negotiation: in this case, the client (my browser) is just saying that it will speak HTTP/2 if available, but that it will fall back to "plain old" HTTP/1.1 connection otherwise.

  0x00f0:       7550 0000 000b 0002 0100 000a 000a
               |channelid|EC points     |elliptic curves
  0x0100:  0008 1a1a 001d 0017 0018 1a1a 0001 00
                                   | grease

The next extension advertises the client's ability to support the channel ID TLS extension which is designed to cryptographically bind an authentication token (such as a cookie) to a specific "channel" (such as a TLS connection). It's an interesting, and very useful, capability, but not widely deployed as of yet. There are also a couple of extensions toward the end that describe how the server should use elliptic curve cryptography, should it choose to do so (which, as we'll see in a minute, it did).

Now, the server, if it accepts the connection at all, is responsible for selecting a cipher suite and providing all of the data that it needs to provide in order for the client to complete a secure key exchange under the selected cipher suite. With the TCP and IP headers omitted as before, the server hello message is shown below.

16:41:51.224463 IP text-lb.ulsfo.wikimedia.org.https > 10.32.2.6.55956: Flags [.], seq 1:1449, 
                  ack 206, win 59, options [nop,nop,TS val 420531442 ecr 570453704], length 1448
	0x0040:       16 03 03 00 6a 02 000066 0303 0863
           | ver | len |  | len  |ver |(random starts)
 	     type=handshake      server hello

This begins just as the client hello packet did: advertising a TLS handshake packet (0x16), but now agreeing to version 3.3, followed by 006a = 106 bytes of content length. The first of these content bytes is 0x02, indicating that this is a server hello, rather than a client hello. This is followed by a length 0x000066 (102) and version 3.3. As part of the handshake's defense against replay attacks, the server must select its own 32 bytes of random data.

	0x0050:  87e9 d4f7 26dd d2ce 267b 06c2 e853 cfdf
	0x0060:  22f9 ed5f 9f1f 223d 7793 94c4 d820 b07d
	             random                     | session ID

This is followed by a session ID. Remember that the session ID was empty in the client hello: if the client hello includes a session ID, that indicates that the client is trying to re-establish a previously negotiated TLS session. However, if the client doesn't provide one, the server will create a new one and return its value in the hello message, as it does here.

	0x0070:  4f80 6a20 4eab b616 d330 5fa2 6969 e5c7
	0x0080:  f97a 01a9 6ff8 d27d 320b 417a 1c71 cca9
	                                           | cipher suite

After the session ID is the selected cipher suite. Here, the server has selected cipher suite ECDHE/ECDSA/ChaCha20/Poly-1305/SHA2 (defined in RFC 7905). This means that ECDHE (elliptic-curve diffie-hellman) will be used to negotiate a symmetric encryption key for the ChaCha20 encryption algorithm and ECDSA (elliptic-curve digital signature algorithm) will be used to validate the handshake. The SHA2 algorithm will be used as the secure hashing mechanism for the HMAC that signs each exchanged packet. Finally, a compression method must be selected from among those presented by the client. Unsurprisingly, the server here selected the one method that the client provided: 0 (no compression).

  0x0090:  00 00 1e ff01 0001 00 000b 0004 0300 0102
             | ext |renegotiate | ec curves point   |
             | len |            |    format         |
  0x00a0:  0005 0000 0017 0000 0010 0005 0003 0268
          | status  | extended|  ALPN             |
                      master
                      secret
  0x00b0:  32

Like the client hello, the server hello allows for extensions, and practically speaking, all client hellos and server hellos do include extensions. One particularly important server hello extension included here are ec_points_format 000b: since elliptic-curve cryptography was selected by the server for both handshake and digital signature, both sides must agree on a format for the exchange of the elliptic-curve parameters. For the most part, the server is just echoing back the client extensions - in essence, letting the client know that the extensions were understood, allowing it to either take advantage of extended functionality (such as in the case of the status request extension), disabling certain functions (for instance, if the server didn't recognize the renegotiation info extension) or potentially aborting the connection altogether, which might be the case if the extended master secret extension weren't recognized. It's worth noting in this case that the server did not echo the channel ID extension.

This is the end of the handshake message... but the packet continues! Since TCP is a stream-oriented protocol, TLS takes advantage of this and concatenates multiple handshake messages back-to-back. The very next byte begins a new TLS handshake message of length 0bd3: 3027. Digging further into the message, you see that it's of type 0x0b: certificate. Strictly speaking, the server doesn't have to present a certificate in order to perform a Diffie-Hellman key exchange (either elliptic curve or discrete logarithm). However, in order to guard against man-in-the-middle attacks, that Diffie-Hellman key exchange must also be signed, and in order for that signature to be verified, a public key must be exchanged and verified. What follows, then, is 3020 bytes os ASN.1 DER-formatted X.509 certificate. In this case, the certificate is issued to *.wikipedia.org, signed by GlobalSign, and contains a 256-bit elliptic curve public key. The certificate itself is a tad long, even as certificates go, because it's a multi-purpose (VERY multi-purpose) certificate, identifying many alternative names. Also, since the certificate was signed by an intermediate certificate, the full certificate chain is included.

The certificate itself, which begins at 0x00c0, is an ASN.1-encoded DER representation of an X.509 certificate. I won't cover the details here since I talked about this extensively in a previous post.

	0x00b0:    16 0303 0bd3 0b00 0bcf 000b cc00 0760
	0x00c0:  3082 075c 3082 0644 a003 0201 0202 0c10
	0x00d0:  e6fc 62b7 418a d500 5e45 b630 0d06 092a
	0x00e0:  8648 86f7 0d01 010b 0500 3066 310b 3009
....
	0x0110:  032c f316 375d 67f1 a439 7949 a3c0 5dcc
	0x0120:  55f9 2180 0ffb cee2 296a 5850 e9a6 d7eb
	0x0130:  1c32 36b5 62a7 c1fa e6

Recall that my browser did ask for an OCSP status request; this is the next message in the handshake, message type 22. Remember I said that TLS defines "at least" 10 handshake messages? TLS extensions can define new handshake types in addition to new capabilities, and this one does. What follows is ASN.1 DER-encoded (just like a certificate) and signed certificate status; fortunately for this TLS connection, this one indicates that the status is "good".

	0x0130:                        16 0303 064f 1600
	0x0140:  064b 0100 0647 3082 0643 0a01 00a0 8206
	0x0150:  3c30 8206 3806 092b 0601 0505 0730 0101
	0x0160:  0482 0629 3082 0625 3081 bfa2 1604 149c
	0x0170:  4d00 9900 0e8b b001 8175 a1ba f0d0 25d7
	0x0180:  a01c 4718 0f32 3031 3730 3531 3531 3234
	0x0190:  3130 335a 306f 306d 3045 3009 0605 2b0e
...
	0x0160:  39b0 60f1 1717 c9fe 91f9 37e6 8d8f 505c
	0x0170:  49dd 8ca0 70a6 24d5 ebd7 a3e2 5ccc 78c8
	0x0180:  0893 1339 5760 1e08 c6bf b713 3985 cc16
	0x0190:  c35c 6a12 1680 700c 0748 4481 fe63 271d
	0x01a0:  db64 2756 dab8 eaa7 b3ed 4597 c8a2 4a58
	0x01b0:  4dcc bee7 f0a7 db0a bbf7 b7fd 4a5b 8770
	0x01c0:  ccbb 9ec9 7e6c 8f77 be70 610e 4b18 3868
	0x01d0:  111c 64ee adcc b350 9c3f a446 f3c4 373f
	0x01e0:  bfc0 fa4f 3f

The next step is the actual key exchange — in this case, since the server selected an ECDHE ciphersuite, this means that both sides must exchange points on an elliptic curve, as well as agreeing on an actual elliptic curve to use. This final handshake is complex and interesting, which will be the topic of my next post which will complete my byte-level walk-through of a TLS handshake.

Add a comment:

Completely off-topic or spam comments will be removed at the discretion of the moderator.

You may preserve formatting (e.g. a code sample) by indenting with four spaces preceding the formatted line(s)

Name: Name is required
Email (will not be displayed publicly):
Comment:
Comment is required
Waleed, 2018-01-10
Man, its like you came from heaven just for me.
TCP, SSL, HANDSHAKE, all these walkthroughs are things I'm actively looking for exactly the same as you have on the blog.
Thanks a lot man
Josh, 2018-01-11
Glad I could help! Make sure to read through the whole series if you can - there are a lot of idiosyncrasies in the overall process. Of course, TLS 1.3 is right around the corner and all of this is likely to change later this year. I'll do my best to keep up with it here.
HonkyCodeLoader, 2019-01-11
This was interesting and helpful, thank you. As is your book, which I'm still working on...
Josh, 2019-01-23
Glad you enjoyed it! One of these days I hope to get around to a second edition of Implementing SSL - it's overdue now, with TLS 1.3 having been finalized.
Mukesh Chaurasiya, 2019-05-01
This blog was really helpful, I could easily parse the complete packet of TLS Hanshake packets.

One thing i didn't get is the marker for handshake messgae (0x16). Where did it come from, I couldn't find it in RFC 5246
Josh, 2019-05-01
Glad you enjoyed it! You can find the specification for the handshake message in RFC 5246, section 6.2.1, but it shows the enumerations in decimal (22) rather than hex (0x16). I have them in hex here because that's how they appear in tcpdump.
Alex, 2020-04-13
Hi!
Very useful - I've got an IoT system I'm developing for a project using AWS.
I've done some pcaps of the endpoint device publishing a message to AWS IoT, everything as expected with the handshake, however there are 3 TCP exchanges between the server and device in between the client hello, and the sever hello.
I initially thought it was certificates/some form of authentication from looking at the payloads, however cant find any supporting evidence.
Any idea what would be exchanged between the client and server hello?
Josh, 2020-04-13
What version of TLS? TLS 1.3 includes optional pre-shared key messages that go "in between" (or rather, are an extension of) the client hello. You might want to check out my 7/25 post that goes through a TLS 1.3 handshake. If you'd like, you can post the tcpdump output here and I can take a look - don't worry, there won't be anything sensitive in those first few packets.
Eugene O, 2020-04-21
Hi John!

Thank you for you article. It helped me a lot in my work. There is, however, one point that's not quite clear to me. I mean the hello server message dump. Namely, I can't understand why the length of Random does not equal to 32 bytes. As far as I can see, cipher suite at line 0x0080 equals to cca9. I can see that the previous 32 bytes is SessionID. But the random starts (0863b8) and the rest of random give 33 bytes in total. Could you please tell me where I'm wrong?
Josh, 2020-05-07
It's been a while since I wrote this, and I've lost the original sources - I can only conclude that the 0863b8 was a typo (everything else is lined up in 16 bit groups). I've corrected it; my apologies, I try to proofread these as best I can.
Naushad CK, 2021-10-11

This is a wonderful explanation. Great and much helpful to everyone.

I guess, in 3.3 there is some more details. I found this from the packet capture.

After the following line in client hello(01),
0x008a: 000a RSA/RSA/3DES/CBC/SHA (RFC 5246)


It shows as follows.

FF 1 0 1 0


Ee Min Chia, 2021-11-16
Hi Josh, I devoured your article with great interest, great thanks. I wonder if you mind giving some advice to a situation I am involved in - though sadly I cannot tell too much of the context.

Situation:
1. HTTPS simple GET endpoints, works fine with browsers
2. But if telco uses GGSN Header Enrichment to add a custom TLS extension to clienthello
3. then even the the HTTPS endpoints stop working (SSL_PROTOCOL_ERROR)
(tech says the server sent a reset cmd)
4. I figure that shouldn't be the case: the extension would at most be ignored, the https endpoint should not 'break' just because of the extra extension.

5. Then that must mean something not quite done right in the HeaderEnrichment, you agree?
6. But how to triage this client side? My gut feel is some checksum not updated even though extension is inserted...? Any gut feel from you?

Not much help from server side. but when trying golang as the https server then there is
i/ a spit out of message from golang tls library that TLS handshake error from <ipaddress>:
tls: client's Finished message is incorrect"

ii/ The enrichment at least worked partially right coz the server side actually saw the data in the extension (...just that something fail down the line in the handshake process: about the Finished)

(tried 3 different https server: simple NODEJS, GOLANG..)
Mukul Manikandan, 2022-02-12
Excellent explanations of each byte in the packet flow. I did not know about the GREASE check, and thats pretty neat! :) I'm a fan of your posts, keep writing these!
James E, 2022-09-20
In IETF RFC 8446 (TLS v1.3) ยง4.1.2 (ClientHello), they define

    struct {
        ProtocolVersion legacy_version = 0x0303;    /* TLS v1.2 */
        Random random;
        opaque legacy_session_id<0..32>;
        CipherSuite cipher_suites<2..2^16-2>;
        opaque legacy_compression_methods<1..2^8-1>;
        Extension extensions<8..2^16-1>;
    } ClientHello;


However, when trying to encode a single extension (supported_versions) with a single value (0x0304), I'm running up against the issue that the resulting encoded extension is just 7 bytes, not 8 as the spec has minimum:

    002b # extension type: supported_versions(43)
      0003 # extension data length: 3
        02 # vector, variable length: 2
          0304 # ProtocolVersion element: 0x0304

I'm having trouble getting a clean TLS 1.3 capture in WireShark, so do you have any advice on how this would look in a "real-world" ClientHello?

Thanks.
James E, 2022-09-21
(Specifically, I'm wondering about a hypothetical TLS 1.3 ClientHello that does NOT include TLS 1.2 in the supported_versions extension, and does not have any other extensions. I cannot see how that would or could reach the 8-byte minimum value for the "extensions" vector, but I don't see anything in the spec that would otherwise legitimately mandatorily inflate it up to 8 bytes.)
My Book

I'm the author of the book "Implementing SSL/TLS Using Cryptography and PKI". Like the title says, this is a from-the-ground-up examination of the SSL protocol that provides security, integrity and privacy to most application-level internet protocols, most notably HTTP. I include the source code to a complete working SSL implementation, including the most popular cryptographic algorithms (DES, 3DES, RC4, AES, RSA, DSA, Diffie-Hellman, HMAC, MD5, SHA-1, SHA-256, and ECC), and show how they all fit together to provide transport-layer security.

My Picture

Joshua Davies

Past Posts