Preventing replay attack over UDP

[cyberfish]

I am currently working on a remote desktop program that uses a UDP connection for streaming video and audio only, and a TCP connection for everything else (including mouse input, etc).

The TCP connection is currently secured using AES-256-CBC (pre-shared key), with half the initialization vector generated by the server, and the other half generated by the client (to prevent replay attacks on either side). There is also SHA-256-HMAC encrypted with each message for authenticity and integrity.

1. Does anyone see any vulnerability with that scheme? Network security is not really my thing, but unfortunately is important for this project.

2. How do I secure the UDP channel (uni-directional from server to client)?

Since the UDP "connection" is always "established" after the TCP one, it can use the same IV and key as the TCP connection.

The problem is datagrams can be dropped or duplicated or come out of order, or corrupted, which breaks most block cipher modes.

A simple scheme would be to randomly generate an IV per datagram. That will make the same plaintext encrypt to different ciphertext, but doesn't prevent replay attacks, unless the client remembers all IVs that have ever been used (not practical).

One solution I can think of is to have the server request, say, 512 randomly-generated one time initialization vectors from the client, and the client keep track of which ones have been used, and not allow any IV to be used twice.

But that's pretty complicated. Is there an easier way?

[Yarin]

with half the initialization vector generated by the server, and the other half generated by the client (to prevent replay attacks on either side)

That sounds needlessly complicated, wouldn't just using a nonce during session establishment (post channel establishment) suffice?

[cyberfish]

Wouldn't that still allow a replay attack on the client (someone posing as the server)?

[phantomotap]

[Edit]Sorry about the duplicate; I couldn't delete it.[/Edit]

SHA-256-HMAC

>_<

Please, please put the "HMAC" identifier first: `HMAC-SHA-256'.

Wouldn't that still allow a replay attack on the client (someone posing as the server)?

I agree with Yarin.

The secret is using a negotiated "nonce" and something of a "OTP" for each packet; using something as simple as a "time-based one time pad" makes "replaying" non-trivial without knowledge of the "PSK"/generator.

Client (AES-256-CBC):SID(HMAC-SHA-256(PSK, LFSR(NONCE), (NOW-EPOCH)/SLICES)): "Hello, I am USERNAME! My session token is `NONCE'."
Server (AES-256-CBC):(HMAC-SHA-256(PSK, LFSR(NONCE), (NOW-EPOCH)/SLICES) == SID): "How may I help you USERNAME?"

If you would like to allow for more security at the protocol level, you may instead optionally require a unique "PSK" or "symmetric key" for each client.

You may also follow the complete requirements of the relevant "RFC", but that may be overkill for this project.

The problem is datagrams can be dropped [...] not allow any IV to be used twice.

I'd advise using a good cypher designed for streaming and seeking. (I hate to advertise on anyone's behalf, but I've had success with "Salsa20".) However, if you use the above strategy, your idea of simple encrypting each block with a fresh "IV" is valid. You don't then need to worry about replay--beyond "DOS"--on either side unless the "PSK" is compromised as the packet "HMAC" includes an "expiration date" of sorts which can't be adjusted without knowledge of the "PSK"; both sides discard packets with an invalid/"expired" "HMAC".

Soma

[cyberfish]

Ah ok. Lot's to read up on!

I always thought of nonce as almost like IV. Are they different? Googling doesn't seem to help too much.

I did think about using a timestamp based approach, but it feels to me like it's not really a "clean" solution, because of the window of vulnerability required to allow for network delays. And it's also difficult to get right, for example, if the user or the OS on one side decides to change the system clock.

My current scheme is like this -

Server: [first half of IV (IV1)]
Client: AES-256(IV1) + [second half of IV (IV2)]
Server: AES-256(IV2)

At this point, both party has verified that the other one has the PSK, and then server->client can use "IV1 + IV2" as the IV, and client->server can use "IV2 + IV1" as the IV.

It seems very simple to me. Is there something wrong with this? It feels like I'm just having the IV double as the nonce as described in your scheme?

If you would like to allow for more security at the protocol level, you may instead optionally require a unique "PSK" or "symmetric key" for each client.

That makes sense. Maybe I'll implement that for a future protocol version. It's not very important because 99% of the time, there will only be 1 user, since it's for remotely logging into one's PC. I'm planning to just have the user type a password, and use PBKDF2 to derive a key from that. Though that means if the server wants to store the password hashed, it will have to transmit the salt to the client in plain text, so the client can derive the key. That should technically be OK? (*1)

I'd advise using a good cypher designed for streaming and seeking. (I hate to advertise on anyone's behalf, but I've had success with "Salsa20".) However, if you use the above strategy, your idea of simple encrypting each block with a fresh "IV" is valid. You don't then need to worry about replay--beyond "DOS"--on either side unless the "PSK" is compromised as the packet "HMAC" includes an "expiration date" of sorts which can't be adjusted without knowledge of the "PSK"; both sides discard packets with an invalid/"expired" "HMAC".

Didn't hear about stream ciphers until now. Just read up on them. Pretty cool! It does seem like a better solution than a block cipher in this case.

*1: Basically, if the password is P, the server would store salt S, and PBKDF2(S, P, ...). And when client connects, the server would send S to the client.

[phantomotap]

I always thought of nonce as almost like IV. Are they different?

O_o

That would depend on the algorithm; an "IV" might be generated by combining a "nonce" and a counter for a stream cypher.

I did think about using a timestamp based approach, but it feels to me like it's not really a "clean" solution, because of the window of vulnerability required to allow for network delays. And it's also difficult to get right, for example, if the user or the OS on one side decides to change the system clock.

The time isn't secret so the system clock is not relevant. In the event they are indeed different, you may use "NTP" from a third-party.

[Edit]
Please disregard the original strategy listed here; a good "stream cypher" will transparently do the same job.
[/Edit]

Of course, as I said, using a cypher designed for streaming and seeking would be preferred.

Is there something wrong with this? It feels like I'm just having the IV double as the nonce as described in your scheme?

You are directly communicating the initialization vector.

Though that means if the server wants to store the password hashed, it will have to transmit the salt to the client in plain text, so the client can derive the key.

No. You are still missing a step. You negotiate a common, public "key" in advance--your server specific "PSK"--and use a derivative--not the mathematics one--as a base for communicating a new encrypted key for further encrypted communication.

Diffie–Hellman key exchange - Wikipedia, the free encyclopedia

[Edit]
I'm at 29 hours and a bit off; the "log" I posted had two significant "copy-pasta" errors.

I'll try again after some sleep.
[/Edit]

I've included a more elaborate "log" for the sake of example.

Soma

[cyberfish]

The time isn't secret so the system clock is not relevant. In the event they are indeed different, you may use "NTP" from a third-party.

What I meant is, for example, if the user/OS changes the system clock at some point in the middle of a session to 1 hours in the past, without careful programming, the other side may start rejecting every packet from that point on due to timestamp in the past. Not a security risk, but can functionally break the program (requiring a re-connection and re-synchronization every time the clock on one side changes). That's why I want to avoid it if possible. It's hard to get it right functionally in all situations.

I just read up on Diffie–Hellman. It sounds like a way to exchange a key with no prior knowledge.

But in this case I do have prior knowledge - both parties have the secret (password) already, and can derive a key from that. Why exchange another key then?

You are directly communicating the initialization vector.

Is there anything wrong with that? I thought IVs (for CBC at least) only need to be random and unpredictable, and does not need to be secret. In a sense, in CBC, isn't every ciphertext an IV for the next block?

[phantomotap]

That's why I want to avoid it if possible. It's hard to get it right functionally in all situations.

O_o

If you didn't want difficult you shouldn't have started trying to design your own protocol.

But in this case I do have prior knowledge - both parties have the secret (password) already, and can derive a key from that. Why exchange another key then?

You've said as much; you are talking about the "TCP" channel of communication; I am not. You are advocating a global shared key; I am not.

Is there anything wrong with that?

You are free to use a "CBC" with a globally shared key if you choose.

A strategy that is both more secure and simpler to implement correctly has been explained.

Soma

[cyberfish]

Ah OK I didn't realize we were talking about different things.

For the UDP connection, a stream cipher does make more sense, so I'll probably use that.

I see your concern with global shared keys. That is a decision I made early on to simplify things since this is a single user system, and not designed to be scalable. It's a "home solution" kind of thing where security is important, but simplicity is almost as important.

If I decided to not use a global shared key I would have just used TLS + DTLS, in which case I would just need to handle authentication myself.

I should probably look into TLS and DTLS again. Last time I evaluated it was quite a while ago, and I've done a lot of reading since then.

[Elysia]

Global keys are a really bad idea as as security is concerned.
You are exchanging a lot of data over the net, so eventually someone may just be able to break the key through various exploits. If you are using your global key, then you've just let the hacker into your system: the hacker can see every past and future messages exchanged.
On the other hand, if you derive a new session key, the hacker can only monitor the current session. Past sessions are still unavailable and since a new key is generated next session, the hacker will be unable to get into that either unless they decrypt that too.

You will find that modern protocols always generate a session key. There are various protocols available dealing with how to create and exchange this session key and can be extremely complicated themselves.

[phantomotap]

can be extremely convulsed themselves

O_o

I'm not being picky, but I really need that changed so I can make sense of it.

I can't for all my Pokémon figure out what you are actually saying.

Soma

[Elysia]

I'm not being picky, but I really need that changed so I can make sense of it.

I can't for all my Pokémon figure out what you are actually saying.

Avoiding complicated words that are impossible to spell in the future. Noted.
Post updated.

[cyberfish]

Global keys are a really bad idea as as security is concerned.
You are exchanging a lot of data over the net, so eventually someone may just be able to break the key through various exploits. If you are using your global key, then you've just let the hacker into your system: the hacker can see every past and future messages exchanged.
On the other hand, if you derive a new session key, the hacker can only monitor the current session. Past sessions are still unavailable and since a new key is generated next session, the hacker will be unable to get into that either unless they decrypt that too.

You will find that modern protocols always generate a session key. There are various protocols available dealing with how to create and exchange this session key and can be extremely complicated themselves.

That makes sense. I'll either implement that or just use TLS/DTLS, which seem to do pretty much the same thing.