Recently, I've been doing some work involving cryptography, and encountered the standard .NET CryptographicException: 'Padding is invalid and cannot be removed.' Searching on StackOverflow produces 57 questions concerning this exception; it's a very common problem encountered. So I decided to have a closer look.

To test this, I created a simple project that decrypts and encrypts a byte array:

// create some random data
byte[] data = new byte[100];
new Random().NextBytes(data);

// use the Rijndael symmetric algorithm
RijndaelManaged rij = new RijndaelManaged();
byte[] encrypted;

// encrypt the data using a CryptoStream
using (var encryptor = rij.CreateEncryptor())
using (MemoryStream encryptedStream = new MemoryStream())
using (CryptoStream crypto = new CryptoStream(
    encryptedStream, encryptor, CryptoStreamMode.Write))
{
    crypto.Write(data, 0, data.Length);
    encrypted = encryptedStream.ToArray();
}

byte[] decrypted;

// and decrypt it again
using (var decryptor = rij.CreateDecryptor())
using (CryptoStream crypto = new CryptoStream(
    new MemoryStream(encrypted), decryptor, CryptoStreamMode.Read))
{
    byte[] decrypted = new byte[data.Length];
    crypto.Read(decrypted, 0, decrypted.Length);
}

Sure enough, I got exactly the same CryptographicException when trying to decrypt the data even in this simple example. Well, I'm obviously missing something, if I can't even get this single method right! What does the exception message actually mean? What am I missing?

Well, after playing around a bit, I discovered the problem was fixed by changing the encryption step to this:

// encrypt the data using a CryptoStream
using (var encryptor = rij.CreateEncryptor())
using (MemoryStream encryptedStream = new MemoryStream())
{
    using (CryptoStream crypto = new CryptoStream(
        encryptedStream, encryptor, CryptoStreamMode.Write))
    {
        crypto.Write(data, 0, data.Length);
    }
    encrypted = encryptedStream.ToArray();
}

Aaaah, so that's what the problem was. The CryptoStream wasn't flushing all it's data to the MemoryStream before it was being read, and closing the stream causes it to flush everything to the backing stream. But why does this cause an error in padding?

Cryptographic padding

All symmetric encryption algorithms (of which Rijndael is one) operates on fixed block sizes. For Rijndael, the default block size is 16 bytes. This means the input needs to be a multiple of 16 bytes long. If it isn't, then the input is padded to 16 bytes using one of the padding modes. This is only done to the final block of data to be encrypted.

CryptoStream has a special method to flush this final block of data - FlushFinalBlock. Calling Stream.Flush() does not flush the final block, as you might expect. Only by closing the stream or explicitly calling FlushFinalBlock is the final block, with any padding, encrypted and written to the backing stream. Without this call, the encrypted data is 16 bytes shorter than it should be.

If this final block wasn't written, then the decryption gets to the final 16 bytes of the encrypted data and tries to decrypt it as the final block with padding. The end bytes don't match the padding scheme it's been told to use, therefore it throws an exception stating what is wrong - what the decryptor expects to be padding actually isn't, and so can't be removed from the stream.

So, as well as closing the stream before reading the result, an alternative fix to my encryption code is the following:

// encrypt the data using a CryptoStream
using (var encryptor = rij.CreateEncryptor())
using (MemoryStream encryptedStream = new MemoryStream())
using (CryptoStream crypto = new CryptoStream(
    encryptedStream, encryptor, CryptoStreamMode.Write))
{
    crypto.Write(data, 0, data.Length);

    // explicitly flush the final block of data
    crypto.FlushFinalBlock();

    encrypted = encryptedStream.ToArray();
}

Conclusion

So, if your padding is invalid, make sure that you close or call FlushFinalBlock on any CryptoStream performing encryption before you access the encrypted data. Flush isn't enough. Only then will the final block be present in the encrypted data, allowing it to be decrypted successfully.