We have 2 options:
A: Default to Serializable whitelist checks on both freeze and thaw
B: Default to Serializable whitelist checks only on thaw
Before this commit, Nippy was taking option A.
As of this commit, Nippy is taking option B.
Both are equally safe re: the risk of Remote Code Execution in #130:
- Freezing a malicious payload is *not* a security risk
- Thawing a frozen malicious payload *is* a security risk.
But option B has the benefit of not throwing exceptions by default
against a whitelist that has not [yet] been properly configured.
This is especially helpful for other libraries or applications that
may be using Nippy as an underlying dependency.
Behaviour under our two options against a whitelist that has not
[yet] been properly configured:
A: Throw exception on freeze
B: Freeze successfully, and thaw successully as
{:nippy/unthawable {:class-name <> :content <quarantined-ba> :cause :quarantined}}
I think this is probably less of a nuissance, and so a better default.
Fix a Remote Code Execution (RCE) vulnerability identified in an
excellent report by Timo Mihaljov (@solita-timo-mihaljov).
You are vulnerable iff both:
1. You are using Nippy to serialize and deserialize data from an
UNTRUSTED SOURCE.
2. You have a vulnerable ("gadget") class on your classpath.
Notably Clojure <= 1.8 includes such a class [1].
Many other libraries do too, some examples at [2].
To prevent this risk, a Serialization whitelist has been added.
Any classes not *explicitly* authorized by the whitelist to use
Serialization will NOT be permitted to.
The default whitelist is EMPTY, meaning this is a BREAKING
change iff you make use of Nippy's Serialization support. In
this case, you'll need to update the whitelist for your needs.
For more info see the `*serializable-whitelist*` docstring.
[1] https://clojure.atlassian.net/browse/CLJ-2204
[2] https://github.com/frohoff/ysoserial
Further info below provided by Timo:
------------------------------------
Deserialization vulnerabilities are exploited by constructing objects of classes
whose constructors perform some action that's useful to the attacker. A class like
this is called a gadget, and a collection of such classes that can be combined to
reach the attacker's goal is called a gadget chain.
There are three prerequisites for exploiting a deserialization vulnerability:
1) The attacker must be able to control the deserialized data, for example,
by gaining write access to the data store where trusted parties serialize
data or by exploiting some other vulnerability on the other end of a
communications channel.
2) The deserializer must construct objects of classes specified in the
serialized data. In other words, the attacker must have full control over
which classes get instantiated.
3) The classpath must contain gadgets that can be combined into a gadget chain.
The vulnerable code is in Nippy's function `read-serializable`, which calls the
`readObject` method of `ObjectInputStream`.
I have only tested the PoC with the latest stable version, 2.14.0, but looking at
Nippy's Git history, I believe all versions starting with the following commit
are vulnerable:
commit 9448d2b3ce
[Thu Oct 24 13:47:25 2013 +0700]
For a user to be affected, they must:
1) use Nippy to serialize untrusted input, and
2) have a gadget chain on their classpath.
I suspect (but haven't verified) that using Nippy's encryption feature prevents
exploitation in some cases, but if it's used to encrypt the communications between
two systems, one compromised endpoint could send encrypted but
attacker-controlled data to the other.
Ysoserial [4] contains a list of some Java libraries with known gadget chains.
If any of those libraries can be found on the user's classpath, they are known
to be vulnerable. (Ysoserial's list is not exhaustive, so even if a user doesn't
have these particular libraries on their classpath, they may still have some
other gadget chains loaded.)
Unfortunately Clojure versions before 1.9 contained a gadget chain in the
standard library [5][6], so all Nippy users running Clojure 1.8 or earlier
are vulnerable. (Note that users of later Clojure versions may or may not
be vulnerable, depending on whether they have gadget chains from other
libraries on their classpath.)
[4] https://github.com/frohoff/ysoserial
[5] https://groups.google.com/forum/#!msg/clojure/WaL3hHzsevI/7zHU-L7LBQAJ
[6] https://clojure.atlassian.net/browse/CLJ-2204
Why?
- AES-GCM is faster and can be more secure, Ref. https://goo.gl/Dsc9mL, etc.
- AES-GCM is an authenticated[1] encryption mechanism, providing
automatic integrity checks. This is relevant to [#101].
What's the issue with #101?
- We compress then encrypt on freeze ; Reverse would make compression useless
- So we decrypt then decompress on thaw
Attempting CBC decryption with the wrong password will often but not
*always* throw. Meaning it's possible for decompression could be
attempted with a junk ba. And this can cause some decompressors to
fail in a destructive way, including large allocations (DDoS) or even
taking down the JVM in extreme cases.
Possible solutions?
- We could add our own HMAC, etc.
- And/or we could use something like AES-GCM which offers built-in
integrity and will throw an AEADBadTagException on failure.
There may indeed be reasons [2,3,4] to consider adding a custom HMAC -
and that's still on the cards for later.
But in the meantime, the overall balance of pros/cons seems to lean
in the direction of choosing AES-GCM as a reasonable default.
Note that the change in this commit is done in a backward-compatible
way using Nippy's versioned header: new payloads will be written using
AES-GCM by default. But old payloads already written using AES-CBC will
continue to be read using that scheme.
References
[1] https://en.wikipedia.org/wiki/Authenticated_encryption
[2] https://www.daemonology.net/blog/2009-06-24-encrypt-then-mac.html
[3] https://blog.cryptographyengineering.com/2011/12/04/matt-green-smackdown-watch-are-aead/
[4] HMAC vs AEAD integrity, https://crypto.stackexchange.com/q/24379
[5] AES-GCM vs HMAC-SHA256 integrity, https://crypto.stackexchange.com/q/30627
Housekeeping includes:
* Importing useful encryption+compression stuff into primary ns
for lib consumers.
* Promoting a number of things from Alpha status.
* Exceptions are now all `ex-info`s.
* Simplification of `thaw` API: Nippy v1 support is now automatic
& configuration-free (performance impact in most cases is negligible).
PROBLEM: :legacy-mode :auto/true thawing was resulting in JVM core
dumps when attempting to use Snappy to decompress encrypted data.
CAUSE: The org.iq80.snappy implementation seems to choke on the
random IV byte data being generated by the AES128 encrypter. This
may or may not be a bug (still awaiting feedback from lib's authors).
SOLUTION: We're only susceptible to this issue when trying to
decompress data that is: a) encrypted, b) being thawed in legacy mode.
In particular, we're _not_ susceptible to this issue when thawing
in non-legacy mode because in that case we have a header explicitly
warning us that the data is encrypted.
An obvious work-around, therefore, is just to disable decryption when
attempting to thaw legacy-mode data. In practice this isn't a problem
because older versions of Nippy didn't support encryption anyway.
