[cryptography] [zfs] [Review] 4185 New hash algorithm support

Eugen Leitl eugen at leitl.org
Tue Oct 8 02:09:03 EDT 2013

----- Forwarded message from Saso Kiselkov <skiselkov.ml at gmail.com> -----

Date: Mon, 07 Oct 2013 23:18:21 +0100
From: Saso Kiselkov <skiselkov.ml at gmail.com>
To: zfs at lists.illumos.org
CC: Zooko Wilcox-OHearn <zooko at leastauthority.com>
Subject: Re: [zfs] [Review] 4185 New hash algorithm support
Message-ID: <5253332D.6090509 at gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8
Reply-To: zfs at lists.illumos.org

On 10/7/13 10:17 PM, Zooko Wilcox-OHearn wrote:
> Hi folks:
> I just joined this list because I saw this thread. I'm one of the
> architects of a distributed storage system named Tahoe-LAFS
> (https://Tahoe-LAFS.org). It has quite a few things in common with
> ZFS, architecturally, but it is also very different from ZFS, because
> it's not so much a real *filesystem* as it is like a BitTorrent that
> has an upload button as well as a download button. But it is like ZFS
> inasmuch as they both involve a heck of a lot of hashing for
> error-detection.
> [..snip..]


Thanks for joining, I'm glad we got somebody educated on the matters in
on the discussion! I studied BLAKE for inclusion instead of Skein, but
its performance was worse at the time. BLAKE2 is a different beast
(explained below).

> So, before I go on with my pitch for why you should consider BLAKE2,
> first please clarify for me whether ZFS really needs a
> collision-resistant hash function, or whether it needs only a MAC. I
> had thought until now that ZFS doesn't need a collision-resistant hash
> unless dedup is turned on, and that if dedup is turned on it needs a
> collision-resistant hash.

The reason is purely for dedup and pretty much nothing else. As such, we
only need a hash with a good pseudo-random output distribution and
collision resistance. We don't specifically need it to be super-secure.
The salted hashing support I added was simply to silence the endless
stream of wild hypotheticals on security.

> But this thread seems to indicate that even when dedup is turned on,
> it might be possible to use a MAC, by having a pool-wide secret to use
> for the MAC key… If I understand correctly (which I probably don't),
> that would make it impossible for anyone who doesn't know the secret
> to cause collisions during dedup, but still possible for someone who
> knows the secret (presumably root on that system, or someone who stole
> the secret) to generate blocks that would collide during dedup. If you
> used a collision-resistant hash for that purpose, then nobody would be
> able to cause collisions.

Oh boy, I already regret having referred to the feature in the block
comments as a MAC... it's not and was never intended to be. It's really
just a way to mitigate fears of the potential exploitation of a
not-quite-secure hash function (Edon-R mostly). It is not meant to
authenticate data on the pool to the machine itself (how could it if we
store the key unencrypted on the pool anyway?), or protect us from
somebody who had physical access to the underlying pool. It's just to
prevent attacks on the dedup tables.

> If you need a MAC, I suggest Poly1305-AES. It is very efficient, has a
> nice proof that it is as secure as AES is, and it is part of a new
> proposed cipher suite for TLS ².
> ² http://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-01

Looked at it and looks fairly similar to the VMAC that Radio młodych
bandytów suggested. This is a MAC, whereas we need a hash (for dedup).

> If you need a collision-resistant hash function, I suggest BLAKE2. It
> is more efficient than SHA-256, Skein, or Keccak (see ³), and it has a
> better reputation among cryptographers than Edon-R has. In fact,
> BLAKE2's parent, BLAKE, was rated by NIST as being even more
> well-studied than Keccak was — see my slides, linked above, for quotes
> from NIST's final report on the SHA-3 contest.
> [...snip...]
> You can get the academic papers, source code (both simple reference
> implementations and optimized implementations in various languages),
> test vectors, and so on:
> https://blake2.net

I tried BLAKE2 on my machine and it is true that it provides a slight
performance boost over Skein, though it doesn't even approach Edon-R.
For non-trivial reasons we can't use floating point registers in kernel
code without jumping through many hoops (so no SSE/AVX), plus we need to
be able to run on SPARC, but the pure 64-bit C implementation is a bit
faster than Skein, ~5.1 CPB for BLAKE2 vs ~5.9 CPB for Skein on my Core
i5 test rig. On an Athlon II 1.3 GHz it's actually a bit slower, at 7.75
CPB for BLAKE2 vs 7.45 CPB for Skein. Used the blake2b-ref.c
implementation, compiled GCC 4.4.4 with "-fno-builtin -O2 -std=c99" to
mirror in-kernel build environment.

Overall, I'd say: I don't know. I've already sunk a fair amount of
effort to get Skein to work well and integrated with the KCF and ZFS and
I'm not exactly willing to jump in and expend more effort to get another
-0.1 to +0.7 CPB...


Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/22842876-6fe17e6f
Modify Your Subscription: https://www.listbox.com/member/?member_id=22842876&id_secret=22842876-a25d3366
Powered by Listbox: http://www.listbox.com

----- End forwarded message -----
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
ICBM: 48.07100, 11.36820 http://ativel.com http://postbiota.org
AC894EC5: 38A5 5F46 A4FF 59B8 336B  47EE F46E 3489 AC89 4EC5
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.randombit.net/pipermail/cryptography/attachments/20131008/4286c8d7/attachment.asc>

More information about the cryptography mailing list