Lattice & FHE cryptography, and assorted TCS. Zola 2026-07-02T00:00:00+00:00 https://mark.schultz-wu.com/atom.xml 2026-07-02T00:00:00+00:00 2026-07-02T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/authoring-kit/ <p>A hidden reference page exercising every shortcode in the authoring kit. Not listed anywhere; reachable only by URL. Use it as a smoke test after template changes.</p> <h1 id="math"><a class="zola-anchor" href="#math" aria-label="Anchor link for: math">🔗</a>Math</h1> <p>Inline math like <span class="math">\(e = mc^2\)</span> and a prime <span class="math">\(f'(x)\)</span> (curly-quote-safe), plus a display block:</p> <span class="math">\[\Pr[\,\|e\| > t\,] \le 2\exp\!\left(-\frac{t^2}{2\sigma^2}\right).\]</span> <p>Math is authored as plain <code>$...$</code> / <code>$$...$$</code> in <code>content-src/</code> and wrapped into raw shortcodes by <code>scripts/wrap_math.py</code>, so <code>smart_punctuation</code> never touches it.</p> <h1 id="callouts"><a class="zola-anchor" href="#callouts" aria-label="Anchor link for: callouts">🔗</a>Callouts</h1> <aside class="callout callout-note"> <p class="callout-title">Aside</p> <div class="callout-body"> <p>Asides hold tangential remarks. They render <strong>Markdown</strong>, including math such as <span class="math">(\sigma^2)</span> and links like <a href="https://mark.schultz-wu.com/blog/lattices-for-programmers-pt1/">the LWE post</a>.</p> </div> </aside> <aside class="callout callout-warning"> <p class="callout-title">Warning</p> <div class="callout-body"> <p>Warnings flag caveats. Math here is also safe: <span class="math">(|x|_\infty \le q/2)</span>.</p> </div> </aside> <h1 id="figure"><a class="zola-anchor" href="#figure" aria-label="Anchor link for: figure">🔗</a>Figure</h1> <figure class="figure"> <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Lattice_of_the_divisibility_of_60.svg/320px-Lattice_of_the_divisibility_of_60.svg.png" alt="A *figure* with a Markdown caption." style="max-width: 240px" /> <figcaption>A <em>figure</em> with a Markdown caption.</figcaption> </figure> <h1 id="interactive-demo"><a class="zola-anchor" href="#interactive-demo" aria-label="Anchor link for: interactive-demo">🔗</a>Interactive demo</h1> <p>The demo below is a real Rust → wasm demo (<code>demos/noise-budget/</code>) — an LWE noise-budget visualizer. Build it with <code>just demos</code>; until then it degrades gracefully to the fallback text.</p> <div class="demo" id="demo-noise-budget" data-demo="noise-budget"> <noscript>This interactive demo requires JavaScript and WebAssembly.</noscript> <div class="demo-fallback"><p>LWE noise-budget demo (build with <code>just demos</code>).</p> </div> </div> <script type="module"> (() => { const el = document.getElementById("demo-noise-budget"); const src = "https://mark.schultz-wu.com/demos/noise-budget/noise-budget.js"; let started = false; const boot = async () => { if (started) return; started = true; try { const mod = await import(src); await (mod.default ? mod.default() : Promise.resolve()); if (typeof mod.mount === "function") mod.mount(el); } catch (e) { el.classList.add("demo-error"); el.innerHTML = "<p>Demo failed to load. Run <code>just demos</code> to build it.</p>"; console.error("demo noise-budget failed:", e); } }; if ("IntersectionObserver" in window) { const io = new IntersectionObserver((ents) => { for (const e of ents) if (e.isIntersecting) { io.disconnect(); boot(); } }, { rootMargin: "200px" }); io.observe(el); } else { boot(); } })(); </script> <h1 id="footnotes"><a class="zola-anchor" href="#footnotes" aria-label="Anchor link for: footnotes">🔗</a>Footnotes</h1> <p>Footnotes collect at the bottom.<sup class="footnote-reference" id="fr-ex-1"><a href="#fn-ex">1</a></sup></p> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-ex"> <p>Like this one. <a href="#fr-ex-1">↩</a></p> </li> </ol> </section> 2023-11-14T00:00:00+00:00 2023-11-14T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/hintless-pir/ <p>This summer, I was hosted by Mariana Raykova and Baiyu Li at Google to work on Single Server PIR. This internship led to a <a rel="external" href="https://eprint.iacr.org/2023/1733">paper</a>. Rather than try to put together some tweet-thread (or X-thread? who knows) talking about the result, I thought it best to write it up somewhere that supports latex, e.g. here.</p> <h1 id="private-information-retrieval"><a class="zola-anchor" href="#private-information-retrieval" aria-label="Anchor link for: private-information-retrieval">🔗</a>Private Information Retrieval</h1> <p>Private Information Retrieval is the problem of “securely querying” a database. In particular, you have a client, and a server, who holds a <em>public</em> database. The client wants to retrieve a particular record from this database without the server learning which record was retrieved. Note that there is an obvious trivial protocol — the server transmits the whole database! The client can then locally retrieve the record they want, and the server learns nothing.</p> <p>This scheme is actually fairly (computationally) efficient. The server has a linear scan through the database, and the client can stream the database and only store their desired record, e.g. the memory requirements on the client are fairily small. The bandwidth of the scheme is very large though, and the main goal in PIR research is to reduce these bandwidth requirements.</p> <p>This is done various ways. Typically, authors like to add some “extra tools” to the standard PIR setting, for example</p> <ol> <li><a rel="external" href="https://eprint.iacr.org/2022/949">SimplePIR</a>: Having a (query-independent) “hint” the client downloads beforehand, that they can amortize over all queries.</li> <li><a rel="external" href="https://eprint.iacr.org/2023/204">TreePIR</a>: Assuming the database is held by two non-colluding servers.</li> <li><a rel="external" href="https://eprint.iacr.org/2023/452">Piano</a>: Assume the entire database is streamed to a memory-bound client beforehand.</li> </ol> <p>Under these assumptions, one can get practically efficient schemes, where efficiency metrics tend to be</p> <ul> <li>total bandwidth cost, and</li> <li>server computational cost.</li> </ul> <p>For example, SimplePIR has throughput comperable to the memory bandwidth of certain processors, or in other words server’s computational cost in SimplePIR is comperable to a linear scan through the database. And Piano is even faster!</p> <p>There do exist schemes that avoid these assumptions (for example <a rel="external" href="https://eprint.iacr.org/2022/368">SpiralPIR</a>) which are still pretty fast, but there’s a pretty noticable slowdown — in our measurements SpiralPIR was at least an order of magnitude slower than SimplePIR. Our goal was to bridge this gap — give a scheme that avoided the above three assumptions, but had better server computation than SpiralPIR. We ended up with a scheme that achieves this goal “asymptotically”, meaning the cost to execute it is (in the limit of large database sizes) identical to that of SimplePIR. We measured that this leads to concretely small overhead (for certain “large enough” databases of practically relevant sizes), e.g. we can remove SimplePIR’s hint at low cost.</p> <p>In the interest of brevity, I’ll focus on describing the main idea behind our main construction, as well as a particular optimization that we introduced that I like quite a bit (and is theoretically an order-of-magnitude speedup, though practically I think we measured it as a 6x speedup).</p> <h1 id="single-server-pir-from-lwe"><a class="zola-anchor" href="#single-server-pir-from-lwe" aria-label="Anchor link for: single-server-pir-from-lwe">🔗</a>Single Server PIR from LWE</h1> <p>We start with the SimplePIR (and FrodoPIR) construction. This leverages the following simple idea. If you format an <span class="math">\(m\)</span>-record database <span class="math">\(\mathsf{DB}\)</span> as a matrix <span class="math">\(\mathsf{DB}\in \mathbb{Z}_p^{\sqrt{m}\times \sqrt{m}}\)</span>, then one can retrieve the <span class="math">\(i\)</span>th column of this database by multiplying it via the <span class="math">\(i\)</span>th basis vector <span class="math">\(e_i\)</span>. This can then be computed using homomorphic encryption, namely</p> <span class="math">\[\mathsf{DB}, \mathsf{Enc}(e_i)\mapsto \mathsf{Enc}(\mathsf{DB}\cdot e_i).\]</span> <p>All one needs is an encryption scheme that supports efficient homomorphic matrix-vector multiplication. Learning with Errors (LWE)-based encryption does just this! I’ve written some <a href="https://mark.schultz-wu.com/blog/nist-standard-out/">blog posts</a> on this in the past geared towards a general audience, so I will be brief below. LWE-based ciphertexts take the form</p> <span class="math">\[C := [A, A\cdot s + f + (q/p)m]\in\mathbb{Z}_q^{\sqrt{m}\times n} \times \mathbb{Z}_q^{\sqrt{m}\times 1}\]</span> <p>where <span class="math">\(A\)</span> is a uniformly-random matrix, <span class="math">\(s\)</span> is a (typically short) secret vector, <span class="math">\(f\)</span> is a (short and secret) “noise” vector, and <span class="math">\(m\in\mathbb{Z}_p^{\sqrt{m}}\)</span> is the message one is encrypting. One can efficient compute matrix-vector products using this, as</p> <span class="math">\[\mathsf{DB}\cdot C := [\mathsf{DB}\cdot A, \mathsf{DB}\cdot A\cdot s + \mathsf{DB}\cdot f + (q/p)\mathsf{DB}\cdot m]\]</span> <p>There are some (routine) details that one has to get right with the above scheme, but for the most part the fact that it yields a <em>correct</em> homomorphic scheme is standard. What is harder is getting it to be <em>bandwidth-efficient</em>.</p> <p>The issue is in the transmission of <span class="math">\(A\in\mathbb{Z}_q^{\sqrt{m}\times n}\)</span>, and in particular its dependence on <span class="math">\(n\)</span>. This is the lattice dimension, and is required to be <span class="math">\(\geq 500\)</span> for security purposes (though depending on how large <span class="math">\(q\)</span> is this can grow much higher, e.g. in the thousands). This means that the “<span class="math">\(A\)</span>” component of ciphertexts is more than <span class="math">\(500\times\)</span> larger than the component containing the message. There is a standard trick to “compress” <span class="math">\(A\)</span> to a PRG <sup class="footnote-reference" id="fr-prg-1"><a href="#fn-prg">1</a></sup> seed <span class="math">\(sd\)</span>, which is then expanded to <span class="math">\(A := \mathsf{PRG}(sd)\)</span>. This works to minimize the size of the client’s encryption. The issue is that when the server computes</p> <span class="math">\[\mathsf{DB}\cdot A = \mathsf{DB}\cdot \mathsf{PRG}(sd)\]</span> <p>there is no way to compress this back down to a similarly short seed <span class="math">\(sd'\)</span>. The typical way to circumvent this issue is to using something like Ring Learning with Errors-based schemes, where the “<span class="math">\(A\)</span>” component is the same size as the message-dependent component. This has two issues in our setting though.</p> <ol> <li>Our homomorphic computation is a matrix-vector multiplication. This is quite efficient for LWE, but less so for RLWE.</li> <li>For LWE-based encryption, one can freely choose the modulus <span class="math">\(q\)</span>. For example, one can set <span class="math">\(q \in \{2^{32}, 2^{64}\}\)</span> for fast hardware-based arithmetic. For RLWE-based encryption, there are often restrictions on <span class="math">\(q\)</span>, and <span class="math">\(q\in\{2^{32}, 2^{64}\}\)</span> didn’t work for our protocols.</li> </ol> <p>In our setting we are able to (almost) completely solve the first problem. In particular, we give an RLWE-based protocol to securely compute matrix-vector multiplication that is practically (when measured in <span class="math">\(\mathbb{Z}_q\)</span> operations) only a small constant factor larger than the corresponding plaintext computation. This amounts to <em>heavily</em> optimizing a standard protocol (the <a rel="external" href="https://eprint.iacr.org/2014/106">“Diagonally Dominant” multiplication algoriothm</a>), including shaving a <span class="math">\(O(\log n)\)</span> factor that I will describe at the end of this post. When measured in <span class="math">\(\mathbb{Z}_q\)</span> operations, our protocol is within constant factors of the cost of the LWE-based protocol, which is within constant factors of the underlying <em>plaintext</em> computation. While this is great, recall that existing protocols have server compute comperable to a <em>linear scan</em> through the database. So losing constant factors (which I expect to be <span class="math">\(\approx 10\times\)</span>) isn’t good enough in this setting. There are some other limitations of our protocol (it defers certain operations to the client post-decryption, and some more techincal things), in our setting of PIR it is theoretically quite compelling.</p> <h2 id="simplepir-as-a-reduction-from-pir-to-linpir"><a class="zola-anchor" href="#simplepir-as-a-reduction-from-pir-to-linpir" aria-label="Anchor link for: simplepir-as-a-reduction-from-pir-to-linpir">🔗</a>SimplePIR as a Reduction from PIR to LinPIR</h2> <p>As mentioned in the above, at a high level one can build PIR from LWE by having</p> <ol> <li>Clients upload <span class="math">\(C := [A, A\cdot s + f + (q/p)e_i]\)</span>,</li> <li>The server compute <span class="math">\(\mathsf{DB} \cdot C\)</span>, and</li> <li>return this to the client, who decrypts.</li> </ol> <p>This requires transmitting <span class="math">\(A\)</span> and <span class="math">\(\mathsf{DB}\cdot A\)</span>, which are large. SimplePIR optimizes this simple protocol by “standardizing” a single matrix <span class="math">\(A := \mathsf{PRG}(sd)\)</span>, and having the client download <span class="math">\((sd, \mathsf{DB}\cdot \mathsf{PRG}(sd))\)</span> before making queries. Modulo certain technicalities, this allows the client to omit transmitting <span class="math">\(A\)</span> from their query, and the server to omit transmitting <span class="math">\(\mathsf{DB}\ast A\)</span>, fixing the issue.</p> <p>There is an alternative way to view what SimplePIR is doing, which I find useful in describing our protocol. Recall that one way of viewing a standard PIR scheme is as a way of securely computing</p> <span class="math">\[M, x \mapsto M\cdot x\]</span> <p>where <span class="math">\(M = \mathsf{DB}\)</span> is a database, formatted as a matrix, and <span class="math">\(x =e_i\)</span> is a basis vector. SimplePIR reduces this to securely computing</p> <span class="math">\[M, x\mapsto M\cdot x\]</span> <p>where <span class="math">\(M = \mathsf{DB}\ast A\)</span> is the “hint”, and <span class="math">\(x = s\)</span> is the LWE secret. We call this a <em>linear</em> PIR scheme. So SimplePIR reduces a standard PIR query to the database <span class="math">\(\mathsf{DB}\in\mathbb{Z}_p^{\sqrt{m}\times \sqrt{m}}\)</span> to a linear PIR query to the “hint” <span class="math">\(H:=\mathsf{DB}\cdot A\in \mathbb{Z}_q^{\sqrt{m}\times n}\)</span>. For large-enough <span class="math">\(m\)</span> this is a smaller database, yielding a concrete savings. SimplePIR itself can be viewed as using the trivial PIR protocol of transmitting <span class="math">\(H\)</span> to the client to answer these linear PIR queries.</p> <p>Our protocol modifies this to use our RLWE-based PIR scheme to answer these linear PIR queries. As mentioned before, this (concretely) loses constant factors compared to SimplePIR. This yields to a scheme that has server computational costs of the form</p> <ul> <li><span class="math">\(m + O(\sqrt{m}n)\)</span> operations in <span class="math">\(\mathbb{Z}_{2^{32}}\)</span>, compared with</li> <li><span class="math">\(m\)</span> operations in <span class="math">\(\mathbb{Z}_{2^{32}}\)</span> for SimplePIR.</li> </ul> <p>So, for large-enough <span class="math">\(m\)</span>, we can replace the trivial linPIR query to <span class="math">\(H\)</span> with a linPIR query to our scheme at no asymptotic cost. This is (theoretically) the content of the main construction of our paper. Practically, we use several additional optimizations to make sure that “large-enough” is not too large. I’ll mention my favorite such optimization below.</p> <h1 id="an-speedup-for-many-rlwe-based-linearly-homomorphic-computations"><a class="zola-anchor" href="#an-speedup-for-many-rlwe-based-linearly-homomorphic-computations" aria-label="Anchor link for: an-speedup-for-many-rlwe-based-linearly-homomorphic-computations">🔗</a>An <span class="math">\(O(\log n)\)</span> Speedup for many RLWE-based (Linearly) Homomorphic Computations</h1> <p>RLWE-based cryptography often describes computations in terms of two isomorphic rings. There is</p> <ul> <li><span class="math">\(R_{n,q} = \mathbb{Z}_q[x] / (x^n+1)\)</span>, which is a ring under addition and multiplication of polynomials modulo <span class="math">\((q, x^n+1)\)</span>, and</li> <li><span class="math">\(\widehat{R}_{n,q}\)</span>, e.g. the set <span class="math">\(\mathbb{Z}_q\)</span> under coordinate-wise addition and multiplication.</li> </ul> <p>These rings are only isomorphic for certain <span class="math">\(q\)</span> (“NTT-friendly”), but they allow you to replace polynomial multiplication (naively <span class="math">\(O(n^2)\)</span> time) with a linear-time operation. One can convert between the two rings at cost <span class="math">\(O(n\log n)\)</span> operations in <span class="math">\(\mathbb{Z}_q\)</span>. This cost is often the asymptotically dominant cost in homomorphic computations, to the point that counting how many conversions (NTTs) one uses is generally a decent proxy for overall running time.</p> <p>We find a way to precompute all NTTs that occur in our protocol in a way that is independent of things like</p> <ul> <li>the particular client making a query, and</li> <li>the particular database one is issuing a linear PIR query to.</li> </ul> <p>Practically, after executing the protocol once (say during a preprocessing phase) and producing state that may be kept <em>locally</em> on the server, we get an <span class="math">\(O(\log n)\)</span> speedup.</p> <p>The high-level idea is simple. Recall that RLWE-based ciphertexts <span class="math">\([a(x), b(x)]\)</span> consist of <em>pairs</em> of polynomials, where</p> <ul> <li><span class="math">\(a(x)\)</span> is independent of the message (and should be “random”), and</li> <li><span class="math">\(b(x) = a(x)\ast s(x) + f(x) + (q/p)m(x)\)</span> is “structured”.</li> </ul> <p>One almost always wants to keep polynomials in the evaluation representation <span class="math">\(\widehat{R}_{n,q}\)</span>, as it has more efficient multiplication. The only time we want to switch back to <span class="math">\(R_{n,q}\)</span> is to compute operations that depend on particular coefficients of the polynomial, which happens when multiplying a ciphertext by a large plaintext polynomial with low noise growth. For simplicity, say that multiplying <span class="math">\([a(x), b(x)]\)</span> by the large plaintext polynomial <span class="math">\(d(x)\)</span> <sup class="footnote-reference" id="fr-polyhom-1"><a href="#fn-polyhom">2</a></sup> produces a ciphertext <span class="math">\(c' = [F_0^d(a), F_1^d(b)]\)</span>. Our insight is that if one knows <span class="math">\(d\)</span> beforehand, the maps <span class="math">\(F_0^d, F_1^d\)</span> can (after preprocessing that is <span class="math">\(O(\log n)\)</span> slower) be computed in linear time. Moreover, the map <span class="math">\(F_0^d\)</span> is <em>deterministic</em>, and therefore</p> <ol> <li>if you start knowing <span class="math">\(a(x)\)</span> (say because <span class="math">\(a(x)= \mathsf{PRG}(sd)\)</span>), and</li> <li>also know <span class="math">\(d(x)\)</span> (which is generally the <span class="math">\(a(x)\)</span> polynomial of a different ciphertext, for example during key-switching),</li> </ol> <p>then the result is a ciphertext <span class="math">\(c' =[F_0^d(a), F_1^d(b)] = [a'(x), b'(x)]\)</span> that we also know the polynomial <span class="math">\(a'(x)\)</span> for. This leaves one in the position to iterate this argument, and remove all NTTs from (certain) computations.</p> <p>We haven’t tried to describe the precise class of circuits which our preprocessing optimization works for. My current understanding of the limits are that</p> <ul> <li>Homomorphic multiplications (either “tensor-based” or “gadget-based”) do not preserve our “knowing <span class="math">\(a(x)\)</span>” invariant <sup class="footnote-reference" id="fr-gadget-1"><a href="#fn-gadget">3</a></sup></li> <li>Homomorphic additions work fine</li> <li>gadget-based key-switching works fine.</li> </ul> <p>So we can preprocess complex linearly-homomorphic computations (for example the matrix-vector multiplication protocol in our work), but not things like FHEW-type bootstrapping. There are also some costs for this optimization. We require that <em>the server</em> knows <span class="math">\(a(x)\)</span> beforehand. It is therefore natural to have the server pick the seed <span class="math">\(sd\)</span>, and transmit it to all clients. This works, but requires the the clients resample an RLWE key <span class="math">\(s(x)\)</span> for each query to maintain security. More importantly, they must also resample any encrypted key material. In our particular setting this comes at low cost, but in other settings it is likely something that must be balanced against the potential <span class="math">\(O(\log n)\)</span> speedup.</p> <p>Anyway, this is skipping plenty of other stuff in our paper (for example, an additional PIR scheme!), but seems like a decent analogue for putting together some hard-to-read-twitter-thread.</p> <hr /> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-prg"> <p>Technically one needs this to be a hashing-based primitive to be able to get provable security out of it. The term to search is “Extendible output function”, but I won’t belabor the point. <a href="#fr-prg-1">↩</a></p> </li> <li id="fn-polyhom"> <p>This is not as simple as computing <span class="math">\([c(x)\ast a(x), c(x)\ast b(x)]\)</span>. This works for “small” polynomials <span class="math">\(c(x)\)</span>, but increases the error too much when <span class="math">\(c(x)\)</span> is large. As I am trying to avoid discussions of error in (R)LWE here, I will solely point interested readers to <a rel="external" href="https://eprint.iacr.org/2020/086">this</a>. <a href="#fr-polyhom-1">↩</a></p> </li> <li id="fn-gadget"> <p>This is to say that we can preprocess NTTs in a circuit using gadget-based operations until the first gadget-based homomorphic multiplication, after which we can no longer preprocess things. <a href="#fr-gadget-1">↩</a></p> </li> </ol> </section> 2022-07-05T00:00:00+00:00 2022-07-05T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/nist-standard-out/ <p>Today, <a rel="external" href="https://csrc.nist.gov/publications/detail/nistir/8413/final">NIST put out its preliminary choice of PQC algorithms to standardize</a>. I wrote an introduction to lattice-based KEMs (for non-cryptographers) this last weekend in preparation for the report, where one implements an (aggressively unoptimized) variant of FrodoKEM. FrodoKEM ended up not being standardized, but the “broad picture” of the construction should still be useful to understand Kyber.</p> <p>Anyway, I am writing this small post to collect links to the three posts on a single page.</p> <ul> <li><a href="https://mark.schultz-wu.com/blog/lattices-for-programmers-pt1/">Part 1</a> discusses the base underlying hardness assumption (Learning with Errors) some.</li> <li><a href="https://mark.schultz-wu.com/blog/lattices-for-programmers-pt2/">Part 2</a> constructs a simple lattice-based (private-key) cryptosystem, and</li> <li><a href="https://mark.schultz-wu.com/blog/lattices-for-programmers-pt3/">Part 3</a> constructs a lattice-based (public-key) cryptosystem.</li> </ul> <hr /> 2022-07-02T00:00:00+00:00 2022-07-02T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/lattices-for-programmers-pt1/ <p>This is a first post in a series of blog posts where I plan on building a relatively simple lattice-based public-key encryption scheme. This construction is not novel at all [<sup class="footnote-reference" id="fr-frodoKEM-1"><a href="#fn-frodoKEM">1</a></sup>], but the intention is to</p> <ul> <li>simplify certain optimizations that aren’t needed for security, and</li> <li>present things in straightforward language, in terms of simple mathematical concepts.</li> </ul> <p>Throughout, I will first present (insecure) constructions (that are <em>exceedingly</em> simple), before describing how we make them secure. These insecure constructions correspond to “noiseless learning with errors”. As we will see later in this particular post, the “noise” is central to the overall security of constructions.</p> <p>Mathematically, this series of posts will be rather simple. The main things required will be matrix multiplication, and modular arithmetic. Lattice-based encryption can involve more complex topics (such as lattices, and modules over polynomial rings), but these are needed solely to investigate the security of optimized constructions. Our construction is (intentionally) far from optimized, allowing us to ignore these complex topics.</p> <p>In this post I’ll first talk about other elementary approaches to teaching cryptography (namely Textbook RSA), before introducing the underlying hard problem we will use (the Learning with Errors, or LWE problem). I’ll finish by giving some informal (but formalizable) justifications for why the LWE problem is hard. Given the practical (and elementary) approach I am taking though, this justification will necessarily be at a high level.</p> <h1 id="textbook-rsa"><a class="zola-anchor" href="#textbook-rsa" aria-label="Anchor link for: textbook-rsa">🔗</a>Textbook RSA</h1> <p>If you’re a programmer interested in learning about cryptography, there are some decent resources available. One in particular is a scheme that cryptographers call <em>Textbook RSA</em>. Most websites you might try to learn about RSA from cover this scheme (see <a rel="external" href="https://www.geeksforgeeks.org/rsa-algorithm-cryptography/">GeeksForGeeks</a>, <a rel="external" href="https://en.wikipedia.org/wiki/RSA_(cryptosystem)#Operation">Wikipedia</a>, or <a rel="external" href="https://www.simplilearn.com/tutorials/cryptography-tutorial/rsa-algorithm">Simplilearn</a>). Really, if you haven’t formally studied cryptography (but have tinkered around with it a little bit), “textbook RSA” is likely exactly what you think the RSA cryptosystem is.</p> <p>Explicitly:</p> <ol> <li>The public-key is a product of two primes <span class="math">\(N = pq\)</span>, along with some value <span class="math">\(e\)</span> (typically hard-coded to either 3, or the better choice [<sup class="footnote-reference" id="fr-better-1"><a href="#fn-better">2</a></sup>] of <span class="math">\[2^{16} + 1 = 65537\]</span>).</li> <li>The secret-key is the pair <span class="math">\((p, q)\)</span>, along with an algorithm to compute “<span class="math">\(e\)</span>th roots <span class="math">\[\bmod N\]</span>”, namely the map <span class="math">\(c\mapsto c^{1/e}\bmod N\)</span>.</li> <li>Encryption maps <span class="math">\(m\mapsto m^e\bmod N\)</span>, and</li> <li>Decryption maps <span class="math">\(c\mapsto c^{1/e}\bmod N\)</span>.</li> </ol> <p>It is straightforward to see that this scheme is <em>correct</em>. This is because</p> <span class="math">\[m\stackrel{\mathsf{Enc}}{\mapsto}m^e\bmod N\stackrel{\mathsf{Dec}}{\mapsto}(m^e)^{1/e}\bmod N \equiv m\bmod N.\]</span> <p>In words, encryption raises things to the <span class="math">\(e\)</span>th power, and decryption takes <span class="math">\(e\)</span>th roots. Provided these notions are defined sensibly, they are inverses, and everything works out.</p> <p>One then shows a variety of things, such as</p> <ul> <li>if you know <span class="math">\((p,q)\)</span>, you can efficiently take <span class="math">\(e\)</span>th roots, and</li> <li>if you don’t know <span class="math">\((p,q)\)</span>, the “obvious” way to take <span class="math">\(e\)</span>th roots is factoring <span class="math">\(N\)</span> first [<sup class="footnote-reference" id="fr-inequivalent-1"><a href="#fn-inequivalent">3</a></sup>].</li> </ul> <h2 id="merits-of-textbook-rsa"><a class="zola-anchor" href="#merits-of-textbook-rsa" aria-label="Anchor link for: merits-of-textbook-rsa">🔗</a>Merits of Textbook RSA</h2> <p>Clearly, the above scheme is <em>very simple</em> to describe. Mathematically, both encryption and decryption are barely a line each. Key generation is similarly (conceptually) simple. Moreover, the underlying hard problem can be presented as being “Factoring”. This isn’t really true, but it’s somewhat close to the truth, and this problem is (conceptually) simple to understand.</p> <h2 id="issues-with-textbook-rsa"><a class="zola-anchor" href="#issues-with-textbook-rsa" aria-label="Anchor link for: issues-with-textbook-rsa">🔗</a>Issues with Textbook RSA</h2> <p>There are two broad issues I’d like to highlight, namely <em>implementation complexity</em> and <em>security</em>.</p> <h3 id="implementation-complexity"><a class="zola-anchor" href="#implementation-complexity" aria-label="Anchor link for: implementation-complexity">🔗</a>Implementation Complexity</h3> <p>One issue with the above (simple) description of RSA is that it hides a good deal of complexity. Standard implementations of RSA require the following algorithms be implemented</p> <ol> <li>the extended GCD algorithm, and</li> <li>primality testing.</li> </ol> <p>Neither of these are that bad to do (having beginning programmers do them can be a good exercise). For example, the following code snippet is a perfectly reasonable approach at implementing primality testing, as well as a simple rejection sampling algorithm to sample random (large) primes.</p> <pre class="giallo" style="color: #CDD6F4; background-color: #1E1E2E;"><code data-lang="plain"><span class="giallo-l"><span>import random</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>def isprime(n):</span></span> <span class="giallo-l"><span> if n % 2 == 0:</span></span> <span class="giallo-l"><span> return False</span></span> <span class="giallo-l"><span> else:</span></span> <span class="giallo-l"><span> i = 3</span></span> <span class="giallo-l"><span> while i**2 &lt; n:</span></span> <span class="giallo-l"><span> if n % i == 0:</span></span> <span class="giallo-l"><span> return False</span></span> <span class="giallo-l"><span> i += 2</span></span> <span class="giallo-l"><span> return True</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>def sampleprime(n):</span></span> <span class="giallo-l"><span> while True:</span></span> <span class="giallo-l"><span> p = random.randint(1, 2**n)</span></span> <span class="giallo-l"><span> if isprime(p):</span></span> <span class="giallo-l"><span> return p</span></span></code></pre> <p>This leverages that Python’s standard library contains native support for big (2048-bit) integer arithmetic, so we can ignore implementing all of that. Yay!</p> <p>The issue with this naive approach is that it is utterly infeasible. As mentioned, RSA requires 2048-bit integers, or 1024-bit primes. To verify that a number is prime, the above primality test requires (roughly) <span class="math">\(\sqrt{n}\)</span> operations, or <span class="math">\(2^{512}\)</span>. This is to say that the above (natural) primality testing algorithm that many students happily write down is secretely an exponential-time algorithm, and therefore cannot be used for any actual instantiation of RSA.</p> <p>Therefore, to <em>actually</em> implement RSA you need to discuss fast primality testing algorithms (for example <a rel="external" href="https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test">Miller-Rabin</a>). This brings a great deal of complexity into the story (and we even got to skip the big integer implementation!).</p> <p>This is all to say that while RSA is conceptually simple, actually <em>running</em> RSA for parameters that are thought to be secure requires a great deal of complexity.</p> <h3 id="security"><a class="zola-anchor" href="#security" aria-label="Anchor link for: security">🔗</a>Security</h3> <p>The second issue is that RSA (as presented above) is not secure. Cryptography has a minimal acceptable notion of security known as <a rel="external" href="https://en.wikipedia.org/wiki/Ciphertext_indistinguishability#Indistinguishability_under_chosen-plaintext_attack_(IND-CPA)">Indistinguishability under Chosen-Plaintext Attack</a>. Among other things, this implies that you cannot tell if two ciphertexts are encryptions of the same message. But for the above “textbook RSA”, if you encrypt the same message twice, you get back the same ciphertext (encryption is <em>deterministic</em>). This is typically fixed by introducing a <em>padding scheme</em>, but these are both complex, and easy to get wrong (there have been <em>many</em> attacks in practice on improper RSA padding).</p> <p>So Textbook RSA is simple to describe, but moderately complex to implement in a way that is reasonably efficient for parameters people actually use, and quite complex to implement in a way that is known to be secure. RSA also has the issue that (even when properly padded) it is insecure against quantum computers, so once those develop further it’ll be insecure in all settings.</p> <h2 id="the-solution-textbook-lwe"><a class="zola-anchor" href="#the-solution-textbook-lwe" aria-label="Anchor link for: the-solution-textbook-lwe">🔗</a>The Solution: Textbook LWE</h2> <p>While in this blog series I will present a simplified variant of lattice-based encryption (you might call it “Textbook LWE”), the simplifications will mean that the scheme is <em>less efficient</em>, rather than <em>less secure</em>. While it will be less efficient, it’ll still be <em>plenty</em> efficient compared to things like Textbook RSA — for “real parameters” (actually <em>larger</em> parameters than those used in practice, as I’m somewhat lazy), all operations take less than a second. There will be inefficiences to the scheme (in particular, the size of the ciphertexts will be poor), but these are things that can be easily fixed if one cares.</p> <p>I will actually present two constructions of each algorithm under discussion, which I will call the <em>noiseless</em> and <em>noisy</em> constructions. The noiseless construction will always be conceptually simpler, and capture “the essence” of what is happening (while being insecure). The noisy construction will be secure, albient somewhat more complex than the noiseless construction.</p> <p>With that, we’ll discuss the underlying hard problem the constructions will be based on (akin to factoring for RSA, or more properly taking modular roots for RSA).</p> <h1 id="the-learning-with-errors-assumption"><a class="zola-anchor" href="#the-learning-with-errors-assumption" aria-label="Anchor link for: the-learning-with-errors-assumption">🔗</a>The Learning with Errors Assumption</h1> <p>Similarly to the constructions I describe, I’ll go over noiseless and noisy versions of the Learning with Errors Assumption. Both are most simply phrased in terms of modular matrix arithmetic. Throughout, all operations are done modulo <span class="math">\(q\)</span> for some (not necessairly prime) number <span class="math">\(q\in\mathbb{N}\)</span>. This is true even for “implicit” operations, for example the additions and multiplications that occur as part of a matrix-vector product <span class="math">\(A\vec v\)</span>.</p> <h2 id="noiseless"><a class="zola-anchor" href="#noiseless" aria-label="Anchor link for: noiseless">🔗</a>Noiseless</h2> <blockquote> <p>Let <span class="math">\(n, q\in\mathbb{N}\)</span> be integers. Let <span class="math">\(\vec s\gets (\mathbb{Z}/q\mathbb{Z})^n\)</span> be a uniformly random (secret) vector. Then, the Noiseless Learning with Errors assumption is that, for <span class="math">\(A\gets (\mathbb{Z}/q\mathbb{Z})^{n\times n}\)</span> a uniformly random (public) matrix, then <span class="math">\((A, A\vec s)\)</span> is hard to distinguish from uniformly random.</p> </blockquote> <p>So, the noiseless Learning with Errors assumption is about the pseudo-randomness of matrix-vector products <span class="math">\(A\vec s\)</span>. Without wasting too many words on this assumption, I’ll simply say that standard linear-algebraic techniques (for example Gaussian elimination) suffice to show that this is insecure. In particular,given <span class="math">\((A, A\vec s)\)</span>, it is straightforward to recover <span class="math">\(\vec s\)</span>. This is what we call in cryptography “bad”.</p> <h2 id="noisy"><a class="zola-anchor" href="#noisy" aria-label="Anchor link for: noisy">🔗</a>Noisy</h2> <p>One way to attempt to stop this attack is to introduce noise (we will discuss later how Gaussian elimination behaves with respect to noise).</p> <blockquote> <p>Let <span class="math">\(n, q\in\mathbb{N}\)</span> be integers, and let <span class="math">\(\chi\)</span> be a distribution on “small” integers modulo <span class="math">\(q\)</span>. Let <span class="math">\(\vec s\gets (\mathbb{Z}/q\mathbb{Z})^n\)</span> be a uniformly random (secret) vector. Then, the Learning with Errors assumption is that, for <span class="math">\(A\gets (\mathbb{Z}/q\mathbb{Z})^{n\times n}\)</span> a uniformly random (public) matrix and <span class="math">\(\vec e\gets \chi^n\)</span>, then <span class="math">\((A, A\vec s +\vec e)\)</span> is hard to distinguish from uniformly random.</p> </blockquote> <p>The hardness of this problem depends <em>heavily</em> on the underlying choice of parameters <span class="math">\(n, q, \chi\)</span> (and there are many variants of this problem I will not describe). As mentioned before, if there is no noise (i.e. <span class="math">\(\chi\)</span> is concentrated on <span class="math">\(\{0\}\)</span>) the problem is broken. If <span class="math">\(\chi\)</span> is uniform on <span class="math">\(\mathbb{Z}/q\mathbb{Z}\)</span>, is is also straightforward to show that the problem is “perfectly secure” (but unfortunately, it is not known to be useful for cryptographic constructions). This leaves the intermediate case, where <span class="math">\(\chi\)</span> is a distribution on some subset of <span class="math">\(\mathbb{Z}/q\mathbb{Z}\)</span> (that is not solely <span class="math">\(\{0\}\)</span>). For even fairly small subsets (say <span class="math">\([-10, 10]\)</span>), this is thought to be secure. If you actually want to get parameters that are plausibly secure, tools [<sup class="footnote-reference" id="fr-estimator-1"><a href="#fn-estimator">4</a></sup>] have been built by the community, but I won’t bother formally checking the parameters I use are fine (they’re larger than what everyone else uses, so it should be fine).</p> <h1 id="reasons-we-think-lwe-is-hard"><a class="zola-anchor" href="#reasons-we-think-lwe-is-hard" aria-label="Anchor link for: reasons-we-think-lwe-is-hard">🔗</a>Reasons we think LWE is Hard</h1> <p>There are a <em>number</em> of perspectives on the LWE problem for why it is plausibly hard. I’ll describe a few below, where again given my elementary focus I will keep things</p> <ul> <li>at a high level, and</li> <li>non-technical.</li> </ul> <p>This is to say that the rest of this section will be “informal but formalizable”.</p> <p>Before proceeding, I will say that the “real reason” why LWE is thought to be hard is because a lot of people have tried breaking it, and haven’t had much success. In particular, the best attacks take roughly time <span class="math">\(2^{cn}\)</span> for <span class="math">\(c\approx 0.292\)</span>. This is better than RSA (<span class="math">\(2^{\sqrt[3]{n}}\)</span>), and worse than elliptic-curve discrete logarithm (<span class="math">\(2^{n/2}\)</span>).</p> <p>It’s an open secret in cryptography that we can’t have “real reasons” why any particular problem is hard, as it is consistent with what is known in complexity theory that P = NP, and cryptography doesn’t exist. Of course, few cryptographers <em>believe</em> this, but we can’t rule it out! Still, there are a few interesting things you can say at a high-level about the hardness of LWE, so I’ll briefly recount them.</p> <h2 id="it-is-a-problem-of-practical-interest"><a class="zola-anchor" href="#it-is-a-problem-of-practical-interest" aria-label="Anchor link for: it-is-a-problem-of-practical-interest">🔗</a>It is a Problem of Practical Interest</h2> <p>This isn’t a reason to think that LWE is hard <em>per se</em>, but it is a reason that a certain community would be happy if LWE was easy. From this perspective, you could say we have a win-win situation — either cryptographers are happy (LWE is hard), or coding theorists are happy (LWE is easy). Moreover, coding theorists have been trying to solve problems like LWE for a while (because it would be good for them), but have been so far unable to. With that teaser, I’ll go over the applications of LWE to coding theory.</p> <p>Coding theory studies the construction of (among other things) <em>error-correcting codes</em>. It is well-known in coding theory that</p> <ol> <li><em>Random constructions</em> often are “essentially the best” (in terms of amount of information transmitted, amount of errors that can be corrected, etc) among all possible constructions, and</li> <li>Random <em>linear</em> constructions are often essentially as good as <em>random constructions</em>.</li> </ol> <p>This is to say that when I talk about “random linear codes” below, this can be interpreted as “effectively optimal codes”.</p> <p>Anyway, LWE can be interpreted as a problem regarding the decodability of random linear codes. In particular, the problem of going from <span class="math">\((A, A\vec s +\vec e)\)</span> can be interpreted as being given a description of a random linear code (<span class="math">\(A\)</span>), and a noisy encoding of a message (<span class="math">\(A\vec s + \vec e\)</span>), and being asked to decode it to get back <span class="math">\(\vec s\)</span>. Note that this is slightly different than the statement of LWE that I used (which was simply that of <em>distinguishing</em> <span class="math">\((A, A\vec s+\vec e)\)</span> from random — a maybe easier task), but in most settings these problems are known to be equivalent.</p> <p>In summary, if LWE is easy, coding theorists are able to efficiently decode a certain class of effectively optimal codes [<sup class="footnote-reference" id="fr-AWGN-1"><a href="#fn-AWGN">5</a></sup>]. Note that this is different from other cryptographic problems — factoring large numbers is not practically useful outside of attacking cryptographic constructions.</p> <h2 id="gaussian-elimination"><a class="zola-anchor" href="#gaussian-elimination" aria-label="Anchor link for: gaussian-elimination">🔗</a>Gaussian Elimination</h2> <p>As we discussed before, Gaussian Elimination breaks noiseless LWE. How does introducing noise change things?</p> <p>For a (very) brief refresher on Gaussian elimination, you are given some system of equations</p> <span class="math">\[ax+by = e,\qquad a'x+b'y = e',\]</span> <p>and must try to find some solution <span class="math">\((x,y)\)</span> to both of them. Gaussian Elimination proceeds by taking linear combinations of the equations to “cancel out” unknowns. For example, multiplying the first equation by <span class="math">\(a'\)</span>, the second by <span class="math">\(a\)</span>, and subtracting leads to</p> <span class="math">\[(a'b-b'a)y = a'e-e'a.\]</span> <p>Roughly speaking, this can let us “remove a dimension” from an LWE instance, i.e. convert a dimension <span class="math">\(n\)</span> LWE instance to a dimension <span class="math">\(n-1\)</span> LWE instance. Removing this dimension comes at a cost though — the error goes from <span class="math">\(|e|\)</span> to <span class="math">\(|a'e-e'a|\)</span>, which is quite likely larger than <span class="math">\(2|e|\)</span>. This is to say that each dimension we double we (probably) lose a factor 2 in the error, so trying to solve LWE with Gaussian Elimination makes the error grow exponentially, so only works for <em>very</em> small error settings.</p> <h2 id="a-warning-regarding-np-hardness"><a class="zola-anchor" href="#a-warning-regarding-np-hardness" aria-label="Anchor link for: a-warning-regarding-np-hardness">🔗</a>A Warning Regarding NP-hardness</h2> <p>While there are many theoretical justifications for lattice-based encryption, I won’t cover many of them (many are useful “asymptotically but not concretely”). Still, I’ll explicitly discuss the issue of NP-hardness, as it is one that many people casually get wrong.</p> <p>First, it is worth mentioning that NP-hardness is subtly the wrong notion for cryptography. This is because it is a <em>worst-case</em> notion, while cryptography needs <em>average-case</em> hardness. For example, SAT is both</p> <ol> <li>a worst-case hard problem (if you can solve it you can solve any NP problem!)</li> <li>fairly easy to solve on “practical instances”.</li> </ol> <p>This is to say that for many distributions over SAT instances, SAT is actually <em>easy</em>, despite being worst-case hard. This can happen for cryptographic problems as well — for example it is usually rather straightforward to exhibit <em>some</em> prime factors of random integers (half of them are divisible by 2), which breaks the factoring problem (at least the standard way it is formulated).</p> <p>Now, the LWE problem is <em>sort of</em> related to an NP-hard problem. It’s one of the things cryptographers get most excited about (called the “worst-case to average-case reduction”). The “sort of” there carries a lot of weight though, so lets talk about it.</p> <p>There is a problem called <span class="math">\(\mathsf{SVP}_\gamma\)</span> (the Gap Shortest Vector Problem with approximation factor <span class="math">\(\gamma\)</span>). This is a decision problem, where given a matrix <span class="math">\(A\)</span> and number <span class="math">\(t\)</span>, one has to decide if</p> <ol> <li><span class="math">\(\min_{\vec z \in \mathbb{Z}^n\setminus\{0\}} \lVert A\vec z\rVert < t\)</span>, or</li> <li><span class="math">\(\min_{\vec z\in\mathbb{Z}^n\setminus\{0\}} \lVert A\vec z\rVert \geq \gamma t\)</span>.</li> </ol> <p>If <span class="math">\(t \leq \min_{\vec z\in \mathbb{Z}^n\setminus\{0\}} \lVert A\vec z\rVert < \gamma t\)</span>, the algorithm can say whatever (both “true” and “false” count as correct behavior). The idea is roughly as follows — when <span class="math">\(\gamma = 1\)</span>, the algorithm has to <em>perfectly</em> distinguish between the two cases, while for <span class="math">\(\gamma > 1\)</span>, the algorithm only has to distinguish between instances that are “far apart” in a way that is measured by <span class="math">\(\gamma\)</span>. So the problem has gotten easier in some controlled way.</p> <p>Anyway, <span class="math">\(\mathsf{SVP}_1\)</span> is (under some reasonable technical assumptions) NP-hard. With some other technical details omitted, if you can solve LWE you can also solve <span class="math">\(\mathsf{SVP}_{\Omega(\sqrt{n})}\)</span>. Some people mis-interpret this, and say “lattice-based encryption is based on an NP-hard problem”. This is wrong though — formally, <span class="math">\(\mathsf{SVP}_{\Omega(\sqrt{n})}\)</span> is known to be in NP and coNP, so (under even more technical assumptions [<sup class="footnote-reference" id="fr-PHcollapse-1"><a href="#fn-PHcollapse">6</a></sup>]) <em>can’t</em> be NP-hard.</p> <p>There are other caveats about concretely applying the aforementioned connection to <span class="math">\(\mathsf{SVP}_\gamma\)</span> (that have to do with “tightness” of the reduction). Roughly speaking, the issue is one of “hidden constants in <span class="math">\(O(\cdot)\)</span> notation”. For example, <a rel="external" href="https://eprint.iacr.org/2016/360.pdf">one analysis</a> of the aforementioned reduction is that if one can solve LWE in time <span class="math">\(T\)</span>, then one can solve <span class="math">\(\mathsf{SVP}_{\Omega(\sqrt{n})}\)</span> in time <span class="math">\(\approx 2^{500}T\)</span>. It is possible that more efficient reductions are found, but the current ones are mostly asymptotically interesting.</p> <h2 id="quantum-hardness"><a class="zola-anchor" href="#quantum-hardness" aria-label="Anchor link for: quantum-hardness">🔗</a>Quantum Hardness?</h2> <p>Lattice-based encryption (unlike RSA, or the elliptic-curve discrete logarithm problem) is thought to be hard for <em>quantum</em> computers to break as well. I’m no quantum complexity theorist, so I’ll briefly describe the justification I know for the hardness (although, similar to classical hardness, it really boils down to “many smart people have thought about it and failed to get fast algorithms”).</p> <p>Roughly speaking, many quantum algorithms that obtain <em>large</em> speed increases proceed by solving a very general problem (the <a rel="external" href="https://en.wikipedia.org/wiki/Hidden_subgroup_problem">Hidden subgroup problem</a>). In particular, Factoring and the Discrete Logarithm problem can both be phrased as HSP instances and then solved in (quantum) polynomial time. Can lattice-based encryption be phrased as an HSP instance? The answer is (perhaps surprisingly, given the claimed quantum security) <em>yes</em>.</p> <p>HSP instances are parameterzed by a choice of group <span class="math">\(G\)</span>. The HSP instances we know how to solve efficiently are mostly over <em>abelian</em> groups <span class="math">\(G\)</span>. Lattice-based encryption can be phrased as an HSP over a <em>non-abelian</em> group (the Dihedral group). There are some reasons to think this abelian/non-abelian difference should be a structural “switching point” in the complexity of the problem (the HSP heavily relies on the quantum fourier transform, and fourier transforms are <em>much</em> nicer over abelian groups).</p> <p>That all being said, there is nothing stopping someone from giving a quantum attack on LWE tomorrow, similarly to how there’s nothing stopping someone from giving a linear-time algorithm for SAT tomorrow [<sup class="footnote-reference" id="fr-SAT-1"><a href="#fn-SAT">7</a></sup>]. Such uncertainty is a core part of cryptography.</p> <hr /> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-frodoKEM"> <p>For experts, the intention is to present <a rel="external" href="https://frodokem.org/">FrodoKEM</a>, without any of the various compression optimizations, and with a simpler noise distribution. <a href="#fr-frodoKEM-1">↩</a></p> </li> <li id="fn-better"> <p>Small choices of <span class="math">\(e\)</span> can <a rel="external" href="https://en.wikipedia.org/wiki/Coppersmith%27s_attack#Low_public_exponent_attack">lead to attacks</a> (if other mistakes are made as well). <a href="#fr-better-1">↩</a></p> </li> <li id="fn-inequivalent"> <p>It is known that factoring <em>suffices</em> to take <span class="math">\(e\)</span>th roots, but it is not known to be <em>necessary</em>, i.e. there may be faster approaches. The above is (roughly) a simple presentation of the RSA cryptosystem in a “textbook” form. Standard presentations describe in detail how you actually take <span class="math">\(e\)</span>th roots given the factorization <span class="math">\((p, q)\)</span>, but this won’t be needed in my series. <a href="#fr-inequivalent-1">↩</a></p> </li> <li id="fn-estimator"> <p>https://lattice-estimator.readthedocs.io/en/latest/?badge=latest <a href="#fr-estimator-1">↩</a></p> </li> <li id="fn-AWGN"> <p>These codes are for the “Additive White Gaussian Noise Channel”, roughly speaking codes for <em>analog</em> problems (like wireless communications) rather than <em>digital problems</em>. <a href="#fr-AWGN-1">↩</a></p> </li> <li id="fn-PHcollapse"> <p>Formally, it is not NP-hard unless the polynomial hierarchy collapses all the way. <a href="#fr-PHcollapse-1">↩</a></p> </li> <li id="fn-SAT"> <p>See <a rel="external" href="https://cstheory.stackexchange.com/questions/93/what-are-the-best-current-lower-bounds-on-3sat">here</a> for some discussion on explicit bounds on the running time of a SAT algorithm. None preclude an algorithm of both <span class="math">\(O(n)\)</span> running time and space. <a href="#fr-SAT-1">↩</a></p> </li> </ol> </section> 2022-07-02T00:00:00+00:00 2022-07-02T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/lattices-for-programmers-pt2/ <p>In the previous post, we discussed</p> <ol> <li>how “Textbook RSA” (as presented in tutorials) is often not able to be executed for parameters large enough to be plausibly secure,</li> <li>what the LWE problem is, and</li> <li>heuristic justifications for why it is hard.</li> </ol> <p>We’ll next construct an exceedingly simple (private key) lattice-based encryption scheme. Throughout all of our implementions, we will need some basic functions, for example</p> <ul> <li>sampling uniformly random vectors/matrices,</li> <li>sampling vectors/matrices with bounded coordinates,</li> <li>matrix operations (multiplications, transpose, and addition).</li> </ul> <p>These as straightforward to implement, but code is available <a rel="external" href="https://github.com/mark-schultz/lwe-for-programmers/blob/main/utils.py">here</a>. Anyway, in terms of these operations it is straightforward to implement private-key encryption. We first go over the (conceptually) simpler noiseless case, before modifying the construction to handle noise.</p> <h1 id="noiseless-private-key-lattice-based-encryption"><a class="zola-anchor" href="#noiseless-private-key-lattice-based-encryption" aria-label="Anchor link for: noiseless-private-key-lattice-based-encryption">🔗</a>Noiseless (Private-key) Lattice-based Encryption</h1> <p>Given <span class="math">\((A, A\vec s)\)</span>, the (noiseless) LWE assumption states that this is indistinguishable from random. While this assumption is <em>false</em>, we can still pretend it is true, and construct a cryptosystem that would be secure (were it true). The main idea we need to leverage is that</p> <ol> <li>under the noiseless LWE assumption, <span class="math">\((A, A\vec s)\)</span> looks like <span class="math">\((A, \vec u)\)</span> for uniform <span class="math">\(\vec u\)</span>.</li> <li>given a uniform vector <span class="math">\(\vec u\)</span>, one can obtain secure encryption by an argument similar to the one-time pad.</li> </ol> <p>This second point deserves some more discussion. First, one might be tempted define <span class="math">\(\mathsf{Enc}_{\vec s}(\vec m) = (A, A\vec s \oplus \vec m)\)</span>, where <span class="math">\(\oplus\)</span> is an operation such as XOR. While this is a good instinct, it doesn’t work out well here — all of the arithmetic here is (standard) arithmetic modulo <span class="math">\(q\)</span>, which XOR doesn’t behave particularly well with respect to.</p> <p>Another natural idea is to do the same thing, but with addition modulo <span class="math">\(q\)</span> replacing XOR. This ends up working! Namely, under the (false) Noiseless LWE assumption, one can show that <span class="math">\(\mathsf{Enc}_{\vec s}(\vec m) := (A, A\vec s+\vec m)\)</span> defines a secure encryption function. Moreover, one can define decryption such that this function is correct — it is straightforward to verify that <span class="math">\(\mathsf{Dec}_{\vec s}(A, \vec b)= \vec b - A\vec s\)</span> works (where all arithmetic is done modulo <span class="math">\(q\)</span>).</p> <p>This gives (after we have defined the aforementioned matrix/vector/sampling operations) the following simple cryptosystem.</p> <pre class="giallo" style="color: #CDD6F4; background-color: #1E1E2E;"><code data-lang="plain"><span class="giallo-l"><span>import random</span></span> <span class="giallo-l"><span>from utils import *</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>class NoiselessPrivKey:</span></span> <span class="giallo-l"><span> def __init__(self, n, q):</span></span> <span class="giallo-l"><span> self.n = n</span></span> <span class="giallo-l"><span> self.q = q</span></span> <span class="giallo-l"><span> def key_gen(self):</span></span> <span class="giallo-l"><span> self.s = sample_unif_vector(self.n, self.q)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def enc(self, m):</span></span> <span class="giallo-l"><span> # Sampling A</span></span> <span class="giallo-l"><span> A = sample_unif_matrix(n, self.q)</span></span> <span class="giallo-l"><span> # Computing b := As</span></span> <span class="giallo-l"><span> b = matrix_vector_multiply(A, self.s, self.q)</span></span> <span class="giallo-l"><span> # Adding m to As</span></span> <span class="giallo-l"><span> b = vector_vector_add(b, m, self.q)</span></span> <span class="giallo-l"><span> return (A, b)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def dec(self, ctxt):</span></span> <span class="giallo-l"><span> (A, b) = ctxt[0], ctxt[1]</span></span> <span class="giallo-l"><span> # Recomputing As</span></span> <span class="giallo-l"><span> As = matrix_vector_multiply(A, self.s, self.q)</span></span> <span class="giallo-l"><span> # Recovering m = b - As</span></span> <span class="giallo-l"><span> for i in range(self.n):</span></span> <span class="giallo-l"><span> b[i] = (b[i] - As[i]) % q</span></span> <span class="giallo-l"><span> return b</span></span></code></pre> <p>Note that this code works for cryptographically large parameters (which is <span class="math">\(n\in [500,1000]\)</span> roughly and <span class="math">\(q\approx 2^{10}\)</span>). This is because lattice-based encryption is essentially just simple linear algebra on word-size integers [<sup class="footnote-reference" id="fr-RING-1"><a href="#fn-RING">1</a></sup>]! With this idea for how lattice-based encryption should work, we can re-introduce the noise, and fix the issues it brings about.</p> <h1 id="noisey-private-key-lattice-based-encryption"><a class="zola-anchor" href="#noisey-private-key-lattice-based-encryption" aria-label="Anchor link for: noisey-private-key-lattice-based-encryption">🔗</a>Noisey (Private-key) Lattice-based Encryption</h1> <p>Introducing noise both doesn’t impact the security of our construction at all, and changes everything. This is because the natural noisy variant of our protocol (where we add <span class="math">\(\vec m\)</span> to the second component of <span class="math">\((A, A\vec s +\vec e)\)</span>, rather than <span class="math">\((A, A\vec s)\)</span>) is secure under the LWE assumption (similarly to how the prior version of our protocol can be shown to be secure under the noiseless LWE assumption). The main difference is that the noiseless LWE assumption is clearly false, but the LWE assumption is thought to be true, i.e. we now have a secure protocol!</p> <p>The issue we have to fix is correctness. If we use the above modified encryption algorithm with our prior decryption algorithm, we get that</p> <span class="math">\[\mathsf{Dec}_{\vec s}(\mathsf{Enc}_{\vec s}(\vec m)) = \mathsf{Dec}_{\vec s}(A, A\vec s + \vec e +\vec m) = \vec m + \vec e\neq \vec m.\]</span> <p>From this, we can see that the error being zero was key to the correctness of our prior (insecure) protocol. Can we fix this correctness issue somehow?</p> <p>The answer is (obviously) yes. The solution is straightforward — we need a way to encode the message <span class="math">\(\vec m\)</span> to tolerate additional noise <span class="math">\(\vec e\)</span>. While one can appeal to fancy techniques from coding theory, typically people just scale <span class="math">\(\vec m\mapsto (q/2)\vec m\)</span> (when the coordinates of <span class="math">\(\vec m\in\{0,1\}^n\)</span>), or <span class="math">\((q/p)\vec m\)</span> for <span class="math">\(\vec m\in\{0,\dots,p-1\}^n\)</span> more generally.</p> <p>This is to say that encryption of the form</p> <span class="math">\[\mathsf{Enc}_{\vec s}(\vec m) = (A, A\vec s +\vec e + (q/2)\vec m)\]</span> <p>suffices. For decryption, we again subtract <span class="math">\(A\vec s\)</span> from the second component, but then need a way to map <span class="math">\((q/2)\vec m+\vec e\mapsto \vec m\)</span>. To do this, we first divide by <span class="math">\(q/2\)</span> to get <span class="math">\(\vec m + \vec e / (q/2)\)</span>, and then round to the nearest integer. Provided the coordinates of <span class="math">\(\vec e / (q/2)\)</span> are small enough (concretely less than <span class="math">\(1 / 2\)</span>, or equivalently the coordinates of <span class="math">\(\vec e\)</span> must have absolute value at most <span class="math">\(q/4\)</span>), this will be correct, and we’ll be done!</p> <p>This leads to the (slightly different) code than before</p> <pre class="giallo" style="color: #CDD6F4; background-color: #1E1E2E;"><code data-lang="plain"><span class="giallo-l"><span>import random</span></span> <span class="giallo-l"><span>from utils import *</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>class LWEPrivKey:</span></span> <span class="giallo-l"><span> def __init__(self, n, q, B):</span></span> <span class="giallo-l"><span> self.n = n</span></span> <span class="giallo-l"><span> self.q = q</span></span> <span class="giallo-l"><span> self.B = B</span></span> <span class="giallo-l"><span> def key_gen(self):</span></span> <span class="giallo-l"><span> self.s = sample_unif_vector(self.n, self.q)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def enc(self, m):</span></span> <span class="giallo-l"><span> # Sampling A, e</span></span> <span class="giallo-l"><span> A = sample_unif_matrix(n, self.q)</span></span> <span class="giallo-l"><span> e = sample_bounded_vector(self.n, self.B)</span></span> <span class="giallo-l"><span> # Computing b := As + e</span></span> <span class="giallo-l"><span> b = matrix_vector_multiply(A, self.s, self.q)</span></span> <span class="giallo-l"><span> b = vector_vector_add(b, e, self.q)</span></span> <span class="giallo-l"><span> # Scaling m -&gt; (q//2) m</span></span> <span class="giallo-l"><span> scaled_m = [(self.q//2) * m[i] % self.q for i in range(self.n)]</span></span> <span class="giallo-l"><span> # Adding (q//2)m to b = As + e</span></span> <span class="giallo-l"><span> b = vector_vector_add(b, scaled_m, self.q)</span></span> <span class="giallo-l"><span> return (A, b)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def dec(self, ctxt):</span></span> <span class="giallo-l"><span> (A, b) = ctxt[0], ctxt[1]</span></span> <span class="giallo-l"><span> # Recomputing As</span></span> <span class="giallo-l"><span> As = matrix_vector_multiply(A, self.s, self.q)</span></span> <span class="giallo-l"><span> # Recovering scaled_m = b - As</span></span> <span class="giallo-l"><span> for i in range(self.n):</span></span> <span class="giallo-l"><span> b[i] = (b[i] - As[i]) % q</span></span> <span class="giallo-l"><span> # Scaling (q//2)m + e -&gt; m</span></span> <span class="giallo-l"><span> m = [0 for _ in range(self.n)]</span></span> <span class="giallo-l"><span> for i in range(self.n):</span></span> <span class="giallo-l"><span> m[i] = round(b[i] / (self.q//2)) % 2</span></span> <span class="giallo-l"><span> return m</span></span></code></pre> <p>This gives us a complete implementation of (private-key) lattice-based encryption! There is technically a security issues with this code (it uses pythons default RNG), but this is simple enough to fix. Note that, even if one uses a secure RNG, you probably don’t want to use this in production — there are <em>much</em> more efficient options if one uses the optimizations we are (intentionally) omitting.</p> <hr /> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-RING"> <p>Here, “integers” is somewhat particular to FrodoKEM. Optimized schemes (based on structured lattices) such as Saber and Kyber instead do “linear algebra” with vectors/matrices of <em>polynomials</em>. <a href="#fr-RING-1">↩</a></p> </li> </ol> </section> 2022-07-02T00:00:00+00:00 2022-07-02T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/lattices-for-programmers-pt3/ <p>This is a continuation on my series on a simple construction of (public-key) lattice-based encryption. We’re finally ready to construct public-key encryption! Similarly to the private-key setting, we will first give a construction in the (simpler, but insecure) <em>noiseless</em> setting, before adapting the construction to handle noise.</p> <h1 id="noiseless-public-key-lattice-based-encryption"><a class="zola-anchor" href="#noiseless-public-key-lattice-based-encryption" aria-label="Anchor link for: noiseless-public-key-lattice-based-encryption">🔗</a>Noiseless (public-key) Lattice-based Encryption</h1> <p>For private-key encryption, our construction was easiest to understand in terms of the one-time pad. For public-key encryption, things are instead easiest in terms of <em>Diffie-Hellman Key Exchange</em>. Recall that, for a fixed element <span class="math">\(g\in \mathbb{F}_p\)</span>, this works by</p> <ol> <li>sampling <span class="math">\(r, s\gets \mathbb{F}_p\)</span> (technically <span class="math">\(\mathbb{Z}/(p-1)\mathbb{Z}\)</span>, but ignore this detail),</li> <li>exchanging <span class="math">\(g^r, g^s\)</span>, and then</li> <li>having each party compute <span class="math">\((g^r)^s = g^{rs} = (g^s)^r\)</span>.</li> </ol> <p>Diffie-Hellman works by computing <span class="math">\(g^{rs}\)</span> in two ways, where eacy way involves the “secret key” (<span class="math">\(r\)</span> or <span class="math">\(s\)</span>) and “public key” (<span class="math">\(g^r\)</span> or <span class="math">\(g^s\)</span>). For the (noiseless) LWE assumption, our secret key is <span class="math">\(\vec s\)</span>, and public key is <span class="math">\(A\vec s\)</span>. Can we use this to “compute something in two ways” again?</p> <p>The answer is yes! In particular, we will compute <span class="math">\(\vec r^tA\vec s\)</span> in two ways, namely via <span class="math">\(\vec r^t(A\vec s)\)</span>, and <span class="math">\((\vec r^t A)\vec s\)</span>. This means that we need the noiseless LWE sample <span class="math">\((A, A\vec s)\)</span>, and the <em>transposed</em> sample <span class="math">\((A^t, A^t\vec r)\)</span> (as <span class="math">\((A^t\vec r)^t = \vec r^t A\)</span>). One of them we will store long-term (in the public key), while the other we will generate “fresh” (and store in the ciphertext). Without further ado, we have the following (noiseless) public-key cryptosystem</p> <pre class="giallo" style="color: #CDD6F4; background-color: #1E1E2E;"><code data-lang="plain"><span class="giallo-l"><span>import random</span></span> <span class="giallo-l"><span>from utils import *</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>class NoiselessPubKey:</span></span> <span class="giallo-l"><span> def __init__(self, n, q):</span></span> <span class="giallo-l"><span> self.n = n</span></span> <span class="giallo-l"><span> self.q = q</span></span> <span class="giallo-l"><span> def key_gen(self):</span></span> <span class="giallo-l"><span> A = sample_unif_matrix(self.n, self.q)</span></span> <span class="giallo-l"><span> self.sk = sample_unif_vector(self.n, self.q)</span></span> <span class="giallo-l"><span> As = matrix_vector_multiply(A, self.sk, self.q)</span></span> <span class="giallo-l"><span> self.pk = (A, As)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def enc(self, m):</span></span> <span class="giallo-l"><span> (A, As) = self.pk</span></span> <span class="giallo-l"><span> # Sampling r</span></span> <span class="giallo-l"><span> r = sample_unif_vector(self.n, self.q)</span></span> <span class="giallo-l"><span> # Computing u := A^tr</span></span> <span class="giallo-l"><span> At = matrix_transpose(A)</span></span> <span class="giallo-l"><span> u = matrix_vector_multiply(At, r, self.q)</span></span> <span class="giallo-l"><span> # Computing r^t(As)</span></span> <span class="giallo-l"><span> rAs = vector_vector_inner_product(r, As, self.q)</span></span> <span class="giallo-l"><span> # Using r^t(As) as a random pad</span></span> <span class="giallo-l"><span> v = (rAs + m) % self.q</span></span> <span class="giallo-l"><span> return (u, v)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def dec(self, ctxt):</span></span> <span class="giallo-l"><span> (u, v) = ctxt[0], ctxt[1]</span></span> <span class="giallo-l"><span> s = self.sk</span></span> <span class="giallo-l"><span> # Computing (r^tA)s</span></span> <span class="giallo-l"><span> rAs = vector_vector_inner_product(u, s, self.q)</span></span> <span class="giallo-l"><span> # Subtracting rAs from v</span></span> <span class="giallo-l"><span> m = (v - rAs) % q</span></span> <span class="giallo-l"><span> return m</span></span></code></pre> <p>So ciphertexts contain the term <span class="math">\(v = \vec r^tA\vec s + m\)</span>, and the share <span class="math">\((A^t, A^t\vec r)\)</span> needed to decrypt. Actually, we can fully omit <span class="math">\(A^t\)</span> (as it already in the public key) for some small space savings.</p> <p>While encryption/decryption are quite fast for this scheme (well under <span class="math">\(.10\)</span> seconds), it has a <em>significant</em> flaw, namely that it can only encrypt a <em>single</em> element from <span class="math">\(\mathbb{Z}/q\mathbb{Z}\)</span>, which for <span class="math">\(q\approx 3000\)</span> is only like 12 bits. An (unoptimized) way to fix this is to simply break your plaintext up into 12-bit long blocks, and encrypt multiple things. This introduces quite a bit of redundency that one can optimize away (and this is precisely what FrodoKEM does compared to what I have presented), but I will defer details of this to later. The main idea is to use the above template scheme, but with many of the matrices/vectors “resized” (i.e. with certain parameters tuned).</p> <p>Well, one shouldn’t use <em>exactly</em> the above scheme, as it isn’t secure! With that, we’ll discuss how one can make it secure.</p> <h1 id="noisy-public-key-lattice-based-encryption"><a class="zola-anchor" href="#noisy-public-key-lattice-based-encryption" aria-label="Anchor link for: noisy-public-key-lattice-based-encryption">🔗</a>Noisy (public-key) Lattice-based Encryption</h1> <p>We again switch every usage of <span class="math">\((A, A\vec s)\)</span> to <span class="math">\((A, A\vec s + \vec e)\)</span>. This is fairly mechanical (and simple to verify that you got right), but as this verification step is somewhat technical (write down a security proof!) I will omit it.</p> <pre class="giallo" style="color: #CDD6F4; background-color: #1E1E2E;"><code data-lang="plain"><span class="giallo-l"><span>import random</span></span> <span class="giallo-l"><span>from utils import *</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>class LWEPubKey:</span></span> <span class="giallo-l"><span> def __init__(self, n, q, B):</span></span> <span class="giallo-l"><span> self.n = n</span></span> <span class="giallo-l"><span> self.q = q</span></span> <span class="giallo-l"><span> self.B = B</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def key_gen(self):</span></span> <span class="giallo-l"><span> A = sample_unif_matrix(self.n, self.q)</span></span> <span class="giallo-l"><span> self.sk = sample_bounded_vector(self.n, self.B)</span></span> <span class="giallo-l"><span> As = matrix_vector_multiply(A, self.sk, self.q)</span></span> <span class="giallo-l"><span> e = sample_bounded_vector(self.n, self.B)</span></span> <span class="giallo-l"><span> b = vector_vector_add(As, e, self.q)</span></span> <span class="giallo-l"><span> self.pk = (A, b)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def enc(self, m):</span></span> <span class="giallo-l"><span> (A, b) = self.pk</span></span> <span class="giallo-l"><span> # Sampling r</span></span> <span class="giallo-l"><span> r = sample_bounded_vector(self.n, self.B)</span></span> <span class="giallo-l"><span> e_prime = sample_bounded_vector(self.n, self.B)</span></span> <span class="giallo-l"><span> # Computing u := A^tr + e&#39;</span></span> <span class="giallo-l"><span> At = matrix_transpose(A)</span></span> <span class="giallo-l"><span> u = matrix_vector_multiply(At, r, self.q)</span></span> <span class="giallo-l"><span> u = vector_vector_add(u, e_prime, self.q)</span></span> <span class="giallo-l"><span> # Approximately computing r^t(As)</span></span> <span class="giallo-l"><span> rAs = vector_vector_inner_product(r, b, self.q)</span></span> <span class="giallo-l"><span> # Adding e_double_prime so it is a &quot;LWE encryption&quot;</span></span> <span class="giallo-l"><span> e_prime_prime = sample_bounded_vector(1, self.B)[0]</span></span> <span class="giallo-l"><span> v = rAs+ e_prime_prime % self.q</span></span> <span class="giallo-l"><span> # Adding an encoding of m to the &quot;random pad&quot;</span></span> <span class="giallo-l"><span> v = (v + (self.q//2)*m) % self.q</span></span> <span class="giallo-l"><span> return (u, v)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> def dec(self, ctxt):</span></span> <span class="giallo-l"><span> (u, v) = ctxt[0], ctxt[1]</span></span> <span class="giallo-l"><span> s = self.sk</span></span> <span class="giallo-l"><span> # Approximately (r^tA)s</span></span> <span class="giallo-l"><span> rAs = vector_vector_inner_product(u, s, self.q)</span></span> <span class="giallo-l"><span> # Subtracting rAs from v</span></span> <span class="giallo-l"><span> v = (v - rAs) % self.q</span></span> <span class="giallo-l"><span> # Decoding (q//2)*m + error_terms</span></span> <span class="giallo-l"><span> return round(v / (self.q//2)) % 2</span></span></code></pre> <p>One can compute that the scheme is correct provided that <span class="math">\(\lfloor (\langle sk, e'\rangle + \langle r, e\rangle + e'') / (q//2)\rceil =0\)</span>. If we choose <span class="math">\(sk, \vec r\)</span> as they were before (uniformly at random from <span class="math">\(\mathbb{Z}/q\mathbb{Z}\)</span>, this will barely ever happen. Instead, we use the known result that one loses no security if one samples the LWE secret from the same distribution as the LWE error to sample <span class="math">\(\vec{sk}, \vec r\)</span> with each coefficient at most <span class="math">\(B\)</span>. This yields correct encryption provided that <span class="math">\(2nB^2 + B < q/4\)</span>. In practice you’ll get some cancellations between the random terms, but I won’t bother giving a refined analysis — the intention is to point out that one can still get correct encryption, although in the public key setting you have to correct more error than the private key setting.</p> <p>Note that the noisy setting can encrypt even less useful data than the noiseless setting — a single element of <span class="math">\(\{0,1\}\)</span> now, rather than <span class="math">\(\mathbb{Z}/q\mathbb{Z}\)</span>. Again, this is merely an issue with the unoptimized scheme, and one can easily optimize things to encrypt more data.</p> <h1 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion">🔗</a>Conclusion</h1> <p>With that, we’ve constructed public-key lattice-based encryption! The construction is an (aggressively unoptimized) version of the FrodoKEM cryptosystem, a NIST PQC round 3 (finalist/alternate, I forget). The main optimizations we are missing out on are</p> <ol> <li>the aforementioned technique of encrypting more than just a single bit, and</li> <li>a technique to compress the random matrix <span class="math">\(A\)</span> in the public key (which is <span class="math">\(n^2\log_2 q\)</span> bits large — fairly big) to a short 128-bit seed, and</li> <li>optimizing the choice of bounded error distribution to be a <em>concentrated</em> bounded error distribution (think a Gaussian), allowing one to get tighter bounds on the noise one must correct.</li> </ol> <p>Everything else is essentially the same. Note that even among NIST candidates, FrodoKEM is an intentionally “unoptimized” one, meaning that it intentionally uses weaker security assumptions (at some cost in potential optimizations). The (main) optimization it misses out on is that of appealing to <em>algebraically structured lattices</em>. I might write up a blog post on this topic, but it is fairly advanced in comparison, as it most naturally phrased in terms of modules over quotients of polynomial rings. While this topics are very possible to concretely work with (i.e. they aren’t “abstract nonsense”), it is decidedly more advanced than simple matrix multiplications and modular arithmetic.</p> <hr /> 2021-03-05T00:00:00+00:00 2021-03-05T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/primelattice/ <p>Earlier this week, a <a rel="external" href="https://eprint.iacr.org/2021/232">paper was posted on eprint</a> claiming to be authored by <a rel="external" href="https://en.wikipedia.org/wiki/Claus_P._Schnorr">Schnorr</a>, and claiming (in the abstract at least) to break RSA. The paper itself appears to have been a work-in-progress (a version of it made the <a rel="external" href="https://twitter.com/StefanoMTessaro/status/1366914461886386176?s=20">Crypto 2009 rump session</a>), and some comments by <a rel="external" href="https://twitter.com/DucasLeo/status/1366957088954601473?s=20">Ducas</a> (who is an expert on lattices) make it sound like a known strategy that doesn’t seem to work out (he pointed to the paper <a rel="external" href="https://arxiv.org/pdf/1003.5461.pdf">here</a> from 2010, and also <a rel="external" href="https://github.com/lducas/SchnorrGate">experimentally tested some of the claims</a>, demonstrating a gap between Schnorr’s claim and experimental evidence). Of course, this does not mean that RSA is completely safe (see <a rel="external" href="https://twitter.com/matthew_d_green/status/1366961403161174021?s=20">Heninger’s comments on the matter</a>), but it does dampen much of the excitement.</p> <p>As I am giving a talk to the Crypto student seminar on Friday, I leafed through the paper before deciding that I would not be able to present it well (both due to the short notice, and the fairly technical nature of the claimed result). In doing this, I saw a lattice which is quite similar to one from <a rel="external" href="https://eprint.iacr.org/2018/146.pdf">one of Ducas and Pierrot’s (DP) papers</a>. The lattice itself seems to be defined in my advisor’s book from 2002, but I can’t seem to get access to a digital copy via institutional access.</p> <p>In this post I will try to discuss both lattices, although I will focus on DP’s contruction (which I understand better). Both are related to the complexity of factoring — Schnorr’s is designed such that short vectors help <em>break</em> factoring, while DP’s is designed such that the ability to <em>break factoring</em> allows one to solve the “Bounded Distance Decoding” problem (which is a typical hard problem on lattices). In what follows, I will focus on the <span class="math">\(\ell_2\)</span> norm (although DP also has results for the <span class="math">\(\ell_1\)</span> norm). They are also both related to “logarithms of primes”, although the Schnorr construction uses the (standard) logarithm, while DP uses a <em>discrete</em> logarithm.</p> <h1 id="lattice-preliminaries"><a class="zola-anchor" href="#lattice-preliminaries" aria-label="Anchor link for: lattice-preliminaries">🔗</a>Lattice Preliminaries</h1> <p>A <em>lattice</em> is a discrete subgroup <span class="math">\(\mathcal{L}\subseteq \mathbb{Z}^n\)</span>. Equivalently, it is the image <span class="math">\(\mathbf{B}\mathbb{Z}^k\)</span> of the integers under <span class="math">\(\mathbf{B}\in\mathbb{R}^{n\times k}\)</span> (a <em>basis</em> of the lattice). Here, <span class="math">\(n\)</span> is the dimension, and <span class="math">\(k\)</span> is the rank. Generally, one restricts to “full-rank” lattices, i.e. where <span class="math">\(k = n\)</span>.</p> <p>An important quantity associated with a lattice is the length of its <em>shortest vector</em> <span class="math">\(\lambda_1(L) = \min_{\ell\in L\setminus\{0\}}\lVert \ell\rVert_2\)</span>. One can place balls of radii <span class="math">\(\lambda_1(L) / 2\)</span> centered at each lattice point in a non-overlapping way. This means that information theoretically, one can recover <span class="math">\(\ell\)</span> from a perturbation <span class="math">\(\ell+e\)</span> for <span class="math">\(e\)</span> in one of the aforementioend balls. The computational problem of doing this is known as the <em>bounded-distance decoding problem</em>, and being able to solve it efficiently on input a random lattice already suffices to solve the LWE problem (in fact, the LWE problem can be interpreted as doing precisely this).</p> <p>From a different perspective, lattices with “large <span class="math">\(\lambda_1(L)\)</span>” can correct “larger amounts of noise”, and have applications in the construction of error-correcting codes. There is a boring way to get a lattice with <em>arbitrarily large</em> <span class="math">\(\lambda_1(L)\)</span> though, which is based on the identity:</p> <span class="math">\[\lambda_1(\alpha L) = \alpha\lambda_1(L)\]</span> <p>where <span class="math">\(\alpha L\)</span> is the same lattice <span class="math">\(L\)</span> with its basis scaled by <span class="math">\(\alpha\)</span>. For this reason, when trying to build “dense” lattices people often work with a scale-invariant quantity related to <span class="math">\(\lambda_1(L)\)</span>. There are a variety you can choose<sup class="footnote-reference" id="fr-centerdensity-1"><a href="#fn-centerdensity">1</a></sup>, but I will discuss the normalized shortest vector:</p> <span class="math">\[\overline{\lambda}_1(L) = \frac{\lambda_1(L)}{\sqrt[n]{\det L}}\]</span> <p>A fundamental result in the theory of lattices is <em>Minkowski’s Theorem</em>, which upper bounds <span class="math">\(\overline{\lambda}_1(L) \leq 2\mathsf{vol}(\mathcal{B}_n(1))^{-1/n}\sim \sqrt{2n/\pi e}\)</span>. There is a natural question — how <em>close</em> can one get to this bound? Read DP for a summary, but the main idea is that random lattices are within a constant factor of the bound, and that there exist explicit constructions that are similarly close (but that the decoding problem is open for).</p> <p>Before DP, the highest-density lattice one could decode was the Barnes-Wall lattice of density <span class="math">\(\Theta(\sqrt[4]{n})\)</span>, with decoder by Micciancio-Nicolosi. The lattice of DP can be decoded in <span class="math">\(\Theta(\sqrt{n}/\log n)\)</span> time, which is a large improvement.</p> <h1 id="the-lattices-of-schnorr-and-dp"><a class="zola-anchor" href="#the-lattices-of-schnorr-and-dp" aria-label="Anchor link for: the-lattices-of-schnorr-and-dp">🔗</a>The Lattices of Schnorr and DP</h1> <h2 id="schnorr"><a class="zola-anchor" href="#schnorr" aria-label="Anchor link for: schnorr">🔗</a>Schnorr</h2> <p>Throughout this, let <span class="math">\(p_1,\dots, p_n\)</span> be distinct primes. Schnorr introduces<sup class="footnote-reference" id="fr-adleman-1"><a href="#fn-adleman">2</a></sup> the lattice with basis:</p> <span class="math">\[B_{n, c} = \begin{pmatrix} \sqrt{\ln p_1}& 0 & 0\\ 0 & \vdots & 0\\ 0 & 0 & \sqrt{\ln p_n}\\ N^c\ln p_1 & \dots & N^c \ln p_n \end{pmatrix}\]</span> <p>Here <span class="math">\(B_{N, c}\in\mathbb{R}^{(n+1)\times n}\)</span>, and <span class="math">\(N = \prod_i p_i\)</span>. Schnorr then claims that solving the closest vector problem on this lattice with target point <span class="math">\(t = e_{n+1}N^c\ln N\)</span> allows one to construct a triple <span class="math">\[(u, v, |u - vN|)\]</span> that is <span class="math">\(p_n\)</span>-smooth, meaning each member of the triple has all prime divisors <span class="math">\(\leq p_n\)</span>.</p> <p>Given <span class="math">\(n+1\)</span> triples <span class="math">\((u_i,v_i, |u_i-v_iN|)\)</span>, one proceeds as follows. By smoothness, we have that:</p> <span class="math">\[u_j = \prod_i p_i^{e_{i,j}},\qquad u_j-v_jN = \prod_i p_i^{e_{i,j}'}\]</span> <p>It is simple to see that <span class="math">\((u_j-v_jN)/u_j\equiv 1\bmod N\)</span>, so <span class="math">\[\prod_i p_i^{e_{i,j}-e_{i,j}'}\equiv 1\bmod N\]</span>. Any solution <span class="math">\(t_1,\dots,t_{n+1}\in\{0,1\}\)</span> which solves:</p> <span class="math">\[\sum_i t_i(e_{i,j}-e_{i,j}')\equiv 0\bmod 2\]</span> <p>leads to a solution of <span class="math">\(X^2 - 1\equiv 0\bmod N\)</span>, which (provided such <span class="math">\[X\neq \pm 1\bmod N\]</span>) yields two non-trivial factors of <span class="math">\(N\)</span>. In particular, one sets:</p> <span class="math">\[X = \prod{i = 1}^{n+1}p_i^{\frac{1}{2}\sum_{j = 1}^{n+1} t_j (e_{i,j} - e_{i,j}')}\]</span> <p>One can see similarities between this and the <a rel="external" href="https://en.wikipedia.org/wiki/Quadratic_sieve">quadratic sieve</a>, although they do not appear to be the same (and the quadratic sieve does not appear to have anything to do with lattices).</p> <p>Concretely, the above approach <em>does</em> work, and Leo Ducas has explicit experiments showing parameter ranges that it works in. But these parameters are much worse than Schnorr claims (from a cryptanalytic point of view at least).</p> <h2 id="dp"><a class="zola-anchor" href="#dp" aria-label="Anchor link for: dp">🔗</a>DP</h2> <p>The lattice of DP is broadly similar, although it is now defined in terms of <em>discrete</em> logarithms of primes, rather than traditional logarithms (there are of course other differences as well). It fixes <span class="math">\(n, m\in\mathbb{N}\)</span> such that <span class="math">\((\mathbb{Z}/m\mathbb{Z})^*\)</span> is a cyclic group<sup class="footnote-reference" id="fr-cyclic-1"><a href="#fn-cyclic">3</a></sup>, and sets:</p> <span class="math">\[\mathcal{F} = \{p\in\mathbb{N}\mid p\text{ prime}, p\nmid m, p \leq B\}\]</span> <p>where <span class="math">\(B\)</span> is chosen such that <span class="math">\(|\mathcal{F}| = n\)</span>. Easy arguments show that <span class="math">\(B\sim n\log n\)</span>.</p> <p>Now, consider the group homomorphism:</p> <span class="math">\[\psi : \mathbb{Z}^n \to (\mathbb{Z}/m\mathbb{Z})^*,\qquad (x_1,\dots,x_n)\mapsto \prod_{i = 1}^n p_i^{x_i}\bmod m\]</span> <p>The kernel of this is a subgroup of <span class="math">\(\mathbb{Z}^n\)</span>, and therefore a lattice. One can write it as:</p> <span class="math">\[\mathcal{L} = \{(x_1,\dots,x_n) \in\mathbb{Z}^n \mid \prod_i p_i^{x_i}\equiv 1\bmod m\} = \{(x_1,\dots,x_n)\in\mathbb{Z}^n \mid \sum_{i = 1}^n x_i\log_g p_i\equiv 0\bmod \varphi(m)\}\]</span> <p>Here, <span class="math">\(\log_g p_i\)</span> is the <em>discrete log</em> of <span class="math">\(p_i\)</span> within the group <span class="math">\((\mathbb{Z}/m\mathbb{Z})^*\)</span>. Lattice cryptographers have a compact way to describe lattices of the above form, namely via the notation:</p> <span class="math">\[\Lambda_q^\perp(g) = \{\vec x\in\mathbb{Z}^n\mid \langle \vec x,\vec g\rangle\equiv 0\bmod q\}\]</span> <p>where <span class="math">\(q\)</span> need not be prime. One might call this a “rank 1 <span class="math">\(q\)</span>-ary lattice”. Such lattices satisfy a sort of universality property, in that one can approximate a large class of lattices from lattices of this type<sup class="footnote-reference" id="fr-approximate-1"><a href="#fn-approximate">4</a></sup>.</p> <p>From this description, it is fairy standard to write down an explicit basis:</p> <span class="math">\[\begin{pmatrix} 1 & 0 & \dots &0& 0\\ 0 & 1 & \dots &0& 0\\ \vdots & & \ddots & \vdots\\ 0 & 0 & \dots & 1 & 0\\ -\frac{\log_g p_1}{\log_g p_n} & -\frac{\log_g p_2}{\log_g p_n} &\dots & -\frac{\log_g p_{n-1} }{\log_g p_{n}} & \varphi(m) \end{pmatrix}\]</span> <p>Besides the last</p> <p>Anyway, DP writes down an explicit (up to determining <span class="math">\(\mathcal{F}\)</span>) lattice, and shows that:</p> <ol> <li>It is quite dense</li> <li>One can efficiently decode it</li> </ol> <p>The first point is not that surprising — there are various results that essentially say that <span class="math">\(\log_g t\)</span> is distributed “randomly” (where I mean this in an informal way) within <span class="math">\((\mathbb{Z}/m\mathbb{Z})^*\)</span>. As random lattices tend to have good density properties, it is perhaps not surprising that a “random ish” lattice has good density (as actually random<sup class="footnote-reference" id="fr-random-1"><a href="#fn-random">5</a></sup> lattices get within a <em>constant</em> factor of Minkowski’s bound). For this reason, I will focus on presenting the decoding algorithm. The proof of correctness of the decoding algorithm itself can be seen as a proof that <span class="math">\(\lambda_1(L)\)</span> is not too small.</p> <p>DP frame their results in terms of the <span class="math">\(\ell_1\)</span> norm, and then use the generic inequality <span class="math">\(\lVert x\rVert_1 \leq \sqrt{n}\lVert x\rVert_2\)</span>. I will mimic the presentation in their paper, where they first discuss the <span class="math">\(\ell_1\)</span> norm where the errors are <em>positive</em> (i.e. in <span class="math">\(\mathbb{N}^n\)</span>), before extending to arbitrary (discrete) errors.</p> <p>Say that you are given a perturbed lattice vector <span class="math">\(t = x + e\)</span>. Then, one can compute:</p> <span class="math">\[\prod_{i = 1}^n p_i^{t_i} = \prod_{i = 1}^n p_i^{x_i}\prod_{i = 1}^n p_i^{e_i}\equiv \prod_{i = 1}^n p_i^{e_i}\bmod m\]</span> <p>This essentially the same as “syndrome decoding” if you are familiar with that. One then has that <span class="math">\(\prod_{i = 1}^n p_i^{e_i} \leq B^{\sum_i e_i} = B^{r}\)</span></p> <p>where <span class="math">\(r\)</span> is an <span class="math">\(\ell_1\)</span> upper bound on the size of decodable errors. As long as this quantity does not “overflow” (i.e. as long as <span class="math">\(B^r < m\)</span>), everything works great. This is because <span class="math">\(p_i^{e_i} < m\)</span> is a smooth number, so one can efficiently factor it, and recover the <span class="math">\(e_i\)</span>’s. This leads to an <span class="math">\(\ell_1\)</span> bound of <span class="math">\(r < \frac{\log_2 m}{\log_2 B}\)</span>.</p> <p>For arbitrary errors, the above decomposition no longer works. This is because one has that:</p> <span class="math">\[f = \prod_i p_i^{t_i} \equiv \prod_{i : e_i > 0}p_i^{e_i} \prod_{i : e_i <0} p_i^{e_i}\equiv u/v\bmod m\]</span> <p>Is now a “fraction”. One can recover this fraction by looking at the two-dimensional lattice <span class="math">\[x - fy\]</span>, where <span class="math">\(x, y\in\mathbb{Z}\)</span>. Under certain conditions, the shortest vector <span class="math">\((x, y)\)</span> in this lattice is precisely <span class="math">\((u, v)\)</span>. So recovering <span class="math">\((u, v)\)</span> can be reduced to a 2-dimensional SVP instance, which is efficient to compute. The “certain conditions” mean that the decoding bound is now <span class="math">\(r < \frac{\log_2(m/2)}{2\log_2(B)}\)</span>, but this is essentially the same as before.</p> <h1 id="extensions"><a class="zola-anchor" href="#extensions" aria-label="Anchor link for: extensions">🔗</a>Extensions?</h1> <p>If one examines the equations, <span class="math">\(B\sim n\log n\)</span> is required to get <span class="math">\(n\)</span> distinct primes in the construction, which by the prime number theorem requires that <span class="math">\(B\sim n\log n\)</span>. The <span class="math">\(\ln n\)</span> factor away from optimal in this construction is implicitly a <span class="math">\(\ln B\)</span> factor. So there is a natural question — can one generalize this construction to allow for smaller <span class="math">\(B\)</span>, to close this gap?</p> <p>I personally suspect that the answer is no. I thought about it a bit two years ago, by appealing to generalizations of the notion of primality. Essentially, if one was working in a system that had “many more primes” than one would expect, it is possible that one could port over this argument (say if one could take <span class="math">\(B\sim \ln n\)</span>). This may seem like nonsense at first, but recall that <span class="math">\(B\)</span> is a norm bound on some objects, so while within a “1D” object like <span class="math">\(\mathbb{Z}\)</span> there are only at most <span class="math">\(\ln n\)</span> elements of norm below <span class="math">\(\ln n\)</span>, in a higher dimensional object there can be many more.</p> <p>Unfortunately, I was not able to make this strategy work. The field of <a rel="external" href="https://en.wikipedia.org/wiki/Abstract_analytic_number_theory">abstract analytic number theory</a> studies such “generalized primes”, and proves things like prime number theorems for them. I was unable to find a reasonable system that had dense enough primes (I initially was hoping working over number fields would work, and only searched for more abstract things after this failed).</p> <p>Since then, I like to think of the construction more in terms of “de-randomizing” a construction that meets Minkowski’s bounds by (implicitly) combining:</p> <ol> <li>The fact that <span class="math">\(\log_g p_i\)</span> is “pseudorandom”, so <span class="math">\[\Lambda_m^\perp((\log_g p_i))\]</span> is “pseudorandom” as well.</li> <li>That rank 1 <span class="math">\(q\)</span>-ary lattices approximate arbitrary lattices fairly well, so <span class="math">\(\Lambda_m^\perp((\log_g p_i))\)</span> is “fairly close” to a random general lattice</li> <li>That random (general) lattices are within constant factors of Minkowski’s bound</li> </ol> <p>It would be interesting to explore this point of view further, and see if the construction can be formalized in these terms. In particular, it would be interesting to see where one loses a <span class="math">\(\ln n\)</span> factor in the above steps. I would guess step 2, but I do not have a great understanding of how strong the approximation one gets is quantitatively (it is of course stated in the papers I discussed, but it is generally stated in terms of approximating the entries of the Gram matrix of the lattice coordinate-wise. I do not have a great intuition how this impacts things like <span class="math">\(\lambda_1(L)\)</span>).</p> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-centerdensity"> <p>Namely, you can also work with what I believe is called the <em>center density</em>, and defined as <span class="math">\[\frac{\mathsf{vol}(\mathcal{B}_n(\lambda_1(L)/2))}{\det L}\]</span> Where <span class="math">\(\mathcal{B}_n(R)\)</span> is the <span class="math">\(n\)</span>-dimensional ball of radius <span class="math">\(r\)</span>. This is roughly the “<span class="math">\(n\)</span>th power” of the quantity we work with, although there is an additional term related to the volume of the ball of radius 1. Note that this name might be slightly off, but I believe it is the one that <em>Conway and Sloane</em> refer to, but will not check for this post. <a href="#fr-centerdensity-1">↩</a></p> </li> <li id="fn-adleman"> <p>I have heard that this construction is initially due to Adleman (although there are slight differences between Schnorr’s and Adleman’s lattices). Unfortunately, people have indicated that Adleman’s paper may not be easily available online. <a href="#fr-adleman-1">↩</a></p> </li> <li id="fn-cyclic"> <p>This assumption is not strictly necessary, but relaxing it does not lead to better a construction. <a href="#fr-cyclic-1">↩</a></p> </li> <li id="fn-approximate"> <p>This is discussed in <a rel="external" href="https://arxiv.org/abs/1604.07800">Shor and Eldar’s Systematic Normal Form paper</a>, which is morally similar to <a rel="external" href="http://neilsloane.com/doc/FATS.pdf">Sloane and Vaishampayan’s investigations of projecting <span class="math">\(\mathbb{Z}^n\)</span> orthogonal to a single vector</a>. In fact, these kinds of results were behind <a rel="external" href="https://arxiv.org/abs/1611.06999">Eldar and Shor’s attempt to put <span class="math">\(\mathsf{CVP}\)</span> in <span class="math">\(\mathsf{BQP}\)</span> in 2016</a>, which was itself another exciting time. <a href="#fr-approximate-1">↩</a></p> </li> <li id="fn-random"> <p>Here, while “random” is formal, you have to be careful what you mean by it. In particular, the space of all lattices can be described as a ``double coset’’ space, and there is a natural probability distribution on this space known as the <em>Haar measure</em>. Many spaces unrelated to lattices have Haar measures though (such as any compact group), and I personally have never gone down this rabbithole deep enough to have good pointers to learn more, or to have a good idea about what such a distribution explicitly looks like for lattices. <a href="#fr-random-1">↩</a></p> </li> </ol> </section> 2020-10-30T00:00:00+00:00 2020-10-30T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/qkd/ <p>After attending a talk by Psi Vessely on Quantum Key Distribution (in particular on <a rel="external" href="https://en.wikipedia.org/wiki/BB84">BB84</a>), the UCSD crypto group discussed some of the alleged benefits of QKD. We mentioned the “standard list”:</p> <ol> <li> <p>No-cloning theorem provides protection against evesdroppers that “standard” crypto cannot achieve <sup class="footnote-reference" id="fr-wiretap-1"><a href="#fn-wiretap">1</a></sup></p> </li> <li> <p>The protocol itself only requires authenticated channels, so can be viewed as “key exchange within Minicrypt”</p> </li> </ol> <p>In our discussion, Adam Suhl pointed out that we don’t <em>strictly</em> need to be within Minicrypt, as (using previously-shared randomness) you could use a <a rel="external" href="https://en.wikipedia.org/wiki/Message_authentication_code#One-time_MAC">one-time MAC</a> to authenticate messages. This leads to a natural question:</p> <blockquote> <p>If you run QKD using a one-time MAC to authenticate communications, can you “gain” on the amount of shared random bits you have?</p> </blockquote> <p>If you could do the above, it gives a construction of information-theoretic secure cryptography which both <em>doesn’t</em> assume that you’re in minicrypt, and <em>doesn’t</em> have a limit on the amount of communication you can do.</p> <h1 id="the-heuristic-answer"><a class="zola-anchor" href="#the-heuristic-answer" aria-label="Anchor link for: the-heuristic-answer">🔗</a>The Heuristic Answer</h1> <p>Even without describing the BB84 protocol, it should be clear that the answer depends on the (classical) round complexity of the protocol. In particular, all classical communication must be MAC’d, and therefore expends <sup class="footnote-reference" id="fr-one-time-mac-1"><a href="#fn-one-time-mac">2</a></sup> some constant amount of the key material both parties have. This means that a protocol which is <span class="math">\(O(f(n))\)</span> classical rounds must <em>also</em> have <span class="math">\(O(f(n))\)</span> bits of preshared randomness to execute, where <span class="math">\(n\)</span> is the amount of <em>quantum</em> bits communicated.</p> <p>Provided the algorithm can agree on <span class="math">\(\omega(f(n))\)</span> random bits, we would therefore have a strict “gain” in the number of shared bits upon a single execution of the protocol. We could then “bootstrap” this procedure to get an arbitrary number of pre-shared bits, which we can use with things like the one-time pad to discard <em>all</em> computational assumptions!</p> <h1 id="the-actual-answer"><a class="zola-anchor" href="#the-actual-answer" aria-label="Anchor link for: the-actual-answer">🔗</a>The Actual Answer</h1> <p>At this point, I would normally do some computations. For better or worse, while searching for the round complexity of parts <sup class="footnote-reference" id="fr-info-reco-1"><a href="#fn-info-reco">3</a></sup> of the BB84 protocol, I found <a rel="external" href="https://eprint.iacr.org/2018/264.pdf">this paper</a>. Section 4 in particular describes a relatively simple protocol which seems to achieve “OTP-like” encryption by using QKD with one-time MACs to generate enough key material. Note that I haven’t read the full paper (and do not work in this area), but the plausible idea we had seems to have been fleshed out by this paper, and they claim it works.</p> <h1 id="issues-with-full-qkd-without-minicrypt"><a class="zola-anchor" href="#issues-with-full-qkd-without-minicrypt" aria-label="Anchor link for: issues-with-full-qkd-without-minicrypt">🔗</a>Issues with full QKD without Minicrypt</h1> <p>The aforementioned scheme <em>seems</em> to provide a construction that allows you to exchange enough key material to do the one-time pad on arbitrarily large messages with <em>no</em> computational assumptions. This is what I was hoping for, and we found it!</p> <p>Unfortunately, there’s the following <em>dumb</em> attack an evesdropper can always do to thwart this plan — measure every quantum message! This measurement can lead to the key exchange protocol failing, and a reduction in the jointly shared randomness of Alice and Bob, which will eventually force them to fallback on using techniques that do not deplete their jointly shared randomness (such as computationally secure MACs to authenticate the channel).</p> <p>So the answer to “Can you do QKD without Authenticated Channels”? seems to be <em>yes, but no</em>. Still, the possibility of “bootstrapping” QKD is something I haven’t seen in casual discussions of it, and was quite interesting to learn about.</p> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-wiretap"> <p>It’s worth mentioning that QKD is not the only form of key exchange which works information-theoretically based on some non-standard assumptions on the communication channel — the “Wire-tap channel” assumes that evesdroppers recieve “noisier” versions of messages than actual participants, and can achieve similar results. This is usually studied by coding theorists, but there can be some overlap with cryptography — for example, there are lattice-based constructions which rely on a parameter (known as the “flatness factor”) which looks <em>suspiciously</em> like the smoothing parameter from lattice cryptography, but I digress. <a href="#fr-wiretap-1">↩</a></p> </li> <li id="fn-one-time-mac"> <p>This is because one-time MACs are information-theoretically secure provided that you don’t reuse the keys, similarly to the one-time pad. <a href="#fr-one-time-mac-1">↩</a></p> </li> <li id="fn-info-reco"> <p>In particular of the “information reconciliation” step — all other steps seem to be obviously a constant number of rounds to me. <a href="#fr-info-reco-1">↩</a></p> </li> </ol> </section> 2020-10-16T00:00:00+00:00 2020-10-16T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/sensitivity-calculus/ <p>In my <a href="https://mark.schultz-wu.com/blog/dpsensitivity/">previous post</a>, I discussed how for an appropriate “discrete” form of the derivative <span class="math">\(\nabla\)</span>, that we have that:</p> <span class="math">\[\Delta_{\vec x, p}(f) = \lVert \nabla f(\vec x)\rVert_p\]</span> <p>Where <span class="math">\(\Delta_{\vec x, p}(f)\)</span> is the local <span class="math">\(\ell_p\)</span> sensitivity of <span class="math">\(f\)</span> at <span class="math">\(\vec x\)</span>, and <span class="math">\(\lVert\cdot\rVert_p\)</span> is a suitably-defined operator norm. At the end of that post I discussed the potential for a “sensitivity calculus”. I will further develop this concept in this post, with the aim of developing the prerequisites for a programmatic sensitivity analysis. Along the way, I’ll run into a significant roadblock. I’m writing all of this up to show <em>why</em> the idea I had at the end of the last post ended up not working out. As a result, this likely isn’t very interesting to anyone besides myself, but it means that I’ve tied up a loose end, and can move onto other things after being <em>waaay</em> too interested in this in the last 24 hours.</p> <h1 id="the-rules-of-discrete-differentiation"><a class="zola-anchor" href="#the-rules-of-discrete-differentiation" aria-label="Anchor link for: the-rules-of-discrete-differentiation">🔗</a>The rules of discrete differentiation</h1> <p>The idea here is to mimic the “differentiation” rules for standard calculus. Some basic examples of these are the:</p> <ol> <li>Sum rule</li> <li>Product rule</li> <li>Quotient rule</li> <li>Chain rule</li> </ol> <p>If we had equivalents of all of these, we would be in a rather good place. I will first develop equivalents for these for the 1D discrete derivative, before trying to generalize these to higher dimensions.</p> <h2 id="the-1d-case"><a class="zola-anchor" href="#the-1d-case" aria-label="Anchor link for: the-1d-case">🔗</a>The 1D case</h2> <p>Recall that the 1D discrete derivative (also known as the forward difference operator) is defined as:</p> <span class="math">\[f'(n) = f(n+1) - f(n)\]</span> <p>Its easy to see by inspection that this behaves well with respect to sums (and is in fact linear). As a result, I will ignore that case, and examine products. One can easily compute:</p> <span class="math">\[\begin{aligned} (fg)'(n) &= f(n+1)g(n+1) - f(n)g(n)\\ &= (f'(n) + f(n))g(n+1) - f(n)g(n)\\ &= f'(n)g(n+1) + f(n)(g(n+1)-g(n))\\ &\stackrel{1}{=} f'(n)g(n+1) + f(n)g'(n)\\ &= f'(n)(g'(n) + g(n)) + f(n)g'(n)\\ &= f'(n)g(n) + f(n)g'(n) + f'(n)g'(n) \end{aligned}\]</span> <p>We see in the above line marked 1 that we come <em>close</em> to the definition of the standard product rule, but unfortunately fall short. We still get a well-defined product rule though, so we move on to quotients.</p> <span class="math">\[\begin{aligned} (f/g)'(n) &= \frac{f(n+1)}{g(n+1)} - \frac{f(n)}{g(n)}\\ &= \frac{f(n+1)g(n) - f(n)g(n+1)}{g(n+1)g(n)}\\ &= \frac{f(n)g(n) + f'(n)g(n) - f(n)g(n) - f(n)g'(n)}{g(n+1)g(n)}\\ &= \frac{f'(n)g(n) - f(n)g'(n)}{g(n)^2 + g'(n)} \end{aligned}\]</span> <p>We again get something <em>close</em> to what we would expect, but not the same. We finally move onto the chain rule. We note that this is slightly more subtle than before — the domain and codomain of <span class="math">\(f\)</span> and <span class="math">\(g\)</span> are not the same. There are two possible solutions to this:</p> <ol> <li>Restrict composition to functions <span class="math">\(f : \mathbb{N}^k\to\mathbb{N}^j\)</span></li> <li>Do not make this restriction, and only restrict the “inner” function to have domain <span class="math">\(\mathbb{N}^k\)</span>.</li> </ol> <p>We can worry about this later though, and first try to simplify the expression <span class="math">\((f\circ g)'(n)\)</span>. One (again) gets a chain rule, but of a slightly different form:</p> <span class="math">\[\begin{aligned} (f\circ g)'(n) &= f(g(n+1)) - f(g(n))\\ &= f(g(n) + g'(n)) - (f\circ g)(n) \end{aligned}\]</span> <p>Note that we require <em>very little</em> requirements on <span class="math">\(f\)</span>, solely that its domain matches up with <span class="math">\(g\)</span>’s codomain. We quickly mention that this doesn’t mean suddenly that we use standard “continuous” derivatives in our analysis (even if <span class="math">\(f : \mathbb{R}^n\to\mathbb{R}^m\)</span>). This is because the “top” of our analysis is the discrete Jacobian <span class="math">\(\nabla f\)</span>, and the rules we develop will propegate <em>discrete</em> derivatives “down” our expression in particular. So <em>even</em> if we are analyzing a continuous function, we will be analyzing it in terms of the forward difference operator (i.e. discrete derivative).</p> <h2 id="higher-dimensions"><a class="zola-anchor" href="#higher-dimensions" aria-label="Anchor link for: higher-dimensions">🔗</a>Higher Dimensions</h2> <p>What does the calculus of derivatives for a function <span class="math">\(f : \mathbb{N}^k\to\mathbb{R}\)</span> look like?</p> <p>Recall that we define:</p> <span class="math">\[(\partial/\partial x_i)f(\vec x) = f(\vec x + e_i) - f(\vec x)\]</span> <p>It is routine to check that this leads to (except for the chain rule) precisely the same “rules”, but in terms of the partial derivatives rather than the standard derivatives. I will repeat the analysis for the chain rule, as it is both simple, and illustrates this point fairly well.</p> <span class="math">\[\begin{aligned} (\partial/\partial x_i)(f\circ g)(\vec x) &= (f\circ g)(\vec x + e_1) - (f\circ g)(\vec x)\\ &= f(g(\vec x + e_1)) - (f\circ g)(\vec x)\\ &= f(g(\vec x) + (\partial/\partial x_i)g(\vec x)) - (f\circ g)(\vec x) \end{aligned}\]</span> <p>Given that this situation was simple, what about higher dimensions? Once our codomain is <span class="math">\(\mathbb{R}^m\)</span>, we seem to lose the product and quotient rules. This is simply because <span class="math">\(\mathbb{R}^m\)</span> does not have a great multiplication or division operation to use in the first place. The sum rule will still work for boring reasons, so what about the chain rule?</p> <p>Let <span class="math">\(g : \mathbb{N}^k\to \mathbb{R}^n\)</span>, and let <span class="math">\[f : \mathbb{R}^n\to\mathbb{R}^m\]</span>. What can we say about <span class="math">\(\nabla (f\circ g)(\vec x)\)</span>? Well, at least the following:</p> <span class="math">\[\begin{aligned} \nabla (f\circ g)(\vec x) &= ((\partial/\partial x_1)(f\circ g)(\vec x),\dots, (\partial/\partial x_k)(f\circ g)(\vec x))\\ &= ((f\circ g)(\vec x + e_1) - (f\circ g)(\vec x),\dots, (f\circ g)(\vec x + e_k) - (f\circ g)(\vec x))\\ &= ((f\circ g)(\vec x + e_1),\dots, (f\circ g)(\vec x + e_k)) - \mathbf{1}\otimes (f\circ g)(\vec x)\\ &= (f(g(\vec x) + (\partial/\partial x_1)g(\vec x)),\dots, f(g(\vec x) + (\partial/\partial x_k)g(\vec x))) - \mathbf{1}\otimes (f\circ g)(\vec x) \end{aligned}\]</span> <p>Here, by <span class="math">\(\mathbf{1}\otimes (f\circ g)(\vec x)\)</span>, I mean the tensor product of <span class="math">\(\mathbf{1} = (1,1,\dots,1)\in\mathbb{R}^k\)</span> with <span class="math">\((f\circ g)(\vec x)\)</span>. This is just the matrix <span class="math">\(((f\circ g)(\vec x),\dots, (f\circ g)(\vec x))\)</span>, or equivalently the “vector” of that shape where each entry is an <span class="math">\(m\)</span>-dimensional (column) vector.</p> <p>This suggests that we <em>do</em> have another chain rule. It may not be able to be written in a pretty way, but our initial goal was to use computers to calculate this all anyway, so who cares? We (in principle) have a chain rule, so can bravely go on to the next step.</p> <h1 id="the-brick-wall-you-run-into"><a class="zola-anchor" href="#the-brick-wall-you-run-into" aria-label="Anchor link for: the-brick-wall-you-run-into">🔗</a>The brick wall you run into</h1> <p>At this point, I admit defeat. This may seem premature, given that one <em>should</em> be able to write a computer program to apply the above rules. I haven’t yet computed discrete derivatives of simple functions, but this also seems relatively easy to do.</p> <p>What isn’t simple is the operator norm. I was hoping that the sub-additivity and sub-multiplicativity combine with the calculus we develop to allow us to get (potentially loose) upper bounds on the various derivatives. This intuition is right for <em>continuous</em> calculus, but fails in our discrete setting.</p> <p>Why is that? Because of the chain rule. The multivariable form of it is particularly nice — specificially that:</p> <span class="math">\[\nabla (f\circ g)(\vec x) = (\nabla f)(g(\vec x))\cdot (\nabla g)(\vec x)\]</span> <p>Note that this expresses the derivative of the <em>composition</em> in terms of the (matrix) product of the derivatives (which are linear maps). While this allows for compact notation, it <em>also</em> has an important property that it behaves well with respect to sub-multiplicative norms (such as the operator norm) — in particular:</p> <span class="math">\[\lVert \nabla (f\circ g)(\vec x)\rVert_p \leq \lVert (\nabla f)(g(\vec x))\rVert_p \lVert \nabla g(\vec x)\rVert_p\]</span> <p>It is not clear how to upper bound the norms of some of the expressions I was developing. So of the two steps:</p> <ol> <li>Rewrite derivative in terms of “simpler” derivatives automatically using a “differential calculus”</li> <li>Upper bound this rewritten derivative using sub-additivity and sub-multiplicativity of the operator norm</li> </ol> <p>The second step is the one which seems problematic. In general, one would need a way to upper bound the operator norm of expressions of the form:</p> <span class="math">\[\lVert (f(g(\vec x) + (\partial/\partial x_1)g(\vec x)),\dots, f(g(\vec x) + (\partial/\partial x_k)g(\vec x)))\rVert_p\]</span> <p>This seems intrinsically difficult. If we unwrap what the operator norm means, this is bounding the expression:</p> <span class="math">\[\begin{aligned} \lVert \nabla (f\circ g)(\vec x)\rVert_p &= \sup\{\lVert \nabla (f\circ g)(\vec x)e_i\mid i\in[|\mathcal{X}|]\rVert_p\\ &= \max_i\lVert (\partial/\partial x_i) (f\circ g)(\vec x)\rVert_p\\ &= \max_i \lVert f(g(\vec x) + (\partial/\partial x_i)g(\vec x)) - f(g(\vec x))\rVert_p \end{aligned}\]</span> <p>One can use triangle inequality to separate this out, but I run into issues even when trying to upper bound <span class="math">\(\lVert f(g(\vec x))\rVert_p\)</span>. The issue here (compared to the continuous case) is that this is a composition of <em>potentially non-linear</em> functions. It looks like with enough “continuous” data, there are areas of math that attempt to get bounds like this for non-linear functions (for example <a rel="external" href="https://arxiv.org/pdf/math/0506424v2.pdf">this</a>). You could even do simpler things, and find a “suitably good” polynomial approximation to <span class="math">\(f\)</span>. These techniques feel like they’re starting to lead to incredibly loose bounds though, so I’ll decide to walk away from the brick wall now rather than continue to hit my head against it until something breaks.</p> 2020-10-15T00:00:00+00:00 2020-10-15T00:00:00+00:00 Unknown https://mark.schultz-wu.com/blog/dpsensitivity/ <p>Differential Privacy is an area which has seen wide interest (both in academia and real-life) in recent years. But what <em>is</em> differential privacy? As I don’t work in this area, I’m rather ill-suited to say anything — but at least why does it have that name? Frank McSherry (one of the founders of the area, who shares the <a rel="external" href="https://en.wikipedia.org/wiki/G%C3%B6del_Prize">Godel Prize</a> as a result) <a rel="external" href="https://crypto.stackexchange.com/questions/77324/what-does-the-term-differential-in-differential-privacy-mean">states that it was one of several names considered</a>, including others like “Marginal Privacy” or “Incremental Privacy”. He further states that he thinks the name “stuck” due to an analogy with differential cryptanalysis.</p> <p>This post is a relatively quick note on how “differential” objects show up (implicitly) in differential privacy. This is somewhat unsurprising, but formalizing this has been sufficiently weird journey that it seemed like it should be shared with others.</p> <h1 id="what-is-differential-privacy"><a class="zola-anchor" href="#what-is-differential-privacy" aria-label="Anchor link for: what-is-differential-privacy">🔗</a>What is Differential Privacy?</h1> <p>To avoid this post dragging on too long, I’m going to answer “I’m not telling”. My plan is to do this <em>regardless if I really know</em>, so this response is actually differentially private (ha).</p> <p>For people who actually want to know, there are a variety of excellent technical resources to learn. In particular <a rel="external" href="https://privacytools.seas.harvard.edu/files/privacytools/files/the_algorithmic_foundations_of_differential_privacy_0.pdf">The Algorithmic Foundations of Differential Privacy</a> by Dwork and Roth is great. I’ll defer to them for an introduction, and even adopt their notation. Really, the entirety of this post is pointing out that a common parameter within differential privacy can be interpreted in terms of a deceptively simple equation. To cut to the chase:</p> <span class="math">\[\Delta_{\vec x, p}(f) = \lVert \nabla f(\vec x)\rVert_p\]</span> <p>Where <span class="math">\(\Delta_{\vec x, p}(f)\)</span> is the local <span class="math">\(\ell_p\)</span> sensitivity of <span class="math">\(f\)</span> at <span class="math">\(\vec x\)</span>. This immediately implies the following for the global sensitivity <span class="math">\(\Delta_p(f)\)</span>:</p> <span class="math">\[\Delta_p(f) = \max_{\vec x\in\mathbb{N}^{|\mathcal{X}|}} \lVert \nabla f(\vec x)\rVert_p\]</span> <p>So in effect, if you define triangles correctly, you can “flip” one upside down (or right-side up — I do not know the natural orientation of triangles). Before discussing the “discrete derivative” definitions which will be needed to interpret the right-hand side, we should probably talk about the quantity on the left-hand side.</p> <h1 id="the-sensitivity-of-a-function"><a class="zola-anchor" href="#the-sensitivity-of-a-function" aria-label="Anchor link for: the-sensitivity-of-a-function">🔗</a>The sensitivity of a function</h1> <p>As mentioned before, I will use the notation of Dwork and Roth’s book. Definition 3.1 and 3.8 define the (global) <span class="math">\(\ell_1\)</span> and <span class="math">\(\ell_2\)</span> sensitivities. Upon reading both of them, there is a straightforward generalization to <span class="math">\(\ell_p\)</span> sensitivities, which I give below:</p> <blockquote> <p>The global <span class="math">\(\ell_p\)</span> sensitivity of a function <span class="math">\[f : > \mathbb{N}^{|\mathcal{X}|}\to\mathbb{R}^k\]</span> is:</p> <span class="math">\(\Delta_p(f) = \max_{\vec{x}, \vec{y}\in\mathbb{N}^{|\mathcal{X}|} : \lVert \vec x - \vec y \rVert_1 = 1} \lVert f(\vec x) - f(\vec y)\rVert_p\)</span></blockquote> <p>One can also speak of the “local” sensitivity of a function <em>at</em> at point <span class="math">\(\vec{x}\in\mathbb{N}^{|\mathcal{X}|}\)</span>. This can be viewed as the global sensitivity where <span class="math">\(\vec{x}\in\mathbb{N}^{|\mathcal{X}|}\)</span> is <em>fixed</em>. An <span class="math">\(\ell_1\)</span> version of this is mentioned in definition 7.1. As no notation is given, I will write <span class="math">\(\Delta_{p, \vec x}(f)\)</span>.</p> <blockquote> <p>The local <span class="math">\(\ell_p\)</span> sensitivity of a function <span class="math">\(f : \mathbb{N}^{|\mathcal{X}|} \to \mathbb{R}^k\)</span> is:</p> <span class="math">\[\Delta_{p, \vec x}(f) = \max_{\vec y\in\mathbb{N}^{|\mathcal{X}|} : \lVert > \vec x - \vec y\rVert_1 = 1} \lVert f(\vec x) - f(\vec y)\rVert_p\]</span></blockquote> <p>One can easily see that <span class="math">\(\Delta_p(f) = \max_{\vec x\in\mathbb{N}^{|\mathcal{X}|}}\Delta_{p, \vec x}(f)\)</span>. As the global sensitivity is more common, I’ll quickly describe the motivation for it.</p> <p>Generally this is explained in a few steps. First, it is stated (or already known) that <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span> is the natural domain of an arbitrary database. Intuitively, a <em>record</em> in a database is an element <span class="math">\(r\in\mathcal{X}\)</span>, for some finite set <span class="math">\(\mathcal{X}\)</span>. Then the <em>database</em> is a collection of records, where you count the multiplicity of each record. It therefore lives in <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span>. Next, one notes that the condition that <span class="math">\(\lVert \vec x-\vec y\rVert_1 = 1\)</span> can be interpreted as “the databases <span class="math">\(\vec x\)</span> and <span class="math">\(\vec y\)</span> are neighboring”, in that they differ in how many times a particular record occurs by a count of 1. Finally, the condition <span class="math">\(\lVert f(\vec x) - f(\vec y)\rVert_p\)</span> is interpreted as quantifying how much <span class="math">\(f(\vec x) - f(\vec y)\)</span> can change on neighboring records. The local sensitivity still bounds how much <span class="math">\(f\)</span> can change on neighboring records, but it <em>in particular</em> bounds how much <span class="math">\(f\)</span> can change on records which neighbor <span class="math">\(\vec x\)</span>.</p> <p>Given language like “The <span class="math">\(\ell_p\)</span> sensitivity quantifies how much a function <span class="math">\(f\)</span> can change on adjacent points”, it seems entirely unsurprising that you can characterize it in terms of a derivative-like concept. But what <em>is</em> a derivative for a function with the domain <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span>?</p> <h2 id="discrete-derivatives"><a class="zola-anchor" href="#discrete-derivatives" aria-label="Anchor link for: discrete-derivatives">🔗</a>Discrete derivatives</h2> <p>Throughout this section, I will define “discrete” analogus of the:</p> <ol> <li>Derivative <span class="math">\(f'\)</span></li> <li>Partial Derivative <span class="math">\((\partial/\partial x_i)f\)</span></li> <li>Gradient <span class="math">\(\nabla f\)</span></li> </ol> <p>I will do this by defining a (reasonable) “discrete” analogue of <span class="math">\(f'\)</span>, and then showing that in the continuous case, one can define <span class="math">\[(\partial/\partial x_i)f\]</span> abstractly in terms of <span class="math">\(f'\)</span>, and <span class="math">\(\nabla f\)</span> abstractly in terms of <span class="math">\((\partial/\partial x_i)f\)</span>. I’ll then shove the discrete version of <span class="math">\(f'\)</span> through this set of transformations to end up with a discrete version of <span class="math">\(\nabla f\)</span>. This “pipeline” is the sense in which I mean that my definition of the discrete gradient is <em>natural</em>.</p> <p>To highlight the similarities throughout this, I will use the same notation for discrete and continuous derivatives. If one wanted to actually compute with these, you could just check the domain of <span class="math">\(f\)</span> to see which of the two methods to apply. Anyway, onwards to technical content.</p> <p>First, I’ll restrict to “1 dimensional” functions <span class="math">\[f : \mathbb{N}\to\mathbb{R}\]</span>, before generalizing to something more useful.</p> <p>Recall that the (continuous) derivative is traditionally defined via the <em>difference quotient</em>:</p> <span class="math">\[f'(x) = \lim_{h\to 0, h\neq 0}\frac{f(x+h) - f(x)}{h}\]</span> <p>If we want to restrict to discrete domains, we have to make sure that <span class="math">\(x+h\)</span> is discrete (as it must be in the domain of <span class="math">\(f\)</span>). In our setting, this means that <span class="math">\(h\in\mathbb{N}\)</span>. As we need <span class="math">\(h\neq 0\)</span>, this itself really means that we need <span class="math">\(h = 1\)</span>, so can define:</p> <span class="math">\[f'(x) = \frac{f(x+1) - f(x)}{1} = f(x+1) - f(x)\]</span> <p>This definition is fairly standard (known as the <a rel="external" href="https://en.wikipedia.org/wiki/Finite_difference">forward difference operator</a>), and denoted <span class="math">\(\Delta(f)\)</span>. As <span class="math">\(\Delta\)</span> is <em>already</em> a super overloaded symbol in this post, I will instead refer to it as <span class="math">\(f'(x)\)</span>.</p> <p>How do we generalize (continuous) derivatives to higher dimensions? The way that most people learn is the <a rel="external" href="https://en.wikipedia.org/wiki/Partial_derivative#Formal_definition">partial derivative</a>:</p> <blockquote> <p>Let <span class="math">\(f : \mathbb{R}^n\to \mathbb{R}\)</span>. The <em>partial derivative</em> of <span class="math">\(f\)</span> with respect to the variable <span class="math">\(x_i\)</span> is defined as:</p> <span class="math">\[\frac{\partial}{\partial x_i}f(\vec{x}) = \lim_{h\to 0, h\neq 0} > \frac{f(\vec{x} +he_i) - f(\vec{x})}{h}\]</span></blockquote> <p>I will interpret the partial derivative as the 1D derivative of a particular unary function. The particular function is:</p> <span class="math">\[F_{\vec x, i}(t) = f(\vec x - \vec{x}_i + e_i + te_i)\]</span> <p>One can check that in the continuous case this is such that <span class="math">\[F_{\vec x, i}'(\vec x_i) = (\partial/\partial x_i)f(\vec x)\]</span>.</p> <p>When applied to the discrete derivative, one gets that:</p> <blockquote> <p>Let <span class="math">\(f : \mathbb{N}^n\to \mathbb{R}\)</span>. For any <span class="math">\(\vec{x}\in\mathbb{N}^n\)</span> and <span class="math">\(i\in[n]\)</span>, let <span class="math">\(F_{\vec{x}, i}(y) = f(\vec x - \vec{x}_ie_i + te_i)\)</span> The <em>discrete partial derivative</em> of <span class="math">\(f\)</span> with respect to the variable <span class="math">\(x_i\)</span> is defined as:</p> <span class="math">\[\begin{aligned} (\partial/\partial x_i)f(\vec{x}) &= F_{\vec{x}, i}'(\vec{x}_i)\\ &= f(\vec x + e_i) - f(\vec x) \end{aligned}\]</span></blockquote> <p>I’ll quickly mention that (something close to this) <a rel="external" href="https://en.wikipedia.org/wiki/Finite_difference#Multivariate_finite_differences">seems like it is used <em>some places</em></a>. For further motivation for why this may be a sensible “discrete partial derivative”, I’ll point to a conceptually close definition of “<span class="math">\(\mathbb{F}_2\)</span>-partial derivatives”, which take a similar form (such as in <a rel="external" href="http://web.cs.ucla.edu/~sherstov/pdf/directional.pdf">this paper</a>, near theorem 1.5).</p> <p>Of course, sensitivity is defined for functions <span class="math">\[f : \mathbb{N}^{|\mathcal{X}|}\to\mathbb{R}^k\]</span>, so we have one final step of generalization to do. Recall that the gradient is just a vector containing the partial derivatives in each direction:</p> <span class="math">\[\nabla f(x) = ((\partial/\partial x_1)f(x),\dots, (\partial/\partial x_n)f(x))\]</span> <p>I will then define the “discrete gradiant” in the same way below:</p> <span class="math">\[\nabla f(x) = ((\partial/\partial x_1)f(x), \dots, (\partial/\partial x_n)f(x))\]</span> <p>Before proceeding, we need to note that (when the codomain of <span class="math">\(f\)</span> is dimension <span class="math">\(>1\)</span>) <span class="math">\(\nabla f(x)\)</span> is most naturally a <em>matrix</em> (called the <a rel="external" href="https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant"><em>Jacobian</em> matrix</a>), whose <span class="math">\(i\)</span>th column is <span class="math">\((\partial/\partial x_i)f(x)\)</span>. I will still notate this <span class="math">\(\nabla f\)</span> I will define the norm of a matrix to be its <a rel="external" href="https://en.wikipedia.org/wiki/Operator_norm"><em>operator norm</em></a>. The general definition of this is included below:</p> <blockquote> <p>Let <span class="math">\(M : V\to W\)</span> be a linear operator, and let <span class="math">\(\lVert\cdot\rVert_V\)</span>, <span class="math">\(\lVert\cdot\rVert_W\)</span> be norms on <span class="math">\(V\)</span> and <span class="math">\(W\)</span> respectively. Define the norm of <span class="math">\(M\)</span> by:</p> <span class="math">\[\lVert M\rVert = \sup\{\lVert Mv\rVert_W \mid v\in V, \lVert v\rVert_V = > 1\}\]</span></blockquote> <p>Of course, our domain is <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span>. What norm are we picking on it? This choice is key to the entire construction, so I will discuss it in a little more depth.</p> <h2 id="viewing-as-a-normed-space"><a class="zola-anchor" href="#viewing-as-a-normed-space" aria-label="Anchor link for: viewing-as-a-normed-space">🔗</a>Viewing <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span> as a normed space</h2> <p>The natural norm to put on <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span> is a norm induced by viewing it as a subset of <span class="math">\(\mathbb{R}^{|\mathcal{X}|}\)</span>. This is precisely the choice I will make. If this is so straightforward, why does this have its own separate section?</p> <p>What is <em>less</em> straightforward is what the (closed) unit ball on <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span> looks like. Or actually, it’s a little <em>too straightforward</em>. This is clearly the intersection of the unit ball in <span class="math">\(\mathbb{R}^{|\mathcal{X}|}\)</span> with <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span>. What is a little odd is that, for any <span class="math">\(p\in (1,\infty)\)</span>, it is <em>also</em> clear that the only points in this ball are 0, along with the “standard basis vectors” <span class="math">\(e_i\)</span>. Quickly notice that:</p> <ol> <li> <p>In the <span class="math">\(\ell_\infty\)</span> norm this isn’t the case (for example, <span class="math">\[(1,1,\dots, 1)\]</span> is in the ball).</p> </li> <li> <p>This is an “easy” example of where the closed unit ball (things of norm <span class="math">\(\leq 1\)</span>) is not the same as the closure of the open unit ball (things with norm <span class="math">\(<1\)</span>). The open unit ball, in any <span class="math">\(\ell_p\)</span> norm, will just be <span class="math">\(\{0\}\)</span>.</p> </li> </ol> <p>This is all to say that for <span class="math">\(p\in(1,\infty)\)</span>, we have that:</p> <span class="math">\[\begin{aligned} \{\vec x\in\mathbb{N}^{|\mathcal{X}|} \mid \lVert \vec x\rVert_p \leq 1\} &= \{0\} \cup \{\vec x\in\mathbb{N}^{|\mathcal{X}|} \mid \lVert \vec x\rVert_p = 1\}\\ &= \{0\}\cup \{e_i\mid i\in[|\mathcal{X}|]\} \end{aligned}\]</span> <p>The above implies that when we define the operator norm, the <em>particular</em> norm we choose on the domain does not really matter (as long as it is not the <span class="math">\(\ell_\infty\)</span> norm), as the elements of norm exactly 1 will always be precisely the standard basis of the “overlying” vector space. For this reason, we will write:</p> <span class="math">\[\begin{aligned} \lVert M\rVert_p &= \sup\{\lVert Mv\rVert_p \mid v\in\mathbb{N}^{|\mathcal{X}|}, \lVert v\rVert_p = 1\}\\ &= \sup\{\lVert Mv\rVert_p \mid v\in\{e_i\}, i\in[|\mathcal{X}|]\} \end{aligned}\]</span> <p>To denote the operator norm of a linear map <span class="math">\[M : \mathbb{N}^{|\mathcal{X}|}\to\mathbb{R}^k\]</span>.</p> <p>We could be putting <em>any</em> <span class="math">\(\ell_p\)</span> norm on the domain while doing this, and will end up with an equivalent definition. We’ll next show how this leads to a quite natural interpretation of the <span class="math">\(\ell_p\)</span> sensitivity as the maximum of the operator norm of <span class="math">\(\nabla f(\vec{x})\)</span> over <span class="math">\(\vec{x}\in\mathbb{N}^{|\mathcal{X}|}\)</span>.</p> <h1 id="the-sensitivity-as-a-norm-bound-on-the-jacobian-matrix"><a class="zola-anchor" href="#the-sensitivity-as-a-norm-bound-on-the-jacobian-matrix" aria-label="Anchor link for: the-sensitivity-as-a-norm-bound-on-the-jacobian-matrix">🔗</a>The <span class="math">\(\ell_p\)</span> sensitivity as a norm bound on the Jacobian matrix</h1> <p>Let <span class="math">\(f : \mathbb{N}^{|\mathcal{X}|} \to \mathbb{R}^k\)</span>. We will next show the following:</p> <blockquote> <span class="math">\(\Delta_{\vec x, p}(f) = \lVert \nabla f(\vec x)\rVert_p\)</span></blockquote> <p>By taking maxes over <span class="math">\(\vec x\)</span>, this also implies our expression for the global sensitivity. The proof is rather straightforward. By the characteriation of norm-1 elements of <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span>, we have that the operator norm will be:</p> <span class="math">\[\lVert \nabla f(\vec x)\rVert_p = \sup\{\lVert [\nabla f(\vec x)] e_i\rVert_p\mid i\in[|\mathcal{X}|]\}\]</span> <p>This is clearly equivalent to <span class="math">\(\max_{e_i}\lVert \nabla [f(\vec x)] e_i\rVert_p\)</span>. Note that <span class="math">\([\nabla f(\vec x)] e_i\)</span> is the <span class="math">\(i\)</span>th column of <span class="math">\(\nabla f(\vec x)\)</span>, which is precisely <span class="math">\((\partial/\partial x_i)f(\vec x)\)</span>. It follows that:</p> <span class="math">\[\begin{aligned} \lVert \nabla f(\vec x)\rVert_p &= \max_{i}\lVert (\partial/\partial x_i)f(\vec x)\rVert_p &= \max_i \lVert f(\vec x + e_i) - f(\vec x)\rVert_p \end{aligned}\]</span> <p>This is <em>nearly</em> the <span class="math">\(\ell_p\)</span> sensitivity, except we are maximizing over the choice of <span class="math">\(\vec x + e_i\)</span>, rather than <span class="math">\(\vec y\)</span> such that <span class="math">\(\lVert \vec x - \vec y\rVert_1 = 1\)</span>. But these can readily be seen to be equivalent — as norm 1 elements in <span class="math">\(\mathbb{N}^{|\mathcal{X}|}\)</span> are precisely standard basis vectors, we have that <span class="math">\(\vec x, \vec y\)</span> that are adjacent are of the form <span class="math">\[\vec x = \vec y \pm e_i\]</span> for some <span class="math">\(i\)</span>.</p> <p>It follows that:</p> <span class="math">\[\Delta_{\vec x, p}(f) = \lVert \nabla f(\vec x)\rVert_p\]</span> <p>and:</p> <span class="math">\[\Delta_{p}(f) = \max_{\vec x\in\mathbb{N}^{|\mathcal{X}|}} \Delta_{\vec x, p}(f) = \max_{\vec x\in\mathbb{N}^{|\mathcal{X}|}} \lVert \nabla f(\vec x)\rVert_p\]</span><h1 id="an-explicit-calculation-in-this-framework"><a class="zola-anchor" href="#an-explicit-calculation-in-this-framework" aria-label="Anchor link for: an-explicit-calculation-in-this-framework">🔗</a>An explicit calculation in this framework</h1> <p>To flesh this out a little more, I’ll next do a sensitivity calculation in the above language. I imagine this will look essentially like a sensitivity calculation in the “standard” setting.</p> <p>Recall that a database <span class="math">\(\vec{x}\in \mathbb{N}^{|\mathcal{X}|}\)</span> is a count of the multiplicities of various records <span class="math">\(r\in\mathcal{X}\)</span>. What does a function of a databse look like then? For example, if we have a database of names + ages, and want to compute the average age, how do we do this in the “histogram” representation?</p> <p>There are at least two ways to interpret this — as I do essentially no differential privacy, I have no clue which is correct. But the first is much nicer to work with, so I will use it (after briefly discussing both options).</p> <ol> <li> <p>A database of ages can be identified with a vector <span class="math">\[\vec x = (x_0,x_1,\dots, x_{122})\]</span>, where <span class="math">\(122\)</span> is an absolute upper bound on the age of a person (chosen as it is the <a rel="external" href="https://en.wikipedia.org/wiki/List_of_the_verified_oldest_people">oldest verified age</a>).</p> </li> <li> <p>A database of peoples ages can be identified with a vector of <em>tagged ages</em> <span class="math">\((x_{n_0, 0}, x_{n_0, 1},\dots, x_{n_0, 122}, x_{n_1,0},\dots)\)</span>, where <span class="math">\(n_i\)</span> are the collection of <em>admissible names</em>.</p> </li> </ol> <p>While the second definition defines the database itself better (and is a <em>much</em> more useful definition if we want to query the average number of vowels in each persons name, or something else), the first leads to significantly “smaller” databases generically. Maybe viewing databases as multi-sets instead of vectors is what makes the second the more appropriate definition — I don’t know. All I know is I want to compute the sensitivity of the average age of someone, and the first definition is easier to work with, so I will look at it for this example.</p> <p>The average age under the first definition is:</p> <span class="math">\[\mathsf{mean}(\vec{x}) = \frac{\sum_{i = 0}^{122} i \vec{x}_i}{\sum_{i = 0}^{122} \vec{x}_i} \sim \frac{123\mathbb{E}[\vec x]}{\lVert \vec x\rVert_1}\]</span> <p>Here, the expectation is taken with respect to a uniform distribution on <span class="math">\(\{0,1,\dots, 122\}\)</span> (a set of size <span class="math">\(123\)</span>). I use it mostly as compact notation for the sum. This still has a fairly sensible discrete gradient:</p> <span class="math">\[\begin{aligned} (\partial/\partial x_i)f(\vec x) &= \mathsf{mean}(\vec x + e_i) - \mathsf{mean}(\vec x) \\ &=\frac{123\mathbb{E}[\vec x + e_i]}{\lVert \vec x + e_i\rVert_1} - \frac{123\mathbb{E}[\vec x]}{\lVert \vec x\rVert_1}\\ &= 123\left(\frac{\mathbb{E}[\vec x] + i}{\lVert \vec x\rVert_1 + 1} - \frac{\mathbb{E}[\vec x]}{\lVert \vec x\rVert_1}\right)\\ &= 123\left(\frac{\lVert \vec x\rVert_1\mathbb{E}[\vec x] + i\lVert \vec x\rVert_1 - (\lVert \vec x\rVert_1 + 1)\mathbb{E}[\vec x]}{\lVert \vec x\rVert_1(\lVert \vec x\rVert_1 + 1)}\right)\\ &= 123\frac{i - \mathbb{E}[\vec x]}{\lVert \vec x\rVert_1 + 1}\\ &= 123\frac{\mathbb{E}[e_i - \vec x]}{\lVert x \rVert_1 + 1} \end{aligned}\]</span> <p>Note that we can convert from <span class="math">\[\lVert \vec x + e_i\rVert_1 \to \lVert \vec x\rVert_1 + 1\]</span> as <span class="math">\(\vec x\in\mathbb{N}^{|\mathcal{X}|}\)</span> (so the absolute value in the definition of the <span class="math">\(\ell_1\)</span> norm is immaterial), but we cannot necessairly convert back from <span class="math">\(\lVert \vec x \rVert_1 + 1\)</span> to <span class="math">\(\lVert \vec x - e_i\rVert_1 = \lVert e_i - \vec x\rVert_1\)</span>. We can instead write:</p> <span class="math">\[\begin{aligned}(\partial/\partial x_i)\mathsf{mean}(\vec x) &= 123\frac{\mathbb{E}[e_i - \vec x]}{\lVert \vec x\rVert_1 + 1}\\ &= \frac{\lVert e_i - \vec x\rVert_1}{\lVert \vec x\rVert_1 + 1}\mathsf{mean}(e_i - \vec x)\\ &= -\frac{\lVert \vec x - e_i\rVert_1}{\lVert \vec x + e_i\rVert_1}\mathsf{mean}(\vec x - e_i) \end{aligned}\]</span> <p>If we then want to compute the noise sensitivity from this, we need to compute:</p> <span class="math">\[\begin{aligned}\max_{\vec x \in\mathbb{N}^{|\mathcal{X}|}}\max_i \lVert (\partial/\partial x_i)f(\vec x)\rVert_p&= \max_{\vec x\in \mathbb{N}^{|\mathcal{X}|}}\max_i \frac{\lVert 123\mathbb{E}[\vec x - e_i]\rVert_p}{\lVert x + e_i\rVert_1} \end{aligned}\]</span> <p>The mean is over the set of ages <span class="math">\(\{0,1,\dots, 122\}\)</span>. This mean can be maximized when <span class="math">\(i = 122\)</span> and <span class="math">\(\vec{x} = 0\)</span> (there are some small issues with the resulting count being negative which I will ignore), where the mean is <span class="math">\(1\)</span>, and thus the sensitivity is 123, as expected. I get the impression this is essentially the same computation you would do in the “standard” framework though.</p> <h1 id="potential-extension"><a class="zola-anchor" href="#potential-extension" aria-label="Anchor link for: potential-extension">🔗</a>Potential extension</h1> <p>One application of this representation is to the development of a <em>sensitivity calculus</em>. Here, “calculus” means a framework for calculating things. One reason differential calculus has been so successful is that derivatives give a (mechanically) simple way to do computations with complex functions. In particular, as derivatives behave well with respect to:</p> <ol> <li>Sums</li> <li>Products</li> <li>Quotients</li> <li>Composition</li> </ol> <p>One can usually compute the derivative of an arbitrary function in terms of the above “rules”, along with the knowledge of some “simple” derivatives. By framing sensitivity in terms of derivatives, there is a hope that one could derive such a set of rules for <em>discrete derivatives</em>. This could “automate” the computation of <span class="math">\(\nabla f\)</span> in sensitivity analysis, by giving a way to write <span class="math">\(\nabla f\)</span> as an expression in (known) “simple” derivatives.</p> <p>Of course, sensitivity analysis still requires computing the operator norm of this expression. The operator norm is sub-additive and sub-multiplicative, which one can leverage to get an <em>upper bound</em> on the sensitivity in terms of the sensitivities of the aforementioned “simple” derivatives. This upper bound is potentially loose though, which would hurt the applicability of this method. If the upper bound is tolerable, this should be relatively easy to program.</p> <p>Unfortunately, the above reasoning seems to be erronous. I’ve written up some details <a href="https://mark.schultz-wu.com/blog/sensitivity-calculus/">here</a>, but “short story” is that the chain rule one can derive involves terms of the form:</p> <span class="math">\[f(g(n) + g'(n))\]</span> <p>Its unclear how to upper bound the <span class="math">\(\ell_p\)</span> norm of this, which would be required to derive even (loose) upper bounds on the operator norm. Note that for “continuous” calculus the chain rule has terms of the form:</p> <span class="math">\[f'(g(n))g'(n)\]</span> <p>So for a sub-multiplicative norm one could hope to get some upper bounds. Feel free to read the writeup for more details, but as it ends in “failure” it is likely not very interesting.</p>