This is an adaptation of a talk I gave at Microsoft Research in November 2018. [PDF]
I exposit the sampling techniques used in my recommendation systems work and its followups in dequantized machine learning:
 Tang – A quantuminspired algorithm for recommendation systems
 Tang – Quantuminspired classical algorithms for principal component analysis and supervised clustering;
 Gilyén, Lloyd, Tang – Quantuminspired lowrank stochastic regression with logarithmic dependence on the dimension;
 Chia, Lin, Wang – Quantuminspired sublinear classical algorithms for solving lowrank linear systems.
The core ideas used are super simple. This goal of this blog post is to break down these ideas into intuition relevant for quantum researchers and create more understanding of this machine learning paradigm.
Table of Contents
 An introduction to dequantization
 Quantum for the quantumless
 Implications
 Appendix: More details
 Glossary
 References
Notation is defined in the Glossary.
The intended audience is researchers comfortable with probability and linear algebra (SVD, in particular). Basic quantum knowledge helps with intuition, but is not essential: everything from The model onward is purely classical. The appendix is optional and explains the dequantized techniques in more detail.
An introduction to dequantization
Motivation
The best, most soughtafter quantum algorithms are those that take in raw, classical input and give some classical output. For example, Shor’s algorithm for factoring takes this form. These classicaltoclassical algorithms (a term I invented for this post) have the best chance to be efficiently implemented in practice: all you need is a scalable quantum computer. (It’s just that easy!)
Nevertheless, many quantum algorithms aren’t so nice. Most wellknown QML algorithms convert input quantum states to a desired output state or value. Thus, they do not provide a routine to get necessary copies of these input states (a state preparation routine) and a strategy to extract information from an output state. Both are essential to making the algorithm useful.
An example of an algorithm that is not classicaltoclassical is the swap test. If we have many copies of the quantum states , then the swap test estimates their inner product in time polylogarithmic in dimension. While this routine seems much faster than naively computing classically, we can only run this algorithm if we know how to prepare the states and . It may well be the case that state preparation is too expensive for input vectors, making the quantum algorithm as slow as the classical algorithm. This illustrates the format and failings of most QML algorithms.
You might then ask: can we fill in the missing routines in QML algorithms to get a classicaltoclassical algorithm that’s provably fast and useful? This is an open research problem: see Scott Aaronson’s piece on QML^{1}. We have a variety of partial results towards the affirmative, but as far as I know, they don’t answer the question unless you’re loose with your definitions of at least one of “classical”, “provably fast”, or “useful”. So let’s settle for a simpler question.
How can we compare the speed of quantum algorithms with quantum input and quantum output to classical algorithms with classical input and classical output? Quantum machine learning algorithms can be exponentially faster than the best standard classical algorithms for similar tasks, but this comparison is unfair because the quantum algorithms get outside help through input state preparation. We want a classical model that helps its algorithms stand a chance against quantum algorithms, while still ensuring that they can be run in nearly all circumstances one would run the quantum algorithm. The answer I propose: compare quantum algorithms with quantum state preparation to classical algorithms with sample and query access to input.
The model
Before we proceed with definitions, we’ll establish some conventions. First, we generally consider our input as being some vector in or , subject to an access model to be described. Second, we’ll only concern ourselves with an algorithm’s query complexity, the number of accesses to the input. Our algorithms will have query complexity independent of input dimensions and polynomial in other parameters. If we assume that each access costs (say) or , the time complexity is still polylogarithmic in input dimension and at most polynomially worse in other parameters.
Now, we define query access to input; we can get query access simply by having the input in RAM.
Definition. We have query access to (denoted ) if, given , we can efficiently compute .
If we have stored normally as an array in our classical computer’s memory, we have because finding the th entry of can be done with the code x[i]
.
This notion of access can represent more than just memory: we can also have if is implicitly described.
For example, consider the vector of squares: for all .
We can have access to without writing in memory.
This will be important for the algorithms to come.
Definition. We have sample and query access to (denoted ) if we have query access to ; can produce independent random samples where we sample with probability ; and can query for .
Sampling and query access to will be our classical analogue to assuming quantum state preparation of copies of . This should make some intuitive sense: our classical analogue has the standard assumption of query access to input, along with samples, which are essentially measurements of in the computational basis. Knowledge of is for normalization issues, and is often assumed for quantum algorithms as well (though for both classical and quantum algorithms, often approximate knowledge suffices).
Example. Like query access, we can get efficient sample and query access from an explicit memory structure. To get for a bit vector , store the number of nonzero entries and a sorted array of the 1indices . For example, we could store as
Then we can find by checking if , we can sample from by picking an index from uniformly at random, and we know , since it’s just . This generalizes to an efficient binary search tree data structure for for any .
We can also define sample and query access to matrices as just sample and query access to vectors “in” the matrix.
Definition. For , is defined as for the rows of , along with for the vector of row norms (so ).
By replacing quantum states with these classical analogues, we form a model based on sample and query access which we codify with the informal definition of “dequantization”.
Definition. Let be a quantum algorithm with input and output either a state or a value . We say we dequantize if we describe a classical algorithm that, given , can evaluate queries to or output , with similar guarantees to and query complexity .
That is, given sample and query access to the inputs, we can output sample and query access to a desired vector or a desired value, with at most polynomially larger query complexity.
We justify why this model is a reasonable point of comparison two sections from now, in Implications. Next, though, we will jump into how to build these dequantized protocols.
Quantum for the quantumless
So far, all dequantized results revolve around three dequantized protocols that we piece together into more useful tasks. In query complexity independent of and , we can perform the following:

(Inner Product) For , given and , we can estimate to error with probability ;

(Thin MatrixVector) For , given and , we can simulate with queries;

(Lowrank Approximation) For , given and some threshold , we can output a description of a lowrank approximation of with queries.
Specifically, our output is for , , and (), and this implicitly describes the lowrank approximation to , (notice rank ).
This matrix satisfies the following lowrank guarantee with probability : for , and (using ’s SVD),
This guarantee is nonstandard: instead of , we use . This makes our promise weaker, since it is useless if has no large singular values.
For intuition, it’s helpful to think of as multiplied with a “projector” that projects the rows of onto the columns of , where these columns are “singular vectors” (approximately orthonormal, and with corresponding “singular values” that are encoded in the diagonal matrix ).
The first two protocols are dequantized swap tests and the third is essentially a dequantized variant of phase estimation seen in quantum recommendation systems^{2}.
Now, we describe how these techniques are used to get the results for recommendation systems, supervised clustering, and lowrank matrix inversion. We defer the important details of models and error analyses to Implications, instead focusing on the algorithms themselves and how they use dequantized protocols.
Supervised clustering
We want to find the distance from a point to the centroid (average) of a cluster of points . If we assume sample and query access to the data points, computing reduces to solving the following:
Problem. For , and , approximate to additive error.
Algorithm. We can write as the inner product of an order three tensor; through basic tensor arithmetic, it is equal to , where are
Applying the algorithm for inner product (1) gives the desired approximation with samples and queries.
Recommendation systems
We want to randomly sample a product that is a good recommendation for a particular user , given incomplete data on userproduct preferences. If we store this data in a matrix with sampling and query access, in the right model, finding good recommendations reduces to:
Problem. For a matrix along with a row , given , approximately sample from where is a sufficiently good lowrank approximation of .
Remark. This task is essentially a variant of PCA, since a lowrank decomposition is dimensionality reduction of the matrix, viewed as a set of row vectors. This is the “dequantized PCA” I refer to in other work^{3}.
Algorithm. Apply (3) to get for a lowrank approximation . It turns out that this lowrank approximation is good enough to get good recommendations. So it suffices to sample from , where with .
Approximate to norm using inner product protocols (1). Next, compute with naive matrixvector multiplication. Finally, sample from , which is a thin matrixvector product (2).
An aside. This gives an exponential speedup over previous classical results from 1520 years ago^{4}. The story here is quite odd. From what I can tell, researchers at the time knew the important (read: hard) part of the algorithm, how to compute lowrank approximations fast, but didn’t notice that the resulting knowledge of and could be used to sample the desired recommendations in sublinear time, which I think is much easier to understand. This gave me anxiety during research, since I figured there was no way this would have been overlooked. I’m glad these fears were unfounded; it’s cool that this quantum perspective made this step natural and obvious!
Lowrank matrix inversion
The goal here is to mimic a quantum algorithm that can solve systems of equations for lowrank. The dequantized version of this is:
Problem. For a lowrank matrix and a vector , given , (approximately) respond to requests for , where is the pseudoinverse of .
Algorithm. Use the lowrank approximation protocol (3) to get . From applying the matrixvector protocol (2), we have , where ; with some analysis we can show that the columns of behave like the right singular vectors of . Further, $\hat{\Sigma}_{ii}$ behaves like their approximate singular values. Using this information, we can approximate the vector we want to sample from:
We approximate to additive error for all by noticing that is an inner product of the order two tensors and . Thus, we can apply (1), since being given implies for viewed as a long vector. Finally, using (2), sample from the linear combination using these estimates and .
Implications
We have just described examples of dequantized algorithms for the following problems:
 Recommendation systems^{5}^{2} (this classical algorithm exponentially improves on the previous best!)
 PCA^{3}^{6}
 Supervised clustering^{3}^{7}
 Lowrank matrix inversion^{8}^{9}^{10}
We address here what to take away from these results.
For quantum computing
The most important conclusion, in my opinion, is a heuristic:
Heuristic 1. Linear algebra problems in lowdimensional spaces (constant, say, or polylogarithmic) likely can be dequantized.
The intuition for this heuristic is that, if your problem operates in a subspace of such low dimension, the main challenge is “finding” this subspace and rotating to it. Then, we can think about our problem as lying in where is small, and can solve it with a simple polynomialtime (in ) algorithm. Finding the subspace is an unordered search problem if you squint, so can’t be sped up much by exploiting quantum.
Remark. There are highdimensional problems that cannot be dequantized; for example, given , it takes queries to approximately sample from , where is the Hadamard matrix (this is the Fourier Sampling problem^{11}).
Why do we care about dequantizing algorithms? As the name suggests, I argue that this is a reasonable classical analogue to quantum machine learning algorithms.
Heuristic 2. For machine learning problems, SQ assumptions are more reasonable than state preparation assumptions.
That is, the practical task of preparing quantum states is probably always harder than the practical task of preparing sample and query access. Practically, this makes sense, since for state preparation we need, well, quantum computers.
Quantum computing applications that are realizable with zero qubits!
Even assuming the existence of a practical quantum computer, there is evidence that state preparation assumptions are still harder to satisfy than sample and query access, up to polynomial slowdown. For example, preparing a generic quantum state corresponding to an input vector takes quantum queries to in general, while responding to accesses takes classical queries. Because dequantized algorithms are polynomial in , this means that getting SQ access to a generic vector is much more expensive than running the algorithm.
Of course, we can also consider special classes of vectors where quantum state preparation is easier, but generally SQ access gets proportionally faster as well. For example, we can quickly prepare vectors where all entries have roughly equal magnitude (think vectors whose entries are either or ), but correspondingly, we can compute SQ accesses to such vectors similarly quickly.
On the classical side, the assumption of SQ access is on par with other typical assumptions to make machine learning algorithms sublinear:
 There is a classical dynamic data structure that supports SQ access, fast updates, and sparsity in log time.
 Given an input vector as a list of nonzero entries, sampling from it takes time linear in sparsity.
 independent samples can be prepared with one pass through the data in space.
To summarize these heuristics: quantum machine learning for lowdimensional datasets will probably never get speedups as significant as, say, Shor’s algorithm, even in bestcase scenarios. Unfortunately, QML for lowdimensional problems were the most practical algorithms in the literature, so with this research it’s unclear what the state of the field is today.
The story might not be over, though. We know that quantum computers can “efficiently solve” highdimensional linear algebra problems^{12}; however, this assumes that we have some way to evolve a quantum system precisely according to input data, a much harder problem than the linear algebra itself. Nevertheless, I hold out hope that this result can be applied to achieve exponential speedups in machine learning or elsewhere.
For classical computing
I am cautiously optimistic about the implications of this work for classical computing. The major advantage of dequantized algorithms is sheer speed (asymptotically, at least). However, the issues listed below prevent dequantized algorithms from being strict improvements over current algorithms.
 Gaining SQ access to input typically requires preliminary data processing or the use of a data structure. This means that dequantized algorithms can’t be plugged into existing systems without large amounts of computation.
 SQ access to output might not always be useful or practical.
 Current dequantized algorithms have large error compared to standard techniques.
 Current algorithms have large theoretical exponents, so right now we don’t know whether they run quickly in practice. I expect we can cut down these exponents greatly.
If I had to guess, the best chance for success in dequantized techniques remains recommendation systems, since speed matters significantly in that context. I view the other algorithms as significantly less likely to see use in practice, though probably more likely than their corresponding quantum algorithms.
Regardless, these works fit nicely into the classical literature: dequantized quantum machine learning is just a nicely modular, quantuminspired form of randomized numerical linear algebra.
Appendix: More details
As a reminder, here are the three techniques:
 Inner Product
 Thin MatrixVector
 Lowrank Approximation
Below, we explain (1) and (2) fully, and give a rough sketch of (3).
1. Estimating inner products
First, we give a basic way of estimating the mean of an arbitrary distribution with finite variance.
Fact. For i.i.d random variables with mean and variance , let
Then with probability , using only copies of .
Proof sketch. The proof follows from two facts: first, the median of is at least precisely when at least half of the are at least ; second, Chebyshev’s inequality (applied to the mean).
Estimating the inner product is just a basic corollary of this estimator.
Proposition. For , given and , we can estimate to error with probability with query complexity .
Proof. Sample from and let . Apply the Fact with being independent copies of .
2. Thin matrixvector product with rejection sampling
We first go over rejection sampling, a naive way to efficiently generate samples from a specified distribution from samples from another distribution.
Input: samples from distribution
Output: samples from distribution
 Pull a sample from ;
 Compute for some constant ;
 Output with probability and restart otherwise.
Fact. If for all , then the above procedure is welldefined and outputs a sample from in iterations in expectation.
Proposition. For and , given and , we can simulate with expected query complexity , where
We can compute entries with queries.
We can sample using rejection sampling:
 is the distribution formed by sampling from with probability proportional to ;
 is the target .
Notice that we can compute these ’s (in fact, despite that we cannot compute probabilities from the target distribution), and that the rejection sampling guarantee is satisfied (via CauchySchwarz).
The probability of success is . Thus, to estimate the norm of , it suffices to estimate the probability of success of this rejection sampling process. We can view this as estimating the heads probability of a biased coin, where the coin is heads if rejection sampling succeeds and tails otherwise. Through a Chernoff bound, we see that the average of “coin flips” is in with probability , where each coin flip costs queries and samples.
3. Lowrank approximation, briefly
Proposition. For , given and some threshold , we can output a description of a lowrank approximation of .
Specifically, our output is for , (), and this implicitly describes the lowrank approximation to , (notice rank ).
This matrix satisfies the following lowrank guarantee with probability : for , and (using SVD),
This algorithm comes from the 1998 paper of Frieze, Kannan, and Vempala^{13}. See the recent survey^{14} by Kannan and Vempala for a survey of these techniques, and see Woodruff’s textbook^{15} for a discussion of more general techniques. The form I state above is a simple variant that I discuss in my recommendation systems paper^{5}.
It would be nice to find an exposition that made this result look quantumic, but I haven’t figured out a way to do this yet, so instead I’ll exposit following the original reference.
We now sketch the algorithm along with intuition: it’s most useful to consider the lowrank approximation task as one of finding large approximate singular vectors. First, sample rows of according to norm, and consider the matrix of these rows, all renormalized to have the same length. This is the that we output. One can show that with good probability, which implies that the large right singular vectors of approximate the large right singular vectors of .
Next, we can perform the same process to : sample rows of and get a normalized submatrix such that . Since is a constantsized matrix, we can compute and , the large left singular vectors and values of , which approximate the large left singular vectors and values of . Then, translates these large left singular vectors to their corresponding right singular vectors and rescales them accordingly, giving the approximate singular vectors of as desired.
Glossary
For natural numbers , vector and :
denotes ; and is big O notation; and denotes the th row of and the th column of ; denotes the norm of , ;
is braket notation: kets are column vectors , bras are row vectors , standard basis vectors are denoted , and the tensor product of and is denoted . Of course, these are all really quantum states, but that’s only relevant for quantum algorithms: for my purposes, I use and interchangeably to refer to vectors. (I ignore normalization, but those issues can be dealt with.)
The singular value decomposition (SVD) of is a decomposition , where and are unitary and is diagonal. In other words, for and the columns of and , respectively, and the diagonal entries of , . By convention, .
Using ’s SVD, we can define basic linear algebraic objects. is the spectral norm of . is the Frobenius norm of . is an optimal rank approximation to in both spectral and Frobenius norm. is ’s pseudoinverse.
I define , , and in An introduction to dequantization.
References

Scott Aaronson. Read the fine print. Nature Physics 11.4, 2015. Link ↩

Iordanis Kerenidis, Anupam Prakash. Quantum recommendation systems. arXiv:1603.08675. ↩ ↩^{2}

Ewin Tang. Quantuminspired classical algorithms for principal component analysis and supervised clustering. arXiv:1811.00414, 2018. ↩ ↩^{2} ↩^{3}

Petros Drineas, Iordanis Kerenidis, Prabhakar Raghavan. Competitive recommendation systems. STOC, 2002. Link. ↩

Ewin Tang. A quantuminspired algorithm for recommendation systems. arXiv:1807.04271, 2018. ↩ ↩^{2}

Seth Lloyd, Masoud Mohseni, Patrick Rebentrost. Quantum principal component analysis. arXiv:1307.0401, 2013. ↩

Seth Lloyd, Masoud Mohseni, Patrick Rebentrost. Quantum algorithms for supervised and unsupervised machine learning. arXiv:1307.0411, 2013. ↩

Patrick Rebentrost, Adrian Steffens, Iman Marvian, Seth Lloyd. Quantum singularvalue decomposition of nonsparse lowrank matrices. arXiv:1607.05404 . ↩

András Gilyén, Seth Lloyd, Ewin Tang. Quantuminspired lowrank stochastic regression with logarithmic dependence on the dimension. arXiv:1811.04909, 2018. ↩

NaiHui Chia, HanHsuan Lin, Chunhao Wang. Quantuminspired sublinear classical algorithms for solving lowrank linear systems. arXiv:1811.04852, 2018. ↩

Scott Aaronson and Lijie Chen. Complexitytheoretic foundations of quantum supremacy experiments. arXiv:1612.05903, 2016. ↩

Aram W. Harrow, Avinatan Hassidim, Seth Lloyd. Quantum algorithm for solving linear systems of equations. arXiv:0811.3171, 2008. ↩

Alan Frieze, Ravindran Kannan, Santosh Vempala. Fast montecarlo algorithms for finding lowrank approximations. Journal of the ACM, vol. 51, no. 6, 2004. Link. ↩

Ravindran Kannan and Santosh Vempala. Randomized algorithms in numerical linear algebra. Acta Numerica 26, 2017. Link. ↩

David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science 10.1–2, 2014. Link. ↩