|Think about Loose Coupling|
You're quite right, Fermat's method will only find two factors: it is important to note that all the fastest factoring algorithms (with the exception of the elliptic curve method) use this as a basis.
The Quadratic Sieve works as follows:
If n is the number to be factored, the QS tries to find x and y such that x^2 = y^2 (mod n) and x is not equal to plus or minus y; this implies that (x-y)(x+y)=0 and we compute GCD(x-y,n) to see if this is a non-trivial divisor. There is at least a 1/2 chance that the factor will be non-trivial. The first step is to define
and compute Q(x_1),Q(x_2),...,Q(x_k). Determining the x_i requires sieving over a factor base (using primes such that the Legendre symbol (n/p)=1). From the evaluations of Q(x) we want a subset Q(x_i1)Q(x_i2)...Q(x_ir) that is a square, y^2. Then note that for all x, Q(x)=x'^2 (mod n). So what we now have is that
Q(x_i1)Q(x_i2)...Q(x_ir)=(x_i1*x_i2*...*x_ir)^2 (mod n)
Which are the factors of n.