That's a lot of memory ... oh - you're also going to have to work out things like blocking pieces (for non-knights) and the like. To tell you the truth, I'm not seeing a huge gain here. Pawns have to have the state of the board to determine ep. Granted, you have a set of optimizers to make the ep check pretty cheap, but it's still there.
Frankly, I would drop into XS and write the move generation code there. That is, of course, if the speed is that poor.
Also, a question no-one (including me) has approached is whether or not your original algorithm was good or not. 90% of all speedups happen in better algorithm choice.