Perhaps an Array of Arrays is a better kind of storage?
There will be "holes" in your AoA of course, but you can use a sliding window of (for example) length 4 and it will be easy to see what needs to be dropped or added. The low limit needs to be dropped and the upper limit needs to be added and it is easy to check if they contain anything or not.
CountZero
A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James