I agree with this. At least in part, I understand that this probably is the most sought after product, and therefore is hard to remove from the game.
However, I think it could deter some players, perhaps a function that automatically empties resource buildings after 150% of the original time.
Meaning, if it takes 3 hours for them to fill up, they could empty automatically every 4h 30m. This will leave the bonus for those with gems, as well as it will make it much better for those who choose not to.