When I started driving my RAV over a year ago, I did notice that it was not as quiet as I would like, but initially I dismissed it thinking that I would get used to it. Well, this did not work out. As time went, I found both the level and the "tone" of the noise to be too obtrusive, eventually being one of the issues that made me sell the RAV.
I don't think my RAV got progressively more noisy with time, but I certainly became more aware about the issue.
In my opinion, Toyota just cheapened out on the noise insulation in the RAV. Camry, which has exactly the same engine, is much more quiet. My new Venza's engine is about as noisy as the RAV's with the hood open, but once inside the car the difference is HUGE. Also, any external noises (from tires, other cars, etc.) are much better dampened in the Venza, Camry, HIghlander, ... It's not rocket science, Toyota just saved a few dollars by skimping on sound insulation in the RAV. Unfortunately the only fix would be to add sound insulating / absorbing material to doors, headliner, firewall and wheel wells to a similar level as Toyota does in the Camry, Highlander, ... etc.