Well, the RAV4 is AWD, not 4x4. It has no low range, and it has a limited ground clearance. We can do something about the 4x4 issue by putting locking differentials in, and we can raise the chassis using lift kits. However, there is nothing that we can do to put low range in the RAV.
Mention's been made about other more serious 4x4's using ladder frames - the RAV4 uses a monocoque construction which is inherently weaker. However, Mitsubishi saw fit to build the Pajero this way, so I'm not all too sure that for recreational and leisure purposes, a monocoque construction is such a bad thing. We're not driving 2 tonne pick-ups here that are being put to hard work. We're not climbing over rocks where individual wheels will find themselves in the air and be unable to support the vehicle's weight either.
I think that the RAV uses limited slip differentials, front, rear and central, so it's drive system is perhaps better called 4WD than AWD. This 4WD system fails if two wheels on the same side of the vehicle lose traction. So, if you drive through a portion of terrain with firm ground under one side of the vehicle, and loose mud on the other, you risk losing all traction. A locking differential will help this by making sure the power doesn't get fed to the wheels that have no traction. With AWD, if only one wheel loses traction, then all the engine power goes to that single wheel.
Lifting the RAV is easy. I'm waiting to get some Old Man Emu springs for mine, for an extra 40mm ground clearance. A cheaper way to do it is to put some bungs in-between the suspension coils and the housing, though I'm not sure what real disadvantages this has over using proper springs.
So, I believe the ultimate limiting factor is the lack of low range. Low range is the bee's knees for crawling through rivers, over rocks, through mud and other such obstacles. For steep hill descent low range is a help beyond all others. Particularly if there's any risk of losing traction when you touch the brakes. That's the real issue here; when you start to slide going down-hill, do your utmost to never touch the brakes, doing so will risk your traction, and if you lose traction, you will start to slide. Better to go a bit too fast than to use the brakes and potentially getting into a slide you'll never come out of.
The RAV4 advantages that rav4_at and anthonyd have mentioned are a definite! The RAV is the very best thing for loose sand because it is so light. On loose sand, you don't want to use low range anyway, doing so can get you stuck, it has too much digging ability.
So, compared to a Jeep Cherokee, a Land Rover Defender, a Toyota Hilux (kinda like a Tacoma) or such vehicles, the stock RAV falls very, very far short. Best you see it as a Recreational Activity Vehicle with 4WD (RAV4) rather than a off-roader.