It's really simple. You just open the door and remove the black triangular cover on your window that covers where the mirror goes. It's just clipped in there and you just have to give it a nice tug on the edges and it'll pull out. After removing that (if you have tweeters), then un plug it or you can just leave it dangling with the wires. Now you can see the nuts holding your mirror in place. Just unplug all the wires and unscrew the nuts and then remove the mirror assembly. I never did it on the Rav4 before, but I have removed lots of mirror assemblies in the past and they are all like that.
2009 Rav4 V6 4X4 Sport