The sensor at the top of the handle is for locking the door. When you want to unlock the door, you don't touch that (I assume that is what you mean by "push the button spot"). The sensor for unlocking the door is on the inside of the handle. Just wrap your fingers around the handle to activate that spot.
If you grab the handle and immediately pull to open the door, the system doesn't have time to react and the door won't unlock. The proper way is to grab the handle and do not pull it right away; pause for a half second to give the system time to react, then pull the handle.
When I wash my RAV4, water seems to activate the sensors; I will spray water on the car and the car will lock/unlock if I have the key fob in my pocket. I have learned to leave the fob in the house when I wash the car.
If you wear gloves (or maybe even if your hands are very dry), the car will not unlock. That leads me to believer that the sensors actual work by current conducting across the slots, not a capacitive sensing like the radio screen. That also explains why water may trigger the lock/unlock.
When you open the hatch, the other doors are automatically unlocked. (Closing the hatch does not lock the other doors). If you don't touch the lock sensor at the top of the door handle, the doors won't lock.