On mine (2003 model) there are three different part numbers... the top two A/F sensors are Denso 234-9023 (left) and 234-9028 (right, same sensors but different plugs) and the bottom two O2 sensors are Denso 234-4048. I'd expect this is the same for all the 2001-2003 RAVs with 1AZ-FE engines.
Highly recommend using the Denso parts. I've previously made the mistake of installing aftermarket A/F sensors and I had intermittent heater codes as well as some fuel trim oscillation. Non-genuine downstream O2 sensors seem to be OK but I'd still go the Denso for peace of mind... no, not a Denso sales person
My experience has been that the slotted O2 sensor tools, such as the one Snyrav4 posted, work best, but the bottom sensor clearance is tight so you may have some fun getting the tool and your wrench in at the same time. All part of the fun of automobile maintenance. Don't forget to put some anti-seize on the threads when you reinstall, a little tube of it comes with the new sensors. Good luck!