Drivetrain noise when releasing accelerator pedal
So I've noticed this ever since I bought it... if I'm on the throttle to 3k RPMs or more, especially at 65-70 on the highway or starting from a stop quickly, then release the gas, there's a thud thud (two distinct noises) from the rear. The noises are not loud, but prominent, they sound very much like if I ran over a bump on the road. Only when I release the pedal suddenly, otherwise there is no noise.
I had my mechanic listen to it, he says he's noticed this on several Toyotas around the 4.1 years, none seemed to result in any damage. He says it would be a witch hunt trying to pick exactly where it is coming from and would likely require a full rebuild of the rear end transmission components if I did want to try fixing it, but he says it's not worth it, the transmissions he's seen do this have gone longer than the engines they were attached to in most cases. I'm inclining to agree, that shop has been very good to my family, but just double checking.
Anyone have this issue and/or a better explanation?