Or yet another way that is really simple to implement would be to take power from the external supply connection to establish a working supply for a small SPDT relay. The relay could default via the NC contact to connection to USB power and if the external power was present it would power the relay coil. This would simultaneously disconnect you from USB power and connect you to the external power. If both happened to be present then the external supply would override.

If you wanted the opposite default then just change the power source that supplies the relay coil and set the external supply connection to the relay NC contacts.

This solution is low cost, easy to engineer, and will give you essentially zero voltage drop.

Now I know some who read this will frown at the use of a relay like this but sometimes this kind of solution is overlooked because we all think relays should be history. But I say not so. Even your FET will have some voltage drop. Its going to always be the rule following the old adage, "You pay for it one way or the other". In this case it is either the price of a voltage drop across a solid state device or a small current load to drive a relay coil.

Hope this just might be the solution you needed...
Mike Karas