Your beep comes from the startup transient, while the processor comes out of reset. If you invert the output driving the buzzer that will do what you want. Alternatively, you can do the same job by using a PNP transistor between Vcc and the load. Emitter to Vcc, Collector to load and base to your output. The transistor is off if the base is less than 0.6 volts below Vcc, ie when the output is +5.

Steve