I have a few designs that do not have pullups at all. Internal 51 port pullup is enough, but of course the bus capacitance must be kept low in this case (short tracks on PCB).