The original question specifically stated 1 master and many slaves - this is precisely what the serial mode 3 is for:

The 9th bit is used as a Address/Data flag; the master sets the 9th bit at the start of a message to indicate that the byte contains a slave address;

Only the addressed slave takes any notice of the message (the address recognition is done in hardware if your variant has the "enhanced" serial port);

A Slave can transmit only in response to a command addressed to it - thus collisions are avoided.