Since you are too lazy to capitalize and spell correctly (e.g., "cpl", "coz"), I won't say what all is wrong with your routine, but will simply offer this:

void bintodec(byte bth)
{
    serial[2] = bth % 10 + '0';
    bth /= 10;
    serial[1] = bth % 10 + '0';
    bth /= 10;
    serial[0] = bth '0';
}