Hi Deepak,
Weather it is a concern in your case depends on
1. Kind of load are you switching.

Usually when switching inductive loads such as motors, solenoids etc. you should care more about noise than switching resistive loads.

2. Amount of current you are switching.

If the magnitude of current being switched is huge you will get more noise.

3. Voltage of AC cycle at the moment of switching.

The amount of noise generated is in proportion to the instantaneous voltage of AC cycle. If you switch your relay when the voltage is crossing zero line, you will get far lesser noise.

You should carefully implement the Erik Malund's noise supression techniques in order to keep the noise minimum. Precaution is always better than cure.

Regards,
Prahlad Purohit