Ofir,

Let's think about this from a signal processing sense, ignoring the implementation, to stimulate thought about what approaches make sense to try.

The original voice signal has particular spectral properties. It is those properties that give the signal its uniqueness. Altering those properties will alter the perceived character of the voice.

A simple model of the voice might be a carrier, modulated by a filter. An improvement could include a broadband noise source to simulate a whisper, while a variable frequency source simulates tonal variation associated with speaker excitement or emphasis. The filter following the carrier source could be a time variable filter to simulate the attack, decay and amplitude of sounds associated with various letters and combinations of letters.

Based on the above, you could detect the instantaneous frequency of the carrier using a high gain amplifier, and, rectify and low pass the original to find the envelope. Expect to work out some details with the low pass filter since it will have time varying coefficients.

It remains for you to detect the original signal components, alter them in software and re-issue them in altered form.

Bruce