Found an excellent guide (in Swedish) to remove PulseAudio. Basically do this:
alsamixer
Check that your system uses Pulseaudio (it says clearly by Card and Chip) in the top left corner.
sudo rm /etc/X11/Xsession.d/70pulseaudio
killall pulseaudio
sudo apt-get remove pulseaudio
sudo apt-get install esound
Then run:
alsamixer
And check that it is not using PulseAudio any longer.
I will report after thorough testing if it works fine.