Pretty easy, really. As a precaution, make sure you have a recent system restore point!
Click start/run and type in regedit and hit enter (runs program called regedit).
Then just navigate your way to each entry you want to delete. Highlight it by clicking on it and press the Del key.