The memory function will be affected by the illegal use of drugs. It kills brain cells, as does the long-term overuse of alcohol. Two prescription drugs that have had Memory Loss as side effects are Prozac and Zoloft. The patients’ symptoms would get better as far as the reason they were put on these drugs, but once memory loss began to develop, the patients would have to be taken off them.