Your experience is echoed among many FF users however FF is vulnerable to ActiveX attacks and malicious code. (see below) ActiveX is a core component of windows and Windows based browsers (i don’t know about Opera) will include a shim or adapter to execute code. All an attacker has to do is push the malicious code through the ActiveX connector and bam, same exploit used in IE now works for FF or Chrome…
TBH: I use FF and OSX at home, right now I don’t worry about virus or spyware at all.