Just because you're paranoid doesn't mean that someone not out to get you. ;-)
[slashdot] "The people who bring you the DroneBL DNS Blacklist services, while investigating an ongoing DDoS incident, have discovered a botnet composed of exploited DSL modems and routers. OpenWRT/DD-WRT devices all appear to be vulnerable. What makes this worm impressive is the sophisticated nature of the bot, and the potential damage it can do not only to an unknowing end user, but to small businesses using non-commercial Internet connections, and to the unknowing public taking advantage of free Wi-Fi services. The botnet is believed to have infected 100,000 hosts." A followup to the article notes that the bot's IRC control channel now claims that it has been shut down, though the ongoing DDoS attack on DroneBL suggests otherwise.
This could be your Verizon or Comcast DSL modem (probably not); use a good strong (non-default) password and DON'T administer it from the Internet. Just saying.