You do want to be using SSL. Unencrypted connections are far too easily eavesdropped. On the other hand, it’s worth remembering that SSL only gets your login and email encrypted between your computer and your smtp server. Once your email hits the big pipes, it’s unencrypted again, vulnerable to whomever can get access to the transit points. A rogue operative in any ISP or fiber optic supplier could still siphon off huge amounts of data. Even if such a person existed, s/he would be unlikely to be able to regularly get all of your email though. However, random emails, especially if they traverse exotic territories with loose security could be grabbed.
Email is not private. Don’t forget that ever. Email is not private.