I got these symptoms following upgrade 1.2.5a to 1.3.3.
 (capital A with circumflex) characters were appearing at random places in blog posts. In forum post "Upgrade messed up with content" kishorebudha suggested changing the database character set setting:
I think you use the new config file. The original config file has the following
DB_CHARSET(utf8)
I changed it to DB_CHARSET(utf-8)
Can you confirm you have tried the same.
That fixed the display of spurious characters, and I had left it like that, but it was causing this problem.
I've now changed it to latin1, having found that is the default character set for my database. I can now access new blogs, and do not have spurious characters displaying.
mysql> show create database wordpress;
gives
CREATE DATABASE wordpress
/*!40100 DEFAULT CHARACTER SET latin1 */
Now thinking about whether I should be planning to change that to utf8, to keep things standard...