(Note: Donncha may want to look at this.)
I've been debugging a problem the last couple days, related to getting blog options from blogs other than the current blog. (Imagine widgets that render content from other blogs.)
What I noticed in debugging the code is that...
$val = get_option('foo');
...will access the "options" cache group.
While on the other hand...
$val = get_blog_option(1,'foo');
...will read from "site-options" cache group.
This seems like it could be a bug... as I assumed, and I think must plugin developers assumed that get_blog_option(1,'foo') would be the same as get_option('foo') from the blog_id=1 context.
In the old old old code (1.0 and earlier) get_blog_option() in fact did this, it would do a switch_to_blog() and then get_option().
As implemented today though, you could easily have someone setting/getting two different cache options that are out of sync.
Is this a bug? Should I report this on Trac?