Old Evidence and Apologies

If the problem of old evidence for Bayesian epistemology is just the following, then I don't think it's a problem:

Sometimes it seems like we should change our probabilities based on discovering logical consequences of a theory, but Bayesian updating only involves changing probabilities when you make a new observation.

For (it seems to me) this objection has the same ultimate structure as the following, surely bad, objection:

Sometimes it seems like we should apologize, but obeying so-and-so's moral theory involves never wronging anyone - and hence never apologizing.

If old evidence E is logically incompatible with hypothesis H, then Bayesianism says that you should *already* have ruled out all the worlds where H is true, and changed your probabilities accordingly, whenever you observed that E. So, I see no problem for the Bayesian epistemologist in saying that when you discover that you have failed to update in the way required by the theory (by not noticing a logical incompatibility), you should fix the mistake and change your probabilities accordingly.

[Compare this with the following popular intuition in ethics: you should promise to visit your grandmother and then visit her, but given that you aren't going to visit you shouldn't promise to visit her.]