In the programming of a table-based application module (i.e. the user mostly enters tabular data in an already laid-out table), how would you reject user input for a given cell?

The scenario is: the user edits the cell, enters something (text, picture, ...) and you want them to notice when they finish editing (hitting enter, for example) that their entry is not valid for your given "format" (in the wider meaning: it can be that they entered a string instead of a number, that their entry is too long, too short, they include a picture while it's not acceptable, ...).

I can see two different things happening:

1. You can rather easily fit their entry into your format, and you do so, but you want them to notice it so they can change if your guess is not good enough (example: they entered "15.47" in a field that needs to be an integer, so your program makes it "15")
2. You cannot guess what to do with their entry, and want to inform them that it's not valid.

My question specifically is: what visual display can you offer to inform the user that his input is invalid? Is it preferable to refuse to leave the editing mode, or not?

The two things I can imagine are:

• using colors (red background if invalid, yellow background for my case 1 above)
• when you reject an input, do something like Apple does for password entry of user accounts: you make the cell "shaking" (i.e. oscillating left and right) for one second, and keep the focus/editing in their so they don't loose what they've typed.

Let's hear your suggestions.

PS: This question is, at least in my thought process, somehow a continuation and a specialization of my previous question on getting users to read error messages.

PPS: Made this community wiki, was that the right thing to do on this kind of question or not?