Some of my favorite characters that I've created are, of course, my villains. (Or would it be better to refer to them as "antagonists"?) Regardless of how vile they become, I still love them. But I'm afraid... that might be my problem. The main antagonist could be best described as "chaos incarnated", a job he takes great pleasure in, but when he realizes that he's not the biggest threat possible to the story's world, he temporarily assists my protagonist in bringing down that other character, but just for that segment of the story. After that, he ends the temporary alliance and goes back to being the biggest bad the universe is capable of producing. Likewise, 3 of my other villains are each a different type of murderer with no remorse for their actions. However... all 3 of them strongly refuse to kill or seriously harm a specific demographic of people, which of course, is based off of their own personal histories, but everyone else is fair game to them, especially if their targets are of the opposite demographic. They are also under leadership of my main antagonist, so they follow through with whatever orders he gives to them, but they have to be forced into killing if their target is one of the people they refused to harm.

Is this bad? I want my audience to understand that they are "evil" but I don't want them to be flat either. I want to give my villains a reason for being the way they are, and not just throwing them in there because my protagonist and their allies need somebody to put behind bars. I want my readers to understand why these particular characters became evil, or even sympathize with them, but not condone their actions; I merely want them to know that they are the bad guys, and they are not people you would ever want as friends, but there is a reason why they turned out so rotten. It didn't just happen overnight. I also don't want to run the risk of them becoming "anti-heroes", or is that exclusive to protagonists? I want to make my characters (heroes and villains) likable for being them, regardless of their virtues or vices. 

...so when does a villainous character stop being "bad"? Is there a particular line they mustn't cross in order to remain a threat to the protagonist and innocent people?

--edited by VA Diaz on 1/27/2014, 1:28 AM--

A protagonist stops being bad when they stop opposing the protagonist. Having the reader understand what they do and why is good. It gives them depth, but it can still keep them mysterious. As long as they continue to oppose and fight against the protagonist, they will be "bad."

Miyazaki put it best when he said there is no such thing as an antagonist; only two people/groups of people fighting against each other and trying to get what they want. Sometimes that means hurting the general population, which is seen as bad. 

I have an antagonist who I love. He's so sweet, and he's being manipulated by his mother to do her bidding. But he's still trying to kill the main character's brother and best friend, so he's an antagonist. Will the reader feel sorry for him? Most likely. He's still "bad" because of what he does in the story.

I hope this helps!

~Linnea

In practical terms, if everyone is equally "likeable," your story is not likely to grab readers as firmly...it becomes more like a stranger watching a tug-of-war at a family reunion...the stranger doesn't care who wins, the kids on both side are cute.  The family members care, but they're in the story--the reader is outside the story, to start with.   And a perpetual tug of war--one that goes on too long but has equally attractive people on both ends of the rope--bores strangers in the end.  They'll walk off.  They want one side to cheer for, and the other side to cheer against.  The more polarity you can manage without it being silly, the most likely you are to have your readers glued to the page.

It is possible to have an ambivalent character (the "rogue" who is bad-in-law but likeable) as a protagonist, but then the opposition needs to be either smarmy-unlikeable-good, or bad and unlikeable.  If you've read Fritz Leiber's Fafherd & Gray Mouser stories, there are rogues (very likeable rogues) as protagonists, and various oppositions who are unlikeable in some way.   You can have protagonist and opposition who are equally places on the good/evil axis...but not on the likeable/unlikeable axis.   Or rather, you can, but it's harder to grab readers, who are quicker to adhere to likeability than to good/smart/strong/physically attractive.

So if you have a character you personally like and want to keep likeable, oppose him/her/whatever with someone who's less likeable.