To be fair the judges all work in the industry so they understand who is likely to be marketable or not, so they probably have a good idea of who will be the best for getting people to watch the show.
I don't get why people are surprised that it isn't a straightforward singing or talent contest, its an entertainment show that succeeds or fails based on maximising its audience figures so that it can make more money by selling advertising (think what Talk Talk etc are paying), there have been other reality shows that have marketed themselves more on being a straightforward search for talent but they don't get the same audiences. They try to cover as many demographics as possible eg there will always be an Irish contestant on it because that gets Irish viewers, they love the idea of an Irish underdog winning a British show...they will try and spread the others across the regions.
Also the music industry has types of singer/types of image that are 'in' like the fashion industry does, which Gary Barlow pretty much openly admitted, that was why he rejected that lad with fake tan and the veneers on his teeth, who was a decent singer but just looked a bit dated, whereas the skinny indie boy with skintight jeans and the tattoos on his ass and the scouser that always wears a hat were just average singers but have got more marketable images in the current climate. Had Adele not been around, then the fat Scottish girl would have been a good contestant as she has a good voice, but really that space in the market has been filled, there isn't enough space for two of them (and to be fair, she was a poor man's Adele, and Adele herself is a poor man's Alison Moyet).
Every year people say this stuff about "it's a joke, it should be able the best singers going through" but also every year there will be a 'shock eviction' in week 4 or 5 where one of the better singers goes out and some of the joke acts are still in - thats perfectly plausible, people don't vote for the best singers they vote for the entertaining ones that they want to watch on TV. And as for these theories that the public vote is 'fixed'....I very much doubt it. The producers will use the public vote as the guide for which characters people want to watch, and they will make sure they stay in, because the aim of the producers is to maximise their profits, and EVERYTHING will be about the audience rating figures each week, the whole leverage they have with advertisers, what big name stars they can get to perform on the show etc, it all comes down to how the audience rating figures compare to everything else on TV. Given that there is so much multimedia and different distractions these days, to get a show on mainstream TV that has huge ratings is very difficult, so much as people like to slag Simon Cowell off, he has achieved a pretty difficult feat to get the ratings that his shows get.