Genre is a relatively young creation, one spawned in the early 20th Century with the birth of the literary canon and the development of the business of publishing as a result of cheaper paper and ink. Before this, there was certainly high and low fiction but they were personified by the novel, the romance and either Gothic or the penny dreadful (which grew into the pulp magazines of the 1920s and 30s). Genre came later when there more people with the leisure time to read and likely dates to the 1920s. Terms like Science Fiction (a bastardisation of the term 'Scientifiction' which was deemed so awful that nobody took it seriously) come to us from people like Hugo Gernsback, the Belgian magazine proprietor who specialised in that sort of story. I believe, despite its use, genre has become effectively useless as a term because there are so many strands within genres that a flat 'Fantasy' or 'Science Fiction' or even 'Horror' (which arguably has been the most scorned and bastardised, a vampire in a novel does not a horror novel make and a lot of the books out on the other shelves of your local bookstore probably have more scary stuff in them than a great deal of the volumes littering the Horror section).
Over the past decades we have, like it or not, seen genre fiction go from strength to strength, endlessly diversifying and changing to reflect the Zeitgeist of the wider world, picking up its own tics and language; it's own cliches and ways of doing things. In 2014 it has become mainstream, as the success of Game of Thrones attests. Despite this, it is folly to brush the many different kinds of story into one bushel; from the moment that the genres were 'born', as in they were booted out of general fiction and into their own specialised market/ghetto, differences have been apparent. There were already different voices and attitudes apparent in American and British approaches within fantasy fiction to what was suddenly a single genre. The British approach drew from Lord Dunsany and the tradition established by the Pre-Raphaelite poets in their plundering of mythology. This would eventually lead to Tolkien's Middle Earth and the long shadow it casts over a lot of the genre. American writers were far more couched in the pulp tradition and drew as much from Westerns and the Penny Dreadfuls of their time as they did other work, creating Sword and Sorcery as a result. Whilst Tolkien mourned the dying of goodness, cast against the slow decline of the British Empire, writers like Howard were creating muscular heroes who were out for daring do and the spoils of adventure. So even at the start, for Fantasy at least there were divergent cultures.
This is in part because the world has changed, the culture of deference and the focus on brave white, middle class men*, involved in 'Important Things' has gone away for the most part. Our heroes today are often outsiders, often people in the gutter, who have lost almost everything. This is a part of the effect that so much fiction being published has had, over time the genres have become nuanced and now have many different voices within them. Much of this is a question of legacy and reaction of course, as new writers came along they wrote their visions, which often clashed with those of older generations. Ideas travelled too, Moorcock's work is largely nested in Sword and Sorcery but he was obviously influenced by Hippy culture and the moral relativism of the 1960s. Good and Evil vanish, replaced by Law and Chaos, taking a step away from the Christianised visions of previous generations. Within SF similar developments occurred, albeit more slowly. By the 1980s a dark edge of near future SF (in the form cyberpunk but also other dystopian visions) had largely replaced the Utopian edge of earlier writers. Science had fallen from grace as the fabled technocracies failed to emerge, again the outsider and the criminal became the focus as the genre began to ask the question 'what if the future is a busted flush?' Fantasy has followed, shedding the shadow of The Lord of the Rings more and more with time until today, whilst it remains to an extent it is probably the thinnest it has ever been.. The latest trend in the genre has been almost an epic Sword and Sorcery, using the simplicity of characters like Conan but setting them against problems that encompass the entire world. Elves and the like have been banished in favour of political intrigues and an emphasis on some kind of historical realism.
With all these changes the question, for me becomes, is Tolkien's Lord of the Rings really comparable to Mary Gentle's Rats and Gargoyles in anything other than a superficial way? They both take place in other worlds, born purely out of imagination, but at the same time they draw from different sources, build worlds that are starkly different. There is little to unite them. What really connects the Golden Age Foundation Asimov novels like to something like Moxyland by Lauren Beukes? It is true you could not have one without the other and there is a definite legacy in the latter, not only to Aasimov but also to the work in the 1980s. This though is the only link, the societies and assumptions described in each work are fundamentally different. They are such different flavours that to group them under the single genre seems to be a disservice and surely only confuses people new to this kind of fiction.
In this context I can understand why things like Waterstones' lists of authors are so important, though they need to work on making them more inclusive. But I do feel that if you need a guide to a genre not just because of its size but because there are so many movements and tones within it then really the idea of genre has served its purpose and we need a new one. This is before we even consider crossover fiction, which usually blends two genres together, the most famous example probably being supernatural thrillers like the Harry Dresden novels, though we could also consider books like Pride and Prejudice and Zombies as a form of crossover fiction. The trend seems to be kicking away from discrete genres and back to the state of affairs in the 19th Century, where there was only fiction.
And that's probably the best place to be.
* You can probably add straight, conservative and other unflattering adjectives to that list too.