When it comes to the general theme of a universe, I think I've made it clear at this point that I prefer settings with a combination of harsh circumstances as well as established logic. I like scenarios where normal humans struggle against a powerful foe and must overcome them through quick thinking, fast reflexes, or good old-fashioned grit. However, I also like those things to make sense within the setting, rather than being pulled out just for the sake of having "the good guys win".
I'm going to introduce a term here, and that term is "Hard, but Fair". This basically describes what I like in a setting, show, or game: I like things to be difficult, but consistent. To me, both aspects are equally important in creating a sense of believability and drama. There are plenty of things that do one or the other - that make a world that's difficult and grim but without the balancing fairness, or that establish a logical world without a central, conflict-driving problem. So let's take a look at these two concepts, and I'll try to explain why I think they're so valuable.
"Hard": Adversity and Challenge
Easy tasks are not the stuff of legends. It's not a huge stretch to say that a simple or trivial task is generally considered to be boring. Even a work of fiction that makes things easy for the player or the characters is still going to pretend that it's epic and challenging. Whether the characters are afraid or not comes down to the theme of the work, but there's at least a veneer of difficulty. There are very few series where the hero says "Yes, I'm going to murder all of these people who are objectively weaker than I am and give them no chance to fight back" - they at least pretend that the enemies are some sort of threat. Why, even Star Wars: Return of the Sith had a throwaway "We have to get out before more security droids arrive" line, despite the laughable qualities of those particular foes.
While "difficult things" are universal as long as characters are vaguely human and mortal, a setting can encourage it in some ways. Some settings, such as Warhammer 40k, do this more than others. Still, even a relatively light-hearted campaign (one that's not all grim all the time) can establish an ongoing threat or challenge so that it seems like there's more at stake than "some adventurers go out into the wilderness and kill goblins who may or may not be evil". A lack of believable opposition can turn drama into melodrama - meandering and pointless, without any tension.
For example, one fairly simple contrast between Final Fantasy Tactics and Final Fantasy Tactics Advance was the introduction of the judges, who, among other things, prevented permanent death in areas under their control. It's more complex than that on both sides - FFT had healing and resurrection magic, and FFTA had certain areas without judges that behaved more like the original FFT - but this simple change turned a battle into a competition. Nobody dies in a judged battle, so it's more like two sports teams fighting - which, while still intense, is nowhere near as much so as an unavoidable fight to the death.
In this sense, difficulty is a tradeoff of "drama" for "ease". It's a risk analysis situation: the characters value their lives, but there's also valuable rewards and fame for defeating a powerful foe. Without that dynamic, it's just as brave and bold to fight goblins for a thousand levels as it is to fight dragons and giants and all the other classic foes. However, in some cases it goes beyond simply finding bigger things to fight - a properly "grimdark" setting can make it so that a character's entire life is an escalating battle, where even a peasant farmer must fight off rampaging ogres and trolls in addition to tending to the normal problems of life. Something doesn't have to be fantastic to be "difficult" - it's easy to forget that for many people, just surviving is, by itself, a major endeavor.
"Fair": Equality and Comprehension
A "difficult" setting is fine, but it's hard to get audiences involved in it if there's not some semblance of fairness, as well. It's one thing to make things difficult, but it's another to make them impossible. There should be some sort of fighting chance, or at least a logical understanding of why failure occurs. A good setting or game should set expectations and then make them consistent: if you do x you will get y. It may be hard to do x, but that's what you need to do. In contrast, making it unavoidably hard reduces player/audience involvement, because they see no reason to bother trying or caring.
This is a concept I examined in my analysis of Warhammer 40k: if things are more or less "grimdark", they need to be logically connected to something. A separation arises between things that are naturally difficult (the universe is overrun with enemies and humanity must defend itself, even from within) and arbitrarily difficult (everyone is a jerk for the sake of being a jerk, no exceptions). A hard-but-fair situation in 40k would be a unit of guardsmen holding out against a tyranid invasion - an unfair situation would be those same units arbitrarily sentenced to death afterwards for no reason other than "grimdark". Warhammer Fantasy, in contrast, has the world being difficult to live in - orcs, trolls, ogres and all that - but doesn't have as much internal strife.
On the personal level, "fairness" also applies to things like enemies and traps. The cry of "Don't I even get a saving throw?" that was pervasive in many old-school D&D games epitomizes this: there needs to be some chance to do something about a threat, even if it's in a vague luck-based sense. It's not "fair" to get killed by an arrow with no warning - there should be some chance to evade or block, however slim. Imagine a dragon or a demon or one of the other powerful monsters described above - certainly these creatures are more powerful than the average human, but understandably so. Their statistics in a game should reflect that, and a low-level character who goes against them should be swiftly defeated. However, they should be defeated according to the rules of the game, rather than defeated in a cutscene or just told that "you died".
"Hard But Fair": The Balancing Act
So now we have these two concepts. "Hard" is good because difficulty and adversity make for more tension and more interesting stories. "Fair" is good because without a way to properly overcome that difficulty (or at least fail in a consistent, understandable way), the players or audience are separated from the reasons that difficulty is valuable. This can be both an interactive concept (how players react to challenges) or an uninteractive concept (how audiences react to character deaths).
To me, "Hard but Fair" is characterized well by the Forlorn Hope (Dutch for "Lost Troop"), groups of men who volunteered to lead the charge into a wall-breach during a siege. These men would almost certainly be killed, but success - however unlikely - would yield great rewards. It's more accurate to say that the Forlorn Hope is a tug-of-war between "fair" and "unfair": it's unfair because they're cannon fodder thrown into an unavoidably awful situation, but it's fair because there is at least some gleam of hope left. Whether or not it's "fair" depends on what the individual can do to avoid death: charging straight into a volley of musket-fire is hardly "fair", but if there's some way to deal with it then that's less of an issue.
Life, realistically speaking, usually isn't fair. In some cases, people who are born or thrust into unpleasant circumstances can crawl out of it through hard work, skill, and luck. A combatant can survive a battle if they do well, or a poor worker can do their best and pull ahead economically. Sometimes, though, a situation is simply hopeless: there's nothing you can do to avoid negative consequences, and your fate is not in your own hands. Think of the shift from melee combat to long-range artillery. Yes, melee combat is brutal, but ostensibly one's skill as a swordsman or fighter can at least influence the outcome. On the other hand, things like artillery, land mines, and snipers strike without warning, and there's nothing you can do except hope you're not the one hit. That's a huge difference in morale - confidence and training can deal with the former, but the latter is unavoidable.
The same is true of workers and laborers - if there's even a sliver of hope for advancement or escape, then it can be pursued. If there's not, the question becomes "why am I still doing this", which is not always answerable. It's a key issue of motivation: there must be hope if good performance is expected. People who don't feel their situation can change are not going to give more than the minimum requirement. In the same way, audiences who don't feel there's any hope for a character aren't going to invest in them, and players are generally going to be annoyed if they die to something unavoidable like a cutscene death or a "no, you just die" DM decision.
The value of "hard but fair" combines the tension and drama of a difficult battle with the tempering element of "giving the audience a reason to bother caring". It is that spark of hope that draws them in and lets them invest themselves in a character and their well-being. Whether it's a simple traveler making their way in the world or a mighty swordsman cleaving down as many foes as he can, these two elements must work in tandem for the struggle to be worth the audience's time.
"Easy But Fun": The Counter-Argument
Not every setting can run on grimdark dark grimness. Not every game can be bone-shatteringly hard. However, I do think we need to really look at these sorts of games and what they represent. I return here to the concepts of PvP and PvE: "I am defeating another living being" versus "I am overcoming an obstacle". In an "easy" game or setting that revolves around combat, there's two ways to approach things: "Combat isn't a big deal" (as seen in FFTA, as well as many non-lethal combat games), or "There's little chance of protagonists dying" (the majority of RPGs).
The former case has already been addressed: having low consequences makes things cleaner, but also takes out a lot of the drama. It reduces it from a battle to the death to a sports competition. However, it's probably the more logical of the two choices, because both sides are fair: they're bound by the same rules, and they're both in equal danger. Internally, it's perfectly consistent, but it simply ends up lacking the life-and-death drama of a more drastic series. This isn't bad, but it is a limitation.
The latter case, on the other hand, tends to be more troubling. When it can be routinely expected that a hero can slaughter enemies without being harmed themselves, then the hero stops being sympathetic and that sympathy shifts to their enemies. There is no point in rooting for the obvious winner - it is the underdog that garners an audience's favor, because that's what creates tension. A world where protagonists are safe from "mooks" is an inherently unfair one, and it is only because those weak henchmen are assumed to be both evil and inhuman that the audience retains any sense of moral connection with the hero.
This is the effect a character shield can have: it begs the question "Does the hero really need to do this? Can't they just use their superpowers to disarm the enemy, rather than kill them for no reason?" It's hard to justify support for a character who's stronger than everyone they're fighting, especially when their enemies are sentient beings. Basically, an easy setting is going to have to end up making a choice: are things "to the death", or are they "fair"? Serious things are still happening, and people are still dying - they're just not part of the quirky cast of central protagonists who have comic misadventures.
With that established, let's take a look at some "hard" franchises and see how they measure up.
In Warhammer 40k, this unity often does not exist, leading to entire planets being destroyed because of exposure to Chaos or perceived traitorous behavior. This pushes it too far over the edge for some people, because it hits a point where it stops being about heroic individuals doing their best in a crappy universe and starts being about author fiat. It's hard, but it's not fair. Things could be okay, but things find a way to be not okay for fairly non-sensible reasons.
The stuff that usually gets mocked for being "grimdark" follows that line of thinking: things are bad for the sake of being bad. Those soldiers who just fought off the Orkish invasion were sentenced to death because of a bureaucratic error. Of course, the terrifying thing is that such things have happened in real life, which is something I think people miss when discussing 40k: anything awful that happens in it has already been done, and done far more horribly, in reality. It's hard to think of a struggle as "heroic" when most of its heroes have been betrayed by their own government, and that's the kind of reaction that 40k creates. This is why there are so many attempts to try to create those points of light - Ciaphas Cain, the Salamanders chapter, various short stories - and this may or may not work depending on the audience's understanding of the setting's logic.
Demon's Souls' appeal came from this line of thought: "If I made a mistake, it's my fault, and now I can deal with it". Sure, there were some traps that would seem arbitrary or whatever, but even the more unpleasant traps had at least some indication of what was there, and what was going to happen. It was a hard game, sure, but it came down to player skill and awareness, rather than precognition.
Gantz before, but this provides a pretty good opportunity. The setup of Gantz is that, after people die, they're being flash-cloned and teleported into a survival "game" to fight aliens. It's obviously more complicated than that, but in general the recurring setup is that it's a group of people who aren't familiar with each other, don't know why they're there, and don't have any idea how their technology works. People who survive the first round become veterans through experimentation, coordination, and sharing knowledge. In fact, it's a pretty big shift from one arc to another: one arc is about selfish survival, with no regard for teammates, and the other is about organizing a ragtag group into a fighting force. At the end of these incredibly lethal "games", the rewards include being resurrected with no memory of the game, resurrecting another person, or gaining better equipment and resuming the hunt.
Gantz comes off to many people as hard but unfair - characters die all the time, very suddenly, with little warning. However, it's one of the major aspects of the series that teamwork and coordination USUALLY equal survival. Not always, but usually. People rarely "just die" - there's some mistake made, or some unforeseen issue, or just an incredibly powerful foe. There is one glaring exception (giant Monty Python foot out of nowhere) but a lot of the time it felt like a situation had fallen apart, rather than "the author chose that character to die". Of course, that's totally true, as it is with all fiction, but it didn't feel like it to me.
So, To Sum Up:
1) Difficulty in a setting, either in-universe or in gameplay terms, can emphasize the dangers and risks of a setting and create more tension.
2) Fairness in a setting is important in terms of keeping the audience or players attracted and giving them some sense of control, rather than just saying "you died, deal with it".
3) Taking away difficulty can make a setting more accessible, but in logical terms the difference between the players and their enemies should be addressed in some way.