Roll Over vs Roll Under

One of the basic differences between modern and old school D&D is whether ability checks are roll high (modern D&D) or roll under (old school D&D). I’m going to bracket various aesthetic issues of which mechanic is more elegant, simple, etc. and only look at the math of how swingy they are and in particular how much does the ability score matter to the roll. Whether swingy is desirable or not is itself a matter of taste or circumstance, but it seems to me that some tasks should be like arm wrestling, where strength matters way more than luck, and others should be more like looking for a lost set of keys, where luck should matter at least as much as wisdom (in the sense of how observant you are).

My hunch is that the ability scores should matter more in the roll under mechanic than the roll high mechanic, but let’s see if I’m right. (Spoiler: I was right). It’s the kind of thing you could do with probability theory but it’s much easier to do via simulation and so that’s how I did it. You could probably do it in anydice, but I ran it in R.

First, let’s review the relevant mechanics.

The 5e version of roll high has the following rules and guidelines.

  • Roll equal to or higher than a target number. An easy task is 10, a medium task is 15, and a hard task is 20. There are more gradations, some of which are more difficult than the merely “hard” but we’ll leave it there.
  • There is nothing special about a natural 1 or natural 20.
  • Apply the relevant ability score modifier. These are very swingy, with mods ranging from -4 to +4 over the ability score range of 3-18.
    • The ability scores themselves are generated by 4d6 keep 3, so on average mods are positive.
    • Players can assign score arrays to abilities. (I will be ignoring this rule).
  • Apply proficiency modifiers. (I will be ignoring this rule).
  • Savings throws, rogue abilities, etc. are all special cases of ability checks.

The OSE (B/X) version of roll under has the following rules and guidelines.

  • Roll equal to or under your ability score.
    • Ability scores are rolled as 3d6.
    • Scores are rolled down the line. (I will be ignoring this rule).
  • A natural 1 is an automatic success, a natural 20 is an automatic failure.
  • Use difficulty to modify the die. An easy task is -4 and a hard task is +4. Implicitly, a medium task is a straight roll.
  • Savings throws, thief abilities, etc. have their own mechanics unrelated to ability scores.

The first step is creating a set of ability scores. I’ll do the OSE rules as written approach of 3d6 first.

n.sims <- 10000
results.3d6 <- vector(length = n.sims)
for (i in 1:n.sims) {
  results.3d6[i] <- sum(sample(seq(1:6),3,replace=TRUE))
}
hist(results.3d6)

As you’d expect, it’s a pretty much perfect bell curve.

Since I want to compare a pretty low to an average to a pretty high ability score to see how much ability score matters, let’s use 10th percentile, median, and 90th percentile.

deciles.3d6 <- quantile(results.3d6,probs = c(.1,.5,.9))
p10.3d6 <- deciles.3d6[1]
p50.3d6 <- deciles.3d6[2]
p90.3d6 <- deciles.3d6[3]
deciles.3d6
## 10% 50% 90% 
##   7  10  14

Now let’s do the same for 4d6, keep 3, which is both rules as written for 5e (and AD&D) as well as a popular house rule for OSE.

results.4d6 <- vector(length = n.sims)
for (i in 1:n.sims) {
  results.4d6[i] <-  sum(sort(sample(seq(1:6),4,replace=TRUE),
  decreasing = TRUE)[1:3])
}
hist(results.4d6)

The 4d6 keep 3 method gives us a left-skewed distribution.

deciles.4d6 <- quantile(results.4d6,probs = c(.1,.5,.9))
p10.4d6 <- deciles.4d6[1]
p50.4d6 <- deciles.4d6[2]
p90.4d6 <- deciles.4d6[3]
deciles.4d6
## 10% 50% 90% 
##   8  12  16

Now let’s create our mods. The range is 3-18 but I’m creating mods for the range 1-18 for ease of value indexing. We only really need the 5e mods as the OSE mods are irrelevant for this mechanic but including them anyway.

scores <- seq(1:18)
mods.ose <- c(NA,NA,
  -3,
  -2,-2,
  -1,-1,-1,
  0,0,0,0,
  1,1,1,
  2,2,
  3)
mods.5e <- c(-5,
  -4,-4,
  -3,-3,
  -2,-2,
  -1,-1,
  0,0,
  1,1,
  2,2,
  3,3,
  4)
mods <- cbind(scores,mods.ose,mods.5e)

Now let’s create a ton of d20 rolls. We’ll do two sets of rolls and store it as straight rolls, rolls with advantage (keep high), and rolls with disadvantage (keep low).

rolls.a <- sample(seq(1:20),size = n.sims,replace=T)
rolls.b <- sample(seq(1:20),size = n.sims,replace=T)
rolls.adv <- pmax(rolls.a,rolls.b)
rolls.dis <- pmin(rolls.a,rolls.b)
rolls <- cbind(rolls.a,rolls.adv,rolls.dis)

OK, now let’s see how often our players succeed on their rolls for every combination of easy roll/medium roll/hard roll and low score/medium score/high score using OSE checks with the OSE 3d6 scores.

matrix.template <- matrix(nrow = n.sims,ncol = 3)
colnames(matrix.template) <- c("p10","p50","p90")
matrix.template <- as.data.frame(matrix.template)

ose.raw <- list(matrix.template, matrix.template, matrix.template)
names(ose.raw) <- c("easy","medium","hard")

for (i in 1:n.sims) {
  ose.raw[["easy"]][i,1] <- ifelse(rolls[i,1]-4<=p10.3d6,1,0)
  ose.raw[["easy"]][i,2] <- ifelse(rolls[i,1]-4<=p50.3d6,1,0)
  ose.raw[["easy"]][i,3] <- ifelse(rolls[i,1]-4<=p90.3d6,1,0)
  for (j in 1:3) {
    ose.raw[["easy"]][i,j] <- ifelse(rolls[i,1]==1,1,ose.raw[["easy"]][i,j])
    ose.raw[["easy"]][i,j] <- ifelse(rolls[i,1]==20,0,ose.raw[["easy"]][i,j])
  }
  ose.raw[["medium"]][i,1] <- ifelse(rolls[i,1]<=p10.3d6,1,0)
  ose.raw[["medium"]][i,2] <- ifelse(rolls[i,1]<=p50.3d6,1,0)
  ose.raw[["medium"]][i,3] <- ifelse(rolls[i,1]<=p90.3d6,1,0)
  for (j in 1:3) {
    ose.raw[["medium"]][i,j] <- ifelse(rolls[i,j]==1,1,ose.raw[["medium"]][i,j])
    ose.raw[["medium"]][i,j] <- ifelse(rolls[i,j]==20,0,ose.raw[["medium"]][i,j])
  }
  ose.raw[["hard"]][i,1] <- ifelse(rolls[i,1]+4<=p10.3d6,1,0)
  ose.raw[["hard"]][i,2] <- ifelse(rolls[i,1]+4<=p50.3d6,1,0)
  ose.raw[["hard"]][i,3] <- ifelse(rolls[i,1]+4<=p90.3d6,1,0)
  for (j in 1:3) {
    ose.raw[["hard"]][i,j] <- ifelse(rolls[i,1]==1,1,ose.raw[["hard"]][i,j])
    ose.raw[["hard"]][i,j] <- ifelse(rolls[i,1]==20,0,ose.raw[["hard"]][i,j])
  }
}

colMeans(ose.raw[["easy"]]
##    p10    p50    p90 
## 0.5578 0.7011 0.9009

colMeans(ose.raw[["medium"]])
##    p10    p50    p90 
## 0.3601 0.4834 0.7155

colMeans(ose.raw[["hard"]])
##    p10    p50    p90 
## 0.1596 0.3121 0.5074

An easy task is one that a character with a low score can complete a bit over half the time and a high score character can complete about 90% of the time. A hard task is one that a character with a low score can complete about 1/6 of the time and a high score character about half the time. That’s both plausible and shows a strong influence from the score.

I’m not going to include the code for using OSE ability checks with a 4d6 house rule, but here are the results.

colMeans(ose.4d6[["easy"]])
##    p10    p50    p90 
## 0.6042 0.7612 0.9970

colMeans(ose.4d6[["medium"]])
##    p10    p50    p90 
## 0.4104 0.5740 0.8091

colMeans(ose.4d6[["hard"]])
##    p10    p50    p90 
## 0.1970 0.3918 0.5973

(Update: there was a glitch in my code that generated ose.4d6[["hard"]]. Thanks to John for catching it. I have since corrected).

Now, let’s do the 5e system with straight rolls (ie, no advantage or disadvantage).

dnd5e.raw <- list()
dnd5e.raw <- list(matrix.template, matrix.template, matrix.template)
names(dnd5e.raw) <- c("easy","medium","hard")

for (i in 1:n.sims) {
  dnd5e.raw[["easy"]][i,1] <- ifelse(rolls[i,1]+mods.5e[p10.4d6]>=10,1,0)
  dnd5e.raw[["easy"]][i,2] <- ifelse(rolls[i,1]+mods.5e[p50.4d6]>=10,1,0)
  dnd5e.raw[["easy"]][i,3] <- ifelse(rolls[i,1]+mods.5e[p90.4d6]>=10,1,0)
  dnd5e.raw[["medium"]][i,1] <- ifelse(rolls[i,1]+mods.5e[p10.4d6]>=15,1,0)
  dnd5e.raw[["medium"]][i,2] <- ifelse(rolls[i,1]+mods.5e[p50.4d6]>=15,1,0)
  dnd5e.raw[["medium"]][i,3] <- ifelse(rolls[i,1]+mods.5e[p90.4d6]>=15,1,0)
  dnd5e.raw[["hard"]][i,1] <- ifelse(rolls[i,1]+mods.5e[p10.4d6]>=20,1,0)
  dnd5e.raw[["hard"]][i,2] <- ifelse(rolls[i,1]+mods.5e[p50.4d6]>=20,1,0)
  dnd5e.raw[["hard"]][i,3] <- ifelse(rolls[i,1]+mods.5e[p90.4d6]>=20,1,0)
}

colMeans(dnd5e.raw[["easy"]])
##    p10    p50    p90 
## 0.4926 0.5896 0.6879
colMeans(dnd5e.raw[["medium"]])
##    p10    p50    p90 
## 0.2501 0.3471 0.4422
colMeans(dnd5e.raw[["hard"]])
##    p10    p50    p90 
## 0.0000 0.0991 0.2004

We can notice two things about the 3e/5e mechanic as compared to the OSE mechanic.

  1. In 3e/5e, success is more a function of task difficulty and less a function of ability score with the notable exception that “hard” tasks are not just hard but impossible if you have a low ability score.
  2. In 3e/5e, the mechanic is much harsher, not withstanding that RAW 3e/5e ability scores tend to be higher.

There are a few caveats to the second point though. First, OSE has special mechanics for many of the things 3e/5e uses ability checks for, these tend to be based on level, and at low level your character is pretty much useless. For instance, old school thieves are famously useless as thieves (but have a good shot at scaling El Capitan). Second, 3e and 5e are games of stacking buffs. 5e isn’t as bad about this as 3e or Pathfinder, but there’s still the expectation that straight rolls are for suckers. There are other buffs, such as the “guidance” cantrip (add 1d4 to your roll), but the big buff is advantage (roll twice, keep high). And of course sometimes you have disadvantage (roll twice, keep low), with the really important one that everyone fails to apply being that using darkvision gives disadvantage on perception checks.

I’m not gonna give my code for simulating ability checks with advantage and disadvantage as it’s really similar to above except the code for advantage is similar except it has rolls[i,2] in every line. Likewise, disadvantage has rolls[i,3]. Here are the results.

colMeans(dnd5e.adv[["easy"]])
##    p10    p50    p90 
## 0.7486 0.8410 0.9104

colMeans(dnd5e.adv[["medium"]])
##    p10    p50    p90 
## 0.4414 0.5760 0.7001

colMeans(dnd5e.adv[["hard"]])
##    p10    p50    p90 
## 0.0000 0.1899 0.3625

With advantage, the overall difficulty level is similar to OSE checks. Of course with disadvantage it gets even harder.

colMeans(dnd5e.dis[["easy"]])
##    p10    p50    p90 
## 0.2487 0.3539 0.4795

colMeans(dnd5e.dis[["medium"]])
##    p10    p50    p90 
## 0.0630 0.1223 0.1963

colMeans(dnd5e.dis[["hard"]])
##    p10    p50    p90 
## 0.0000 0.0109 0.0423

Anyway, it seems that the old school “roll under” ability check is actually pretty generous aside from the fact that it doesn’t apply to many situations, being used only when there’s not a more specific mechanic. However it’s not too bad for 3e/5e players as they can get similar ease by stacking buffs.

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

Comments (

2

)

  1. John

    It is interesting how 5e gives you a +5% per 2 ability points, while B/X gives it per single ability point, causing odd number ability scores to provide virtually no advantage in 5e. I think in practice – as you allude to – 5e gives many further boosts to players though, including racial ability score bonuses, frequent ability score increases, and enough proficiencies across that party that someone often has a +5 at first level in some key skills (while high-level characters with expertise can get a +17 without magic items).

    I think in terms of how they play, systems from the Gygaxian TSR period (OD&D through BECMI – though the mid-80s did see a proto skill system for 1e) require characters to accumulate magic items, which – along with spells – allow them to succeed at “skill” tasks that would have been difficult at 1st level but should be easy now (thieves being an exception as they have a mechanical system allowing them to improve as they level up).

    A fighting-man, however, requires something like a girdle of hill giant strength to finally start bending bars at high level, as without it they have no mechanism to improve. A flip side to this though is that such a magic item will have a much larger impact on their probability of success that it will for the 5e character, who requires an ability score of 22 to outperform what he will likely have by mid-levels without any items.

    Two nits – I think the medium and hard rows may be switched for the OSE 4d6 house rule, as I expect the probabilities to be monotonic, and the typo “OSR (B/X)” is well suited to troll both “OSRIC was first” and “Mork Bork counts” factions of the every expanding OSR.

    Like

    Reply
    1. gabrielrossman

      That’s a very good point about old school play assuming accumulation of a panoply of magic items. It’s easy to forget as we’ve increasingly gotten interested in Appendix N stuff, which tends to be relatively low magic, but all the 4+ HD monsters that are invulnerable to non-magic weapons (and the barbarian/monk abilities to hit them anyway at mid-level) are strong indications of a presumption that most characters would have magic weapons. And I agree that in practice they serve a similar roll as ability score improvements in WotC era D&D.

      Will correct the typo, really did mean OSE (B/X). And will double check the code for ose.4d6, good catch.
      (Update: the bug was I had roll-4 for hard, thus making it the same rules as for easy, whereas it should have been +4)

      Like

      Reply
%d bloggers like this: