MoP Block Calculations – Part 4

In this final installment, we’re going to use a Monte Carlo simulation to see how accurately the analytical model portrays the results of a more realistic (i.e. discrete event-driven) combat scenario. If the simulation agrees very well with the analytical results, we can feel confident that the derivation and conclusion we’ve drawn from that model are correct. If they differ significantly, we might have to re-visit the model and the assumptions that went into it to figure out what we did wrong.

The Monte Carlo method is a very common technique in statistical simulation. The basic idea is that you have a system with a random element that makes it sensitive to fluctuations. It could be a random process, such that the same input gives different outputs based on the state of the system (some quantum mechanical situations are like this), or it could be a deterministic process that has random inputs. In either case, the Monte Carlo method involves generating a bunch of random numbers and looking at the output of the system. By doing this enough times, you can determine if there are statistically significant trends in the output data.

In our case, we have a random process – combat. We start from the same initial conditions (everything off cooldown, no buffs, etc.), but the process itself contains randomly-determined results based on combat table rolls. Thus, our Monte Carlo simulation basically simulates combat – we track a bunch of cooldowns and buff durations, and make combat table rolls that affect the results. By running the simulation long enough, we can get pretty accurate stochastic averages for the values we’re interested in.

Simulation Details

You can find the code for the Monte Carlo simulation here:
montecarlo.m
montecarlo2.m
montecarlo3.m

The basic idea is that it tracks the boss swing timer, the cooldowns of CS and J, the uptimes of both SotR buffs, Divine Purpose, and Grand Crusader, as well as the GCD. It runs in timesteps of 10 milliseconds and keeps track of whether the boss attack is a hit, miss, or block (as well as the type of block – “gblock,” “sblock,” or regular block). It makes combat rolls for avoidance and block to determine these outcomes, as well as random rolls for Divine Purpose and Grand Crusader. The logic is such that it will prioritize CS>J>AS(only if GC available), and it won’t cast J if there’s <0.2 seconds left on the CS cooldown (I needed to do this because J scales slightly better with haste, resulting in it coming off-cooldown a little earlier and mucking up the rotation).

Finally, I’ve put some code in there to handle our assumptions from Part II. It will cast SotR whenever we have >3 holy power or a Divine Purpose proc, but it won’t re-cast SotR until the guaranteed block is used. This risks “wasting” some holy power generation, but in practice that happened 0 times in any of my simulations because we’re not pooling Holy Power to 5. We can go back later and see how this changes if we do pool holy power simply by changing the conditionals on SotR, but by coding it the way I did, we should see the best possible agreement with the model.

Note that our other assumptions – namely that our $S$ uptime is based on a single application estimate and that everything can be approximated as occuring at nicely-spaced intervals – are not being adjusted for. So if there’s any significant inaccuracy introduced by those assumptions, they should show up in the data.

Simulation Results

Now let’s look at the results. I’ve used the same input values as the analytical derivation. First, let’s look at what we get with those inputs:

$G= 0.417$
$S= 0.831$
$T_{\rm SotR} = 6.7$
$R_{\rm HPG} = 0.3845$

Plot showing the distribution of hit sizes with the current model. 29.8% of attacks are avoided (0 damage), 29.3% are “gblocked” (0.25 damage), 18.7% are “sblocked” (0.5 damage), 3.8% are blocked regularly (0.7 damage), and 18.4% aren’t blocked at all (1 damage). DTPS is out of a possible 50 (1 damage every 2 seconds, multiplied by 100).

If you’ll note, these values are extremely close to the ones in the analytical derivation. With 10,000 minutes of combat, the Monte Carlo manages to match $G$ and $S$ to the analytical model to three decimal places. $R_{\rm HPG}$ differs slightly in the fourth decimal place, but $T_{\rm SotR}$ is a little bit higher than the 6.4 predicted by the analytical model. My guess is that this is the error due to discretization – namely that in the model, we don’t consider the coolown on SotR and use a stochastic time average for Divine Purpose. In the numerical sim, SotR’s 1.5-second cooldown keeps it from bunching too tightly, and ends up pushing back SotR casts in cases with multiple DP procs.

The graph also shows some of the features that made mastery so weak in our analytical model. Only 40.9% of attacks are even eligible for block as a result of the two-roll system, so that inherently almost halves block’s effectiveness. We’re only blocking 22.5% of all attacks (18.7% “sblocks,” 3.8% “regular blocks”), giving 22.5%/40.9%=55.01% block, right in line with our input block chance. 59.1% of all attacks are avoided or end up being guaranteed blocks.

Now let’s look at what happens when we add 600 rating of various sources (I’ve skipped armor here, because I didn’t bother to include it in the Monte Carlo). The simulation runs for 10,000 minutes of combat for each stat, which is still not enough to get wonderful accuracy for such a small change in rating, but enough to make sure we’re correct in at least the first significant digit. Here is the reduction in DTPS (multiplied by 100 to make the numbers more reasonable) that occurs when adding 600 rating of each type:

Dodge: 0.9285
Hit: 0.6835
Exp: 0.6226
Haste: 0.4955
Mastery: 0.2948

Again, these seem to validate our values from the analytical calculation. Dodge is well ahead of Hit/Exp. Hit and Expertise should be identical, but even with 10k minutes per stat, we have some statistical fluctuations. Haste lags behind hit/expertise, and mastery still brings up the rear. So this, combined with our pin-point accuracy on $G$ and $S$ suggests that the analytical model is quite accurate indeed.

Finally, let’s take a look at variance. A 10,000-minute fight isn’t realistic, because most fights last on the order of 5 minutes. How do $G$, $S$, and $T_{\rm SotR}$ vary from fight-to-fight for short encounters?

This plot shows the distribution of G and S values for 500 different 5-minute (combat time) simulations. G varies significantly less than S, as S is more dependent on the spacing and timing of SotR casts and boss attacks.

As we see here, $G$ can range from about 0.32 to 0.55 depending on attempt. But $S$ varies quite a bit more than that – from as low as 50% to as high as 100%. The standard deviations here are 3.69% (for $G$) and 9.63% (for $S$). Plots for $T_{\rm SotR}$ and $R_{\rm HPG}$ are shown below, but they don’t really tell us much that we didn’t already know. HPG varies based on hit/expertise and Grand Crusader procs, and SotR casts inherit that randomness and add a little Divine Purpose noise.

This histogram shows the variation of the SotR cast interval T_{SotR} under the same conditions as the previous plot.

This histogram shows the distribution of HPG rates for 500 different 5-minute (combat time) simulations.

Conclusions

This has been a short post – mostly because I’m rushing to finish this before heading out for the weekend. But I think the point was to validate the analytical model with a numerical simulation, and I don’t think there’s any doubt that we’ve done that. There are some interesting statistical questions we could answer with this code, but that will have to wait for another time. In any event, we now have two conclusive pieces of evidence – an analytical model and a numerical simulation – that the current model of Paladin blocking makes mastery our dump stat. Let’s hope that this is enough to convince Blizzard that change is needed.

This entry was posted in Tanking, Theck's Pounding Headaches, Theorycrafting and tagged , , , , , , , , . Bookmark the permalink.

16 Responses to MoP Block Calculations – Part 4

  1. Zaephod says:

    Dr. Theck, you’ve done the tankadin world a great service. We’re all smarter for it, and that LaTeX plugin you’ve got going here is great! Enjoy your weekend and the comment storm that is coming! :) Thank you for all this! /salute

    • Theck says:

      It’s actually not a plugin. WordPress natively supports LaTeX (though badly – it doesn’t seem to recognize a lot of standard formatting and spacing commands, especially in multiline equations).

  2. Newsom says:

    Thanks for taking the time to do this. Let’s hope Blizzard sees it.

  3. Weebey says:

    “In any event, we now have two conclusive pieces of evidence – an analytical model and a numerical simulation – that the current model of Paladin blocking makes mastery our dump stat. Let’s hope that this is enough to convince Blizzard that change is needed.”

    From Ghostcrawler’s post the other day on active mitigation:

    “If you think a particular stat is undervalued, by all means let us know, but you’re probably going to have to provide some math to make your argument. We also don’t spend a great deal of effort balancing all of the numbers in beta until we’re happy with the abilities — there is no point making Shield of the Righteous play really nice with mastery if we decide to redesign the ability.”

    Given that we also know that GC knows Theck, and I believe even reads this blog, I would take this quote as good evidence that Blizzard is well aware of the problem, and we can probably expect some significant adjustments to our mastery if they decide to retain this model of SotR.

  4. Yosaf says:

    Theck,
    This series has been a great read. I actually work for a software company that produces discrete event simulation software. In my spare time I occasionally tinker with my own event driven simulation that was originally conceived to figure out how our rotation worked during Cata Beta. My sim, while not as detailed as yours, supports many of your findings so consider that a third piece of evidence that supports your findings, especially around SotR’s availability. That was one of the things that I saw in my sim that was surprising; Seeing that ~6 second interval between SotR’s made me think my model was wrong.
    -Yosaf

  5. Lakhesis says:

    Thanks some very interesting posts, I’ve enjoyed reading them. It’s pleasing to see that you’ve probably (hopefully) directly affected the design back at bliz hq too.

  6. Hamsda says:

    After asking in the comment section on the last part of your block calculations, Theck, I have one more question, which you might be able to answer (or at least give a good guess):
    Would it be beneficial to let the Prot Mastery increase the Block Value instead of Block Chance?

    Elaborating a bit further: I like the fact, that SotR is an “all-in-one solution”, at least somewhat similar to Death Strike, because the split implementation from the beginning of MoP was rather lackluster as Meloree pointed out in this post: http://sacredduty.net/2011/12/07/another-take-on-active-mitigation/ and as others have about the current MoP WoG implementation in the comments.
    But as you have pointed out, increasing the block chance (besides the heavy nerf coming with it) doesn’t do much, when only 40% of incoming attacks are influenced by it. If we had (say by choosing prot spec) 20% base block chance and 30% block value, SotR keeps its current implementation of a guaranteed block followed by an increased chance to block (say upping the chance to a total of 50%) for a limited timeframe, but mastery would instead increase the block value. Wouldn’t this make mastery a more desireable stat again, because it effectively scales with other stats?
    Having more mastery would make all “parts” of SotR stronger, be it the guaranteed block or the increased chance, while still providing at least a little benefit to the non-SotR-block. This would also further emphasize the active mitigation part, since having the guaranteed block or increased chance more often would let you benefit more from your mastery than just having more mastery but a fixed (relatively low-ish) block chance (thus still keeping the value of hit/expertise quite high, but maybe pushing mastery at least over haste if not right below dodge/parry).

    I would appreciate a comment on those thoughts, maybe you could even model it (though I think that would require a bit too much effort just for an idea, since a change in mastery would change a few equations).

    Thanks in advance :)

    • Theck says:

      It would certainly make mastery stronger. I’d have to do some modeling to figure out exactly how much, though. But having mastery just increase block value instead of chance would couple it with the guaranteed block, which should very significantly increase its value.

      • Hamsda says:

        Apparently Blizzard either reads your posts or did some thinking themselves, but with the removal of the guaranteed block and an increase in block value SotR should be in a place where mastery is one of the better stats again.

  7. Pingback: Warrior Block Calculations – Part 1 | Sacred Duty

  8. Wonton says:

    Beautiful. I am in awe of your analysis skills, Theck. With these results, it’ll be interesting to see where the design team goes with SotR so as to both make it fun and leave mastery as something of a high-priority stat.

  9. Pingback: Radio Silence and SotR Musings | Sacred Duty

  10. Pingback: The Light and How to Swing It: The framework for our Mists rotation | Owned Gamers

  11. Pingback: MoP Mastery/Block Simulations – Build 15739 | Sacred Duty

  12. Pingback: L90 Mitigation Stat Weights for Warriors | Sacred Duty

  13. Pingback: MoP Mitigation Stat Weights – Monte Carlo – b16048 | Sacred Duty

Leave a Reply