In the last few weeks, I’ve been spending a lot of time working on my MATLAB/C# simulations for Mists of Pandaria (which, unfortunately, has also meant a lot less time for blogging). While there’s still a lot to do, things are settling down on beta, and we should be able to start producing results relevant for 5.0 very soon.
Today, I want to share with you an interesting quirk I stumbled across yesterday while doing some code testing. This will also give you a little insight into what goes into theorycrafting simulations, as well as how you troubleshoot bugs (and sometimes discover interesting new things).
For a little background: our code base includes a C# executable (fsm.exe) which performs the finite state machine calculations that describe a particular rotation. For example, if we want to know how the following priority queue works:
Shield of the Righteous > Crusader Strike > Judgment > Avenger’s Shield
we feed a shorthand for the queue (“SotR>CS>J>AS”) to the executable, along with other relevant inputs (hit/expertise, haste, talents, and so on) and it spits out data about how often we cast each ability. For example, the output of “SotR>CS>J>AS>Cons>HW” looks like this:
'AS' 'AS(GC)' 'Cons' 'CS' 'HW' 'J' 'SotR' [0.0370] [0.0404] [0.0880] [0.2222] [0.0806] [0.1481] [0.1303]
The values are the number of casts per second for each ability. So for example, we’re casting Crusader Strike 0.2222 times per second, or every 1/0.2222=4.5 seconds. You can see that we have it differentiating between casting AS when it naturally comes off of cooldown and casting AS during a Grand Crusader proc, which helps us track Holy Power generation.
To calculate how much DPS a rotation does, we take these outputs from the executable, dump them into MATLAB, and combine them with damage-per-cast values, as you’d expect. To calculate scaling with different stats, we might run the executable for hundreds or thousands of different configurations to generate a plot.
While doing just that, I got the following graph:
What this plot is trying to tell us is how each of our stats scales with strength – in other words, as my strength increases, do the other stats change in relative value? The strange part is that this graph is saying that we actually lose DPS when we add haste rating. That’s right, adding 10 haste rating decreases our DPS by about 50. You heard it here first!
Obviously, my first instinct was that this is some sort of bug with the simulation. There’s no possible way that doing stuff faster would reduce our DPS, right? So I started fishing around in the code looking for the cause of the bug. When troubleshooting, I generally work backwards. The DPS seemed wrong, so I took one step back and looked at the output of the executable. And I was a bit surprised by the result – here’s the data:
cast rates with 0 haste rating 'AS' 'AS(GC)' 'Cons' 'CS' 'HotR' 'HW' 'J' 'SotR' 'SS' [0.0383] [0.0363] [0.0848] [0.1756] [0.0416] [0.0709] [0.1484] [0.1200] [0.0332]
cast rates with 30 haste rating 'AS' 'AS(GC)' 'Cons' 'CS' 'HotR' 'HW' 'J' 'SotR' 'SS' [0.0342] [0.0362] [0.0863] [0.1754] [0.0415] [0.0724] [0.1482] [0.1199] [0.0331]
If you look carefully there, you’ll see a few things:
- The cast rates of CS, HotR, J, SotR, and Sacred Shield are basically unchanged, as is the cast rate of Grand-Crusader-procced Avenger’s Shields. The small variations
($\pm 0.0001$ casts per second) aren’t the major feature here.
- The cast rate of Avenger’s Shield went down significantly
- The cast rates of Holy Wrath and Consecration both went up.
In other words, by adding 30 haste rating, we’ve reduced the number of AS casts and increased the number of HW and Cons casts.
At this point, I had a guess as to what was wrong, and it wasn’t a bug in the code. To test it, I wrote up a quick script (because the simulation that produces the graph above is pretty slow, and I wanted to just isolate haste as the lone variable). It simply calculated DPS for a variety of haste rating values, from 0 to 150ish, and plotted them. Here was the result:
So the DPS dips as soon as one point of haste rating is added, and then starts going up again (this plot was generated using steps of 10 haste rating, which makes the dip easier to see). That helped confirmed my suspicion about what was happening.
The problem here, which the title of the graph probably gave away, is that Avenger’s Shield is not affected by Sanctity of Battle. It has a 15-second cooldown, which is exactly 10 GCDs. With no haste, it’s available for use as a filler again 10 GCDs after you cast it (provided Grand Crusader doesn’t proc). But as soon as you have even one point of haste rating, your GCD from all of our other abilities gets shorter. And that means that at the end of that 10th GCD, Avenger’s Shield still has a few milliseconds left on its cooldown, and isn’t available to cast. So the algorithm continues down the priority list and casts a different filler that is available. In essence, your first point of haste rating turns Avenger’s Shield into an 11-GCD cooldown instead of a 10-GCD one.
Similar effects happen at higher haste values, when the problem repeats itself. For example, at 30% haste, we see a little more than a 300 DPS loss from one additional point of haste rating due to exactly the same effect.
Just to check that this was so, I performed another quick test. One of the neat things about writing your own simulations is that you don’t have to adhere to the game rules as they exist on live or beta. If I want to add Avenger’s Shield to Sanctity of Battle, I change one “false” to “true” and it happens! So that’s exactly what I did. And unsurprisingly, this is what I get from the script:
That was the final confirmation – it was definitely the interaction between Sanctity of Battle and Avenger’s Shield (or more accurately, the lack thereof) that was causing the DPS loss. So my afternoon went from trying to track down a weird bug to discovering a strange quirk of the rotation mechanics. Not bad for a lazy Sunday afternoon.
What does this mean in a practical sense? If the developers see this and add Avenger’s Shield to Sanctity of Battle, then the problem won’t exist. So of course, that’s my recommendation, should they see this.
However, let’s say that they don’t. It means that the first ~50-60 haste rating we get is essentially wasted returning us to our zero-haste DPS. It also means that during Heroism/Bloodlust, a paladin with 1 haste rating will do ~400 DPS less than a paladin with 0 haste rating. Haste wasn’t our go-to stat anyway, but this will make it even less attractive in small quantities.
It’s also worth noting that this simulation isn’t modeling latency, which might muddy the waters. And these DPS changes are pretty miniscule in the overall picture; 60 DPS out of around 50k is only about 0.1%. It may not even be worth their time to fix it, depending on how hard it is to add Avenger’s Shield to Sanctity of Battle.
Plus, players might just delay their cast by a few milliseconds and use AS anyway. I suspect that a queue with conditionals that specifies that behavior (for example: only cast Holy Wrath or Consecration if the Avenger’s Shield cooldown is greater than half a second) would pull ahead slightly under these conditions. So perhaps the problem is automatically solved by a human factor – namely that we’re not as good as a computer at making split-second decisions or determining whether an ability will be off-cooldown milliseconds before or after the GCD.
However, it’s sort of unintuitive behavior, even if its only on paper. And in a practical sense it still nullifies some of the benefit from the first few points of haste. Since the solution is so simple, provided there’s no technical issue with adding Avenger’s Shield to Sanctity of Battle, I hope the developers decide to fix it properly (much like they did with Hammer of the Righteous by changing it to use weapon damage scaling, such that we never see it out-scale Crusader Strike again).