As I mentioned on twitter last week, version 530-5 of Simulationcraft has been released. This is the first version to include the Theck-Meloree Index, a damage smoothing metric we developed in a series of previous blog posts.
However, Simcraft is a bit daunting to some players. The program is very versatile, but that also means there are lots of options, and it can be confusing to understand exactly what’s going on. Sometimes, it helps to have someone guide you through the process.
This is the first in a series of “Simulationcraft 101″ blog posts designed to do exactly that. The hope is that by the end of this blog post, a new user will be able to:
- Download and run the program.
- Import their character from the armory
- Run a quick test simulation and calculate their TMI
- Generate stat weights using the new TMI metric
In future installments, we’ll break down some of the options in more detail and talk about how to interpret the myriad of statistics Simcraft provides in its reports.
First, I want to start off with some disclaimers. Simulationcraft is not perfect, and the paladin module especially so. While I have most things implemented, there are a number of bugs that have slipped through into 530-5. Some of them have already been fixed for 530-6, but I’m sure there are some I haven’t even discovered yet. One of the goals of this blog post is to get more people running the program and simulating their characters so that the remaining bugs can be identified and corrected. So if you get funny results, please share them with me. Either post in the comments here, upload the output HTML files and send me a link, or give me the contents of the Simulate window text via a pastebin link so that I can verify the results.
Here are the paladin-specific bugs that I am aware of in 530-5, as well as a few talents that aren’t properly implemented:
- Selfless Healer isn’t implemented
- Sanctified Wrath’s +20% healing received bonus isn’t implemented
- Shield of Glory (T15 2-piece bonus) duration is fixed at 5 seconds rather than scaling with Holy Power spent
- Holy Prism’s cooldown is not being invoked, allowing it to be spammed every GCD
- Vengeance gain is a little wonky, though it should have little effect on TMI results
Outside of those, I think everything is working. The only other thing that’s missing is the ability to perform shifting queues (i.e.SH1), which is something I’m working on for 530-6. For now, we’re limited to a simple SotR spam queue.
Downloading and Unpacking
The first step is to obtain and install the program. To do that, we go to Simulationcraft’s Google Code page and click on the Downloads tab. Pick the appropriate Windows, Linux, or Mac download for your system. If you’re not sure about whether you have 32-bit or 64-bit Windows, just grab the 32-bit one to be safe. Save this file somewhere convenient.
After downloading the file, you’ll need to unzip it to a location. This can be anywhere you like, but since the program doesn’t need to be installed in the traditional sense, you may as well choose the final location you want to put it. In our example, I unzip it into D:\Simcraft\
If you’re using Linux, you’ll have to build the program yourself. I’m not going to provide instructions for that here; the Google code wiki has fairly clear instructions on how to do this if you need them.
To run the program, we open the \simc-530-5-win32\ folder and run SimulationCraft.exe:
which brings up the SimulationCraft GUI. If you’re on Linux or Mac you’re on your own for this step, as I’m not sure what the files are called off the top of my head.
Importing Your Character
You navigate the GUI by moving between tabs. The “Welcome” tab has a pretty good introduction to the overall layout if you’re interested, but we’re going to skip around to quickly get our Sim on. From the top tab menu, choose “Import.” This opens a set of sub-tabs with different options for importing. You can import directly from the battle.net armory in addition to other sources. For this example we’re going to use the armory.
This interface should look fairly familiar – it’s literally the armory webpage loaded in a browser, complete with an address bar at the very bottom. You can navigate it as usual to find your character (if you’re EU, change the URL in the address bar first). Once you do, hit the “Import” button at the bottom right. Make sure you’re in protection spec, though!
When you click Import, Simcraft will grab your character information and generate a simulation file from it. This is displayed in the Simulate tab. SimC will automatically use the default action priority list that I’ve programmed into it, so you don’t need to tweak this tab at all.
There is a big “Simulate” button at the bottom right of this screen. Don’t push it.
You pushed it, didn’t you. All right then, let’s just see what it spits out.
This is the html file Simcraft produces when I sim Theck:
Let’s briefly look at a few features of this report. First, this is the section containing the broad overview of the results:
The first line gives the character name and a bunch of information: our DPS, DTPS, and TMI score. The tables under “Results, Spec, and Gear” give us a more detailed breakdown of these quantities, including error estimates. This sim was only 1000 iterations, which is the default size, but in a few minutes we’ll see how we can increase that to improve accuracy.
The next section contains a bunch of charts showing damage per execute time (DPET), DPS and Vengeance timelines, damage source breakdown, and a slew of other statistics in chart form.
Further down the report are breakdowns of ability usage, buff uptimes and details, resource gains and losses, even more charts, proc counters, and then a bunch of statistics. We’re going to skip over the rest of that for now, because for today we’re only interested in calculating TMI and smoothness scale factors. To do that, we need to change some of the options.
The Options Screen
Go back up to the top tab bar and choose “Options.” It should bring up a set of sub-tabs, with the “Globals” sub-tab displayed:
There are a lot of choices here, some of which are obvious and some of which aren’t. We’ll explore all of the choices here at a later date, but for now, we want to make the following changes:
- Iterations – increase to 10k or higher. Larger numbers of iterations give more accuracy, but also take longer.
- Threads – This increases the number of threads SimC can use to run the simulation, which increases simulation speed. If you have a quad-core processor or higher, set this to 4. If you have a dual-core, set it to 2. If you’re not sure, leave it at 1 and plan on grabbing a drink while the program simulates.
- TMI Standard Boss – this drop-down lets you select one of the standardized TMI boss configurations. Pick the option that’s most appropriate to the content level you usually play at. All of the standard bosses assume 25-man raiding (i.e. T15H hits as hard as Lei Shen does on 25-man heroic mode), so you may want to drop back one category if you’re a 10-man raider. The “custom” option uses the SimC default.
Next, we want to shift over to the “Scaling” sub-tab. This has all of the different options for testing scaling:
As you can see, I’ve checked the boxes for Strength, Stamina, Expertise, Hit, Crit, Haste, Mastery, Armor, Dodge, and Parry. Most importantly though, at the very bottom, I’ve changed the Scale Over option to “tmi” to tell Simcraft that I want scale factors based on the Theck-Meloree Index.
Ok, now we’re ready. Hit the simulate button again. Note that with this many stats, it may take a while unless you’re using 4+ threads. With 10k iterations and 4 threads, it takes about a minute on my i7-2600k. Here’s the result:
In addition to a more accurate estimate of my TMI (because we used more iterations), I now have a new chart in the Charts section:
These are my smoothness scale factors, complete with handy error bars to tell us how confident SimC is about those values. If we increased the number of iterations to 25k or 50k, we’d get even smaller error bars and a better estimate ofeach scale factor.
Note that these scale factors are all negative. That’s because TMI uses “golf rules,” meaning that a lower score is better. So the scale factors are negative because, for example, each point of haste reduces my TMI score by about -1. On the other hand, critical strike rating has almost no effect on TMI, which is reassuring because it shouldn’t have any effect on damage smoothing.
Note that unlike what we usually do here on Sacred Duty, these scale factors are not normalized for itemization. In other words, this is directly comparing 1 stamina to 1 haste to 1 armor, and so forth. So for example, with these stat weights a stamina trinket would hold more value than a haste trinket because the weights are pretty close but you get 50% more stamina on a trinket than you get haste. On the other hand, a haste gem would be worth more smoothing than a stamina gem, because you get 33% more haste than stamina on gems.
Saving and Exporting
You may have noticed that there’s a nice “Save!” button at the bottom right of the report tab, so that you can save these results for future reference. Unfortunately, the button doesn’t do anything. Oops. This bug should be fixed in 530-6. For now, though, you’ll have to save the results manually. You can do that by going to the \simc-503-5-win32\ folder and finding the “simc_report.html” file, which is your latest simulation result. You can rename that file to something memorable (like “theck_10k_scale.html”) to save the results.
You can also automatically export the results to a number of websites. If you check under the “Results, Spec, and Gear” section of the report, you’ll notice a new table containing scale factor information:
As you can see, this table contains both the unnormalized scale factors as well as a normalized (i.e. positive) set that you can use in gear ranking websites. In 530-5 these are normalized to strength, but in 530-6 they’ll be normalized to stamina (since it’s sort of silly to normalize scale factors to strength for the agility tanks).
There are also a bunch of links to gear ranking sites and online optimizers. Each of these links includes all of the normalized scale factors, so they’ll automatically load up in the site when you click the link. For example, clicking the wowhead link brings up a wowhead item ranking page personalized with the stat weights from this run. The “(caps merged)” version sets the hit and expertise values to the lowest simmed value (in this case crit). There’s also a wowreforge link, and I hope to add an AskMrRobot link in the near future.
Rough Rules of Thumb
Without a little context, it’s tough to make heads or tails of the TMI number that Simcraft puts out. For example, is a TMI of 10k good or bad?
The answer to that is somewhat relative, of course. It doesn’t matter what your TMI is if all you care about are scale factors. But the intent is that, when using the standard boss of the appropriate content type for your gear level, you *should* get a TMI value of around 5k-10k. As you start to overgear a tier of content, your TMI should go down
For example, the T15H boss is expecting an ilvl of about 535. But I have an ilvl of 546, so I already overgear the T15H boss. Thus, my TMI is relatively low at around 2-3k. If I compared myself to the T15N boss, it would be even lower (around 700), which would make the stat weight calculations a lot more sensitive to noise. So it’s generally good to sim against the boss that most closely approximates your gear level, and when in doubt, aim high.
If I sim Rhidach, who has an ilvl of 527, I’d probably want to use the T15N boss, because that boss expects an ilvl of ~522. If I do that, I get a TMI of around 5300, which is about right because he’s starting to overgear normal content. If I sim him against the T15H boss, though, I get a TMI of around 67k, about an order of magnitude worse!
The reason the difference is so large is that the metric is normalized to the paladin’s health. Poor Rhidach only has about 720k hit points fully buffed, so if you pit him against a boss that can melee for 340k after armor mitigation, he’s in danger of death from 2 full melees plus a stiff gust of wind, or even a full melee plus some blocked/mitigated attacks. The 6-second moving average is going to include a fair number of these events clocking in at 120% or more of his health, and since the weight function is exponential in percentage health, they cause a significant increase in score.
This is by design of course – the point is to heavily penalize large spikes that put you in danger of death, and a 120% health spike certainly fits that description. Note that the scale factors are still going to be pretty similar, though. The relative rankings will be the same, though the values may shift around some because not all stats scale similarly with boss hit size. So even if you get a TMI score in the millions, the stat weights will still be very reliable.
Finally, note that I’ve only tested this extensively for paladins. A warrior, DK, druid, or monk tank at a similar ilvl may not fall into the 5k-10k range that we do. Again, that’s by design, because we don’t want to normalize across tanking classes. If a DK takes spikier damage intake than a paladin, that’s something we want to know, and TMI should properly reflect that by giving us a larger value.
This was a quick-and-dirty introduction to Simcraft. I plan on going more in-depth about many of the options and reported statistics in later blog posts, but if you don’t feel like waiting, the Simulationcraft Wiki has lots of information on how to get started, tweak options, and interpret results.
Also note that the simulation output is only as good as what you put into it. I think the prot warrior module is fairly complete, but I’m not sure about the DK, druid, or monk modules. The DK module in particular suffers from the lack of a good “recent damage taken” conditional, because it limits them to spamming Death Strike rather than reacting to large health changes. Since that’s very similar to the sort of information we would like to have for shifting queues, I’m working with another dev to get an action priority list option implemented for that in the near future.
In addition, for the more technically-minded, I’ve written up a TMI Standard Reference Document. This outlines the official calculation method and specifies standard conditions for comparing TMI between different tanks. Since this is the initial version of the SRD, feedback on the details is greatly appreciated.