As I’ve mentioned a few times on Twitter already, I’ve been working on refining the formula used to calculate the Theck-Meloree Index. The current version certainly works, or at least, gives me the numerical effects that I initially wanted. But in the 6+ months since we defined the metric, we’ve learned a lot more about the quirks involved with having a raw exponential metric. Several of which are more rooted in psychology than mathematics!
(If you’re keeping score, that’s Wrathblood – 1, Theck – 0)
In any event, after a bit of playing with the possibilities I’ve finally decided how I want to modify the formula. It’s all but finished, in fact. The only thing left to do is fine-tune some constants, which I think I’ve done sufficiently well already. But the only good way to test that is to generate a lot of data and see if it’s working the way I want it to.
Which normally would be fine, but there are a few issues with doing that myself.
- It takes a long time to generate the amount of data I’m looking for. Think several hundred simulations, each with 25k iterations, and calculating 10 scale factors.
- I want to test it on a variety of gear sets. Again, it takes a lot of time to put together gear sets, and I don’t really want to troll the armory looking for random players to import.
- I want to test it on all five tanking classes (or at least, the ones that SimC supports). Again, short of trolling the armory, it would be a formidable task to find an appropriate number of players to get a proper sample. And would take a long time.
- I want to test it against multiple different TMI bosses… so multiply all of those time investments by a factor of four or five.
I’m busy enough as it is with all sorts of other projects (*cough* and Diablo III), not to mention my job, that it’s not feasible for me to generate all of this data myself. Unless you want to wait for the new TMI definition until December. Of 2017.
I could just release the new metric into the wild, of course. I’m pretty sure it’s functioning properly, after all. But I’d much rather be able to do some rigorous testing of it in case there are weird problems that I didn’t anticipate.
This is where you come in. Instead of running several hundred simulations myself, I’m asking each of you to run a simulation or two for me. Basically, you could consider this the public beta test of TMI 2.0.
How To Contribute Data
I’ve coded the new TMI definition into Simulationcraft, and it’s available as an option in version 547-2. By default, it will calculate stat weights using the old formula. However, you can enable the new formula with the argument new_tmi=1.
You can do this by adding that line to the Simulate tab as shown in the screenshot below:
The results page will then report TMI as calculated using the new formula.
If every reader of the blog runs their own character through the sim, I will have a veritable sea of data to swim through (as in, many thousands of simulations). I’m not that optimistic about a 100% reader-to-data-submission conversion rate, so if you can run your character several times with different options (i.e. against different TMI bosses), that’s even better.
Here are the basic guidelines that I’m looking for in submissions:
- 25000 iterations
- Standard Patchwerk fight (these should all be SimC defaults)
- Length: 450
- Vary Length: 20%
- Style: Patchwerk
- Level: Raid Boss
- Target Race: humanoid
- Num Enemies: 1
- Challenge Mode: Disabled
- Standard Player settings (again, defaults)
- World Lag: Low
- Player Skill: Elite
- Scale Factors (make sure you choose to scale over “tmi”)
- Strength or Agility (depending on your class)
All of these options can be found on either the Options->Globals tab or the Options->Scaling tab. First, a quick look at the Globals tab:
You can see that I have all of the settings at default here. The only two I want you to play with are the TMI Standard Boss and the TMI Window.
For the TMI Boss, pick one (or more) ilvl-appropriate bosses. For example, if you’re in heroic T16 gear, then you shouldn’t bother simming against the T15 bosses at all, and probably not against T16N10. Stick to T16H bosses or the 17Q boss. Please do not use “custom” – that will pit you against Fluffy Pillow, who is not so fluffy anymore now that he learned how to perform melee and spell nukes.
For TMI Window, the standard is six seconds. Feel free to leave it at that if you’re getting reasonable results. If you get really weird-looking stat weights or your TMI is below, say, 1000, consider dropping this a little, maybe to four seconds. Please submit the wonky stat weight data anyway, because that’s also useful to me, but then submit the (hopefully) normal-looking data you get using the lower TMI window.
On to the Scaling tab:
As you can see, I’ve checked all of the stats I’m interested in. If you’re a druid or monk tank, please check the Agility box too (in that case you can skip Strength if you want to). Above all though, make sure you’ve chosen to scale over “tmi.” I can’t stress this enough, because if you scale over DPS you’ll get scale factors that are useless to me, and it will just mean I have to spend time filtering the data to eliminate those useless data points.
Once you’ve completed the simulation, you can enter the data in the form below. Please also attach the html results (which you can get by using the “Save” button at the bottom right of the results pane in SimC) using the “Upload” button at the bottom of the form. I’m requesting the html so that I can sanity-check the data and figure out what’s happening with outliers, so you can’t submit data without first attaching that file.
There’s no limit to how many times you can submit data, so you can run several different characters through the simulation if you want to. In fact, that’s encouraged, because data from undergeared alts is just as valuable to me (if not more) as data from overgeared mains. And of course, you can run a character against several different TMI bosses and submit each result separately.
Just don’t keep re-submitting a single simulation result multiple times, because each submission after the first would be useless for obvious reasons.
If the embedded form below isn’t working for you for some reason, you can also access it directly via this link: http://goo.gl/SY36xu. Note that you’ll have to reload the page (or open the link in another new tab) in order to use the submission form again.
Thanks in advance for your help! Depending on how quickly the data comes rolling in, I may be able to have this all wrapped up as early as next week.
As soon as that’s done, I’ll be making a much longer post detailing what changes I’ve made, why I’ve made them, and how the new formula works, including simulated data that I used to develop the metric and actual data from this exercise.