Primer on calculating DPS stat weights

Primer on calculating DPS stat weights - Printable Version

+- SWTOR Mechanics Forums (http://mmo-mechanics.com/swtor/forums)
+-- Forum: General (/forum-1.html)
+--- Forum: Game Mechanics (/forum-5.html)
+--- Thread: Primer on calculating DPS stat weights (/thread-689.html)

Pages: 1 2 3 4 5 6 7 8

RE: Primer on calculating DPS stat weights - giiper - 02-26-2012 08:08 AM

(02-25-2012 02:20 PM)LagunaD Wrote:
(02-25-2012 02:10 PM)giiper Wrote: I tried with 102 accuracy rating on my char and i had 94.54% Basic Accuracy on my Main Hand and 61.54% on my off hand. I have 1 point on Steadfast so without points there it would be 93.54% and 60.54%. When i go to your spreadsheet and put 102 accuracy rating the accuracy that is calculated for Standard Attack is 0.995447813 and for the off hand is 0.665447813. I just want to know if Standard Attack should be the same of Basic Accuracy and if so is it an error on the spreedsheet or am i doing something wrong?

The spreadsheet assumes you have 3/3 in Steadfast, and Ataru stance active.

Thks for the fast reply that makes sense then. And good work on the spreedsheet.
It would be nice if we could chose our spec and rotation Tongue

thks again

RE: Primer on calculating DPS stat weights - kray - 03-03-2012 03:14 AM

Wanted some input really quick from you guys. Recently did a coding overhaul on my simulator, increased the speed per sim by 10-20x, and am now able to run stat weights properly.

With my settings, I'm running 1000 sims of 300sec each with a deltaStat of 50. Take a look at my results after generating values with these parameters:

Aim Equivalence Points (deltaDPS per stat divided by deltaDPS of Aim):

Code:

Aim    1.00    1.00    1.00    1.00    1.00    1.00    1.00    1.00

Crit    0.79    0.68    0.64    0.73    0.87    0.76    0.66    0.77

Surge    0.13    0.07    0.00    0.04    0.06    0.00    0.10    0.08

Accuracy    0.28    0.38    0.38    0.18    0.27    0.27    0.22    0.35

Power    0.92    0.89    0.86    1.05    1.11    0.94    0.87    1.00

Tech Power    0.60    0.63    0.72    0.69    0.68    0.64    0.66    0.74

DPS Equivalence Points (deltaDPS / deltaStat, change in DPS per point of stat):

Code:

Aim    0.44    0.43    0.39    0.39    0.38    0.42    0.43    0.39

Crit    0.35    0.29    0.25    0.28    0.33    0.32    0.28    0.30

Surge    0.06    0.03    0.00    0.01    0.02    0.00    0.04    0.03

Accuracy    0.12    0.16    0.15    0.07    0.10    0.11    0.09    0.14

Power    0.40    0.38    0.34    0.41    0.42    0.39    0.37    0.39

Tech Power    0.26    0.27    0.28    0.27    0.26    0.27    0.28    0.29

I know it's a bit hard to read, but essentially what I'm seeing in DEP (what I really care about) is a variation of +/- .03 or so, which is exacerbated when looking at the easier-to-read AEP format. In practice, this variation is absolutely minimal, but I'd like to do what I can to get rid of it at this point. Any recommendations, or is the answer "run more sims"?

Caveat: The stats I'm using right now have Surge almost at the cap, to explain the low values for that.

RE: Primer on calculating DPS stat weights - RuQu - 03-03-2012 03:33 AM

(02-23-2012 03:15 PM)LagunaD Wrote:
(02-23-2012 01:37 PM)Kor Wrote: The best way is generally the instantaneous stat weight, which can be calculated as the dps change when y = 1, y=-1, or averaging the two. The gives the best idea of current stat weights.

The problem is that changing a stat by one unit results in a DPS change so small that the simulation time required to isolate it from statistical noise becomes extremely large.

If you want your stat weights accurate to 1%, you need to measure a tiny difference between two large numbers to 1%, when the difference itself is less than 1%.

1 point of (say) Power increases your DPS by (roughly) +0.33. So you need to measure the difference between (say) 1500 and 1500.33 to 1%, which means measuring the DPS at each to the two points to within +/- 0.002 or so - roughly one part in a million...

The only practical way I can think of to do this is the obvious one described by Kray: pick the smallest increment large enough to make a statistically distinguishable difference in the DPS.

I do think it's better to use +x and -x, as Kor suggests. Calling the weight

w = (y(+x) - y(-x)) / (2*x)

is equivalent to drawing a quadratic curve through the points (-x,0,+x) and finding the slope at 0. So really you have managed to take into account (through cancellation) second-order effects.

I'm not sure I follow.

If +1 Power amounts to an increase of +X DPS for ability A and +Y DPS for ability B, and you know the ratio of A to B usage, should it not be trivial to simply say that that increase in total DPS is (X*A_casts+Y*B_casts)/(A_cast + B_casts)?

I ran into this a bit when I started trying to simulate rotations for Gunnery, due to the RNG of the procs in that tree. The outputs would vary by so much as to make assigning a value quite difficult. But since what we want to know is how much it would change if we change the stat, but hold everything else equal, should we not, actually, hold everything else equal? I simply set the seed for my RNG as a fixed seed, and reseed every time the relevant button is pushed. So the first rotation might have very few procs, and the second lots of procs, but that pattern will be true for every stat setup. The rotations vary internally within a computation run, but not from run to run.

RE: Primer on calculating DPS stat weights - kray - 03-03-2012 04:03 AM

That brings up a good point, related to my post. Would seeding the random generator during stat weight calculation be beneficial?

I see the point you're getting at, and it seems like a good way to remove the rng-based variance that I get while simming my stat weights. Even with 2000 sims, variance can still noticeably occur.

RE: Primer on calculating DPS stat weights - Cosmic Osmo - 03-03-2012 04:09 AM

Why is it necessary to run a simulation at all? I have been calculating DPS equivalent points using a very simple 1-page excel sheet that works on an assumed rotation (eg. in 18 seconds I will use 1 gut, 2 stockstrikes, 1 pulse cannon, etc), with the effects of procs and cooldowns smoothed out over time, or just omitted, depending on their relevance. This method is easily "good enough" for giving stat weights that are useful in picking gear (eg. balancing crit vs power, balancing accuracy vs surge).

I'm not trying to attack you here, I'm just wondering if there's some complexity I've missed that makes it necessary to simulate with random factors included.

RE: Primer on calculating DPS stat weights - kray - 03-03-2012 04:29 AM

(03-03-2012 04:09 AM)Cosmic Osmo Wrote: Why is it necessary to run a simulation at all? I have been calculating DPS equivalent points using a very simple 1-page excel sheet that works on an assumed rotation (eg. in 18 seconds I will use 1 gut, 2 stockstrikes, 1 pulse cannon, etc), with the effects of procs and cooldowns smoothed out over time, or just omitted, depending on their relevance. This method is easily "good enough" for giving stat weights that are useful in picking gear (eg. balancing crit vs power, balancing accuracy vs surge).

I'm not trying to attack you here, I'm just wondering if there's some complexity I've missed that makes it necessary to simulate with random factors included.

I think it was linked earlier, but here it is again for posterity... Sims vs Formulas.

http://code.google.com/p/simulationcraft/wiki/FormulationVsSimulation

Quote:Accuracy-vs-Precision
One might say that the difference between formulation and simulation is the difference between precision and accuracy. The former puts the proverbial bullet in the same spot (12 inches off center) every time. The latter puts the bullet within 3 inches of the center, but never hits the same place twice.

RE: Primer on calculating DPS stat weights - RuQu - 03-03-2012 04:46 AM

(03-03-2012 04:29 AM)kray Wrote:
Quote:Accuracy-vs-Precision
One might say that the difference between formulation and simulation is the difference between precision and accuracy. The former puts the proverbial bullet in the same spot (12 inches off center) every time. The latter puts the bullet within 3 inches of the center, but never hits the same place twice.

And here I think is the advantage of seeding the RNG.

Within your chosen number of iterations you never hit the same place twice, but every time you choose to rerun that set of iterations, they hit all the same places.

RE: Primer on calculating DPS stat weights - Cosmic Osmo - 03-03-2012 04:54 AM

Okay, using that analogy, this is how I view formulation vs simulation:

[Image: 6KDDc.png]

Can you actually say that the green dots represent a more useful picture of what the red dots are doing than the blue dot does (in other words, does it seem that simulation gives us something useful that formulation doesn't)?

The obligation of a formulation-calculator is merely to be within the player's performance deviation. If you're within that deviation and generating stat weights, then those stat weights are going to be absolutely optimal for some of that player's boss attempts, and pretty damn close on the rest—probably so close that theoretical (impossible) optimization would not even result in changing a piece of gear. The marginal gain of putting any more effort into it than that is negligible. (Surely someone will come in here and tell me "lol your red dots are so spread out, you must be a terrible player!" Adorable.)

Remember, it is not our task to calculate the DPS of the player. We only need to calculate stat weights. Getting the rotation slightly wrong, or averaging out something like a power proc, will actually not affect the stat weights generated by very much at all, because different attacks tend to respond in very similar ways to increases in stats (notable exception: basic level 1 free attack).

RE: Primer on calculating DPS stat weights - kray - 03-03-2012 05:08 AM

(03-03-2012 04:54 AM)Cosmic Osmo Wrote: Okay, using that analogy, this is how I view formulation vs simulation:

Can you actually say that the green dots represent a more useful picture of what the red dots are doing than the blue dot does (in other words, does it seem that simulation gives us something useful that formulation doesn't)?

The obligation of a formulation-calculator is merely to be within the player's performance deviation. If you're within that deviation and generating stat weights, then those stat weights are going to be absolutely optimal for some of that player's boss attempts, and pretty damn close on the rest—probably so close that theoretical (impossible) optimization would not even result in changing a piece of gear. The marginal gain of putting any more effort into it than that is negligible. (Surely someone will come in here and tell me "lol your red dots are so spread out, you must be a terrible player!" Adorable.)

Remember, it is not our task to calculate the DPS of the player. We only need to calculate stat weights. Getting the rotation slightly wrong, or averaging out something like a power proc, will actually not affect the stat weights generated by very much at all.

That's where you're wrong. It is my task as someone who writes simulation to generate stat weights based off of DPS. I run thousands of sims with changes in various stats, and calculate their value off of that. In that case, the calculated DPS needs to absolutely be accurate and precise.

The issue with your graphic (awesome by the way!) is that the top end player -- the one who really cares about stat weights the most -- is going to be doing everything in his/her power to reach that bullseye. You also don't include (nor does the link I sent you) the fact that as you run more and more simulations and average the results, those dots for simulations get closer and closer to the bullseye.

But, simulation vs formulation is an argument as old as time. Fact is, I'm doing it this way and want to find out a way to improve this method!

RuQu, I like the idea. Dunno if I'm going to include it in mine though.

RE: Primer on calculating DPS stat weights - RuQu - 03-03-2012 05:08 AM

(03-03-2012 04:54 AM)Cosmic Osmo Wrote: Okay, using that analogy, this is how I view formulation vs simulation:

Can you actually say that the green dots represent a more useful picture of what the red dots are doing than the blue dot does (in other words, does it seem that simulation gives us something useful that formulation doesn't)?

The obligation of a formulation-calculator is merely to be within the player's performance deviation. If you're within that deviation and generating stat weights, then those stat weights are going to be absolutely optimal for some of that player's boss attempts, and pretty damn close on the rest—probably so close that theoretical (impossible) optimization would not even result in changing a piece of gear. The marginal gain of putting any more effort into it than that is negligible. (Surely someone will come in here and tell me "lol your red dots are so spread out, you must be a terrible player!" Adorable.)

Remember, it is not our task to calculate the DPS of the player. We only need to calculate stat weights. Getting the rotation slightly wrong, or averaging out something like a power proc, will actually not affect the stat weights generated by very much at all.

Agreed, and I think the image sums it up well.

I think we also need to keep in mind that there is a customer, the player, and that the results need to be meaningful and useful to them. Using an RNG to simulate 1000 runs produces a reasonable number (with some error on it), but if the stat weights differ from click to click for the same stats, you can really confuse the user who may not know much about any of the math on the backend.

You either need to simply give them an order (Power>Crit>Surge for example) or give them numerical weights that won't change just because they click the Calculate button again. Hence, feeding a fixed seed.

Saw someone on the SWTOR.com forums referring someone to my Google Code project to answer a gearing question, and they said their testing of my results were within 3-5% of what they experienced in game. Considering the high variability of in-game situations and player skill, and the assumptions that are required for any of this kind of work, I'd say that's close enough, as did they.

That said, I think for some people this type of thing is more fun than the game, so getting that extra bit of accuracy out of it is more important than how useful the actual efforts are.