Survey data efficacy in compensation management

by on June 25, 2009 · 0 Comment POSTED IN: HR Info Center

Key questions will find out if the data works for your compensation management system

Survey data questions need to link back to your compensation management plan
We kind of ask these questions to relate back to our compensation management strategy. First, who responded? Are the participants in the survey similar in demographics to the organization, in this case our client, but is it similar to our organization? Are they similar in size, similar in industry, similar in geographic area?

The next consideration with compensation management data is sample size. How many participants are there? There are some surveys out there that have tens of thousands of participants. They’re very statistically valid. There’s a huge respondent base. Other surveys have three, four, five, ten participants. They’re much less reliable. Not only from a statistical perspective. As you look at that in a single year or particularly, if you start to look at trends from one year to the next, the turnover of one organization choosing to participate and one organization choosing not to participate can significantly change the data on a year to year basis, which leads me to the next point, the stable respondent base.

Survey responses can change your compensation management plan
Does the survey tend to have similar sets of respondents year after year so that we can look at trending analysis? One of the worst things that can happen as we go to this market pricing process – and for us to market price a job one year and we come up with a market price of $60,000 using our data.

And then we don’t have a stable respondent base so we use the exact, same data, the exact, same matches the following year and now it says the market’s $45,000 or $50,000. But we know that that’s not right but which is the right number? We just had a change in the respondent base that created some noise in our market data. So we want to make sure that we control that as much as possible.

Is the survey regularly conducted?
Generally speaking, surveys are conducted on an annual basis. That is the typical time frame. You’ll find some surveys that are conducted every two years. That is not very common. And generally speaking, high quality surveys are not conducted every two years. Just too much time goes by. Too much is changing in the compensation management market over a two-year period.

What is the value of the survey in regards to our compensation management planning?
What is the cost of this survey, what is the quality of the data, how many jobs can we match to this, is it the appropriate labor markets in which we compete for staff comparing all those things to the cost of the survey – can be a critical decision in selecting market data.

Scrubbing methodology is critical to the compensation management data
With the high quality surveys, you’ll find that they have a scrubbing methodology. They’re looking at outliers in the participation and they’re following up with those participants to make sure that the data that was provided is accurate.

Reliable data versus unreliable data in compensation management. Generally speaking, high quality data comes from the major consulting and surveying firms. They are generally going to be statically validated. They’re going to have that scrubbing methodology to really have high quality results.

Conduct a standard deviation analysis of the data.
Let’s say that we’re market pricing our accountants and we have three different survey sources that all have appropriate data for an accountant in our local area. And the three data sources — one says $40,000, one says $42,000 and the other one says $75,000.

Well, by running standard deviation of the different surveys, the different numbers that we included in that aggregate match rather than just average those three surveys together, if we run that standard deviation analysis, they will jump out of the play, “Hey, we have a real high standard deviation for these matches. Let’s go in and let’s double check to make sure that something screwy isn’t going on.”
From the Rapid Learning Institute webinar: “How to Set Pay Ranges That Are Fair and Effective” by Ed Rataj

Leave a Reply

Close

Request a Free Demo

We'd love to show you how this industry-leading training system can help you develop your team. Please fill out this quick form or give us a call at 877-792-2172 to schedule your one-on-one demo with a Rapid Learning Specialist.