The three major types of unreliable data in employee compensation surveys

Data quality and employee compensation surveys

Self-reported data in employee compensation surveys.

These tend to be internet-based free employee compensation survey sources. And they tend to be employee-focused, not employer-focused. They are designed so that employees can go on to the internet and find the appropriate compensation level. They’ve started to branch out and tried to sell separate products to employers.

The first red flag that I have when I see that is that they give two very different answers. If you look up two identical searches, two identical market pricings and you one on the employee-oriented side and one on the employer-oriented side, you have but two different answers to which I ask, which one is the right one?

The second issue is that it’s self-reported data. What I mean by that is I could go on to one of these websites and I can ask, “How much do the compensation consultant based in St. Louis earn?” and I could maybe even tell them the size of my practice so on and so forth.

And they say, “Okay. And we’d be happy to share some data with you but first tell us how much you earn so we can incorporate that into our database.” So I tell them my salary and they gave me a report on how to go beat up my boss for a pay raise.

If I don’t like the numbers that they gave me, I can go right back on to that same, exact website, ask them the exact same question, “How much do the compensation consultant based in St. Louis earn?” By entering junk into their database, I can drive the results that I want. There’s also no quality control in the self reported employee compensation surveys

Department of Labor employee compensation survey.
The Bureau of Labor Statistics and Compensation Survey conducts an employee compensation survey for each state. The problem with this data is really two-fold.
First and foremost, the people collecting the employee compensation data for the Department of Labor are literally paid by the participant, not paid by the quality of participants. The person soliciting information for the DOL will literally say, “I don’t care what you put down as your participation, just put something down in a piece of paper and send it back in.” That’s all they care about.

Secondly, they have this very broadly defined job catagories. So, for example, in the you could look in the Department of Labor Texas state employee compensation survey and they had a job category called, “Teachers, professors and coaches.” The data included in that is everybody from the University of Texas’ head football coach at somewhere around $1.2 million a year through an entry level grade school teacher at the time in some areas around $24,000 a year.

Start averaging all of these data points together and the final result looks reasonable, I think it’s at $60,000 or so. But the methodology they used to get there made absolutely no sense whatsoever. It made no sense to put those two jobs, much less the scope of those two jobs in the same data point and that’s what they did.
Employee compensation data from competitors.
These often occur outside of HR and the managers will call and say, “Hey, we need to give somebody a raise because so and so down the street is paying more.” There’s a problem for a number of reasons. First and foremost, if you are in the private sector, meaning you’re not working for the government, it is illegal to call your competitors and ask them what they pay. It’s considered collusion.

Secondly, it’s not statistically reliable. Even if that is exactly right, that the shop down the street is paying 50% more for the exact, same job, that’s only one data point. It’s entirely possible if you go the other way down the street they’re paying 50% less. So, one or two competitors does not define the labor market.

From the Rapid Learning Institute webinar: “How to Set Pay Ranges That Are Fair and Effective” by Ed Rataj