Estimating cost of labor for a controlled experiment

Posted by Lorin Hochstein on Programmers See other posts from Programmers or by Lorin Hochstein
Published on 2013-04-28T02:22:48Z Indexed on 2013/06/27 10:29 UTC
Read the original article Hit count: 440

Filed under:
|

Let's say you are a software engineering researcher and you are designing a controlled experiment to compare two software technologies or techniques (e.g., TDD vs. non-TDD, Python vs. Go) with respect to some qualities of interest (e.g., quality of resulting code, programmer productivity).

According to your study design, participants will work alone to implement a non-trivial software system. You estimate it should take about six months for a single programmer to complete the task. You also estimate via power analysis that you will need around sixty study participants to obtain statistically significant results, assuming the technologies actually do yield different outcomes.

To maximize external validity, you want to use professional programmers as study participants. Unfortunately, it isn't possible to find professional programmers who can volunteer for several months to work full-time on implementing a software system. You decide to go the simplest route and contract with a large IT consulting firm to obtain access to programmers to participate in the study.

What is a reasonable estimate of the cost range, per person-month, for the programming labor?

Assume you are constrained to work with a U.S.-based firm, but it doesn't matter where in the U.S. the firm itself or the programmers or located.

Note: I'm looking for a reasonable order-of-magnitude range suitable for back-of-the-envelope calculations so that when people say "Why doesn't somebody just do a study to measure X", I can say, "Because running that study properly would cost $Y", and have a reasonable argument for the value of $Y.

© Programmers or respective owner

Related posts about freelancing

Related posts about research