What's the best way to unit test code that generates random output?

Posted by Flynn1179 on Stack Overflow See other posts from Stack Overflow or by Flynn1179
Published on 2010-06-18T09:30:20Z Indexed on 2010/06/18 9:33 UTC
Read the original article Hit count: 258

Filed under:

best-practices

|

unit-testing

|

statistics

|

probability

Specifically, I've got a method picks n items from a list in such a way that a% of them meet one criterion, and b% meet a second, and so on. A simplified example would be to pick 5 items where 50% have a given property with the value 'true', and 50% 'false'; 50% of the time the method would return 2 true/3 false, and the other 50%, 3 true/2 false.

Statistically speaking, this means that over 100 runs, I should get about 250 true/250 false, but because of the randomness, 240/260 is entirely possible.

What's the best way to unit test this? I'm assuming that even though technically 300/200 is possible, it should probably fail the test if this happens. Is there a generally accepted tolerance for cases like this, and if so, how do you determine what that is?

© Stack Overflow or respective owner

Related posts about best-practices

Batch Best Practices and Technical Best Practices Updated

as seen on Oracle Blogs - Search for 'Oracle Blogs'
The Batch Best Practices for Oracle Utilities Application Framework based products (Doc Id: 836362.1) and Technical Best Practices for Oracle Utilities Application Framework Based Products (Doc Id: 560367.1) have been updated with updated and new advice for the various versions of the Oracle… >>> More
Partial template specialization of free functions - best practices

as seen on Stack Overflow - Search for 'Stack Overflow'
As most C++ programmers should know, partial template specialization of free functions is disallowed. For example, the following is illegal C++: template <class T, int N> T mul(const T& x) { return x * N; } template <class T> T mul<T, 0>(const T& x) { return T(0); } //… >>> More
Object Oriented PHP Best Practices

as seen on Stack Overflow - Search for 'Stack Overflow'
Say I have a class which represents a person, a variable within that class would be $name. Previously, In my scripts I would create an instance of the object then set the name by just using: $object->name = "x"; However, I was told this was not best practice? That I should have a function set_name()… >>> More
Good place to look for example Database Designs - Best practices

as seen on Stack Overflow - Search for 'Stack Overflow'
I have been given the task to design a database to store a lot of information for our company. Because the task is rather big and contains multiple modules where users should be able to do stuff, I'm worried about designing a good data model for this. I just don't want to end up with a badly designed… >>> More
Notification Email Best Practices--From Server Setup to Programming

as seen on Stack Overflow - Search for 'Stack Overflow'
All, I'm in the process now of building a SaaS tool that allows network admins to generate notification emails to the members of the end-users of our platform (among many many other things). I'm running into a bit of an "out of my expertise" wall, as I know there are a lot of variables involved… >>> More

Related posts about unit-testing

Unit testing best practices for a unit testing newbie

as seen on Programmers - Search for 'Programmers'
In recent years, I have only written small components for people in larger projects or small tools. I have never written a unit test and it always seems like learning how to write them and actually making one takes a lot longer than simply firing up the program and testing for real. I am just about… >>> More
About unit testing a function in the zend framework and unit testing in general

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello people, I am diving into the world of unit testing. And i am sort of lost. I learned today that unit testing is testing if a function works. I wanted to test the following function: public function getEventById($id) { return $this->getResource('Event')->getEventById($id); } So… >>> More
Unit testing a database connection and general questions on database-dependent code and unit testing

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, If I have a method which establishes a database connection, how could this method be tested? Returning a bool in the event of a successful connection is one way, but is that the best way? From a testability method, is it best to have the connection method as one method and the method to get… >>> More
What is the objective of unit testing?

as seen on Programmers - Search for 'Programmers'
I've been working with C# for the last 2 years, and I've never done any unit testing. I just need to know what the objective of unit testing is. What kind of results do we expect from unit testing? Is code quality checked by unit testing? In my view, unit testing is the job of testers; if that is… >>> More
Unit testing newbie team needs to unit test

as seen on Programmers - Search for 'Programmers'
I'm working with a new team that has historically not done ANY unit testing. My goal is for the team to eventually employ TDD (Test Driven Development) as their natural process. But since TDD is such a radical mind shift for a non-unit testing team I thought I would just start off with writing unit… >>> More