Pester testing – and unit testing in general – is interesting. Take, for example, this scenario

Unit testing functions which are *meant* to return random values within limits. Now that’s an interesting task.

— Jason Brown (@cloudyopspoet) October 14, 2015

I mean, you can test a return value is less than a limit over and over and over. But can you ever be totally sure oh no this is philosophy

— Jason Brown (@cloudyopspoet) October 14, 2015

Yep. Unit testing Functions which are designed to return a random value is most tricky. Take, for example, a Function I knocked up a little while ago that’s meant to return a random date and time during working hours in the following week.

Function Get-RandomDate
{
    [CmdletBinding()]
    param()
    # weekday, in the coming week, during business hours

    $now = Get-Date -Hour ((9..17) | Get-Random) -Minute ((0..59) | Get-Random)
    $now = $now.AddDays(7) # move it into next week
    $now = $now.AddDays( ((1..5) | Get-Random) - $now.DayOfWeek.value__)  # randomise the day    
    return $now 
}

Now, I am not 100% sure as I write this blog whether or not I’ve screwed up this function completely. Luckily, I’m using Pester, so I can test it. But because it returns a random value, this makes things a bit… tricky. You may be getting a regressed-to-the-mean middle result while your test runs, but out in the wild you may be returned an outlier and suddenly your function is causing all manner of screw-ups.

The answer, for now at least, rests with running your function multiple times in a test, and thus gaining a reasonable assurance that your code is not returning an odd value. Consider this test, for instance

It "Returns a date in the future" {
            (1..100) | % {            
                $ok = $true
                $d = Get-RandomDate
                if($d -lt (Get-Date))
                {
                    Write-warning $d
                    $ok = $false
                }
                $ok | Should Be $true
            }
        }

If we run our Get-RandomDate function one hundred times, and it doesn’t ever return a date in the past, we can be reasonably – though not totally – sure it works as intended. Maybe one time in a thousand it may return an incorrect value. Maybe if I test it a thousand times, the thousand-and-first iteration might give a dodgy result. Or maybe not.

We also don’t want this function to just return me the same thing over and over. There’s an easy way to do that.

        It "Doesn't give the same answer twice" {
            (Get-RandomDate) -eq (Get-RandomDate) | Should Be $false
        }

Maybe I’ll want to run this in a loop too. I probably should. Then again, we can tolerate this. The test is just a safeguard against a future change making the return values ‘hardcode’, for want of a better word.

Now, with that out of the way, one of the things I don’t want my code to return is a Saturday or Sunday. And given that’s a relatively small problem area, I feel relatively comfortable right now in running it only thirty times per test round. 

        It "Does not return a Saturday or Sunday" {
            $ok = $true
            (1..30) | % {
                $d = Get-RandomDate
                if($d.DayOfWeek.Value__ -gt 5)
                {
                    $ok = $false
                }
            }
            $ok | Should Be $true 
        }

Of course, one time out of 300, it might return me a Sunday. Maybe unlikely, but it might. Would it hurt? Probably not, which is another factor in my decision not to labour the test.

Besides, what we’re really doing with a unit test like this is ensuring that the code returns what I want it to and doesn’t deviate from that if something changes later within a given degree of confidence. It’s insurance that a change probably won’t cause a deviation. And that’s a very good thing.

On a grander scale, of course, you may want to run your code thousands, millions, even billions of times, depending on the level of confidence you’re looking for, as pointed out by my mate Damo here

@cloudyopspoet Run unit tests millions of times, analyse results for statistical anomalies.

— Damian Brady (@damovisa) October 14, 2015

Luckily, computers are really, really good at running repetitive operations. The only limit you’ll reach is how much hardware you’re willing to throw at a test and how much time you have to spare. And whether you do in fact want to ship your code before the heat death of the universe.