Category Archives: Rambles

Where it starts out well but then loses the thread

Pester Mocking, ParameterFilters and Write-Output

I just spend an annoyed 45 minutes or so puzzling over a Pester test I was writing. I needed to test an if/elseif/else branching block of PowerShell code, written by someone else, which did very similar things to install powershell modules, uninstall, reinstall, or not, depending on some evaluated conditions and based on a list of modules and their versions

The final else condition was, in essence, not doing anything other than writing to the output stream to say “I’m not doing anything at this point”, so obviously I figured I could use Assert-MockCalled on Write-Output and I’d know what path the code was going down and I’d be sorted.

Anything but. Continue reading →

The Birthday Paradox and testing random functions

I had cause, recently, to write a little function that randomises a name for a resource from a list of roughly 90 words and a 3-digit postfix, in the form ‘happycat-267′*. Now, I don’t want to re-use a name that’s gone before, so the function checks already-deployed resources and if the name is taken, the function recurses to pick a new name from my ~90,000 possiblities.

Of course, when it came to testing this function, I wanted to be reasonably sure that the recursion worked and would continue to work in the future, so I decided to write a simple Pester test that loops a certain number of times and doesn’t return a duplicate value.

But this raised a question. How many iterations should this test go through to be reasonably sure of hitting a duplicate and thus triggering a recursion?

Obviously to fully test, I’d need to run the function 90,000+ times. But that’s computationally expensive and would slow down my testing. I don’t want to do that. But how many is the least number of tests I can do to be reasonably sure of a duplicate appearing?

Which brings me to the Birthday Problem, aka the Birthday Paradox.

Stated simply, the Birthday Problem tells us that in a group of only 23 people, it’s more likely than not to find two people who share a birthday. And if you have 70 people in a room, the probability of at least one birthday match is up at 99.9%.

The Wikipedia article goes into a lot of detail on why this is mathematically true, but the astute among us whill have noticed that this mathematical phenomenon has an application in unit testing my little “random-name-without-collisions” function.

There are Birthday Problem calculators online, such as this one, so I plugged in my possibility space of 90,000 and started playing with iterations. It turns out that if I call my function 1000 times, there’s a >99.5% chance that I’ll produce (and therefore handle) a duplicate.

At 500 iterations, the probability drops to around 75%, and at 300 iterations, it’s around 40% – so clearly 300 iterations is too low, that is it’s more likely that I won’t hit a duplicate.

So, I wrote my own function to calculate the probabilities for me

I can call this in a loop, and calculate a table of probabilities from 1 iteration to 10,000 (and graph that, if I feel like it).

I can then use that table to zoom in on a probability I feel comfortable with. Let’s say I’m happy with a 99% probability of collision, it turns out that 911 iterations will get me there. If I’m happy with 90%, 644 iterations will do it.

Above 911 iterations, the curve plateaus out and the returns from adding more iterations become smaller. We hit a point where PowerShell rounds the probability up to 1 at 2519 iterations. It’s not mathematically certain at this point, but we’re up around a 99.9999999999999% chance of collision.

So we can see there’s really not much point iterating above 2500 repetitions. The increased probability just isn’t worth the extra processor cycles.

So anyway, with a little mocking, I can write a useful test that has a 99% chance of hitting at least one duplicate in the function, and test that it doesn’t actualy return any dupes, thus:

Anyway, this is what I spent yesterday afternoon researching and playing around with. Hopefully someone other than me will find it useful. If not, well I had fun.

 

* not one of the actual values.

 

AWS Sydney Summit Roundup: Serverless is here

This past week saw AWS’s Sydney Summit at the historic Hordern Pavilion. While many saw the “coming soon” announcements as a slight disappointment (Lambda coming soon. API gateway coming soon. Everything else: coming soon), I – and am not the only one – saw it as a bit of a wakeup call.

In fact, this is really my only major takeaway from the summit:

Serverless architecture is coming. Get on board.

Lambda was launched to beta in late 2014 and to general availability in April 2015 to much fanfare, but as always with AWS, only in a subset of regions. Since then, it’s expanded out to encompass Virginia, Oregon, Ireland, Frankfurt and Tokyo.

It’s been much talked of in Australian Cloud circles – so much so that I freely admit to feeling a degree of Lambda Fatigue at meetups and conferences over the last year or so.

But the actual release of Lambda in ap-southeast-2 is imminent. And with it will come a fundamental shift in the way many of us in Australian Cloud circles need to think about Cloud infrastructure, because the potential advanatages in cost, development time and scalability are just too big to ignore. If you’re going all-in on cloud and your plans don’t include Lambda somewhere, then you’re probably doing something badly wrong. Continue reading →

A tedious and probably totally wrong post about idiomatic approaches to PowerShell

In PowerShell, there are many ways to do stuff.

This is a good thing. It’s what made Perl so attractive to me back when I wrote in unreadable languages. There was even an acronym. TIMTOWTDI. There Is More Than One Way To Do It.

This is good. It’s a great thing.

Which is what I thought of today, when browsing around the interwebs, I stumbled*, not for the first time, over a not-that-common but still sometimes-encountered PowerShell idiom for function declaration.

Continue reading →

Stricter DSC processing under WMF 5.0

If you’re all being good little Windows opscoders, you’ll be using DSC. And if, like me, you’ve upgraded to WMF 5.0 (PowerShell 5.0), you may have noticed a few new warning messages popping up in your logs

You do look at your logs, right?

Good.

Well, you may have been seeing this little warning of late

WARNING: The configuration ‘ExampleDSCConfig’ is loading one or more built-in resources without explicitly importing associated modules. Add Import-DscResource –ModuleName ‘PSDesiredStateConfiguration’ to your configuration to avoid this message.

I’ve certainly been seeing it. It hasn’t done any harm. Nothing breaks, it’s just a warning. I’ll get round to fixing it eventually.

Well, in the last day or so, I rolled WMF 5.0 into my production server fleet – a couple of hundred servers all up – and today I’ve had developers asking me why their Octopus Deploy projects are generating warnings now.  Continue reading →

Chickens not Cattle and definitely not Pets. Or maybe Bees

A few weeks ago I was ruminating on Twitter about the “Cattle not Pets” metaphor for Cloud instances.

I started a vague blog draft on the topic, got a little sidetracked and never fully completed the thought. But it’s something that’s itched at me for some time now. Cattle do in fact get fairly individualised treatment, except on the largest of scales. So they’re not a great metaphor. But I didn’t really have a perfect replacement So when I wandered into the office this morning and checked Twitter, I was gratified to see that Jeffrey Snover had tweeted out an article ruminating on basically the same topic, and which did a lot of the agricultural thinking for me.

Continue reading →

Rightsizing Your AWS Cloud Infrastructure: A Rumination

I am currently engaged in a mid-to-long-term project to rightsize the Cloud infrastructure at work. During our rapid change phase, shifting from a co-located infrastructure to public cloud, our primary priority was to get things done and minimise disruption while shifting services into the cloud quickly. Consider the things that cloud infrastructure should be

  • Cheap
  • Fast
  • Scalable
  • Resilient
  • Reliable*

During our migration phase, we weren’t too bothered about cheap. We wanted everything else, but the price ticket was… flexible, within limits. Now, some months later, we’re considering a number of our core services to be stable and mature, and therefore they’re prime candidates for aggressive cost optimisation. Cost optimisation in this case can imply a few different actions. Continue reading →

Unit Testing Functions that return random values with pester

Pester testing – and unit testing in general – is interesting. Take, for example, this scenario

Yep. Unit testing Functions which are designed to return a random value is most tricky. Take, for example, a Function I knocked up a little while ago that’s meant to return a random date and time during working hours in the following week.

Now, I am not 100% sure as I write this blog whether or not I’ve screwed up this function completely. Luckily, I’m using Pester, so I can test it. But because it returns a random value, this makes things a bit… tricky. You may be getting a regressed-to-the-mean middle result while your test runs, but out in the wild you may be returned an outlier and suddenly your function is causing all manner of screw-ups.

Continue reading →

What isn’t DevOps?

I was sitting on twitter today and a stream of tweets rolled past that made me… a little annoyed at first, then sort of intrigued because of the amount of discussion and brain wrangling they triggered.

realdevops

To be fair to Chris, I agree with a lot of what he was saying in his Real DevOps stream. But I fear that among the good stuff were a few tweets were either worded badly or wildly off the mark. I’ll admit I was initially tempted to dissect individual tweets but I think it’s more interesting – and less churlish – to get the Yet Another What Is DevOps post out of the way.

So, let’s get one thing clear, and then I’ll get started. A lot of people don’t know what “Devops” actually means. Here’s what it is, and here’s an opportunity for you to get your “no it’s not” ready.

Devops is the application of lean manufacturing principles to the IT Lifecycle

Continue reading →

Web-enabled S3 bucket migration using OctopusDeploy and PowerShell

I recently had to move a fairly large, fairly heavily trafficked web-enabled S3 bucket between two different AWS accounts. This turned out to be ever so slightly more than a simple drag-and-drop or copy operation.

Why? Well because it was web-enabled, mostly. Web-enabled S3 buckets have some restrictions on what they can be called, and buckets must be given a unique name. So you can’t just create a new webenabledbucket.com.au bucket in account B, copy the data across, flip the DNS and delete the bucket from account A. AWS won’t let you.

You have to have an intermediate stage where your files can live while the old bucket is deleted and the new one is provisioned. Which is where OctopusDeploy and the mighty Robot Army came to the rescue. Continue reading →