Tag Archives: PowerShell

The PowerShell automation language

CloudFormation’s Update-CfnStack and “No Updates Are To Be Performed”

If you use Cloudformation with Powershell – and I do – you’ve probably run into this error message more than once.

If you’re like me, you’ll have tried to suppress it in several ways. You’ve tried to check your own template for changes, avoiding the call if there are none. And you’ve probably fallen foul of tiny, insignificant differences that CloudFormation doesn’t care about. You’ve probably done the same with your parameters and hit similar problems.

You might even have taken to adding a timestamp to a harmless field of your template so that at least something has updated. At an unnamed previous company, we’d just add a comment to the end of a launchconfiguration’s powershell script, which was enough to force an update and suppress the error. You might even be fine with just ignoring the rain of red text.

In my current project, we can’t just ignore it, as we’re driving our templates out of Octopus Deploy. “No updates are to be performed” throws and aborts our pipeline, which is annoying. But we want to catch other, genuine error messages from Update-CfnStack.

So what to do?

Well, turns out this is an object lesson in correct use of try/catch.

Continue reading →

The Birthday Paradox and testing random functions

I had cause, recently, to write a little function that randomises a name for a resource from a list of roughly 90 words and a 3-digit postfix, in the form ‘happycat-267’*. Now, I don’t want to re-use a name that’s gone before, so the function checks already-deployed resources and if the name is taken, the function recurses to pick a new name from my ~90,000 possiblities.

Of course, when it came to testing this function, I wanted to be reasonably sure that the recursion worked and would continue to work in the future, so I decided to write a simple Pester test that loops a certain number of times and doesn’t return a duplicate value.

But this raised a question. How many iterations should this test go through to be reasonably sure of hitting a duplicate and thus triggering a recursion?

Obviously to fully test, I’d need to run the function 90,000+ times. But that’s computationally expensive and would slow down my testing. I don’t want to do that. But how many is the least number of tests I can do to be reasonably sure of a duplicate appearing?

Which brings me to the Birthday Problem, aka the Birthday Paradox.

Stated simply, the Birthday Problem tells us that in a group of only 23 people, it’s more likely than not to find two people who share a birthday. And if you have 70 people in a room, the probability of at least one birthday match is up at 99.9%.

The Wikipedia article goes into a lot of detail on why this is mathematically true, but the astute among us whill have noticed that this mathematical phenomenon has an application in unit testing my little “random-name-without-collisions” function.

There are Birthday Problem calculators online, such as this one, so I plugged in my possibility space of 90,000 and started playing with iterations. It turns out that if I call my function 1000 times, there’s a >99.5% chance that I’ll produce (and therefore handle) a duplicate.

At 500 iterations, the probability drops to around 75%, and at 300 iterations, it’s around 40% – so clearly 300 iterations is too low, that is it’s more likely that I won’t hit a duplicate.

So, I wrote my own function to calculate the probabilities for me

I can call this in a loop, and calculate a table of probabilities from 1 iteration to 10,000 (and graph that, if I feel like it).

(1..10000) | % {
    $probability = Get-CollisionChance 90000 $_ 
    [pscustomobject]@{Iterations = $_ ; Probability = $probability}
}

I can then use that table to zoom in on a probability I feel comfortable with. Let’s say I’m happy with a 99% probability of collision, it turns out that 911 iterations will get me there. If I’m happy with 90%, 644 iterations will do it.

Above 911 iterations, the curve plateaus out and the returns from adding more iterations become smaller. We hit a point where PowerShell rounds the probability up to 1 at 2519 iterations. It’s not mathematically certain at this point, but we’re up around a 99.9999999999999% chance of collision.

So we can see there’s really not much point iterating above 2500 repetitions. The increased probability just isn’t worth the extra processor cycles.

So anyway, with a little mocking, I can write a useful test that has a 99% chance of hitting at least one duplicate in the function, and test that it doesn’t actualy return any dupes, thus:

Describe "Getting a random name" {
    # Mock this so we don't exhaust API calls by repeatedly calling for a list
    $currentresources = Get-AlreadyDeployedResources
    Mock Get-AlreadyDeployedResources { return $currentresources }

    It "Never returns duplicates (as far as we can tell)" {   
        $reslist = @() 
        (1..911) | % { 
            # get a random resource name
            $reslist += Get-RandomResourceName | tee-object -Variable plusone 
            # add that back to the list we mocked
            $currentresources += [pscustomobject]@{ ResourceName = $plusone; ResourceID = 'fakeidentifier' }
        }
        # count how many names we have
        $all = ($reslist | measure-object | select-object -expand Count)
        #check how many unique names are in that list
        $unique = ($reslist | select-object -unique | measure-object | select-object -expand Count)

        # they should match
        $all | Should Be $unique
    }
}

Anyway, this is what I spent yesterday afternoon researching and playing around with. Hopefully someone other than me will find it useful. If not, well I had fun.

 

* not one of the actual values.

 

The PowerShell Pipeline, explained

So, my previous post on PowerShell has prompted some responses, internally and externally. Sufficient that I did actually re-word some parts of it, and sufficient that I feel a need to be positive and offer something to take away the burn.

So let’s have a go at explaining the pipeline, shall we?

To do this, I’m going to give an example of doing something without the pipeline. I hope that by the end of this post, the value of showing the other way first will be clear. But I’ll say up front, if you have written code like I’m about to show, don’t fret. It still works. There’s just a better way.

The example I’ve chosen is this:

You’re deploying an IIS Web Application using PowerShell, and as part of your deployment process, you want to delete the previous web site(s) from IIS.

So, let’s dig in. I’m going to be quite thorough, and it’s fine to follow along in the PowerShell prompt. You will, of course, need IIS installed if you do, but don’t worry, at the end there’s an example or two that should work for everyone.

Continue reading →

You don’t know PowerShell

We’ve been doing a lot of interviewing at work of late. You see, we’re looking for good Windows DevOps candidates, and the local market is… well… let’s just say that next-gen Windows guys are a little thin on the ground around here.

This is a problem. Because while next-gen windows candidates are thin on the ground, resumés claiming next-gen Windows skills are emphatically not thin on the ground. In fact, I have actually – literally – lost count of the number of resumés I’ve seen where PowerShell is touted as a key skill, only for the candidate to fail some very simple PowerShell questions in the initial screening. So let’s just run through a few basic principles so that we don’t all waste each other’s time.

Continue reading →

HOWTO: Whitelist Pingdom Probe IPs into AWS Security groups

This is something I’ve been meaning to write about for a while.

If you use pingdom for your monitoring, and you have a requirement to lock down your endpoints to a specific set of clients, you may have a painful job on your hands.

Some engineers I’ve spoken to have implemented a kind of proxy to forward pingdom requests through to their locked-down endpoints. Others rely on User-Agent detection to allow Pingdom probes through while denying other traffic.

In my case, I’ve implemented a powershell script that runs at intervals, checking Pingdom’s published Probe IP List and syncing it to my target Security Group. here’s how it’s done.

The very first thing you’ll need to do, if you haven’t already, is contact AWS Support and get your rules-per-group limit increased. By default, you get 50 (at the time of writing), and that’s not enough for this.

Then the code.

First up, you need a list of the IPs you want to whitelist other than pingdom. Not much use only opening your endpoint to the monitoring service, is it?

$whitelist = @(
   "123.123.123.123/32",
   "124.124.124.124/32",
   "52.52.52.0/24"
)

And so on. You may want to store this differently, but for me it’s just straight in the script. For now.

When you have those, you need to grab Pingdom’s probe IPs from their API

$probeIPs = Invoke-RestMethod https://my.pingdom.com/probes/feed

Excellent. Now, the pingdom addresses aren’t in CIDR format, so you need to convert them to CIDR and add them to the $whitelist array you set up earlier. For that, you need a function that does pipeline input.

Function Join-IPs # pipeline function to append to an incoming array of strings
{
    param
    (
        [Parameter(ValueFromPipeline=$true)]
        [string]
        $In,
        [string]
        $JoinTo = "/32"
    )
    process
    {
        return ("" + $_ + "" + $jointo + "")
    }
}

And then you just stick that in your pipeline and get back an array of al the IP ranges that are meant to be in your security group.

$ranges = $whitelist += ($probeIps | select -expand ip | Join-Ips -JoinTo "/32" )

And there you have a list of all the CIDR ranges that are meant to be in your security group’s ingress rule.

My rule literally only opens one port – 443 – so if you have multiple ports, you may want to do this differently. It also does nothing to try and compress down multiple adjacent addresses into a single CIDR, so if you need that, you’re going to need to do a little extra work.

Now, we compare the sec group’s existing rules, and the array we just obtained, like so

$targetGroup = Get-EC2SecurityGroup -Region $region | `
               ? {$_.GroupName -eq "s-fictional-svc-WebElbGroup-H97BD3LE36JI"}


$currentranges = $targetgroup.IpPermissions |`
               ? {$_.FromPort -eq 443} | select -expand IpRanges
$groupID = $targetgroup.GroupId

$diff = Compare-Object $currentranges $ranges 

$diff | % {
    # If the side indicator is =>, we add it
    # if the side indicator is <=, we remove it
    if($_.SideIndicator -eq "=>")
    {
        Write-Host "Granting Ingress perms to" $_.InputObject 
        Grant-EC2SecurityGroupIngress -GroupId $groupID `
                        -IpPermission @{
                                 FromPort = 443; 
                                 IpProtocol = "tcp"; 
                                 IPRanges = $_.InputObject; 
                                 ToPort = 443
                         }
    }

    if($_.SideIndicator -eq "<=")
    {
        Write-Host "Revoking Ingress perms from" $_.InputObject 
        Revoke-EC2SecurityGroupEgress -GroupId $groupId `
                        -IpPermission @{
                                 FromPort = 443; 
                                 IpProtocol = "tcp"; 
                                 IPRanges = $_.InputObject; 
                                 ToPort = 443
                         }
    } 
}

As you can see, we use Compare-Object to determine what needs to be added and what needs to be removed, and push just that rule up – or rip it out of – to the Security Group.

This technique can be used to whitelist any service that publishes its IPs in an API – in fact, if you’re whitelisting a client, you could get your client to publish their IP list to you and literally just put a script like this in place. Why do this crap manually? Let a script do it for you.

Extending Pester for fun and profit

Of late, I’ve been working on a little side project to test a rather complex Akamai Property. We wanted to be confident, after making changes, that the important bits were still working as we expected them to, and for some reason there was no easy, automated solution to test this.

Obviously I decided I’d write my testing project in Pester, and so it was that I began writing a whole bunch of tests to see what URLs returned what status code, which ones redirected, which ones were cache hits and cache misses and what headers were coming back.

First up, I wrote a generic function called “Invoke-AkamaiRequest”. This function would know whether we were testing against Staging or production, and would catch and correct PowerShell’s error behaviour – which I found undesirable – and allow us to send optional Akamai pragma headers (I’ll share this function in a later post).

With that up and running, I could start writing my tests. Here’s a set of simple examples

Describe "An example test, to establish things" {
    Context "Hit up the homepage." {
        It "Should return 200" {
            (Invoke-AkamaiRequest -uristem /).StatusCode | Should Be 200
        }
    }

    Context "Hit up a non-existent page" {
        It "Should return 404" {
            (Invoke-AkamaiRequest -uristem /nonexistent.html).StatusCode | Should be 404
        }
    }

    Context "A redirect works" {
        It "Should gimme 301" {
            (Invoke-AkamaiRequest -uristem /redirectedfolder/nonexist).StatusCode | Should Be 301
        }
    }
}

Now, that last one, testing a 301, is interesting. Not only do you need to test that a 301 or 302 status code is coming back, you also need to test where the redirect is sending you. So I started to write tests like this

It "Should redirect /blog/ to /advice/" {
    $blog = Invoke-AkamaiRequest -path /blog/
    ($blog | select -expand statuscode | Should Be 301) -and 
    ($blog.headers.location | Should Be http://$tld/advice/)
}

And this worked fine. But it was a bit clunky. If only Pester had a RedirectTo assertion I could just throw in there, like so

It "Should redirect /blog/ to /advice/ " {
    Invoke-AkamaiRequest -path /blog/ | Should RedirectTo http://$tld/advice/ 
}

If. Only.

Oh, but it can!

Yes, you can write custom assertions for Pester. They’re pretty easy to do, too. What you need is a trio of functions describing the logic of the test, and what to return if it fails in some way. They are named PesterAssertion, PesterAssertionFailureMessage and NotPesterAssertionFailureMessage, where Assertion is the assertion name, in my case “RedirectTo”

For my particular case, the logic was to take in an HTTP response object, and check that the status was 301 (or 302), and match the Location: header to a specified value. Pretty simple really. Here’s the basic code:

function PesterRedirectTo($value, $expected)
{
    return [bool](($value.statuscode -eq 301 -or $value.statuscode -eq 302) -and 
                    $value.headers.location -eq $expected)
}

function PesterRedirectToFailureMessage($value,$expected)
{
    return "Expected to redirect to {$expected}"
}

function NotPesterRedirectToFailureMessage($value,$expected)
{
    return "Expected not to redirect to {$expected}"
}

I put these into my supporting module (not into the Pester module) and ran my tests. Happy happy days, it worked perfectly. Throwing different URLs at it resulted in exactly the behaviour I wanted.

All that remained was to make the failure messages a little smarter and make the Not assertion more useful, but I figured before I did that I should write this little post with the nice clean code before the extra logic goes in and makes everything unreadable.

You can probably think of several ways you could streamline your tests with assertions right now. I’ve also written RedirectPermanently and ReturnStatus assertions, and I’m looking into HaveHeaders and BeCompressed. I may even release these as an add-on module at some point soon.

You can probably think of things that should go right back into the Pester codebase, too. And there are a number of other ways you can customise and extend pester to fit your own use cases.

To summarise: Pester is not just a flexible and powerful BDD framework for PowerShell. It’s also easily extensible, adding even more power to your PowerShell toolbox.

Now get out there and test stuff.

A tedious and probably totally wrong post about idiomatic approaches to PowerShell

In PowerShell, there are many ways to do stuff.

This is a good thing. It’s what made Perl so attractive to me back when I wrote in unreadable languages. There was even an acronym. TIMTOWTDI. There Is More Than One Way To Do It.

This is good. It’s a great thing.

Which is what I thought of today, when browsing around the interwebs, I stumbled*, not for the first time, over a not-that-common but still sometimes-encountered PowerShell idiom for function declaration.

$func = {
    param($input)
    write "I am a function. Your input was $input"
}

&$func

Continue reading →

Stricter DSC processing under WMF 5.0

If you’re all being good little Windows opscoders, you’ll be using DSC. And if, like me, you’ve upgraded to WMF 5.0 (PowerShell 5.0), you may have noticed a few new warning messages popping up in your logs

You do look at your logs, right?

Good.

Well, you may have been seeing this little warning of late

WARNING: The configuration ‘ExampleDSCConfig’ is loading one or more built-in resources without explicitly importing associated modules. Add Import-DscResource –ModuleName ‘PSDesiredStateConfiguration’ to your configuration to avoid this message.

I’ve certainly been seeing it. It hasn’t done any harm. Nothing breaks, it’s just a warning. I’ll get round to fixing it eventually.

Well, in the last day or so, I rolled WMF 5.0 into my production server fleet – a couple of hundred servers all up – and today I’ve had developers asking me why their Octopus Deploy projects are generating warnings now.  Continue reading →

Introducing xScheduledTaskDSC

So I was hunting round on the web the other day, trying to find a DSC resource which can create and manage Scheduled Tasks on Windows Server 2012. Unfortunately, my searching came up blank. There seemed to be no way of easily managing Scheduled Tasks and Jobs using Desired State Configuration.

So I wrote one.

ScheduledTaskDSC is a public project on GitHub. At the moment, it contains one resource, xScheduledTaskDSC, which can create simple Task Scheduler items, detect and correct configuration drift* and remove unwanted tasks.

At the moment, it’s just good enough for my use case (which is creating simple tasks on transient cloud instances via Octopus Deploy), but I want to expand it significantly to support more scenarios, and maybe add resources for Actions and Triggers, allowing for more complex task creation and management, as well as finishing off some Pester tests on it. But that’s for the future.

For now, feel free to take a look, have a play around with it, suggest or write improvements and send me pull requests. All PRs are considered. Have at it, kids.

 

* drift correction is limited at the time of writing. It detects a subset of drift, and will simply kill broken tasks and recreate them with correct parameters. I’m currently working on more subtle drift correction.

Note: 7 Feb 2016: Carbon includes a Scheduled Task DSC resource, however you need to pass in a chunk of XML. I have been a bit vocal about XML in the past, but Carbon may well do the job for you if you have an XML chunk from, say, schtasks.exe.