Monthly Archives: November 2016

The PowerShell Pipeline, explained

So, my previous post on PowerShell has prompted some responses, internally and externally. Sufficient that I did actually re-word some parts of it, and sufficient that I feel a need to be positive and offer something to take away the burn.

So let’s have a go at explaining the pipeline, shall we?

To do this, I’m going to give an example of doing something without the pipeline. I hope that by the end of this post, the value of showing the other way first will be clear. But I’ll say up front, if you have written code like I’m about to show, don’t fret. It still works. There’s just a better way.

The example I’ve chosen is this:

You’re deploying an IIS Web Application using PowerShell, and as part of your deployment process, you want to delete the previous web site(s) from IIS.

So, let’s dig in. I’m going to be quite thorough, and it’s fine to follow along in the PowerShell prompt. You will, of course, need IIS installed if you do, but don’t worry, at the end there’s an example or two that should work for everyone.

Continue reading →

You don’t know PowerShell

We’ve been doing a lot of interviewing at work of late. You see, we’re looking for good Windows DevOps candidates, and the local market is… well… let’s just say that next-gen Windows guys are a little thin on the ground around here.

This is a problem. Because while next-gen windows candidates are thin on the ground, resumés claiming next-gen Windows skills are emphatically not thin on the ground. In fact, I have actually – literally – lost count of the number of resumés I’ve seen where PowerShell is touted as a key skill, only for the candidate to fail some very simple PowerShell questions in the initial screening. So let’s just run through a few basic principles so that we don’t all waste each other’s time.

Continue reading →

HOWTO: Whitelist Pingdom Probe IPs into AWS Security groups

This is something I’ve been meaning to write about for a while.

If you use pingdom for your monitoring, and you have a requirement to lock down your endpoints to a specific set of clients, you may have a painful job on your hands.

Some engineers I’ve spoken to have implemented a kind of proxy to forward pingdom requests through to their locked-down endpoints. Others rely on User-Agent detection to allow Pingdom probes through while denying other traffic.

In my case, I’ve implemented a powershell script that runs at intervals, checking Pingdom’s published Probe IP List and syncing it to my target Security Group. here’s how it’s done.

The very first thing you’ll need to do, if you haven’t already, is contact AWS Support and get your rules-per-group limit increased. By default, you get 50 (at the time of writing), and that’s not enough for this.

Then the code.

First up, you need a list of the IPs you want to whitelist other than pingdom. Not much use only opening your endpoint to the monitoring service, is it?

$whitelist = @(
   "123.123.123.123/32",
   "124.124.124.124/32",
   "52.52.52.0/24"
)

And so on. You may want to store this differently, but for me it’s just straight in the script. For now.

When you have those, you need to grab Pingdom’s probe IPs from their API

$probeIPs = Invoke-RestMethod https://my.pingdom.com/probes/feed

Excellent. Now, the pingdom addresses aren’t in CIDR format, so you need to convert them to CIDR and add them to the $whitelist array you set up earlier. For that, you need a function that does pipeline input.

Function Join-IPs # pipeline function to append to an incoming array of strings
{
    param
    (
        [Parameter(ValueFromPipeline=$true)]
        [string]
        $In,
        [string]
        $JoinTo = "/32"
    )
    process
    {
        return ("" + $_ + "" + $jointo + "")
    }
}

And then you just stick that in your pipeline and get back an array of al the IP ranges that are meant to be in your security group.

$ranges = $whitelist += ($probeIps | select -expand ip | Join-Ips -JoinTo "/32" )

And there you have a list of all the CIDR ranges that are meant to be in your security group’s ingress rule.

My rule literally only opens one port – 443 – so if you have multiple ports, you may want to do this differently. It also does nothing to try and compress down multiple adjacent addresses into a single CIDR, so if you need that, you’re going to need to do a little extra work.

Now, we compare the sec group’s existing rules, and the array we just obtained, like so

$targetGroup = Get-EC2SecurityGroup -Region $region | `
               ? {$_.GroupName -eq "s-fictional-svc-WebElbGroup-H97BD3LE36JI"}


$currentranges = $targetgroup.IpPermissions |`
               ? {$_.FromPort -eq 443} | select -expand IpRanges
$groupID = $targetgroup.GroupId

$diff = Compare-Object $currentranges $ranges 

$diff | % {
    # If the side indicator is =>, we add it
    # if the side indicator is <=, we remove it
    if($_.SideIndicator -eq "=>")
    {
        Write-Host "Granting Ingress perms to" $_.InputObject 
        Grant-EC2SecurityGroupIngress -GroupId $groupID `
                        -IpPermission @{
                                 FromPort = 443; 
                                 IpProtocol = "tcp"; 
                                 IPRanges = $_.InputObject; 
                                 ToPort = 443
                         }
    }

    if($_.SideIndicator -eq "<=")
    {
        Write-Host "Revoking Ingress perms from" $_.InputObject 
        Revoke-EC2SecurityGroupEgress -GroupId $groupId `
                        -IpPermission @{
                                 FromPort = 443; 
                                 IpProtocol = "tcp"; 
                                 IPRanges = $_.InputObject; 
                                 ToPort = 443
                         }
    } 
}

As you can see, we use Compare-Object to determine what needs to be added and what needs to be removed, and push just that rule up – or rip it out of – to the Security Group.

This technique can be used to whitelist any service that publishes its IPs in an API – in fact, if you’re whitelisting a client, you could get your client to publish their IP list to you and literally just put a script like this in place. Why do this crap manually? Let a script do it for you.