Author Archives: Jason

Web-enabled S3 bucket migration using OctopusDeploy and PowerShell

I recently had to move a fairly large, fairly heavily trafficked web-enabled S3 bucket between two different AWS accounts. This turned out to be ever so slightly more than a simple drag-and-drop or copy operation.

Why? Well because it was web-enabled, mostly. Web-enabled S3 buckets have some restrictions on what they can be called, and buckets must be given a unique name. So you can’t just create a new webenabledbucket.com.au bucket in account B, copy the data across, flip the DNS and delete the bucket from account A. AWS won’t let you.

You have to have an intermediate stage where your files can live while the old bucket is deleted and the new one is provisioned. Which is where OctopusDeploy and the mighty Robot Army came to the rescue. Continue reading →

PowerShell quickie: Setting IIS App Pool IdleTimeout

I just did this and was frankly awash in a sea of misinformation, so here’s how it’s done.

$ValueToSet = 40
$AppPoolPrefix = "*"


$appPools = gci IIS:\AppPools | ? {$_.Name -like $AppPoolPrefix}

$appPools | % {
    $currentTimeout = $_.processModel.IdleTimeOut
    if($ValueToSet -ne $CurrentTimeout.Minutes)
    {
        $_.processModel.IdleTimeout = [TimeSpan]::FromMinutes($ValueToSet)
        $_ | Set-Item 
        Write-Host "Set" $_.Name "to" $_.processModel.IdleTimeout 
    }

}

What are the gotchas here?

Well one, just calling Set-ItemProperty does nothing but apply your change to an in-memory copy of the object. Not the object itself. Forget that solution

Second, you need to pass in a timespan object, not merely a straight number, or you’ll get unexpected results when it tries to set the timeout in ticks

Third: Do not forget the Set-Item call, or you’re utterly lost.

And that’s about it.

[update: I forgot to say, include:

Import-Module WebAdministration

At the start. I don’t have to, because my servers ipmo that module in their PowerShell profiles. But you’ll need to.]

 

Snippet: Extending Connect-MSTSC for fun and profit

If you’ve ever come across Jaap Brasser’s Connect-MSTSC script, you’ll know it’s a pretty cool script. If you have a cloud service such as AWS, and you haven’t leveraged this script in conjunction with Get-EC2PasswordData, then frankly you’re doing something wrong.

I’ve had several variations around this script in our AWS utility script for a while now. There’s Connect-EC2Instance, Connect-RobotArmyv2Group and a few other variations. And there’s the one I put together today, which demonstrates a couple of things, so I thought I’d share

  1. Using Filters to find EC2 instances with particular extended properties
  2. Leveraging open-source code to make your life exponentially easier.

First of all, filters. If you do a Get-Help on Get-EC2Instance, you’ll see the filter property prominently advertised, but what you won’t immediately see is how to use it. What -Filter expects to be handed is an object array of type Amazon.EC2.Model.Filter.

So let’s have a look at what that is

New-Object -typeName Amazon.EC2.Model.Filter | Get-Member 

   TypeName: Amazon.EC2.Model.Filter

Name        MemberType    Definition
----        ----------    ----------
Value       AliasProperty Value = Values
Equals      Method        bool Equals(System.Object obj)
GetHashCode Method        int GetHashCode()
GetType     Method        type GetType()
ToString    Method        string ToString()
Name        Property      string Name {get;set;}
Values      Property      System.Collections.Generic.List[string] Values {get;set;}

OK, so it expects a Name/Value or Name/Values structure. That’s pretty easy to create in an ad-hoc fashion, like so

@{ Name = "private-ip-address"; Values = "10.123.26.144" }

So, we can use this to find an instance with a given IP address, or a given tag, or a given keypair. There’s a whole list of filter properties in the detailed Get-Help output for Get-EC2Instance. So let’s put this together with Get-EC2PasswordData to give us a simple script that will find the Admin password and connect to an instance immediately

Function Connect-ByIp
{
param
(
  $ip
)

  $instance = Get-EC2Instance -filter @{ Name = "private-ip-address"; Values = $ip }
  $instanceid = $instance.RunningInstance.InstanceId
  $password = Get-EC2PasswordData -InstanceId $instanceid -Decrypt -PemFile \\tsclient\c\pemfiles\keypair.pem 
  Connect-Mstsc -ComputerName $ip -User Administrator -Password $password
}

And there you have it. A valuable script now has added value as an AWS tool. Win.

The case of the non-updateable DSC Resource

I’ve been working on something pretty cool here at Domain Tech. Well, I think it’s pretty cool. It’s a Powershell DSC wrapper around our CloudFormation-based server fleet, The Robot Army.

The idea behind doing it this way was to significantly streamline the config management, which in Robot Army v1 was a bit clunky and hard to manage, as well as being based on multiple templates, with multiple parameters specified in multiple places, with a concomitant risk of variance creeping in over time.

Anyway, there’s a full blog post in draft over at Domain Tech explaining Robot Army v2, and that’s not what I want to talk about today. What I want to talk about today is DSC Resource Schemas that don’t update. Continue reading →

The -AcceptEula Gotcha

Maybe we’re an edge case, but our ‘legacy’ Deployments through OctopusDeploy are based on an older batch process, which uses Junction.exe from Sysinternals to refer IIS from a ‘current’ folder to an “application.version.x.x.xx” folder. It’s a pretty common, if a little old-school technique. The original was a batch file, which was replaced by some powershell, which was enhanced a lot end then ended up ported into Octopus. It now deploys the new code, removes the box from the load balancer, switches to the new code using Junction.exe, resets IIS, warms up the new code and finally adds the box back to the load balancer. It’s pretty seamless, if a bit convoluted.

Well recently we had a little change pushed through which rolled back a few system settings on the web servers. And when it came to deploy with Octopus, we saw multiple catastrophic failures. Continue reading →

The Bloatware of the Windows DevOps World

If there’s a piece of software that grows fast and just keeps getting slower, it’s my PowerShell profile. It’s so useful to just cram another module load in there, or add another check script, or Update-Help, or a snippet of code to give a thought of the day. Mine is getting up above the ten second load time mark, and is rapidly approaching fifteen seconds.

Why? Well, I do a lot. I initialize my AWS credentials. I load my warmup scripts and F5 Load Balancer stuff. I make sure PSCX and xWebAdministration are available, and that I’m running at least PowerShell 4.0 (did I mention I ship my profile around from box to box? Yep). I also create a few aliases and read myself a line from my favourite song. I even spin up an asynchronous job to Update-Help, and I’m considering adding a few extra lines of code so I only do updates like that once a day. And code never gets removed, no way. What if I need it?

So it’s getting slow.

Well, I tweeted about this, and a mate tweeted back “you should implement a progress bar, so it at least seems like it’s loading quicker”.

Bloody good idea.

Luckily for me, Powershell supports progress bars natively. If you’ve done an Update-Help, you’ll have seen it. You’ll have also seen it if you’ve fired off an Invoke-WebRequest with an -outfile parameter pointed at a large-ish file. And what’s more, they’re pretty easy to use.

Here’s the basics

Write-Progress -Activity "doing stuff" -percentcomplete 25 -Status "doing a subtask of the stuff activity"

Your activity will appear in the First line. Your Status will appear in the second line. The pecentcomplete, obviously enough, defines how far along the progress bar is filled.

This is so absurdly easy, I’m surprised how few people use it. People still spit output to Write-Host (I certainly do), or at best Write-Verbose. Well, this doesn’t stop you doing that, but it’s a simple, neat way of showing progress. It’s good UX to let people know how much work has been done and how much remains and also it’s cool. Cool things are cool. You should totally use Write-Progress.

Snippet: Getting stuff installed

I sent this little snippet around my team after my aforementioned PowerShell Ninjas training session. It’s an easy way to get a bit of software painlessly installed onto someone’s machine. I just said “Hey, if you need the Powershell commuity extensions, just run this in your powershell prompt”

iex((iwr http://devops.fixt.co/posh/install-pscx.txt).Content)

And now my team have PSCX installed, via this little snippet. There are of course some considerations to be made here. It’s not going to work without having unrestricted scripting permissions, and it might not be great to get your users into the habit of running random code off the internet, but hey, Chocolatey do it, so why can’t you?

This is actually a great technique for power scripters to make your modules and profiles portable. Get them on a server you trust, run a one-liner, hey presto you have a fully configured environment, ready to use. Neat.

In Just One Hour I will make you a PowerShell Ninja

I just delivered the first session of a series here at work, running our C# and mobile devs through the basics of PowerShell, which is a pivotal component of our DevOps strategy. Here’s the slide deck, via SlideShare.

The real meat of the content is outside of the slides themselves, in the presenter notes and in the live code we type as we go through, but the slides will give a feel for what the session is like.