Using PowerShell to extract and report on AWS CloudWatch Metrics

Here at Planet Domain we have a lot of day to day challenges in performance and cost management.

We built a fantastic CD pipeline that allows our developers to rapidly spin up new infrastructure, enabling them to quickly build, test and deploy new features with minimal assistance from the Ops team.

Unfortunately, it also allows our developers to rapidly spin up new infrastructure than then lays idle for the greater part of the day, sucking money out of budget that could otherwise be used on beer, pizza and Xbox accessories.

Now, Cloudwatch is great, but it does tend to involve a lot of clicky-clicky mouse-movey, clicky-clicky stuff in the AWS console. And once you’ve got a lot of points to monitor, the graphs become unreadable. And the filtering options are sometimes finnicky, and it’s not easy to automate things like CloudWatch Dashboards using CloudFormation.

So to get a report on average CPU utilization across our AWS Autoscaling Groups, I turned to PowerShell. Continue reading →

AWS Sydney Summit Roundup: Serverless is here

This past week saw AWS’s Sydney Summit at the historic Hordern Pavilion. While many saw the “coming soon” announcements as a slight disappointment (Lambda coming soon. API gateway coming soon. Everything else: coming soon), I – and am not the only one – saw it as a bit of a wakeup call.

In fact, this is really my only major takeaway from the summit:

Serverless architecture is coming. Get on board.

Lambda was launched to beta in late 2014 and to general availability in April 2015 to much fanfare, but as always with AWS, only in a subset of regions. Since then, it’s expanded out to encompass Virginia, Oregon, Ireland, Frankfurt and Tokyo.

It’s been much talked of in Australian Cloud circles – so much so that I freely admit to feeling a degree of Lambda Fatigue at meetups and conferences over the last year or so.

But the actual release of Lambda in ap-southeast-2 is imminent. And with it will come a fundamental shift in the way many of us in Australian Cloud circles need to think about Cloud infrastructure, because the potential advanatages in cost, development time and scalability are just too big to ignore. If you’re going all-in on cloud and your plans don’t include Lambda somewhere, then you’re probably doing something badly wrong. Continue reading →

NDC Sydney and other various things

I almost forgot to mention. I’ll be speaking at NDC Sydney in August 2016 on the topic of Windows Automation, Continuous Delivery, PowerShell, Robots and World Domination.

The speaker line-up is kind of stellar and this will, I guess, be the highest profile tech conference at which I’ve spoken thus far. I promise to rehearse deeply and not get distracted about trivia such as which idiom in which one should code.

I will be sharing the stage with Fabien Ruffin, who is also speaking at the Sydney AWS Summit on the topic of New Relic. I will be an wandering around that conference and asking people awkward questions while probably sporting a purple hair-dye job and generally making a nuisance of myself while people try to extract sensible answers about Windows workloads in the AWS Cloud.

I will also be around the Sydney DevOps and Cloud meetup scene a bit in the next few months, discussing Windows, Cloud and Automation stuff in-between moments of objectionable surliness and plugging my NDC session.

You have been warned.

A tedious and probably totally wrong post about idiomatic approaches to PowerShell

In PowerShell, there are many ways to do stuff.

This is a good thing. It’s what made Perl so attractive to me back when I wrote in unreadable languages. There was even an acronym. TIMTOWTDI. There Is More Than One Way To Do It.

This is good. It’s a great thing.

Which is what I thought of today, when browsing around the interwebs, I stumbled*, not for the first time, over a not-that-common but still sometimes-encountered PowerShell idiom for function declaration.

Continue reading →

Stricter DSC processing under WMF 5.0

If you’re all being good little Windows opscoders, you’ll be using DSC. And if, like me, you’ve upgraded to WMF 5.0 (PowerShell 5.0), you may have noticed a few new warning messages popping up in your logs

You do look at your logs, right?

Good.

Well, you may have been seeing this little warning of late

WARNING: The configuration ‘ExampleDSCConfig’ is loading one or more built-in resources without explicitly importing associated modules. Add Import-DscResource –ModuleName ‘PSDesiredStateConfiguration’ to your configuration to avoid this message.

I’ve certainly been seeing it. It hasn’t done any harm. Nothing breaks, it’s just a warning. I’ll get round to fixing it eventually.

Well, in the last day or so, I rolled WMF 5.0 into my production server fleet – a couple of hundred servers all up – and today I’ve had developers asking me why their Octopus Deploy projects are generating warnings now.  Continue reading →

.NET Deployment Strategies: The good, the bad, and the ugly

Damian Brady did this excellent talk at NDC London a while ago. It’s a bloody excellent talk in general, and it obliquely mentions some little classifieds company from Sydney who do some things with Continuous Deployment and Teh Cloud and stuff. I thoroughly recommend this. All of it.

.NET Deployment Strategies: the Good, the Bad, and the Ugly – Damian Brady from NDC Conferences on Vimeo.

Introducing xScheduledTaskDSC

So I was hunting round on the web the other day, trying to find a DSC resource which can create and manage Scheduled Tasks on Windows Server 2012. Unfortunately, my searching came up blank. There seemed to be no way of easily managing Scheduled Tasks and Jobs using Desired State Configuration.

So I wrote one.

ScheduledTaskDSC is a public project on GitHub. At the moment, it contains one resource, xScheduledTaskDSC, which can create simple Task Scheduler items, detect and correct configuration drift* and remove unwanted tasks.

At the moment, it’s just good enough for my use case (which is creating simple tasks on transient cloud instances via Octopus Deploy), but I want to expand it significantly to support more scenarios, and maybe add resources for Actions and Triggers, allowing for more complex task creation and management, as well as finishing off some Pester tests on it. But that’s for the future.

For now, feel free to take a look, have a play around with it, suggest or write improvements and send me pull requests. All PRs are considered. Have at it, kids.

 

* drift correction is limited at the time of writing. It detects a subset of drift, and will simply kill broken tasks and recreate them with correct parameters. I’m currently working on more subtle drift correction.

Note: 7 Feb 2016: Carbon includes a Scheduled Task DSC resource, however you need to pass in a chunk of XML. I have been a bit vocal about XML in the past, but Carbon may well do the job for you if you have an XML chunk from, say, schtasks.exe. 

Chickens not Cattle and definitely not Pets. Or maybe Bees

A few weeks ago I was ruminating on Twitter about the “Cattle not Pets” metaphor for Cloud instances.

I started a vague blog draft on the topic, got a little sidetracked and never fully completed the thought. But it’s something that’s itched at me for some time now. Cattle do in fact get fairly individualised treatment, except on the largest of scales. So they’re not a great metaphor. But I didn’t really have a perfect replacement So when I wandered into the office this morning and checked Twitter, I was gratified to see that Jeffrey Snover had tweeted out an article ruminating on basically the same topic, and which did a lot of the agricultural thinking for me.

Continue reading →

Stop Thinking About Servers

Every so often I get a request, from one or more of our developers, for Remote Desktop access to the servers running their code – be it for troubleshooting, configuration or some other arcane purpose.

My answer is almost uniformly “no”.

WAT

“But surely,” says the cat “you’re a super-futuristic DevOps shop spoken of in breathless terms by national IT publications? Don’t you trust your developers??”

Of course we do. But…

Continue reading →