Creating a SublimeText plugin to publish markdown to WordPress

I’ve been using SublimeText for a while now as a no-frills text editor; it has a nice Zen mode which hides all of the tabs and menus and pushes it to full screen which I’ve found perfect for taking notes during seminars:

SublimeText2 Zen mode

I have been trying to take notes using the markdown sytnax, but have found the process between taking the notes in markdown, converting them to HTML, and publishing the HTML to a blog a bit of a pain.

Given that SublimeText supports extension via plugin scripts written in Python, I’ve knocked together an extension which will

  1. convert the markdown to html
  2. push the html directly to wordpress

Before I get into the details of the script, maybe a little background would be useful:

What’s Markdown?

A lightweight markup language for writing human-readable content which can be compiled into HTML. Being able to write a lovely HTML blog post in a very basic text editor instead of the bloated Windows Live Writer is extremely refreshing.

For example, to make a line a <h1> header, prefix it with a #. For <h2>, use ##. For <h3> ###. Easy.

Want a link? Try surrounding it with square brackets and putting the url just after it in round brackets: [Something like this](which links here). It’s a really easy “language” and actually looks decent enough to just read on its own.

To include an image you just need to write ![alt text](image url)

A syntax reference page can be found here.

One problem I do have is the basic support for code highlighting within markdown is poor, obviously, since HTML only has a single <code> block. The Markdown generated HTML conflicts with my wordpress Syntaxhighlighter plugin, hence the ugly clode blocks in this post.

There is support for “fenced-code-blocks” wihtin many implementations of markdown, and with the library that I’m also using, but this is still not playing nicely with Syntaxhighlighter. I’m sure I’ll figure something out eventually – please bear with me until then.

I use the fantastic python-markdown2 library to convert markdown to HTML on the fly; I just had to copy the lib/markdown2.py file from the github repo into the sublimetext “plugins” directory – I created a Markdown subdirectory for it – and reference it from my script. i.e.,

post_content = str(markdown2.markdown(post_content,extras=["code-friendly"]))

What’s Python?

Python

A functional scripting language. I found it really quite tricky to work with and had to install PythonShell as a powershell-ish window for testing out python commands before putting them to work in the plugin script. What’s not great fun is that SublimeText2 installs a different version of Python to PythonShell to version 1 of SublimeText to any version you explicitly install yourself.

Some commands and syntax will work in one version of python and not others

One great example is setting up a proxy; the syntax for this is completely different between versions of python and whatever example code you’re reading might be using the syntax that your version of python doesn’t support.

Working with a functional language took a lot of getting used to; I’d want to return an object from a method but had to instead return a comma delimited list of base types. It’s all a little odd to me, and my messy code shows that!

What’s WordPress?

Wordpress

A popular free blog engine that’s used as a CMS for some small businesses. It has an XML RPC API which conforms to the MetaWeblog spec, and you can really dig into this by looking through the php files that make up a WordPress installation.

Since I’m using Amazon EC2 as my wordpress host I can use Kitty to SSH in and browse to /var/www/html/xmlrpc.php to see the metaweblog service api itself. That file references wp-includes/class-wp-xmlrpc-server.php which contains all of the underlying functionality for the api implementation. This is great reference material to make sure you’re passing the correct values of the correct types to the correct endpoints.

Details

The sublimemarkpress plugin is intended to:

  1. check the contents of a config file for your blog access details
  2. scan the contents of the active window to get the posts details, such as post ID (to make it an Update instead of a Create), tags, and status
  3. convert the text from Markdown to HTML
  4. Push the HTML to the metaweblog endpoint with the correct blog/post details

1. Getting the Blog setup details from a config file

The plugin relies on a settings file called “sublimemarkpress.sublime-settings” using the structure:

{
    "xmlrpcurl": <URL to xml rpc endpoint>,
    "username": <username>,
    "password": <password>
}

to read it:

s = sublime.load_settings("sublimemarkpress.sublime-settings")
mbURL = s.get("xmlrpcurl")
mbUsername = s.get("username")
mbPassword = s.get("password")

2. Get the text and strip out the blog post data

The plugin expects the top of your text file/active window to have optional tags to define blog post details:

<!-- 
#post_id:<id of existing post - optional>
#tags:<comma delimited list of post tags - optional>
#status:<draft or publish - optional>
-->

To get the entire contents of the active window:

all_lines_in_page = self.view.lines(sublime.Region(0, self.view.size()))

Then to extract the header details:

post_id, tags, status, has_header_content = self.GetHeaderContent(all_lines_in_page, header_lines)

where GetHeaderContent is the hack-y:

def GetHeaderContent(self, all_lines_in_page, header_lines):
    page_info = {"has_header_content":False,"post_id":None, "tags":"", "status":""}

    if self.view.substr(all_lines_in_page[0]).startswith("<!--"):
        page_info["has_header_content"] = True
        self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        # post_id
        if self.view.substr(all_lines_in_page[0]).startswith("#post_id"):
            page_info["post_id"] = self.view.substr(all_lines_in_page[0]).split(":")[1]
            self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        #post tags
        if self.view.substr(all_lines_in_page[0]).startswith("#tags"):
            page_info["tags"] = self.view.substr(all_lines_in_page[0]).split(":")[1]
            self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        #post status
        if self.view.substr(all_lines_in_page[0]).startswith("#status"):
            page_info["status"] = self.view.substr(all_lines_in_page[0]).split(":")[1]
            self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        self.MoveCurrentLineToHeader(header_lines, all_lines_in_page) # removes the closing comment tag
    return page_info["post_id"],page_info["tags"],page_info["status"],page_info["has_header_content"]

def MoveCurrentLineToHeader(self, header_lines, all_lines_in_page):
        header_lines.insert(len(header_lines),all_lines_in_page[0])
        all_lines_in_page.remove(all_lines_in_page[0])

3. Convert the rest from Markdown to HTML

As mentioned earlier, thanks to the great python-markdown2 library, this is simply a case of calling the “markdown” method, passing in the content to process:

post_content = str(markdown2.markdown(post_content,extras=["code-friendly"]))

I actually do a test to see if the markdown library can be imported first, then if it fails I don’t even try to convert from markdown to HTML:

can_markdown = False
try: 
    import markdown2 # markdown
    can_markdown = True
except ImportError:
    can_markdown = False

4. Post to metaweblog api

Build up the request using the blog details, post details, and HTML content:

content = self.BuildPostContent(self.view, {"content": post_content, "title": title, "tags": tags, "status": status})

def BuildPostContent(self, view, page_data):        
    return {"description": page_data["content"], "post_content": page_data["content"], "title": page_data["title"], "mt_keywords": page_data["tags"], "post_status": page_data["status"]}

Then submit it to the api:

proxy = xmlrpclib.ServerProxy(mbURL)

if post_id == None:
    post_id = proxy.metaWeblog.newPost(blog_id, mbUsername, mbPassword, content)
else:
    proxy.metaWeblog.editPost(post_id, mbUsername, mbPassword, content)

Extra bits and making it all work

To execute a command from a plugin within SublimeText, firstly you need to import the sublimetext library at the top:

import sublime, sublime_plugin # sublime

then name your class “something<command>” and have it take the “sublime_plugin.TextCommand” parameter:

class PublishCommand(sublime_plugin.TextCommand):

Then to run it you need to hit ctrl+’ to bring up the command window and type:

view.run_command("<name of the class minus the 'Command' suffix>")

e.g. for my plugin:

view.run_command("publish")

You can try this out with loads of the plugins included with a SublimeText install

That’s about it

All of this is in a small github repo. I’d appreciate it if you want to fork the repo and help show me how python should actually be structured. And written.

Known Issues

Categories

These need to be requested, looped through and matched against those associated with a post, added if those specified don’t exist, and the IDs associated with the post. Couldn’t be arsed.

As such, when you post to your blog you’ll see the entry is “uncategorised” and you’ll have to manually edit this.

Images

Since I’m just fooling around with a text editor, uploading images is still a little out of scope.

Random reference material

  • pyblog – for helping to work out the guts of how python interacts with the metaweblog api
  • I used Fiddler & Windows Live Writer for investigating traffic and finding out what the actual parameter being used to specify “tags” is within the metaweblog api: mt_keywords

CSS woes in IE8

Having trouble getting clever CSS to run in IE8? Trying to use webfonts/@font-face and only some of them are being applied? Make sure you don’t have css psuedo-selectors in there, else IE8 dies.

i.e., if you have a list of classes/css selectors having a style applied to them, such as:
[css].modal h2, h1:not(.title), .prod_message .figure h2 {
color:orange;
}[/css]
then the entire section is ignored in IE8 due to the
[css]h1:not(.title)[/css]
entry.

I’ve not been able to track this one down for MONTHS and suddenly I get a new brand of coffee and it all makes sense. Come back to us, IE8, you are once again welcomed.

Node.js @ UKWAUG: MS Cloud Day – Windows Azure Spring Release

The fourth session I attended was the highly energetic and speedy introduction to writing node.js and running it on Azure, presented by the author of Simple.Data and Simple.Web, and one of those voices of the developer community with a great JFDI attitude, Mark Rendle (@markrendle).

I’ve just recently got into node.js development myself and have been very much enjoying node, npm, express, stylus, and nib; there is a fantastic community and expanse of modules already and that can be a bit daunting.

During the session Mark’s short code example shows just how simple it can be to get up and running with node, and also how easy it is to deploy to Azure.

A nice comment was that we are on the road to “ecmascript harmony”! And that “Javascript is a great language so long as you ignore the 90% of it which coffeescript doens’t compile to.”

It was a very fast-paced session; hopefully my notes still make sense though..

What the various aspects of Azure do

  • compute – web, worker, vm
  • websites – .net, node, php
  • storage – blob, tables (distributed nosql, like cassandra), queues
  • sql – sql azure, reporting
  • services – servicebus, caching, acs

What are the Cloud Service types used for

  • web roles – iis, for apps
  • worker – no iis, for running anything

How to peruse the contents of blob or table

General tips for developing sites for use in Azure

  • keep static content in blob storage
  • websites commit and deploy much faster than cloud serviecs commit and deploy process
  • azure/iis needs server.js, not app.js

How to run RavenDB in Azure

  • Spin up a vm and install it!! (this used to be a much trickier process, but the recent Azure update meant that the VM support is mature enough to allow the simpler solution)

Developing node.js

Use jetbrains webstorm for debugging/ or the wonderful online editor, Cloud9IDE. Sublime Text 2 is a great editor for simple code requirements, and has great plugins for Javascript support. I also used this for taking all of the seminar notes as it has a simple “zen” zero-distractions interface

Next up – Hadoop and High Performance Computing

jquery cookies

For anyone else having fun working with cookies in jquery who decided to use jquery.cookie as their plugin of choice; if you create a cookie
[javascript]
$.cookie(‘ro’,’awesome’);
[/javascript]
And then try to change it
[javascript]
$.cookie(‘ro’,’lame’);
[/javascript]
you may actually end up with a cookie that contains BOTH values.

Instead, try jquery.cookies (plural!) which does seem to update correctly. Or just write your own plugin; cookie manipulation isn’t that tough!

Simple.Data “No ADO Provider Found”

Whilst trying to get Simple.Data working with a teeeeny tiny app for work, I couldn’t get past Database.OpenConnection(…) ; each time would result in a “No ADO Provider Found” exception, which had me baffled.

Google wasn’t helping much as it just threw up the symbolsource copy of Simple.Data’s MEFHelper.cs – i.e., the exception itself.

With a bit more digging I found this post on the Google Groups board which looked hopeful.

Eventually I found out that I needed to Nuget (if I can use that as a verb) Simple.Data into whichever projects referenced the class library that uses Simple.Data.

Odd. Seems a bit unnecessary, but it works now so I’m not complaining.

Scripting the setup of a developer PC, Part 4 of 4 – Installing Custom Stuff, Interesting Things Encountered, and Conclusion

This is the final part of a four part series on attempting to automate installation and setup of a development PC with a few scripts and some funky tools. If you haven’t already, why not read the introductory post about ninite, the second part about the command line version of WebPI or perhaps the third instalment about the interesting chocolatey project? Disclaimer: this series was inspired by a blog from Maarten Balliauw

Installing Custom Stuff

There are some other applications out there which I need to be able to install via script as well, such as SQL Server 2008 R2 Tools; although WebPI (and with Chocolately Beta) can install SQL Tools, you’re actually limited to the express versions (AFAIK), and I need the standard install.

Since I have the ISO for this on the network, I can run virtualclonedrive from commandline (after chocolatey installs it) to mount the iso and run the setup application using “vcdmount.exe /l=<drive letter> <iso path>”.

Execute VCDMount with no params to get this helpful dialog for other command line params:

image

So let’s get on with it then:

SQL Server 2008 R2 Tools

It looks like SQL Server has its own command line install options; if I mount the network ISO and pass the parameters “/ACTION=install /IACCEPTSQLSERVERLICENSETERMS /Q /FEATURES=Tools,ADV_SSMS” I should be able to install SQL tools unattended. There is a dependency on Windows Installer 4.5 being installed correctly for this one to work; make sure your WebPI install worked earlier!

 

Visual Studio 2010

It looks like VS2010 has its own command line install options; if I mount the network ISO and pass the parameters “/q /full /norestart” I should be able to install VS2010 unattended. There is an entry for “VS2010SP1Core” in the WebPI xml feeds, and I have tried using that to no avail; see “Interesting Things Encountered” section at the end for a note about WebPI & VS2010.

So the final install script should look like:

@echo off

REM VS2010
"c:\Program Files (x86)\Elaborate Bytes\VirtualCloneDrive\vcdmount.exe" /l=E "Z:\Installation\SetupDevPC\VS2010SP1.ISO" 

E:/Setup/setup.exe /q /full /norestart

REM SQL Tools
"c:\Program Files (x86)\Elaborate Bytes\VirtualCloneDrive\vcdmount.exe" /l=E "Z:\Installation\SetupDevPC\SQLServer2008Tools.ISO"

E:/setup.exe /ACTION=install /IACCEPTSQLSERVERLICENSETERMS /Q /FEATURES=Tools,ADV_SSMS

Something to bear in mind is that this doesn’t work if you haven’t restarted since running the chocolatey powershell script. As such, I’ve edited the chocolatey powershell script to end with:
[powershell]shutdown /r /t 0 /d P:0:0[/powershell]

If all goes well you shouldn’t actually see anything of note; VirtualCloneDrive’s VCDMount mounts each ISO into drive “E” (VCD default install has only one virtual drive defined, in my case that was “E”) and calls the relevant executable with parameters to attempt to force a silent install. VS2010 is completely silent! SQL at least gives a few lines of feedback.

The Bad News

Unfortunately VS2010’s setup.exe doesn’t wait before returning to the script; as such, you would see the call to VS2010’s setup.exe kick off then a few seconds later a call to SQL2008’s setup.exe, which fails since there’s another install already happening.

Again, just as unfortunately, SQL2008 won’t install straight after VS2010 – it demands a restart.

The “Meh” News

My preference is now to install SQL2008 first, since this is a blocking process, then VS2010, then let it restart (remove the “/norestart” flag for VS2010).

Hence the last script is actually:

@echo off

REM SQL Tools
"c:\Program Files (x86)\Elaborate Bytes\VirtualCloneDrive\vcdmount.exe" /l=E "Z:\Installation\SetupDevPC\SQLServer2008Tools.ISO"

E:/setup.exe /ACTION=install /IACCEPTSQLSERVERLICENSETERMS /Q /FEATURES=Tools,ADV_SSMS

REM VS2010
"c:\Program Files (x86)\Elaborate Bytes\VirtualCloneDrive\vcdmount.exe" /l=E "Z:\Installation\SetupDevPC\VS2010SP1.ISO" 

E:/Setup/setup.exe /q /full

Along with the previous powershell script and the beta chocolatey nupkg, the existing script for ninite and webpi and their components, the final directory contents now look like:

281211_autoinstall_iso_dir_contents

The End Result

The Result!

Which brings us FINALLY on to the:

Conclusion

Although it is entirely possible to script the setup of a develop PC without requiring ever seeing a GUI currently, using the tools I’ve chosen to use here, it seems that it can’t be done in a fully automated fashion. Certain products still popped up a confirmation dialog, others required a reboot when I’d specifically suppressed one. Some dependencies were not always resolved correctly.

As such, I hope that you have enjoyed this introduction into my attempt to teach myself some command line WebPI, ninite, chocolatey, and general hackery, and if you have any comments or suggestions please feel free to let me know in the comments or via twitter; I’ve kept various snapshots of my VM I used for this series, so I’ll happily try out any good suggestions!

It would appear that this is a nice set of basic scripts to get a development PC up and running, however once this has been done it makes much more sense to create an image and use that for future setups. There will be a follow up post about creating an image of this configured PC so that future developer PCs can use the image instead of having to reinstall everything, which should be a pretty basic one since it’s nothing new!

Finally, these articles are already out of date! WebPI is now on v4 and the chocolatey “beta” I mentioned is actually now the mainline. No doubt everything else will be out of date in a few more days.

Interesting Things Encountered

  • The webpicmdline tool still raises the odd dialog prompting for a restart (e.g. for MVC3), even with the “suppressreboot” option. Using a really loooong product list I’d specified in one webpi command fails for a lot of them. After rebooting, webpicmd didn’t automatically pick up and carry on as expected; this is why I’ve cut the initial webpi product list to a small number and done the others via chocolatey.

  • Webpicmdline doesn’t install things in the order you list them, which can be a bit odd. i.e., WindowsInstaller45 attempts to install after .Net 4 and promptly fails. Do it on its own and you’re fine.

  • Chocolatey’s webpi support didn’t initially work; I had to restart before I could install anything. I believe this to be related to the webpi installation of WindowsInstaller45 whose required reboot I had suppressed.

  • VS2010’s “/q /full” setup options are incredibly “q” – nothing appears at all; no command line feedback, no GUI. I had to fire off setup.exe without params just to see the GUI load and show me it’s already halfway through the install process! Fantastic.

  • VS2010 exists within the WebPI listing as “VS2010SP1Core” but seems to always fail with an error about needing “VS2010SP1Prerequisite”; this product also exists in the same WebPI feed but was always failing to install via webpicmdline and chocolatey for me. Let me know if you get this working!

The Resulting Scripts

Setup_Step1.cmd

Ninite & WebPI

@echo off

REM Ninite stuff
cmd /C "Z:\Installation\SetupDevPC\Ninite_DevPC_Utils.exe"

REM WebPI stuff
cmd /C "Z:\Installation\SetupDevPC\webpicmdline.exe /AcceptEula /SuppressReboot /Products:PowerShell,PowerShell2,NETFramework20SP2,NETFramework35,NETFramework4"

cmd /C "Z:\Installation\SetupDevPC\webpicmdline.exe /AcceptEula /SuppressReboot /Products:WindowsInstaller31,WindowsInstaller45"

shutdown /r /t 0 /d P:0:0

Setup_Step2.ps1

Chocolatey
[powershell]# Chocolatey
iex ((new-object net.webclient).DownloadString(‘http://bit.ly/psChocInstall’))

# install applications
cinst virtualclonedrive
cinst sysinternals
cinst msysgit
cinst fiddler
cinst tortoisesvn

# getting the latest build for webpi support: git clone git://github.com/chocolatey/chocolatey.git | cd chocolatey | build | cd _{tab}| cinst chocolatey -source %cd%
# I’ve already done this and the resulting nugetpkg is also saved in the same network directory:
cinst chocolatey –source "Z:\Installation\SetupDevPC\"

# Now I’ve got choc I may as well use it to install a bunch of other stuff from WebPI;
# things that didn’t always work when I put them in the looong list of comma delimited installs
# IIS
cinst IIS7 -source webpi
cinst ASPNET -source webpi
cinst BasicAuthentication -source webpi
cinst DefaultDocument -source webpi
cinst DigestAuthentication -source webpi
cinst DirectoryBrowse -source webpi
cinst HTTPErrors -source webpi
cinst HTTPLogging -source webpi
cinst HTTPRedirection -source webpi
cinst IIS7_ExtensionLessURLs -source webpi
cinst IISManagementConsole -source webpi
cinst IPSecurity -source webpi
cinst ISAPIExtensions -source webpi
cinst ISAPIFilters -source webpi
cinst LoggingTools -source webpi
cinst MetabaseAndIIS6Compatibility -source webpi
cinst NETExtensibility -source webpi
cinst RequestFiltering -source webpi
cinst RequestMonitor -source webpi
cinst StaticContent -source webpi
cinst StaticContentCompression -source webpi
cinst Tracing -source webpi
cinst WindowsAuthentication -source webpi

shutdown /r /t 0 /d P:0:0
[/powershell]

Setup_Step3.cmd

VCDMount

@echo off

REM SQL Tools
"c:\Program Files (x86)\Elaborate Bytes\VirtualCloneDrive\vcdmount.exe" /l=E "Z:\Installation\SetupDevPC\SQLServer2008Tools.ISO"

E:/setup.exe /ACTION=install /IACCEPTSQLSERVERLICENSETERMS /Q /FEATURES=Tools,ADV_SSMS

REM VS2010
"c:\Program Files (x86)\Elaborate Bytes\VirtualCloneDrive\vcdmount.exe" /l=E "Z:\Installation\SetupDevPC\VS2010SP1.ISO"

E:/Setup/setup.exe /q /full

Hope you enjoyed the articles, any feedback is appreciated.

Scripting the setup of a developer PC, Part 3 of 4 – Installing.. uh.. everything.. with Chocolatey.

This is part three of a four part series on attempting to automate installation and setup of a development PC with a few scripts and some funky tools. If you haven’t already, why not read the introductory post about ninite or even the second part about the command line version of WebPI? Disclaimer: this series was inspired by a blog from Maarten Balliauw.

Installing.. uh.. everything..: Chocolatey

Chocolatey is sort of “apt-get for windows” using powershell; it doesn’t quite achieve that yet, but the idea is the same; imagine nuget + apt-get. It works exactly like nuget but is meant to install applications instead of development components. The next release will support webpi from within chocolatey, but more on that in a moment.

There’s not much to look at yet, but that’s the point; you just type what you want and it’ll find and install it and any dependencies. I want to install virtualclonedrive, some sysinternals goodies, msysgit, fiddler, and tortoisesvn.

Before you start, make sure you’ve relaxed Powershell’s execution policy to allow remote scripts:
[powershell]Set-ExecutionPolicy Unrestricted[/powershell]

Ok, now we can get on with it. I can now execute a new powershell script to install choc and those apps:

[powershell]# Chocolatey
iex ((new-object net.webclient).DownloadString(‘http://bit.ly/psChocInstall’))

# install applications
cinst virtualclonedrive
cinst sysinternals
cinst msysgit
cinst fiddler
cinst tortoisesvn[/powershell]

This script will download (DownloadString) and execute (iex) the chocolatey install script from the bit.ly URL, which is just a powershell script living in github:
https://raw.github.com/chocolatey/chocolatey/master/chocolateyInstall/InstallChocolatey.ps1

This powershell script currently resolves the location of the chocolatey nuget package:
http://chocolatey.org/packages/chocolatey/DownloadPackage

Then, since a nupkg is basically a zip file, the chocolatey script unzips it to your temp dir and fires off chocolateyInstall.ps1; this registers all of the powershell modules that make up chocolatey. The chocolatey client is essentially a collection of clever powershell scripts that wrap nuget!

Once chocolatey is installed, the above script will fire off “cinst” – an alias for “chocolatey install” – to install each listed application.

What’s even more awesome is that the latest – not yet on the “master” branch – version of Chocolatey can install using webpi. To get this beta version, use the extremely terse and useful command from Mr Chocolatey himself, Rob Reynolds (@ferventcoder):

Adding in the install of this beta version allows me to use choc for a few more webpi components:

[powershell]# Chocolatey
iex ((new-object net.webclient).DownloadString(‘http://bit.ly/psChocInstall’))

# install applications
cinst virtualclonedrive
cinst sysinternals
cinst msysgit
cinst fiddler
cinst tortoisesvn

# getting the latest build for webpi support: git clone git://github.com/chocolatey/chocolatey.git | cd chocolatey | build | cd _{tab}| cinst chocolatey -source %cd%
# I’ve already done this and the resulting nugetpkg is also saved in the same network directory:
cinst chocolatey –source “Z:\Installation\SetupDevPC\”

# Now I’ve got choc I may as well use it to install a bunch of other stuff from WebPI;
# things that didn’t always work when I put them in the looong list of comma delimited installs
# IIS
cinst IIS7 -source webpi
cinst ASPNET -source webpi
cinst BasicAuthentication -source webpi
cinst DefaultDocument -source webpi
cinst DigestAuthentication -source webpi
cinst DirectoryBrowse -source webpi
cinst HTTPErrors -source webpi
cinst HTTPLogging -source webpi
cinst HTTPRedirection -source webpi
cinst IIS7_ExtensionLessURLs -source webpi
cinst IISManagementConsole -source webpi
cinst IPSecurity -source webpi
cinst ISAPIExtensions -source webpi
cinst ISAPIFilters -source webpi
cinst LoggingTools -source webpi
cinst MetabaseAndIIS6Compatibility -source webpi
cinst NETExtensibility -source webpi
cinst RequestFiltering -source webpi
cinst RequestMonitor -source webpi
cinst StaticContent -source webpi
cinst StaticContentCompression -source webpi
cinst Tracing -source webpi
cinst WindowsAuthentication -source webpi[/powershell]

Best bit about this? When you run the first command you’ll download and install the latest version of the specified executable. When this succeeds you’ll get:

[code] has finished successfully! The chocolatey gods have answered your request![/code]

Nice.

You’ll hopefully see your Powershell window update like this a few times:
choc_install (click to embiggen)

But depending on your OS version (I’m using Windows Server 2008 R2) you might see a few alerts about the unsigned drivers you’re installing:
choc_alert

That doesn’t seem to be avoidable, so just click to install and continue.

You might also find that your own attempts to install the beta version of chocolatey fail with errors like these:
choc_install_choc_fail1
or
choc_install_choc_fail2 (click to embiggen)

This is due to how you reference the directory in which your beta choc nuget package lives. If you reference it from a root dir (e.g. “Z:\”) then it’ll fail. Put it in a subdirectory and you’re golden:
choc_install_choc_success2
or using “%cd%” as the source dir (assuming you’re already in that dir):
choc_install_choc_success1

So, with my new powershell script and the beta chocolatey nupkg, along with the existing script for ninite, webpi and their components, my PC Setup directory now looks like this:

281211_autoinstall_choc_dir_contents

The last part of this series focuses on installing other things that either can’t be done or just didn’t work using one of the previous options, a list of “interesting things encountered”, and a conclusion to the whole project; see you back in:

Scripting the setup of a developer PC, Part 4 of 4 – Installing Custom Stuff, Interesting Things Encountered, and Conclusion.

Update: The chocolatey “beta” I mentioned is actually now in the mainline.

Scripting the setup of a developer PC, Part 2 of 4 – Installing Frameworks and Components with WebPI

This is part two of a four part series on attempting to automate installation and setup of a development PC with a few scripts and some funky tools. If you haven’t already, why not read the introductory post about ninite? Disclaimer: this series was inspired by a blog from Maarten Balliauw.

Installing Frameworks and Components: WebPI

Microsoft’s Web Platform Installer makes finding and installing MS applications and components quite easy by bringing the way to find and install them into one central portal (when it works properly, that is.. grumblegrumble..):

image

It also comes in a command line flavour so you can script your component installation, and that’s what I’ll be using.

The list of available packages can be found with the “/list:available” parameter and it’s pretty long.

You can specify custom XML feeds to define your own products and applications to be installed; something that might be interesting to spin up a new instance of your company website on a new server, if you set your web server to run webpi from commandline using a custom feed pointing to the latest stable build of your web application (that’ll be an interesting follow up post at some point I think)!

A summary of the parameters available to you can be found here.

Fire up a command line and run “WebpiCmdLine.exe /List:available” to get a list of available products and applications. It didn’t look exhaustive to me, so I checked the xml locations listed at the beginning of this command:

For the current version of the WebPI (at time of writing this is v3.0) these are:

https://www.microsoft.com/web/webpi/3.0/webproductlist.xml

https://www.microsoft.com/web/webpi/3.0/webapplicationlist.xml

https://www.microsoft.com/web/webpi/3.0/mediaproductlist.xml

https://www.microsoft.com/web/webpi/3.0/toolsproductlist.xml

https://www.microsoft.com/web/webpi/3.0/enterpriseproductlist.xml

 

Trawling through these xml files you can see that apparently you can install VS2010SP1…

292111_autoinstall_webpi_VS2010SP1Core

…but not SQL Tools (other than Express). I’ll get onto this lot shortly.

 

So for the meantime I’m now adding some more to my script to set up the .Net frameworks, Powershell, and Windows Installers:


REM Ninite stuff
cmd /C "Z:\Installation\SetupDevPC\Ninite_DevPC_Utils.exe"

REM WebPI stuff 
cmd /C "Z:\Installation\SetupDevPC\webpicmdline.exe /AcceptEula /SuppressReboot /Products:PowerShell,PowerShell2,NETFramework20SP2,NETFramework35,NETFramework4"

cmd /C "Z:\Installation\SetupDevPC\webpicmdline.exe /AcceptEula /SuppressReboot /Products:WindowsInstaller31,WindowsInstaller45"

Notice the multiple products are comma delimited but without spaces; it bombs if you put a space after the comma.

You can split each product install out into its own webpicmdline.exe call, but this ends up reloading the five xml feeds for each product, which can take an age.

However, I’ve had to split it onto a couple of lines as the install of Windows Installer 4 fails if it’s on the same line as .Net 4. It also requires you to reboot, so since you’ve suppressed this you’ll see some log messages looking like “install: SUCCESS” then “verification: FAILURE”:

[code]Started downloading: WindowsInstaller45
DownloadedWindowsInstaller45
Started installing Products…
Started installing: WindowsInstaller45
Install completed (SuccessRebootRequired): Windows Installer 4.5
Reboot is required for product WindowsInstaller45. Install SUCCESS[/code]

YEY!!

[code]Verifying successful installation…
Windows Installer 4.5 False
Install of Products: FAILURE
Verifying successful log generation…
Windows Installer 4.5 Logs : False
Creation of Product Logs: FAILURE[/code]

uh.. BOOO! So you do actually need to reboot to get this to complete. As such, I’ve added in the final line for this file:

@echo off

REM Ninite stuff 
cmd /C "Z:\Installation\SetupDevPC\Ninite_DevPC_Utils.exe"

REM WebPI stuff 
cmd /C "Z:\Installation\SetupDevPC\webpicmdline.exe /AcceptEula /SuppressReboot /Products:PowerShell,PowerShell2,NETFramework20SP2,NETFramework35,NETFramework4" 

cmd /C "Z:\Installation\SetupDevPC\webpicmdline.exe /AcceptEula /SuppressReboot /Products:WindowsInstaller31,WindowsInstaller45"

shutdown /r /t 0 /d P:0:0

The install directory has grown a little bit too:

281211_autoinstall_webpi_dir_contents

 

It’s getting a bit trickier now. There are issues with this, and I’ll sum it all up in part 4. However, next up is some sweet goodness:

Scripting the setup of a developer PC, Part 3 of 4 – Installing.. uh.. everything.. with Chocolatey.

Update: there is now a v4 of WebPI command line that requires “/install” as the first parameter and is called WebPICmd instead of webpicmdline. Since I’d downloaded v3, that’s the version I’m scripting for. Check out v4 though, as it has added support for “offline mode”; you can download the installers and dependencies for any number of products (to a network share perhaps) and install from there instead of the interwebs – I will do a follow up article doing exactly that 😉

Scripting the setup of a developer PC, Part 1 of 4 – Installing Applications & Utilities with Ninite

Setting up a development PC can be a bit of a pain, unless you’re smart and create an image following the setup of a brand new vanilla install. But who’s organised enough to do that?! I’ll get onto that option in another post, but this one is more an excuse to play with interesting stuff.

I thought I’d have a play with coding up a set of scripts to do as much of this setup as possible instead; there are a few tools out there to do this sort of thing, and I’ve gone with ninite, webpi, and chocolatey.

 

I’ll start with the intended ideal option for each tool, and then go into how this doesn’t work perfectly and why, and what the other options are. Part 1 of this series of 4 is for the easiest tool of all:

 

Installing Applications & Utilities: ninite

This site allows you to create a single exe installer which contains your own selection of a large number of applications/frameworks/utilities:

232111_autoinstall_ninite_web

For an ASP.Net developer PC I’ve gone with Chrome, Safari, Opera, Firefox, Skype, VLC, Flash, Air, Java, Silverlight, Launchy, 7-Zip, WinSCP, PuTTY, Notepad++, WinMerge, Paint.NET, PDFCreator, Reader, DropBox, and Everything Search for my installer. This installer can be called from the command line but the basic version still opens a graphical interface; however no interaction is required. The Pro version comes with a command line installer, but I’ll not be using that.

Ninite Pro is absolutely awesome: you can remotely manage installed software and software patches within your network with a silent install process.

232111_autoinstall_ninite_pro

 

So far my install script set looks like this; pretty bare:

[batch]@echo off
REM Ninite stuff
cmd /C Z:\Installation\SetupDevPC\Ninite_DevPC_Utils.exe[/batch]

And the installation directory is merely one script and one exe:

281211_autoinstall_ninite_dir_contents

That was dead simple! Lovely! Coming up next – something a bit messier:

Scripting the setup of a developer PC, Part 2 of 4 – Installing Frameworks and Components with WebPI.