Monthly Archives: January 2011

Data URI scheme

The Data URI Scheme is a method of including (potentially external) data in-line in a web page or resource.

For example, the usual method of referencing an image (which is almost always separate to the page you’ve loaded) would the one schemes of either html:

[html]<img src="/assets/images/core/flagsprite.png" alt="flags" />[/html]

or css:

[css]background:url(/assets/images/core/flagsprite.png)[/css]

However, this remote image (or other resource) can be base64 encoded and included directly into the html or css using the data uri schema:

[html]<img src="data:image/png;base64,
iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAABGdBTUEAALGP
C/xhBQAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9YGARc5KB0XV+IA
AAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAF1J REFUGNO9zL0NglAAxPEfdLTs4BZM4DIO4C7OwQg2JoQ9LE1exdlYvBBeZ7jq
ch9//q1uH4TLzw4d6+ErXMMcXuHWxId3KOETnnXXV6MJpcq2MLaI97CER3N0
vr4MkhoXe0rZigAAAABJRU5ErkJggg==">[/html]

or

[css]background:url(data:image/png;base64,
iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAQMAAAAlPW0iAAAABlBMVEUAAAD/
//+l2Z/dAAAAM0lEQVR4nGP4/5/h/1+G/58ZDrAz3D/McH8yw83NDDeNGe4U
g9C9zwz3gVLMDA/A6P9/AFGGFyjOXZtQAAAAAElFTkSuQmCC[/css]

So, if you fancy cutting down on the number of HTTP requests required to load a page whilst massively increasing the size of your css and html downloads, then why not look into the data uri scheme to actually include images in your css/htm files instead of referencing them?!

Sounds crazy, but it just might work.

Using the code below you can recursively traverse a directory for css files with “url(“ image references in them, download the images, encode them, and inject the encoded image back into the css file. The idea is that this little proof of concept will allow you to see the difference in http requests versus full page download size between referencing multiple external resources (normal) and referencing fewer, bigger resources (data uri).

Have a play, why don’t you:

[csharp highlight=”72,73,75″]using System;
using System.IO;
using System.Text.RegularExpressions;
using System.Net;

namespace Data_URI
{
class Data_URI
{
static void Main(string[] args)
{
try
{
var rootPath = @"D:\WebSite\";

// css file specific stuff
var cssExt = "*.css";
// RegEx "url(….)"
var cssPattern = @"url\(([a-zA-Z0-9_.\:/]*)\)";
// new structure to replace "url(…)" with
var cssReplacement = "url(data:{0};base64,{1})";

// recursively get all files matching the extension specified
foreach (var file in Directory.GetFiles(rootPath, cssExt, SearchOption.AllDirectories))
{
Console.WriteLine(file + " injecting");

// read the file
var contents = File.ReadAllText(file);

// get the new content (with injected images)
// match css referenced images: "url(/blah/blah.jpg);"
var newContents = GetAssetDataURI(contents, cssPattern, cssReplacement);

// overwrite file if it’s changed
if (newContents != contents)
{
File.WriteAllText(file, newContents);
Console.WriteLine(file + " injected");
}
else
{
Console.WriteLine(file + " no injecting required");
}
}

Console.WriteLine("** DONE **");
Console.ReadKey();
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.ReadKey();
}
}

static string GetAssetDataURI(string fileContents, string pattern, string replacement)
{
try
{
// pattern matching fun
return Regex.Replace(fileContents, pattern, new MatchEvaluator(delegate(Match match)
{
string assetUrl = match.Groups[1].ToString();

// check for relative paths
if (assetUrl.IndexOf("http://") < 0)
assetUrl = "http://mywebroot.example.com" + assetUrl;

// get the image, encode, build the new css content
var client = new WebClient();
var base64Asset = Convert.ToBase64String(client.DownloadData(assetUrl));
var contentType = client.ResponseHeaders["content-type"];

return String.Format(replacement, contentType, base64Asset);
}));
}
catch (Exception)
{
Console.WriteLine("Error"); //usually a 404 for a badly referenced image
return fileContents;
}
}
}
}[/csharp]

The key lines are highlighted: they download the referenced resource, convert it to a byte array, encode that as base64, and generate the new css.

This practise probably isn’t very useful for swapping out img refs  in HTML since you lose out on browser caching and static assets cached in CDNs. It may be more useful for images referenced in CSS files, since they’re static files themselves which can be minified, pushed to CDNs, and take advantage of browser caching.

Comments welcomed.

Quick and Dirty C# Recursive Find and Replace

Say you had a vast Visual Studio solution of something ridunculous like 120+ projects and wanted to test out a few proofs of concept on improving build times.

Now say that one of the proofs of concept was to use a shared bin folder for all projects in a single solution. Editing 120+ proj files is going to make you a little crazy.

How about a little recursive find-and-replace app using regular expressions (my saviour in many menial text manipulation tasks) to do it all for you? That’d be nice, wouldn’t it? That’s what I thought too. So I just did a quick and dirty console app to do just that.

[csharp]using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.IO;
using System.Collections.ObjectModel;

namespace RecursiveFindAndReplace
{
class Program
{
static void Main(string[] args)
{
// where to start your directory walk
var directoryToTraverse = @"C:\VisualStudio2010\Projects\TestSolutionWithLoadsOfProjectsInIt\";

// what files to open
var fileTypeToOpen = "*.csproj";

// what to look for
var patternToMatch = @"<OutputPath>bin\\[a-zA-Z]*\\</OutputPath>;";
var regExp = new Regex(patternToMatch);
// the new content
var patternToReplace = @"<OutputPath>;C:\bin\$(Configuration)\</OutputPath>";

// get all the files we want and loop through them
foreach (var file in GetFiles(directoryToTraverse, fileTypeToOpen))
{
// open, replace, overwrite
var contents = File.ReadAllText(file);
var newContent = regExp.Replace(contents, patternToReplace);
File.WriteAllText(file, newContent);
}
}

// recursive method to return the files we want in all sub dirs of the initial root
static List<string> GetFiles(string directoryPath, string extension)
{
var fileList = new List<string>();
foreach (var subDir in Directory.GetDirectories(directoryPath))
{
fileList.AddRange(GetFiles(subDir, extension));
}

fileList.AddRange(Directory.GetFiles(directoryPath, extension));

return fileList;
}

}
}[/csharp]

No doubt this could be made prettier with a little lambda, but like I said – quick and dirty.

—————–

Edit: I’ve just realised that Directory.GetFiles is inherently recursive. Duh. So the foreach instead becomes:

[csharp highlight=”4″]// get all the files we want and loop through them
foreach (var file in Directory.GetFiles(directoryToTraverse
,fileTypeToOpen
,SearchOption.AllDirectories))
{
// open, replace, overwrite
var contents = File.ReadAllText(file);
var newContent = regExp.Replace(contents, patternToReplace);
File.WriteAllText(file, newContent);
}[/csharp]

So that’s even quicker and slightly less dirty. Ah well.

Hackathon #1: Composite Image

My company recently had a Hackathon day, where the IT dept. was split into teams and had one day to develop something amazing to present to a panel of judges.

Over the next couple of posts I’d like to just show some things that my team hacked together, as I found them quite interesting, if not amazingly applicable to a real-world app!

First up is a very quick hack I created to generate a composite image on the fly given several other images. Apologies that this is in vb.net and not C#, but I was forced to work within our legacy environment! Assumption: all images to composite are of the same dimensions

Here is a function to take an arraylist of image URLs, merge them, and return the byte array:

[vb]Public Function BuildMergedImage(ByVal imgsToMerge As ArrayList) As Byte()

‘build image array, loading each image from a stream into the image array
Dim imagesToMerge As ArrayList = New ArrayList
Dim webClient As System.Net.WebClient = New System.Net.WebClient()
Dim imgStream As Stream

For Each imgToMerge As String In imgsToMerge
imgStream = webClient.OpenRead(imgToMerge.ToString())
imagesToMerge.Add(System.Drawing.Image.FromStream(imgStream))
Next

‘get composite dimensions
Dim height As Integer = CType(imagesToMerge(0), System.Drawing.Image).Height
Dim width As Integer = (CType(imagesToMerge(0), System.Drawing.Image).Width * imagesToMerge.Count())

‘init composite image
Dim mergedImage As Bitmap = New Bitmap(width, height)
Dim g As Graphics = Graphics.FromImage(mergedImage)

‘create composite image
For i As Integer = 0 To (imagesToMerge.Count() – 1)
g.DrawImage(imagesToMerge(i), (CType(imagesToMerge(i), System.Drawing.Image).Width * i), 0)
Next

‘save merged image to mem stream
Dim imgMemStream As New System.IO.MemoryStream
mergedImage.Save(imgMemStream, ImageFormat.Jpeg)

‘convert stream to byte array
Dim bytes(imgMemStream.Length() – 1) As Byte
imgMemStream.Position = 0
imgMemStream.Read(bytes, 0, bytes.Length)
imgMemStream.Close()

‘return byte array
Return bytes
End Function[/vb]

Then to use this method, build up a string arraylist of image urls, pass it to a new function, and output the returned byte array to the response buffer (I used mine in the Page Load, hence the obligatory postback check):

[vb]If Not IsPostBack Then
Response.Buffer = True
Response.ContentType = "image/jpeg" ‘match to the imageformat in the composite image function
Response.BinaryWrite(BuildMergedImage(taggedImg))
End If[/vb]

You’ll need these at the top of your file to get it working though. I realise the “option strict off” is a horrid hack, but it was for a hackathon..

[vb]Option Strict Off

Imports System.IO
Imports System.Drawing.Imaging[/vb]

Bit random, I know, but I found it quite interesting. Have a go with something similar to (but hopefully cleaner than) this:

[vb]Dim imageArrayList As ArrayList = new ArrayList

For Each imageUrl As String In Request.QueryString("i").Split(CChar("-"))
If Not IsNothing(imageUrl) AndAlso Not String.IsNullOrEmpty(imageUrl) Then
imageArrayList.Add(imageUrl)
End If
Next

If Not IsPostBack Then
Response.Buffer = True
Response.ContentType = "image/jpeg" ‘match to the imageformat in the composite image function
Response.BinaryWrite(BuildMergedImage(imageArrayList))
End If[/vb]

Example usage (although in this example it’s not pulling the correct dimensions through, which is odd):

[vb]http://localhost/combinedimages.aspx?i=http://www.docdatastorage.co.uk/Live/flyfiftythree/ProductImages/savage_white_1_medium.jpg-http://www.docdatastorage.co.uk/Live/flyfiftythree/ProductImages/checkit_green_1_medium.jpg-http://www.docdatastorage.co.uk/Live/flyfiftythree/ProductImages/telemaniacs_navy_1_medium.jpg[/vb]