EXIF Quick Reference

Over the past few years, I’ve used the EXIF data stored by modern digital cameras in a couple of programs I’ve worked on. Having access to the date the photo was taken – or which way the camera was rotated when it was – is useful information to have.  Even though it’s relatively easy to do, finding the right identifier for the information you want is much harder than it should be.  This page fills that niche – the most useful/popular EXIF properties in an easy-to-find format.

Read More »

Posted in Miscellaneous | Tagged , , | Leave a comment

Better Printed Photos: Naming with Date Taken

Whenever I’ve needed to print digital photos, I’ve found that Photolab – the photo processing arm of the Canadian grocery chain Loblaws – has always done a great job.  Photos are typically printed in a day or less, are easy to pick up – just stop by your local supermarket! – and have always turned out excellently.  And they are reasonably inexpensive, too.

For as long as I can remember, my parents have been labelling the back of the photos in their photo albums with the month they were taken.  That way, when browsing an album years in the future it is easy to see exactly when something happened and how old I was at the time (they sometimes label the people and the places as well, depending on how non-obvious these attributes are in the photo itself).  Though it takes a bit of work to make all these annotations, the outcome is well worth it in the end.

With traditional film, there are few alternatives to manually annotating photos; ‘burning’ the date into the photo itself is one technique, but obscures some of the image as a result.  But with digital photos, this information – and much more – is stored alongside the photo by the digital camera.  And on the back of the prints we get back from Photolab/Zehrs, they conveniently print the photo filename on the back of the photos.  Wouldn’t it be nice if we could put the date into the filename to make annotating our photos easier?

Naming photos for printing with PowerShell

With my new-found interest in scripting with PowerShell, I figured that there must be a way to automatically name files based on the date they were taken.  Sure enough, there was – the Image class, part of the System.Drawing namespace, allows reading of the EXIF data stored by the camera in the photo.

# Load image and get EXIF date
[System.Reflection.Assembly]::LoadWithPartialName("System.Drawing") > $null
$photo = [System.Drawing.Image]::FromFile($file)
try
{
	$dateProp = $photo.GetPropertyItem(36867)
}
catch
{
	try
	{
		$dateProp = $photo.GetPropertyItem(306)
	}
	catch
	{
		continue
	}
}
$photo.Dispose()

While it is rather straightforward to get EXIF metadata in general from a photo, it takes a bit more work to get the precise data you want.  EXIF data is stored as key:value pairs in the image, with the keys being standardized numerical identifiers.  Here, we first try to access the ‘Date Taken’ value with key 36867, and should that fail, the ‘Date Time’ value with key 306.  These numbers are completely unintuitive, but can be found at various places on the web; an extensive list (if you already know what you are looking for) is in enumeration format here.  If neither can be accessed, we skip renaming the file; this is likely the result editing the photo with a program that does not preserve this data.

Once we’ve retrieved the date the photo was taken from the EXIF metadata, we still have to parse it into a string that we can use for our filename.  The data is stored using UTF8 encoding, so we first transform the raw bytes into a string, and then pull the useful parts of the date out into separate variables.

# Convert date taken metadata to appropriate fields
[System.Reflection.Assembly]::LoadWithPartialName("System.Text") > $null
$encoding = New-Object System.Text.UTF8Encoding
$date = $encoding.GetString($dateProp.Value).Trim()
$year = $date.Substring(0,4)
$month = $date.Substring(5,2)
$day = $date.Substring(8,2)

With that data (and a simple regex), I finished the script by renaming the file into the form ‘YYYY.MM.DD.iNNNN.jpg’, where NNNN is the four-digit image number of the original file (my parent’s camera, a Canon, names files IMG_NNNN.jpg; other manufactures use similar formats).  Now when we print our photos, we don’t need to add the date by hand – the script ensures that a string like ‘2010.01.05.i1234.jpg’ is printed on the back of our photo, which tells us the date (January 5th, 2010) as well as what to search for if we need to print another copy (‘1234’, which finds the original image ‘IMG_1234.jpg’ (again, this example is for Canon cameras)).

Renaming Files with Date Taken: The Full Script

The next time you need to send photos for printing, give the script a try.  Running the script is easy; simply make a copy of all the photos you want to print into a new folder, and save the following PowerShell script into the directory with the photos for printing.  Then, right-click on the script and select ‘Run with PowerShell’.  The script scans the directory, finds all of the images, and then renames them based on the ‘Date Taken’ EXIF field.

PowerShell Date Taken Naming Script [ps1, 6.7K]

Posted in Miscellaneous | Tagged , | Comments closed

Sending Email with PowerShell: Implicit and Explicit SSL

I never quite understood the attraction of scripting; sure, not having to set up the scaffolding code of a more formal language is nice, but the limitations of the environment just seemed too great.  The Unix community has it far better than the Windows community in this regard; shell scripting can accomplish amazing feats – as long as all of the appropriate utilities are strung together in the right configuration, that is.  And though scripting languages like Python and Ruby do hold some allure, it’s one more thing to install on my Windows box and learn the syntax for.  So for most tasks that needed scripting, C# filled the bill – when I could justify the start-up cost.

Enter PowerShell, the Windows scripting language that purportedly brings together the power of Unix shell scripts with the flexibility of the .NET framework. And it’s installed by default on Windows 7, to boot.  I’d seen an interesting demo of how WPF and PowerShell worked together, so when I recently found the need to send a bunch of mark reports out, I figured it was time to give it a whirl.  There were a couple of tricky problems I encountered while I was throwing the script together; here’s a quick overview of sending email with PowerShell.

The Basics

First, we need an email to send.  This should do:

# Mail Message
$from = "user@domain.com"
$to = "user@domain.com"
$subject = "Hello from PowerShell"
$body = "This is a message saying hello from PowerShell."
$hasAttachment = $false
$attachmentPath = "attachmentPath.txt"

The primary requirement for sending email is access to an SMTP server; it needn’t be on the local machine.  We’ll use Gmail’s SMTP server:

# Mail Server Settings
$server = "smtp.gmail.com"
$serverPort = 587
$timeout = 30000          # timeout in milliseconds
$enableSSL = $true
$implicitSSL = $false

Pretty basic stuff; here we’re using smtp.gmail.com on port 587, and we’re using an SSL-encrypted connection.  We’re also not using implicit SSL, and instead using explicit SSL; wait, what’s the difference?

Explicit SSL

Explicit SSL, as described here, means that the client first connects to the server using an unsecure channel, requests that conversations be moved to a secure channel, and then both server and client switch to a secure connection and the rest of the communication is encrypted.  Though this sounds somewhat lengthy, it’s the standard procedure for setting up an SSL connection (see RFC 2228).  Gmail handles explicit SSL without any difficulties, as do many other mail servers; Gmail’s explicit SSL server runs on port 587.

Implicit SSL

In contrast, implicit SSL drops the SSL negotiation and jumps right into the SSL connection to begin with. Often, this is done through a connection to a specific port that only accepts secure connections.  There is no official standard for this mode of communication, though it’s widely implemented; Gmail also handles implicit SSL, this time on port 465.

The Catch

My local university SMTP server only handles implicit SSL, so I needed to find a way to work with this variant.  Unfortunately, the standard .NET library for sending mail, System.Net.Mail – the one often used from PowerShell to send mail – don’t support implicit SSL.  Luckily, System.Web.Mail – an older, now obsolete mail sending library residing in System.Web.dll – does support implicit SSL. 

Putting it all Together

Now that we’ve got a strategy for handling implicit and explicit SSL, let’s code up a section of our script to handle each of these servers so we can switch back and forth easily.

We’re also going to need a set of credentials if we’re logging into a secure server; the Get-Credential cmdlet is just the ticket:

# Get user credentials if required
if ($enableSSL)
{
    $credentials = [Net.NetworkCredential](Get-Credential)
}

We could also prompt at the command line for a username/password combo or just hardcode these values into the script, but the Get-Credential cmdlet is an easy way to get and store just the information we need.

Explicit SSL with System.Net.Mail

We’ll be using the System.Net.Mail namespace to send email to servers using explicit SSL, or to servers that do not require SSL connections.

if (!$enableSSL -or !$implicitSSL)
{
    # Set up server connection
    $smtpClient = New-Object System.Net.Mail.SmtpClient $server, $serverPort
    $smtpClient.EnableSsl = $enableSSL
    $smtpClient.Timeout = $timeout

    if ($enableSSL)
    {
        $smtpClient.UseDefaultCredentials = $false;
        $smtpClient.Credentials = $credentials
    }

    # Create mail message
    $message = New-Object System.Net.Mail.MailMessage $from, $to, $subject, $body

    if ($hasAttachment)
    {
        $attachment = New-Object System.Net.Mail.Attachment $attachmentPath
        $message.Attachments.Add($attachment)
    }

    # Send the message
    Write-Output "Sending email to $to..."
    try
    {
        $smtpClient.Send($message)
        Write-Output "Message sent."
    }
    catch
    {
        Write-Error $_
        Write-Output "Message send failed."
    }

}

Using System.Net.Mail is pretty straightforward – we create an SmtpClient to connect to the mail server with, and make the appropriate settings.  Setting Timeout isn’t necessary, but since the default is really high (something like 100,000 milliseconds) and most servers respond quickly, setting it to 30 seconds ensures that the script will fail in reasonably short order if it cannot connect.

Creating a message is easy too – we just create a new MailMessage and pass in our message to its constructor and attach any necessary files.  Then we call the Send function on the SmtpClient we created, and the message goes on its merry way.

Users of PowerShell version 2 or greater can also make use of the Send-MailMessage cmdlet, which wraps System.Net.Mail in an easy-to-use cmdlet.

Implicit SSL with System.Web.Mail

For servers that only support Implicit SSL, we’ll be using the System.Web.Mail namespace.  This namespace has been made obsolete in favour of the newer System.Net.Mail namespace, but at the current time it’s still included in the .NET Framework and so we can take advantage of it here. 

else
{
    # Load System.Web assembly
    [System.Reflection.Assembly]::LoadWithPartialName("System.Web") > $null

    # Create a new mail with the appropriate server settigns
    $mail = New-Object System.Web.Mail.MailMessage
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/smtpserver", $server)
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/smtpserverport", $serverPort)
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/smtpusessl", $true)
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/sendusername", $credentials.UserName)
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/sendpassword", $credentials.Password)
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/smtpconnectiontimeout", $timeout / 1000)
    # Use network SMTP server...
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/sendusing", 2)
    # ... and basic authentication
    $mail.Fields.Add("http://schemas.microsoft.com/cdo/configuration/smtpauthenticate", 1)

    # Set up the mail message fields
    $mail.From = $from
    $mail.To = $to
    $mail.Subject = $subject
    $mail.Body = $body

    if ($hasAttachment)
    {
        # Convert to full path and attach file to message
        $attachmentPath = (get-item $attachmentPath).FullName
        $attachment = New-Object System.Web.Mail.MailAttachment $attachmentPath
        $mail.Attachments.Add($attachment) > $null
    }

    # Send the message
    Write-Output "Sending email to $to..."
    try
    {
        [System.Web.Mail.SmtpMail]::Send($mail)
        Write-Output "Message sent."
    }
    catch
    {
        Write-Error $_
        Write-Output "Message send failed."
    }
}

As we can see, using System.Web.Mail is quite a bit more convoluted.  First up is loading the System.Web DLL, which is where this namespace resides.  Then, we configure the mail client using CDO fields; most of them have pretty obvious settings, though sendusing and smtpauthenticate are a little obtuse (see the comments in the script for what each value means).  Unlike System.Net.Mail, these server settings are made on the MailMessage class itself, which is also where we set the message properties and add any necessary attachments.  Finally, we send the mail using the Send static function on SmtpMail, and the message goes on its merry way just as before

A Final Note for Gmail Users

Gmail’s authentication settings are a bit strange; in my testing, I’ve found that you need to enter just your username (not username@gmail.com) in order to authenticate with their SMTP servers.  This causes a bit of a problem with Google Apps installations, which also use smtp.gmail.com as their SMTP gateway; username@domain does not work, and I’ve yet to find a workaround.

Also, while Gmail supports explicit and implicit SSL, it does so on different ports – 587 and 465, respectively.

Sending Email with PowerShell – Piece of Cake!

After a bit of hacking, I now understand just what those scripting folks feel when the advocate for scripting environments – it really is quite easy to accomplish what you need to accomplish without a whole lot of code.  And even though I needed to drop into the .NET framework to get classes to deal with explicit and implicit SMTP servers, that was hardly a problem – those classes integrated with the rest of my PowerShell script seamlessly.  PowerShell is certainly a very powerful environment, and I’m glad to add it to my technical bag of tricks.

PowerShell Email Script [ps1, 3.2K].

Posted in Miscellaneous | Tagged , , | Comments closed

Technical Protectionism

There’s a well known concept in economics known as comparative advantage.  A formal definition has many nuances, but the basic principle can be summed up as each party undertaking the work that they are most productive at, such that the overall production of the system is higher than it would be if each party performed tasks they were less productive with.  For example, if Farmer A’s farm could grow 100 bushels of apples or 200 bushels of carrots a year, and Farmer B’s farm could grow 75 bushels of apples or 50 bushels of carrots, then the highest production occurs when Farmer A grows carrots and Farmer B grows apples (75 + 200 = 275 bushels total).  This is the case even though Farmer A can grow more apples than Farmer B; if there were demand for 100 bushels of apples, the best way of growing them is to have Farmer A grow 25 bushels and Farmer B grow 75 – total output is 100 bushels of apples and 150 bushels of carrots, which maximizes total output under the constraint that at least 100 bushels of apples must be grown.  Since Farmer A is better at growing carrots, then it’s always economically beneficial for him/her to do so.

Though this example looks at farmers, it applies to countries as well.  However, the use of regulations can complicate the analysis, as regulations can be applied (through tariffs, import controls, or other measures) to artificially tip the scales in favour of one party.  If Farmer A were restricted to exporting 50 bushels of carrots, and not restricted on apples, then the market realities would result in Farmer A growing primarily apples even though he/she can grow carrots much more easily.  And as a result, the total output of the economy is reduced to 200 bushels, down from 275.  On the surface, applying regulations seems to be a pretty irrational thing to do – why would we work harder and produce less?

The answer to that question lies in the reasons behind applying those regulations, to which there are many.  The one I want to focus on today is protectionism – policies designed to ‘protect’ workers and businesses in a country from outside competition.  If India is able to produce technical workers more cheaply than North America can, then market pressures tend to result in those jobs being performed by Indian workers.  But, you exclaim, they’re stealing our jobs!  Outsourcing is bad!  Of course we should have regulations in place to stop it!

Economists have many good reasons why this isn’t be the case – foremost amongst them the principle of comparative advantage – as even considering these job losses the economy is still better off as a whole.  Though jobs are lost in the short term, in the longer term the local economy picks up the slack in areas the country is more suited to, resulting in a net gain.  The rise of information technology is one example of this; when computers were made domestically in North America, they were relatively rare and obtaining one was expensive.  By manufacturing them in countries with lower manufacturing costs, computers became mainstream and widespread – and we’ve subsequently built a massive business sector on top of them.  For all practical purposes, the Internet as we know it today wouldn’t exist without low-cost computers, and had we regulated the economy to discourage the outsourcing of computer manufacturing, we’d be years behind where we are today.  Granted, we missed out on the jobs that might have existed had we done that, but at the same time we’ve gained an entire sector of jobs in companies like Google, Microsoft, HP, Amazon, and the many, many tech startup companies.  And then we’ve created a whole host of jobs to support those jobs – would Google’s chef’s have a job if Google weren’t around?  Change is always hard, but by allowing it to happen we have indirectly benefitted ourselves – and the rest of the world!

The crux of this post is not comparative advantage and economic protectionism, but rather something I’m going to call technical protectionism.  In the tech field, the players aren’t so much people or companies, but rather, technologies themselves.  Few people would argue that Ruby is a better language to write web applications in than C, or that Photoshop is better than Paint, or that Mac is better than PC (just kidding!).  Actually, it’s that last one that is entirely my point; the Mac versus PC debate will continue indefinitely, as many of those doing the debating are debating for solely protectionist reasons.  There are completely valid technical (and non-technical) reasons why Mac is better than PC, and why PC is better than Mac, but more often than not the discussions seem to take place on a ‘it’s right for me, so it’s right for you’ level that lobs orthogonal arguments toward the opposing side in an attempt to win by attrition or disinterest.

The frustrating part in this whole issue is that, unlike with economic protectionism, technical protectionism is condoned by the tech community.  When the US introduced ‘Buy American’ clauses into some of their economic stimulus packages, experts on both side of the border – and the media to some extent – erupted in condemnation.  Now consider the last time you got into or observed a Mac versus PC, Windows versus Linux, Python versus Ruby style argument: those sorts of arguments are expected – and tolerated – by the technical community!

In my experience, these debates are intrinsically motivated by the participant’s own familiarity with a particular technology: I know how to build programs in technology X, therefore technology X is the right one to solve your problem with.  You see this all over the place – just look at small consulting firms, which often specialize in a single technology stack (Microsoft, IBM, Oracle, LAMP, etc.).  Having one of these firms pitch you a solution inevitably means them pitching a solution that runs on their stack without consideration of the others – if you want a vacation-planning application for 50 employees from an Oracle firm, then you’ll be paying for an enterprise grade Oracle database, regardless of whether a cheap LAMP application would have sufficed.  Sure, it may be unfair for them to pitch the value of another company’s services, but at what point should they be saying ‘this is not something we do well’, and voluntarily withdrawing from consideration so not to waste everyone’s time? 

Even taking a step back from companies, this behaviour is present even in the colleagues I interact with.  Since I worked at Microsoft, in technical debates I am automatically assumed to be the enemy of everything open source and the chief defender of everything Windows.  And while my experience gives me the ability to understand Windows’ capabilities and limitations, I’m not going to side with it automatically – it depends on the situation.  If you need a cheap server with a small application, Windows isn’t right.  If you need an expensive server with  a massive application, Windows isn’t right.  If you need a moderately expensive server with a medium sized application, Windows isn’t right.  And you know what?  You can replace ‘Windows’ in the previous three sentences with ‘Linux’, ‘Mac’, or any other technology – without looking at the precise details of the situation, it’s impossible to tell which combination of features and capabilities are appropriate for the problem at hand.

My fundamental point here is that it is impossible for one technology to solve every problem in the best way possible.  Comparative advantage tells us that the best way of solving all of problems is to choose the right tool for the job, regardless of what that tool might be.  Granted, choosing a tool is nowhere near as straightforward as counting bushels of produce, but the principle still applies: use the combined strength of all parties to the best of their individual abilities.  Sure, we have to make allowances for expertise and to minimize tool diversity to keep overall costs down, but supporting the notion that one can advocate for a particular approach solely because it’s what they know or believe is unproductive.  And yes, that means that you’re favourite solution won’t always win – and that’s OK.  By choosing the best tool for the job, we’ll have a better outcome in the end.

That’s not to say that having opposing products and companies isn’t useful – it is.  Sometimes you need to go against the grain and tackle the hard problems.  But when you do, use the best tools for the job and not the ones you know just for the sake of doing so.  Firefox is the browser it is because IE languished at the top of the market, and made no effort to improve.  Competition from Firefox was good, as it made developers on both teams compete to have the better browser.  At the same time, however, notice that the Firefox team didn’t start from scratch – they took the best technology that was available to them (Netscape) and made it better, rather than taking a protectionist approach and using what they knew.  Innovation occurs when we create something new, not when we take the long way to get to where we’ve already been.

As a community, we need to dispel the notion that technical protectionism is an acceptable attitude to have.  Objective criteria and constraints can be found for every problem we need to solve, and technologies can be evaluated against them impartially.  The next time you encounter an argument from a protectionist position, make sure to call it out, and make it know that this sort of justification is unacceptable.  Just as with outsourcing and the Internet, eliminating technical protectionism can have profound benefits we cannot even imagine, and until we start down that road, we’ll never know what we’re missing.

Posted in Miscellaneous | Comments closed

ECE 150 Programming Samples

This term, I’ve been given a position as a teaching assistant for ECE 150: Introduction to Programming at the University of Waterloo.  Taught in C#, ECE 150 covers most of the basic features of programming languages and how programs are written, and assumes no basic programming knowledge on the part of the students taking it.  Unfortunately for me, that means most of the cool parts of C# and the .NET framework – LINQ, WPF, WCF, ASP.NET MVC, and even language features like generics, anonymous functions, and lambda expressions – are off limits.  However, it also means I need to know the basics really well; one of the other TAs and I were asked today the difference between int.Parse() [used in the lecture slides] and Convert.ToInt32() [used in the textbook]; the answer, as I found out, was quite subtle (NB: for students in the course, the difference isn’t relevant – use whichever you desire).

In that spirit, I’ve decided to put together a handful of applications over the course of the term in order to demonstrate various features of the C# language for programming beginners.  All of the samples will be available via my ECE 150 page, and will be updated as the term progresses.  How many I get done is entirely dependent on the amount of time and number of ideas I have; those that do get made will roughly correspond to each week’s content, and we’ll use them in the final review.  Full source code for each application is also available for those who want a deeper look into some of the features of C# and the .NET platform.

Check them all out at http://nicholasarmstrong.com/projects/teaching/ece150/

Posted in Projects, Waterloo | Tagged , , | Comments closed