Flight Log Version 2.1: Import Flights from Digital Boarding Passes

In general, my Flight Historian has been a big time saver for me as far as tracking my flights – instead of manually generating reports and maps from an Excel file, I can simply add flights to a database and let it do all the work. However, as I’ve started tracking more details about my flights over time, the task of entering the flights has become less simple.

Screenshot 2017-04-09 21.59.30
There are currently 24 fields to fill out in the flight log. Not every field is required, but it can still take several minutes to fill out a flight.

Since I’d been working on parsing boarding pass barcode data, it seemed like a logical next step to write some sort of scanner that would read a boarding pass barcode and import the data as a new flight. Then one of my Twitter followers had a suggestion:

Getting Flight Data from an Apple Wallet Pass

Apple Wallet is an application included with recent iPhones that, among other things, can be used to store boarding passes from various airlines in a single location.

digital-boarding-pass

Even more importantly, Wallet lets you share boarding passes by email.

So I decided I’d do for boarding passes what sites like TripIt do for itineraries. I would set up an email address that I could simply email my Wallet boarding passes to, and then have my server look for emails from me and process any boarding pass attachments.

Getting Boarding Passes from IMAP

As the boarding pass email address I set up supports IMAP, I used Ruby’s Net::IMAP class to have Flight Historian interact with my email account.

When I go to the Import Boarding Pass page on Flight Historian, it checks for new messages (from a specified list of my email addresses) which have a “PKPass” (PassKit Package) attachment.

The PKPass format is how Wallet stores and shares digital boarding passes. It’s actually a zipped file archive with the extension .pkpass. Inside this archive is, among other things, a file called pass.json which contains all of the information about the pass in JSON format, including the visible text and the data encoded in the 2-D barcode.

Thus, I wrote a BoardingPassEmail module for Flight Historian which would loop through any email attachments, and for each attachment, unzip it, navigate to pass.json, and store it as a PKPass object in my database. Once Flight Historian successfully saves the pass, it deletes the email that the pass came from.

Adding a Flight from a Stored Boarding Pass

Since Flights need to be part of a Trip, every trip provides a link to my new Import Boarding Pass view with the Trip’s ID as an argument. However, I included the ability to select a different trip to accept the new flight, if necessary.

Screenshot 2017-04-23 09.35.33

The “Create Flight” links to the same New Flight form as before, but includes a PKPass ID as an argument. If that pass ID is present, the Flights controller knows that it needs to read the JSON data associated with that pass, and use it to populate the appropriate form fields, saving a great deal of manual data entry.

Screenshot 2017-04-23 09.37.07
Fields which were automatically populated are highlighted so that they can be checked for accuracy.

There are three sources for filling in the the fields:

  • Some fields can be populated directly from the pass.json file.
  • Some field data is encoded in the pass’s barcode string. Thus, I created a barcode method for the PKPass model, which uses my Boarding Pass Parser to decode and return the barcode data.
  • The remainder are not found anywhere in the electronic boarding pass. I wrote a method to look up the remaining flight data on FlightAware’s FlightXML API using the boarding pass’s airline, flight number, and UTC departure time.

When the new flight is saved, it also includes a pass_serial_number field, which is a string uniquely identifying each boarding pass as defined in the PKPass format. This way, if a boarding pass is updated, Flight Historian knows that the updated pass needs to be associated with an existing flight rather than a new flight.

Updating a Flight

If Flight Historian finds a boarding pass attachment with a serial number that is already associated with a saved Flight, it shows the pass in a “Flights with Updated Passes” list instead.

Screenshot 2017-04-23 21.43.50

This time, since the Flight already exists, the form only shows the values which have changed, and allows the user to select the new value or keep the existing value.

Screenshot 2017-04-23 21.44.18
In this case, I was upgraded after I received my first boarding pass, so the new boarding pass updates my travel class and barcode data (since the barcode contains seat number).

Aircraft Types

My flight log previously kept track of aircraft families (e.g. any Boeing 737), and didn’t do anything with specific types (e.g. Boeing 737-800), other than to optionally note them in an aircraft variant text field. This was a deliberate decision, as my flight log was generally set up to catalog my flight experiences, and there was no substantial quality of flight difference for me between, say, a 737-800 and a 737-900.

However, one of the fields I needed to look up on FlightXML was the aircraft type. FlightXML does return the specific aircraft type (by ICAO code), so at the very least, I needed to write a lookup function to convert specific aircraft types into the general aircraft families I was using.

However, I decided that it would make more sense to do it right, and actually incorporate aircraft types into my database. I still wanted to maintain my aircraft family structure, and I still had some old flights in my database that I didn’t know the type for anyway. So I eventually decided to add a parent/child relationship to my Aircraft Families table, where both types and families would be stored in the table (and I could assign a flight either a family or a specific type), but the aircraft types would have a parent_id field that linked to the ID of their parent family. That way, I could still summarize all of my flights by their parent family (by looking at all aircraft types that were part of the given family), and for each specific type FlightXML returned, I would know the family.

The side benefit of this structure is that it also made it easy for me to show statistics for the specific types. Each aircraft family page still includes all the flights it did before, but it also shows a list of types, and clicking on one of those types provides details for that type.

Screenshot 2017-04-23 22.17.07.png
Data for all Boeing 737 aircraft I’ve flown
Screenshot 2017-04-23 22.17.29.png
Data specifically for Boeing 737-800 aircraft I’ve flown

Change Log

Front End

  • Created JSON API for parsed BoardingPass data
  • Created Boarding Pass Parser views
  • Created admin view to list any flights with invalid boarding pass data
  • Created view to allow entry of boarding pass IATA codes which were not found in the Flight Historian database
  • Created Import Boarding Passes view
  • Updated Add Flight form to use PKPass data when available
  • Created Edit Flight with Updated Pass form to show and update changed flight data
  • Created new message banner system
  • Added aircraft types to AircraftFamily model and created views to manage and show specific types

Back End

  • Created BoardingPass class for parsing and interpreting IATA Bar Coded Boarding Pass (BCBP) data
  • Created BoardingPassEmail module for connecting to an IMAP email account, finding PKPass attachments, and processing them
  • Created PKPass class for storing PKPass data
  • Added numeric code field to Airline model to support BCBP interpretation
  • Added email fields to User model

Specification

Flight Historian 2.1 Functional and Technical Specification

Source Code

As usual, my project code is available on Github.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s