The Maine Center for Public Interest Reporting launched its Political Party Time series this week. It's powered in part by the Sunlight Foundation's Political Party Time database, which I'll call the PPT.

PPT is a giant database of political fundraisers, filled with information from invitations sent to Sunlight by readers like you. There are a lot of ways to sort the information,

The Pine Tree Watchdog Political Party Time series queries the PPT API (what's an API?) to get a list of events in Maine. Sunlight's API  has a number of ways to limit the events returned (by date, host, beneficiary, etc.), but the sidebar widget we wrote for the series' pages simply lists events that benefit Maine politicians.

The widget lists the benefiting politician or politicians, the date, the place, and a name for the event, for the most-recent 50 events. If a politician has been talked about on the Pine Tree site, readers can click to see stories about that politician. If there's a story about that event, it will be listed in the results. They can also click on the event name to go to the PPT site to see more about the event, including the invitation.

The code for the widget is currently contained in the WordPress child theme for the Pine Tree Watchdog site (they use INN's Largo platform and that provides the parent theme), the relevant parts to this post are the widget itself and the custom page template for the PPT series page.

Here's how it works, and what I learned while building it.

How do you query an API?

The simple way to query an API is to find the right URL and hit it with a browser. It'll return some JSON code that looks like this:

But our readers prefer pretty tables. We do, too, which is why we parse that JSON to get information about the event. Slap some JavaScript into the browser window, load the page and go.

Except it's not that simple. We can't hit the API from the browser because, in addition to requiring an API key to limit access, the Sunlight Foundation APIs deny CORS requests.

CORS is a response to a response to a security threat. In order to prevent Cross-Site Request Forgery attacks, where a malicious website uses your credentials to gain access to another site, browser makers implemented the same-origin policy. Now, code on malicious sites can't access information on good webpages, but neither can good web pages access information on other good web pages.

In situations where both sites want to share information, browser and server developers agreed on the CORS mechanism. When code on a site like Pine Tree Watchdog requests information from the Sunlight Foundation PPT, the browser adds an HTTP header saying that the request originated on the Pine Tree Watchdog site. The Sunlight Foundation API response could include a corresponding HTTP header telling the browser that the Pine Tree Watchdog site is allowed to access the data.

But the response doesn't. Sunlight doesn't have unlimited bandwidth and server space to handle a good, thorough slashdotting. Denying CORS requests cuts down on the amount of information they have to serve.

Since the reader's browser can't retrieve this information, the PTW site does it for them. If the PPT API hasn't been queried in the last 30 minutes, the server will download the API response and save it to disk. Then the site parses that into an HTML table, and saves the table to disk. The HTML table gets placed inside the widget, and the widget is displayed to readers with some CSS that makes the table look like a list.

All this caching happens in order to keep things snappy for readers. By caching the PPT API response, the server doesn't have to hit the API every time it wants to rebuild the list of events. And by caching the event listing HTML, it can insert that in the page instead of laboriously building the listing every single time.

Cross-referencing

The links to the politicians' names is a pretty cool feature. It checks to see if a tag exists in the WordPress database that matches the politician's name. If it does, then their name in the listing is made a link to all stories tagged with the politician.

In a development build, I had it automatically create new tags if the tag didn't exist. This was a little too much information for the readers, so it was removed before the widget launched. The pre-creation of new tags made things easier for the writers, but it gave readers a link to nowhere.

What I learned:

• CORS feels generally useless
• file i/o in PHP (for caching)
• td { display:block; }
• some ways of handling JSON in PHP
• WordPress widget construction
• think about the readers, not just the newsroom users

This was the last project I worked on during my summer internship with INN. Now I'm in Idaho on a 3-month emedia internship for the Progressive Publishing family of magazines.