March 12th, 2011

Public transit data, APIs and City Government

Tom Boutell
Chief Software Architect

Julie Blitzer, Uffer and Jariyasunant, #transitapps A talk at SxSW Interactive

"Why are you doing this?" Julie is a lifelong New Yorker & subway rider, design geek, UX designer to nonprofits & government & good cause projects, former political staffer, still a policy wonk

"Service Design"

Designing an entire service or experience, not the components. Think Apple Store

"Rewind 10-15 years..." everyone walked around with paper maps. We made plans on the phone, we checked the TV news for delays. Running late? Good luck telling your friends!

Now? Everybody has mobile connectivity, wayfinding apps (like Google Maps, which includes transit now and has for some time)

[An artist is drawing her interpretation of the event, nice]

Should we try to change behavior, or improve the efficiency of what we have now?

New York City tried to roll out a cabshare service. Nobody used it because it was poorly designed. You had to walk six blocks to a cabshare location.

Sylvia Harris gave a great presentation about design activism and participatory design at World Usability Day. "There's a relationship between graphic design, activism and getting things done in the community"

Participatory design is a virtuous circle. It improves responses, makes good first impressions, saves time, and reduces errors.

Models of design engagement

1. Crowd sourcing: government asks citizens for ideas.

2. Virtual pitching: citizen publishes ideas, hoping someone in government sees it. (Google searches, twitter searches might bring it up, word of mouth too)

3. Crowd sourced virtual pitching: organization solicits ideas, shows best ideas to government agency. (Heh, managing from below/outside)

That's it for the first presenter. @zhuli

Mike Uffer talking about "GIS in Goverment," data and APIs in some recent NYC projects.

NYC had parking lot data sets, project made that data available to end users.

That led to another project to enable tax assessors to automate some of their work. (This one sounds like pure IT within the government stuff although certainly useful)

They used a mix of interesting data though. Street maps, which were based on the freely available US Census "Tiger" data map set as enhanced by the city ("LION"). This data includes transit, property blocks, etc. They also have a lot of proprietary data they don't share (sometimes for security reasons). The data is provided in vector files, it's not a REST API situation. (I've played with the Tiger data. The format is a bizarre one-off but you can do worthwhile things if you are patient.)

Their goal was to combine public and proprietary information and present it to city staffers who needed it. (OK, so the open part of this is recognizing open data that they didn't have to reinvent and mashing that up with the city-staff-only data)

He is showing slides of J2ME interfaces... ouch. (That's not really fair though because they needed a solution before iPhone app development was even an option... the interfaces do look simple and useful, and they used a lot of open source)

They also did a project to help the department of homeless services count the homeless more effectively, which is important for ensuring resources are adequately distributed.

Technologies used include Google Maps API, ArcGIS, public domain routing algorithms, the U of Minnesota map server, PDF file creation, Windows Mobile and J2ME.


Jerry Blitzer talking about whether mobile technologies are actually changing transit rider behavior or not. Good question!

260 agencies are giving out GTFS transit schedule data now. Realtime data (where is my bus *really*?) is more scattered but it's coming, a lot of agencies use Nextbus and also release feeds (unfortunately some don't share the feeds beyond giving them to Nextbus).

Folks aggregating the data:, the GTFS Data Exchange.

citygoround also has a hall of shame for cities that aren't releasing their data. Awesome.

(GTFS is a very simple open format for transit schedules. It's just CSV files. Works great. I used it on septime, which is narrowly focused on giving you access to the next departure times for your bus while using a primitive webphone in a loud bar. (: There's an iPhone skin I need to deploy real soon now)

Cool bay area apps include Transporter, iBart, and the author's BayTripper. BayTripper actually uses the locations of buses from NextBus feeds to give you realistic plans for right now.

One Bus Away has released an Open Trip Planner which allows you to make complete trip planning apps without writing your own implementation of Dijkstra's shortest path algorithm (:

So what effect has this had? There's lots of studies about real time data and traveler information prior to 2005, but they could not have predicted the new apps. Google Scholar has nor esults for "open transportation data" or "open transit data." (Could that be because they aren't in Google Scholar?)

Previous studies have focused on things like helping people with cognitive impairments...

Travelers overestimate their wait times without real-time information and get frustrated and walk away. Past studies have shown that a stationary sign at a bus stop showing how soon the bus will come does reduce frustration.

So does it lead to increased ridership? The evidence is "mixed." (We need marketing for transit that is as aggressive as the marketing for a car-based lifestyle. SEPTA in Philly has a particularly bad image with people who don't yet use it. There is no user's guide and the system is unnecessarily complex to understand)

The One Bus Away service increases the number of transit trips for the average respondent slightly. Most just get an improvement in the quality of their experience with trips they needed to take anyway. (It stands to reason that existing transit riders are early adopters of this stuff) promotes the idea of a dedicated personal device for transit information. (Wha? An app for the phone you already carry makes a lot more sense here! I won't carry a separate camera even though that clearly enhances my life. I'm frequently surprised that I can't pay at the grocery store with my phone. Why would I carry a transit appliance?)



Panel discussion

Jariyasnunant is talking up the standardization of data (GTFS feeds for example). Benefits all participants.

Uffer is interested in making goverment itself more efficient and cost effective using a mix of public and private data sources and feels that should be left "to the government although contractors do the heavy lifting... there are a lot of political issues." I'll bet.

(My battery is giving up the ghost here but we got through all three talks!)

Discussion of the better use of crowdsourcing to track issues and the problems that come with it.

Jariyasunant: "I write a lot of public transit apps, I don't use public transit. I bike everywhere. Can we make it possible to determine where the buses with bike racks are? How can we disseminate information like 'this bus is full and is going to pass some stops'?" (I don't see that working because the driver doesn't know if it's an option until people do or do not pull the cord to get off at that stop. You shouldn't give up and walk away because the bus *might* be full...)

Challenge question: "What barriers prevent private data from becoming public?"

Uffer: "assessments are not private but the way they are computed is private. Some information about ownership is not automatically public. Also the assessors are often challenged in various decisions that they make." (Went on to indicate that there's a lot of guesswork and they don't want to have to defend it)

Sharing data between agencies is difficult because agencies engage in turf wars. You can use the freedom of information act, it's very expensive and sometimes works.



Now some audience questions

(Plugged in. Yay)

Q. "To what extent do you think that when civic hackers get data, they should be permitted to profit from it?"

A. (Uffer) "I don't see why not." Cited the fact that everyone resells improved census data and that's no bad thing.

(Jariyasunant) "A lot of the public almost expects this type of app to be free. People ask why you don't support this or that agency, they don't get that you built it for free in your own time. Developers should be able to profit from the data (to provide reliable and cheerful service)"

Q. "Is there a standard for realtime arrival data?"

A. (Jariyasunant) "There's lots of academic research on this problem." (The question was about standards for releasing the data though)

This blog post on opentransportation is a good overview of the problem.

Q. "Are agencies cooperating with developers?"

A. (Jariyasunant) "There should be more collaboration."


Tom Boutell
Chief Software Architect