October 3rd, 2011

Wrap it up: storing files in the cloud with PHP

Tom Boutell
Chief Software Architect

Every website has media. Images, MP3 files, even Office documents, cluttering your server's disk space to the point of no return. Amazon Simple Storage Service, aka S3 is a super-popular solution to that problem.

Amazon knows all about storing files- they run some of the biggest, most popular websites in the inner solar system. In fact, they know so much about files that it makes good business sense for them to resell some of that file-storing goodness to the rest of us. Which is extremely handy for those who are not looking forward to building their own server room or overpaying for a crazy-big server. And backups. And backups of the backups. In mutliple time zones.

We all know this. And many of us have used Amazon's official PHP API, which is pretty great. But the trouble comes when you try to marry S3 to your old-school code. Code that likes things like files and directories. 

Smart PHP programmers like to solve problems like this with "stream wrappers." A stream wrapper is code that teaches PHP all about a new way of storing stuff. Most programmers know you can write to a file with file_put_contents(), and many know that you can fetch something over the web with file_get_contents(), even though that something has an http: address and doesn't live on your hard drive. But not many realize that you can teach PHP about entirely new protocols- brand new ways of storing stuff.

So there must already be stream wrappers for S3 already, right? Yes, but those we've tried lack the ability to get down and dirty with subdirectories. You can't list the contents of a folder with the usual PHP functions for doing that. And for us, that's a dealbreaker.

So we built our own. And we taught it to handle subdirectories. And to peek at files faster than is really possible. And then, because if you can't trust your files you're in a whole lot of trouble, we wrote lots and lots and lots of unit tests to make sure the darn thing works. 

Did we build it for Apostrophe? Of course! But there is absolutely nothing preventing you from using it in other PHP projects that don't contain a drop of Symfony or Apostrophe code (*). Even the unit tests are completely standalone PHP. And we'd like to see it become the definitive stream wrapper for Amazon S3.

So we've put it up on github, where all the cool projects hang out these days. And we're looking forward to your comments- and your "issues," "forks" and "pull requests." 

Edit: just pushed our first bug fix up to github! Murphy's Law: you always find a bug not covered by your tests immediately after bragging about your tests. 

Fork aS3StreamWrapper on github

(*) Sad, lonely projects. Just kidding.

Tom Boutell
Chief Software Architect