March 4th, 2011

Liveblogging Symfony Live Paris: Unconference Day

Tom Boutell
Chief Software Architect

Symfony Live unconference schedule

(The unconference is in the opposite wing of the building from the other two conference rooms, there is a sign at the back hall)

First up: Stefan Koopmanschap, "You Are The Community", 9:30am

A small group (since there are some heavy-hitter talks in the other two rooms), so we're turning it into an open discussion about how to grow the Symfony community.

How do you get people to do more than ask for things on a mailing list?

Give people examples of how to help and how easy it is.

"Even if you're not good at Symfony 2 or haven't started with it, you have the honest and objective opinion of a newcomer and you question things others would not"

"Welcome them to the circle, it's friendly, warm hugs. If you can help with documentation, great, that makes it friendly for beginners"

"PHPdoc is pretty easy." (Yes, I know Apostrophe is deficient in phpdoc (: )

"I can sit with Tom and hug him and now he's a member of the community." "When does the hazing happen?"

"Tomorrow's hacking day is a good time to connect with new community members"

"If you're a normal user you think people on the core are geniuses and how could I ever start?"

Josh Holmes has joined us. "In the .NET community people just wait for Microsoft to do everything (even though there is an open source community in .NET as well). I know of a company that is jumping to Rails because they are tired of rewriting Facebook connectors (but if they shared them in a community effort they wouldn't have to)."

(In response to a question from Stefan Josh clarified that they were using an open source framework supported in part by Microsoft, and on which they would love to see more community contributions) "One in 100,000 are giving back, which is too low. But just the fact that people come to a conference means they are more engaged than 95%"

Ryan: "of course we are connecting and need to make an effort to talk not just to each other"

Ryan: "when you say 'contact me if you want to speak on something,' nothing happens. When you say 'Bob, will YOU give a talk on foobar,' they say 'of course, I'd love to.'"

Josh: "can you man the door at registration, etc."

Heh someone arrived to demand a hug from Ryan

Stefan: "three people get paid to work on Zend Framework, most of us don't get paid for doing this stuff... we need to compile lists of bugs that are good candidates for beginners to help with."

Ryan: "more followup. Here are the 10-15 people who contributed. I want to be on that list!"

Tom: "understand what is realistic for community members to do and what you will have to do. (Or refactor your code so it is more realistic for others to do) We had an Apostrophe i18n bug which I wanted the community to address because we don't use i18n much in-house, but when I looked at it more deeply I realized it required a core team member to resolve that particular issue"

Stefan: "compile a list of the easier bugs"

"Trac is a joke for Symfony2, it's out of date."

"They've been setting up Jira since October."

"Wow, that's a big problem, if I don't know where the issues are how do I fix any?"

"Symfony2 doesn't have a stable API yet, it's hard to classify anything as a bug."

Ryan: "treat your community like a business, organize bug-hunt days, prepare in advance, set expectations: Joe makes a blog post at this point, etc."

"Zend Developer Zone (DevZone) is great." "It's outsourced"

Josh: "start small. Writing a blog post with ten things (for people to do) is a lot less effort than a bug hunt day"

Stefan: "60% (of bug hunt day work) is going through Jira and closing old bugs (that have been resolved)"

Josh: "another good one is unit tests, improving or adding"

"Symfony 2 is really well tested"

"There are lots of gaps"

Discussion of plans for tomorrow's hacking day. (Honestly I need a Paris day (: )

"Make sure everyone who hasn't contributed before contributes something. Get them to go through the motions of a pull request"

Stefan: "from what I've heard so far, a big problem is lack of clarity about what to do to contribute. It's hard to know how to contribute in the first place."

Tom: "put yourself in the casual user's place"

Ryan: "I work for a company that uses Symfony2 but is not Sensio. What's my business case (to contribute)?"

Stefan: "it's giving back, but it's also getting your name out and a recruitment benefit and something to brag about to your customers."

Tom: "so keep their names in when they submit patches or report bugs in insightful ways. Include them in commit notes and ticket notes and use those as a basis for changelogs and new-features lists in which you give people credit for their contributions"

Ryan: "if you come across a gotcha and it takes you three hours to figure out something stupid because of poor error messages, improve the error messages or open a ticket about that issue so the next guy does not waste three hours even if it is not technically broken functionality"

Symfony user groups. San Francisco, New York, Knoxville. Josh suggests we might be able to get space at Microsoft offices. "Inconsistent place and time is death to a group"

Tom: "there are just two or three other symfony devs in Philly"

Stefan: "it is awkward to be at a meeting where 90% of attendees work for one company but you keep hiring them..."

Stefan: "the first thing I contributed was organizing a conference, (the original) Symfony camp. You don't have to be an experienced coder to organize something. If there are any ideas on how to make it more transparent..."

Tom: "the Symfony Live online event last year was good for us, people showed up to join us in our offices. How about organizing something similarly but making it an unconference with presentations by the participants"

Next up: ecommerce discussion

What is out there now: OSCommerce, Magento. Nobody is crazy about them

Stefan: the idea is to start a project analogous to the Symfony CMF project

Tom: define the scope. Is this about a common credit card transaction API? Is it about shopping carts?

Stefan: "we can look at Magento to see what is supported (and used). (Start with) something very basic, very minimal"

"Tricky to make things applicable in different contexts and projects"

[We have ten or so people now]

"Eventually we will want to put the ecommerce solution that doesn't really exist on top of the CMF that doesn't really exist!" "Sounds awesome"

"We've started a Braintree payment solutions bundle" (That is cool! How necessary though? Their PHP API is awfully clean already. It does suffer from some naming convention / autoloading difficulties)

"The user bundle lets me have algorithms inside the user object. The whole controller thing is impossible to adapt" (to what you're doing in your project) "It's not something that should be in a bundle"

"It requires a philosophical change. When you get a published bundle you expect to just plug it in and use it"

"There is an existing ecommerce project"

Vespolina is a project with many contributors including docteurklein who is among us. It is already the second generation version of an existing Symfony 2 ecommerce project (:

Many people are moving to MongoDB, using MySQL for transactions

"The last thing you want is to create dependencies on Mongo or whatever. It should be flexible enough that you can use it against anything. A lot of hosting providers don't provide Mongo"

Ryan: "if it's not dependent on Mongo, in theory fairly new developers who need a payment gateway might run into it and find it useful"

ServerGrove MongoDB hosting is an option for those who find it daunting to set up (beta to start soon).

"The ecommerce solution should do something out of the box." "It's like what happened with Netscape when they first open sourced Mozilla, it wasn't a functioning browser out of the box, so most people gave up very quickly on working with the code."

"EMF - EcomMerce Framework?"

Stefan: "we've found our project (Vespolina). We need to go forth and work on it"

Florian (docteurklein): "we are working on it because we need something open that others will use"
 

 


Fifteen minutes to UserBundle - Jeremy Mikola and many other lightning talks!

 

 




Apologies for missing much of the UserBundle talk, I needed caffeine (:

UserInterface, and a summary of UserBundle's features...

Extend the core AdvancedAccountInterface (expire, lock disable), has caonical email/username fields, unique indexing (MongoDB), case folding (RL slugs), character set normalization for localization The hash algorithm is configurable per account There is a drop-in EncoderFactory service for the Security component Supports migrated accounts from legacy projects without limiting new ones to a weak hash algorithm. Roles are assigned directly or inherited via Groups (as in sfGuard) Confirmation token for email verfication and "forgot password" (I wonder how smooth the user experience is here, there needs to be a next step/thank you message after these things are completed, see sfDoctrineApplyPlugin for my implementation on sf1)

Controllers and commands

Registration, email confirmation, reset password Login form for form_login auth listener CRUD actions for user and group administration (secure these in your config! Can't that be a default? Apparently it was and they removed it?) Console commands to create users, etc. similar to sfGuard tasks

The UserManager class, exposed as fos_user.user_manager extends the UserProviderInterface. Loads accounts for configured Security firewalls Used to create/find/update user objects. Before saving, hash plain passwords and canonicalize email & username fields Customizable thanks to DI Canonicalization, encoder factory, default algorithm and user class can all be injected The default implementation of the user class has base model classes for Doctrine ORM and ODM. You can also implement UserInterface from scratch to avoid subclassing

Love how you can inject this stuff

 

Jordi Boggiano: Monolog

Wanted to replace the zend logger with the monolog library. Zend Framework is big

"Making logging fun" - the motto of the Logbook logging library from Python, which is the inspiration for Monolog.

Simplicity! 4 levels: DEBUG, INFO, WARNING, ERROR

Symfony has too nine levels and nobody knows what they do. "I think 4 is already too much. What is INFO? What is WARNING? There is DEBUG information and SOME SHIT HAPPEN (ERROR)"

$logger->addDebug('message') $logger->addInfo('message') $logger->addWarning('message') $logger->addError('message')

You can easily create your own log handlers that log to files or anywhere else, neat example not verbose at all

Channels: separate Logger objects

$routeLog = new Logger('routing'); $dbLog = new Logger('db'); Etc.

Upcoming: handlers for FirePHP, syslog, mail, jira, ...?

http://github.com/Seldaek/monolog

Slides at http://slides.seld.be

Q. To integrate it with Symfony 2 would you need to implement another interface? A. Not supported by default, that would be a next step?

Varnish - Pierre Spring Pierre Spring (http://nelm.io/) will give a very short introduction to Varnish Cache. Varnish makes websites fly!

When you get a crush of people looking at your football-related site you need a way to cope with crazy traffic

They used Okapi on the original project

HTML <--- Okapi <--- XML back end

The XML back end turned out to be a mess, on top of a database, the ORM was slow (Doctrine 1, not Doctrine 2), all kinds of craziness making things slow

The back end was slow- 60 second requests, 45 second time to live... it's sports... 450 requests per second! AAAA

So they looked at Varnish and put that in front, and cut their losses big time.

Varnish is a reverse proxy. Instead of forwarding requests for a web browser, it caches requests on behalf of a back end webserver.

The cache control header gets set by the backend, Varnish spots that and doesn't force the next user to wait for the backend. They aggregated time to live values and used the smallest.

They "open the doors" five minutes before the match starts and all these people flood in. Varnish only lets through the first request and buffers the others until that first request succeeds, rather than allowing a swarm of people all trying to prime the cache (our feed slot cache in Apostrophe would benefit from a "hang on I'm caching" signal like this).

Your clumsy crap on the back end might break, so "saint mode" kicks in, returning the old data for a while in the event of a 500 error because it's better than nothing. The server is down one minute out of three...

Varnish supports ESI

<esi:include src="http://ucla.com/user" onerror="continue" />

(Could we do a Varnish mode ESI plugin for Apostrophe 1.x to cache normal views of slots more cleverly?)

Why do I need VCL (Varnish Control Language, I think)?

Controlling cookies, multiple backends, variants (languages + gzip), load balancing, ACLs, grace & saint modes

http://nelm.io/pierre
 

 


Redis - Jordi Boggiano The no-buzzword key-value store

Redis is a really simple NoDB store (but don't call it that since it's buzzword-free)

In-memory But persistent, so it's more stable than memcache (?) No schema, no db Comes with sixteen stores by default, numbered 0-15, you can change how many. That's as close to a schema as you get (: Human readable

SET key value SET key2 value2 GET key KEYS k*

KEYS k* returns all keys starting with k.

Multiple data types Key -> Value Strings, Hashes, Lists, Sets, Sorted Sets

Redis is often referred to as a data structure server because it can do hashes, lists, sets...

A "sorted set" has elements assigned scores and ordered that way, the scores can be duplicates

Low level appraoch to storage

HMSET user:jordi name Jordi email jordi@nelm.io SADD users:users jordi SADD users:admins jordi HMSET user:bob name bob email bob@example.org SADD users:users bob SMEMBERS users:users // Returns jordi bob SMEMBERS users:admins // Returns jordi

(What does HMSET do?)

Try an interactive tutorial

HGETALL gets back the contents of what HMSET did (?)

Some more features: expiration, pub/sub, transactions, replication, cluster (soon)

(What does it mean to be "in memory" but also "persistent?")

RedisBundle for Symfony 2 predis for generic PHP

(I wonder if it is simple enough that it would make sense to have a pure PHP stub implementation to speed acceptance– plug in real Redis if you need it... same functionality)

 

Let it flow, let it flow, let it flow! - Jacopo Romei It's a 19-slides talk I prepared today for the Ignite Italia event next Saturday. It's about how ideato manages projects, based on flow economics vs. scale economics. Being very short it will be very breadth-first, not too deep in detail.

Author of Pro PHP Refactoring, Agile trainer at ideato

Old fashioned contracts. Force us to write in uncertainty buffers. Pushes strict feature lists into rigid time expectations.

Low quality. Design loopbacks, perceived & conceptual integrity. "Also gold plating, feature creeping. Damaging perceived integrity of our software"

Dependencies. Waiting on upstream providers and features. Maximum occupation. If you are idle I shoot you... seems wise but isn't because it doesn't leave room for spikes in workload

Critical paths. "The incredible machine: the coupling fair. You shoot the balloon, the cat falls..." Forced to cope with schedules coupled with other actors, waiting for their features to be done. Redundancies not provided for

Skyscrapers are not the best metaphor. "Design, build" does not translate to "code, compile" because designing is cheaper than building every time for a building, but in development coding is the real design and the compile time (basically free in PHP) is the "building," basically free, try over and over again etc.

"The Empire State Building is an exception. It took eleven months. After that we never got a building that fast. It was due to the process they put in place"

Cross-skilled teams were kept together. "Pull scheduling:" they planned backwards from the deadline, simplifying problems before solving them. Builders were involved in scheduling. Allowed for decoupled scheduling: they got different timelines for steel structure, window framework, electrical plant, etc. Completely separated those schedules. They had redundant providers, avoiding hangups on delays from one provider or another– they had a company doing floor 3 and a company doing floor 4 and would roughly leapfrog. And of course the famous on-site restaurants (sky-high beams!) to remove time bottlenecks. Getting 3,000 people moving from post to post repeatedly each day would have cost too much time, they fed you sitting on that beam

No old-fashioned contracts, no more fixed price contracts. High quality results. Integrity is built in. Usability testing and automatic testing should be built into the contract. No design loopback (? I think this means no waterfall process).

No dependencies... use user stories (very atomic ones so you can do them in different orders), wireframes, UX, UI professionals working on the project from day zero on their own schedule

No maxing everyone's time out (by original intent anyway), it's OK to read a book if we're not in a spike. Use pull scheduling, use "kanban board" to determine who needs to be working now

No critical paths. Pair programming to avoid getting stuck. Postponing decisions until last responsible moment. Maximize knowledge of where you are before you make a decision.

Great stuff! Breathless pace I think I coped (:

"Pair programming is a way to avoid that critical path— your brain"
 

A Symfony2 Admin Bundle - Thomas Rabaix The talk will show you how the BaseApplicationBundle works with real life examples

Provides CRUD capabilities similar to Symfony 1 admin generator with no code generation. So there's more metacode.

Admin class = metadata at bout a CRUD interface FieldDescription = metadata about one field, can be list field, fitler field or form field Mapper = the glue between Symfony Component (ie, Form) and the admin Builder = Service which knows how to crawte form field element (for instance)

Demo looks very much like classic admin gen

One config file specifies all of the entities you want to administer...

The admin class enumerates the fields to be administered and so on.

He's showing media selection = the ability to pick existing items for a one-to-many relationship in a friendly way, not just a select element like in Symfony 1 admin gen

It's a bit hard to follow on the projector but it amounts to a code generation-free replacement for admin gen, which I applaud wholeheartedly because it is much more maintainable.
 

 


Can't read screen from back here, going to move up... better!

A Symfony2 Grid - Kris Wallsmith A basic grid implementation in Symfony2.

The idea is a grid of form elements I believe, spreadsheet-ish

interface GridInterface extends \IteratorAggregate, \Countable function createForm($key); function load();

createForm() gives you back a form object that has the current gird set as data.

A sample implementation, ProductGrid, specifies public properties name, deleted, sortField, sortOrder, etc. which map to fields in the form (?).

I need a demo of the end result... a little unclear what I'm looking at

The constructor for the controller takes a grid, etc. which works because this is defined in the dependency injection controller with those parameters.

Some rather clean code to save filter changes

This was a "let's look at what's in my textmate for 10 minutes" talk, and pretty cool in that context, I would like to see a few screenshots of the end result sometime.

 

 


Symfttpd, the embedded symfony server - Fabrice Bernhard Symfttpd is a development server which can be embedded in your Symfony projects. It is fast, easy to deploy and based on lighttpd

Symfttpd = a really light embedded server for Symfony projects. It was written by laurentb on github. https://github.com/laurentb/symfttpd

References Fabien's battle with his Apache settings during a live demo yesterday. The idea behind symfttpd is that it is less painful to provision sites for testing. Rails and Django both have servers written in the native language. PHP only has one which is not well maintained. symfttpd instead uses lighttpd which is a very lightweight web server package that is available pretty much everywhere except Windows. symfttpd creates config files for it painlessly

(He acknowledges that VirtualDocumentRoot is an alternative to this)

He shows how easy it is to just fire up an instance of it on a given port and jump in

"Warning: no fork() support will run in single-process mode" why? Not really a problem for single-user testing since listen() can queue requests

You just make a .symfttpd.conf.php file in your homedir, and say where Symfony is installed for various versions, then in your project a config/symfttpd.conf.php file specifies the wanted version and any necessary lib symlinks. (How does lighttpd fire up PHP? Is it plain CGI-ick slow-or mod_php?)

(Would like to know if PHP gets a bytecode cache in this configuration)

 

 


Dependency Resolution with a SAT solver - Nils Adermann This talk discusses the idea behind the dependency resolution used by zypper, the SUSE RPM installer, which inspired the Bundle Management Tool

SAT? Satisfiability

Breaking down satisfiability problems (to figure out the best way to install dependencies in Symfony 2, following a strategy used in SUSE's RPM installer)

It's an NP-complete problem. No known polynomial time algorithm for all cases. "There are still cases (always will be if P=NP) in which you must try every combination of variables"

With 1000 boolean variables that becomes impractical

How does this apply to package management?

The pool is a set of repositories Each package has a name and version and a set of dependencies: requires this, conflicts with this, provides this (so you can use one of several packages to meet the needs of another) What is the best way to satisfy the prerequisites of a newly requested package?

User wants to: install some packages, update some packages, remove some packages, keep some packages

Shows a complex graph in which a conflict exists between a certain version of a tire and a certain set of window glass for a car. Shows the ideal solution, in which you get the newest of everything except the tire; and an acceptable solution, in which a few things are a version back and the tire is version 2

Package managers that don't solve the problem at all include apt (debian/ubuntu) and portage. Others have poor solutions

SUSE's libzypper is able to solve this problem via SAT, with understandable messages and explanations. Based on ideas from minisat.

The Wikipedia article on SATisfiability is helpful.

Depencency rules become boolean logic propositions. (Hey I wrote a simplifier for these in college, gee I wish I had the source code to contribute (: I'm sure he already has one)

Sometimes the algorithm has to plug in some assumptions to see if they lead to a solution iteratively (again, NP-complete, there's no efficient way to get to the perfect solution).

Keep installed packages installed, do not install unnecessary packages, pick newest version as first assumption.

WOW. Symfony 2 bundle dependency management will be best-of-class stuff. Is that necessary though? How many bundles will your project really have? Wouldn't pointing out conflicts be enough? Well, if we can get good suggestions for solutions from the tools too, why not!

 

Symfony2 Debian Packager - Lukas Smith A short introduction to the Debian Symfony2 packager. The idea is to be able to package an application into .deb, leveraging the OS level dependency management and centralized toolchain.

Lukas is talking about package installation at the system level where you can just ask a sysadmin to install a package systemwide

Their old solution at Liip was in Perl and he wasn't crazy about it

So they are now able to make .deb files and install them with dpkg. (Isn't this a dependency on a bunch of stuff you'll have to install on your non-debian box?)

This means Symfony bundle packages can have dependencies on system packages, like php5-cli. That is nice.

Not sure of the relationship between this work and Nils' work (previous talk)

(Quick break to drink from the sink. Green, primitive, vaguely satisfying, does not require trip to other side of building to look for water bottles that have been in short supply)
 

 


Real time event dispatching - Peter Dietrich This talk gives an overview about how events thrown in symfony can be dispatched in real time to web clients. It describes the architecture of the solution and provides examples using the open source comet server APE

Messaging lets you share data across processes. Processes can be part of different apps.

Messages are sent by Producers Messages are delivered to Consumers Messages go through Channels

There are many messaging patterns. http://www.eaipatterns.com/

Talking about a variety of patterns for message passing.

Publish/subscribe mechanisms make it easy to determine what you want to receive

Request/Reply pattern is good when you must have a response clearly connected to the request. A "reply address" mechanism must exist to get the information back via the correct channel.

RabbitMQ can be used to implement these patterns.

 

I am ducking out to do some mental and professional housekeeping before Fabien's keynote. Other livebloggers, feel free to pick up the mantle and comment here with the link (:

Tom Boutell
Chief Software Architect