Scaling Etsy: What Went Wrong, What Went Right
Ross Snyder will be speaking at Surge this Thursday/Friday in Balitmore, on Scaling Etsy: What Went Wrong, What Went Right.
About
At Etsy, our mission is to enable people to make a living making things, and to reconnect makers with buyers. The engineers who make Etsy make our living with a craft we love: software. This is where we'll write about our craft and our collective experience building and running the world's most vibrant handmade marketplace.
Code as Craft is proudly powered by WordPress.com VIP and the SubtleFlux theme.
© Copyright 2013 Etsy
Sounds interesting. Any chance for slides/audio/video to be posted somewhere?
Hi,
saw a write-up of Ross’ talk on arstechnica: http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars
I know you guys sometimes post vids/slides of the talks, any chance of similar treatment?
Just got linked to the arstechnica article from the Etsy Forums and I must say I feel for you guys.
Since joining Etsy, I was pleasantly surprised by the amount of innovation in the site’s design and use of technology but I was not aware of how much that had hurt you in the back-end.
I’ve been part of environments where the scalability of the back-end had limited the front-end for a period of 15-20 years. (Back-end code was written in Cobol). For these type of companies, there is not other choice but to rewrite the whole thing (they just keep avoiding that step).
Clever ideas may come at a high price in the end but on the other hand, without having that ‘unique’, ‘risky’ factor, I don’t know if I would have the level of respect I do for some of the developers at Etsy.
I watched the talk carefully. It seems Etsy started with PostgreSQL and ends with collocating PostgreSQL and MySQL and the intention to phase out PostgreSQL entirely.
I’m very curious to learn the hard facts behind this move. Is it sharding, master-master replication, or something else that PostgreSQL has failed face flat compared to MySQL (in 2011 and beyond)?
Jerry: yes I think the primary reason is that our sharding scheme relies completely on MySQL’s master/master replication. And we have much more expertise in-house with MySQL.
There are probably totally valid ways to shard using postgres (instagram is doing this: http://bit.ly/o2DX0G) but the pedigree of our shards comes straight out of flickr/friendster/etc and is MySQL specific.
I think we owe the community a more thorough explanation of how our sharding works.
I’d love to hear more about how you’re sharding/replicating on MySQL. Additionally though, I don’t feel you owe the community a damn thing, but I’m grateful that you open this sort of information up to the world. Thanks for it.