Reddit open source: applications I highlight...

Translation of the original in Catalan. Read it in: CatalàCastellano

reddit logo

This morning Alfredo has informed me that

reddit

has

opened its source code

. Well then, after looking at each one of the components that it has explained it is using I could not wait to go through each of the applications' websites to have a look at how they are. Below I make a list of what I highlight the most, obviously they are the most unusual and least known applications.

  • HAProxy: it is an application very similar to Pound, its purpose is to balance HTTP requests among different servers. Although they are similar applications Pound and HAProxy can work in a combined way. Let's say that HAProxy is much more oriented to large infrastructures and Pound is for medium load environments. The good thing is that we can use HAProxy and behind it use Pound to make more refined redirections depending on the URL or certain parameters of the HTTP headers. They also differ basically in the treatment they make of HTTPs packets, which also invites working with both tools at the same time. Since we could set up a system to scale HTTPs requests in a secure and simple way.
  • Slony-l:it is a master-slave replication system that supports working in cascade and with failover. It is also designed to work with large databases. Basically used by the PostgreSQL that reddit uses, this allows them to have many database servers for reads, which are basically the requests they make to the database.
  • Pylons: it is an MVC framework that uses Python, it is really light and combines ideas from the Ruby, Python and Perl worlds. Perhaps one of the most surprising things is that Pylons projects do not require a web server if you do not want one, since they themselves already implement one with Python, besides having an installer for the project we create just for the mere fact of creating the project. If you already handle some kind of MVC and are a bit familiar with Python syntax you can watch the screencast: Special Edition Screencast: Pylons with Tesla. With this screencast you will be able to get a very good idea of how this MVC works and of its power. It must be admitted that it promises a lot, do not miss the debug system, I really liked it a lot, I thought that Symfony's debug was good, but this one still beats it.
  • Slor: it is a web information indexing server implemented on top of Tomcat that offers query services via JSON and other webservice systems that hugely simplify its integration with other architectures. It also has a list of very attractive features.
  • Daemontools: a really simple and powerful tool, it basically only serves to launch processes as daemons or directly daemons in themselves. The good thing is that it controls these daemons so that if they fall it brings them up again. The truth is that despite being a bit bigger I would say that monit is better. Although it must be said that it takes much less time to set up monit than daemontools and since the reddit people already use Ganglia as a monitoring system they have not complicated themselves with monit.

It must be admitted that this crowd has made very good choices, and it has also been shown that they are really scalable and that their availability is enviable. A good example of how to do things well and simply from the start. The twitter people could take good note. If you feel like it one of these days I will explain my ideas on how to make twitter highly scalable and not sink with the 2M users it has daily, which does not seem like any exaggeration to me, with a good design behind it surely

Pau

suffers much more with his millions of megs to index daily.

3 comments

Comments from the old blog, as they were written. Comments are closed.

  1. Pau

    Nano ja et dic jo que els de twitter han de patir una mica més :/

    Per cert HAProxy pot estar molt bé pero com a codi sap greu dir que es una veritable xapussa, molt millor Pound. Code nice code :/

  2. Oriol Rius

    et proposo fer un livePodcast i et comento la meva idea de disseny per tal de solucionar el problema de twitter a veure què et sembla.

    Quan quedem?

  3. mTorres

    Buf quan he llegit tot el que usen a can Reddit, immediatament t'he imaginat enimg d'una de les reunions que devien tenir els seus tècincs. Discutint quina seria la millor eina a elegir, provant les diverses opcions, estaries a la teva salsa!

    Per cert interessant article, ara necessitaré temps per fotra-li un cop d'ull a totes aquestes tecnologies!