Genau
A cloud-based search engine built from scratch and deployed on Amazon EC2. A multithreaded crawler that respects robots.txt feeds a custom distributed key-value store, and a Flame batch engine runs a TF-IDF and PageRank ranking pipeline.
The query front-end returns ranked results in well under a second, with key-value lookups parallelized and ranking values pre-computed so they stay cheap at query time.