I had the pleasure of talking with Jay Kreps yesterday about Project Voldemort, here some notes from this conversation.
- Jay is a cool guy, practically bounces off the walls when talking about Project Voldemort. I think I am like that sometimes when talking about things I am working on.
- Project Voldemort is the open development of something that was created to serve a need that LinkedIn had. LinkedIn is supporting the project and supporting the move to make it open.
- There is no apparent short term plan to turn Project Voldemort into anything large, such as a spin-off company or a cloud based service.
- Voldemort is used extensively at LinkedIn for online queries. Volemort is not used for analytical queries, data is instead fed into a data warehouse for this purpose.
- The primary focus of Project Voledmort is scalability and predictability of performance. Voledmort is a key/value store, not document orientated storage.
- Project Voldemort uses a hashing algorithm to allow any node to know that location of data on any other node. This avoids the need to have centralized lookup nodes (which results in multiple hops and potential bottlenecks).
- Project Voldemort doesn’t use REST. During testing a REST interface was found to have too great a latency and instead Project Voldemort has its own native driver interface.
- The name of course comes from Harry Potter. For those not familiar, Voldemort’s sole was broken up in pieces and to kill Voldemort Harry had to kill all the pieces. Project Voldemort aims to mimic the spirit of this, to have scalability but with no single point of failure.
Innovations in information management
