Waffle – A schema-less data storage on top of RDBMS

This was Pycon India‘s First edition and was a two day event in IISc Bangalore. Lot of interesting talks and discussions happened in the conference. I proposed about an open source project I was working on – Waffle.

Waffle is an Open Source Python library for storing data in a schema-less, document oriented way using a relational database. This was an Open Source clone of a similar datastore that was used by FriendFeed as mentioned in the CTO Bret Taylor’s Blog.

The data is stored in tables in a single column as either JSON or Pickled strings. Any keys in the data can be indexed in a separate tables. This library was built on SQLAlchemy 0.5 and can work with SQLite, Postgresql, MySQL and other popular Database systems.

Currently, the project isn’t under active development and library probably wouldn’t work with the latest version of SQLAlchemy. Of course, the database systems has evolved so much, that the need for such a library is non-existent. Databases support JSON types natively and you can index on individual keys to make querying efficient.