Friday, June 19, 2009

As I already mentioned, the main focuses in developing TripTrap were the speed, scalability and easy clustering. Some introduction to technical details now. 

There were some "not very common" decisions made while developing TripTrap.

  1. To improve performance we store binary serialised Java objects in DB and memcached as well. That's fast, easy and universal for all kind of objects (in fact, TripTrap can be used not only for flight searches). The drawback of such approach is that DB is not self-sufficient, one cannot search anything in DB without TripTrap --- DB is a dumb persistent storage in our case. Nevertheless, up to the present moment we have not experienced any problems with that. One more reason: we try to move the work from DB to memcached as much as possible, and memcached is just a map, so we try to avoid involved data structures.
  2. We use InnoDB engine for MySQL. Sounds a little strange, but we have a lot of read-write activity, also tests showed that MyISAM doesn't differ significantly. 

The most tricky thing in cluster environment is the synchronisation. In TripTrap it's done through memcached entries, thus we have relatively small overhead.

We can say TripTrap is good enough in what it made for. Anyway, there's still issues I'm not satisfied with:

  • Load testing: it's done only partially and needs more test cases.
  • More acceptance tests: a couple of dozens exist, but should be more.
  • Threads: more parallel jobs should be used (for example, 16-core servers are more or less common, so TripTrap must use all available resources).
  • Source code tree structure: it's time for review and moving things between packages and simplifying the stuff. We aimed at keeping the project structure as clear as possible.
  • Documentation. Yep, TripTrap suffers from it too.
  • Criticism. There is no ideal systems and solutions, no silver bullets and so on. Usage experience, testing and everything that can help us finding TripTrap problems to make it better. Or ideal :)

TripTrap is getting the feature of multi-point search currently (the search with not only origin and destination, but some intermediate points). That's a kind of challenging feature as it involves extraction of large amount of data to process and filter. Will be ready soon.

So it goes.

No comments:

Post a Comment