SOLR

Wow !

If you have worked with Lucene, then you owe it to yourself to check out SOLR. It's just about to graduate to a full fledged apache project and it's a doozy. Basically take Lucene, add a REST'ful interface on top, basic web administration and goodies like replication & caching. What makes this cool is it is usable by both Java and non-Java tools. So you can run SOLR for your search engine but have a Django based front end. Did I mention it powers the search for some major sites - like shopper.com and was donated by CNET.

Just post xml docs following a dead simple format to SOLR and it takes care of indexing them. Post another xml doc to query it and it shoots you back the search results.

I'm working on scaling up www.getlocalbiz.com and SOLR will definitely be a part of it (we are already using Lucene for search)