Comments on: Fulltext Indexing Wikipedia with Sphinx

By: Hank

Hank — Sun, 16 Sep 2007 04:17:00 +0000

@Andrew:
Thanks for responding so quickly. I updated the post with my latest test using searchd per your suggestion. You were very right – I can’t believe the speed on this thing. Thanks for building it.

By: Hank

Hank — Sun, 16 Sep 2007 04:17:00 +0000

@Paul:
Excellent plan! Thanks for the credit. I am pretty busy so I haven’t had time to pursue this on my own, but I’m glad you’re up to the task. Good luck!

By: David Gerard

David Gerard — Sat, 15 Sep 2007 22:17:00 +0000

If you can beat this into a MediaWiki extension, please put details on http://mediawiki.org/ and the mediawiki-l mailing list. The default MediaWiki MySQL full-text search is literally worse than useless; Wikimedia sites use a Lucene variant; more options would be most welcomed.

By: Andrew Aksyonoff

Andrew Aksyonoff — Sat, 15 Sep 2007 22:17:00 +0000

The actual search should be faster than that – CLI search has a lot of preload overhead which is not there in production mode when using searchd (which preloads data only once at startup) – especially when it warms up.

By: Paul Grinberg

Paul Grinberg — Sat, 15 Sep 2007 22:17:00 +0000

Just to second an earlier post, a MediaWiki extension to enable full text searching with Sphinx would be excellent.

By: Paul Grinberg

Paul Grinberg — Sat, 15 Sep 2007 22:17:00 +0000

Just wanted to point out that I started work on integrating the Sphinx Search Engine into MediaWiki. See more at http://www.mediawiki.org/wiki/Extension:SphinxSearch . I still have quite a ways, to go, so keep your eyes on that page.