
BNL Search is a Search Engine that indexes the web and allows you to search our database to find websites of interest.
It is written solely in PHP and uses MySQL for storage. Due to the massive amount of records we use Sphinx(1) as full-text engine.
There are several routines on how we collect data. The very first method was simply parsing HTML documents to extract URL's and store all valuable information into our database.
Today it works a bit different, and is due for yet another back-end update.
I will not get so much in to detail on how the BNL Search is built up right now, this information may or may not be published in the future.
1. Sphinx is a full-text search engine, distributed under GPL version 2.