Search for database driven site

software development
#1

Hi, I’m developing a dynamic site and is wrestling with how to implement a site search.

As the site is database driven, I will need to crawl my site in order to create a searchable index. I’d like to use existing solutions (open source preferably).

Has anyone implemented a search for a dynamic site? What product(s) did you use? How did you implement?

Suggestions are also welcome.

Thanks,
Pelle

#2

I just want to raise the possibility of leveraging Google search (or any other search provider, for that matter).

Oversimplifying, you make sure that all your content gets crawled by the googlebot, then you just embed Google’s site search widget in your site at appropriately.

The big issues are:

  1. Make sure the crawler doesn’t get lost.
  2. Make sure the crawler doesn’t overload your site.

Obviously, this doesn’t work well for site content that isn’t navigable.

Use the [color=#CC0000]3DOM50[/color] promo code for 3 extra lifetime domains and $50 off
More Dreamhost coupons here!

#3

This doesn’t make any sense to me. If the site is database-driven, there is no need to “crawl” in the way a search engine needs to. You can query your database for any search term. All you would need is a simple form that can accept some search parameters, then after washing it through a few security checks you can query the database and spit out a results page to the user.

– si-blog –
Max discount on any plan with promocode SCJESSEYTOTAL

#4

The main reason I’m looking to use a web crawler is that it makes it fairly uncomplicated if you make changes or add new areas to your site. With a http crawler, what the users can browse, will also be found/indexed and searchable without modification to queries/code.

Any way, after some more research, I found perlfect (http://www.perlfect.com). I installed it and initial tests seems promising.