Internet Search FAQ: Archive

Past additions made month by month to the Internet Search FAQ.

Archive Index:

2001 1999-2000 1998 1997 For the most recent changes go to What's New
Return to Internet Search FAQ or URLs for a rainy day

Most recent at the top. Click on the bouncing to return here

August 2003


1. A fun way of looking up words is the Visual Thesaurus, thought up by those clever people at Plumb who designed the "think-map" I've written about before.

Described by Matthew Mirapaul of the New York Times as "Inventive. Imaginative. Ingenious. Fanciful," and recently featured in the Economist, the online version of the Visual Thesaurus has (the press release tells us) become something of a cult favorite to people who frequent the Web.

For once the press release is right. Use the free online version (or buy the CD-ROM) and when you enter your chosen word it comes up on screen at the centre of a three-dimensional web of related words. You then click on the words or on nodes between them to bring up further related words, sample sentences using those words and definitions. Double-click on a node to bring up a tool for dragging the whole design around to bring key words up into the foreground or demote them into the background. Interactive fun for all.

2. Sad news comes from Jean Manco ODP volunteer editor at the Open Directory that Clearinghouse is no longer being actively maintained. However it is still valuable for value-added searches especially if you are looking to do depth research on a subject and want to be pointed towards resources that have been recommended by their reviewers. Watch out for links that may be out-of-date however.

June 2003


The Internet Search FAQ is back after two years absence - primarily because I've been busy making and finishing a movie - Paradise Grove - www.paradisegrove.co.uk.

But I'm delighted to be back in operation. There's a fair backlog of link checking to be done still, but feel free to get in touch if you find a link that doesn't work or a new site you'd like to recommend, or anything else.

In particular, if you find a website that links to the Internet Search FAQ at the old address (now defunct) please do us all a great favour and tell them the new address: www.search-faq.com.

June 2001

1. Summer Tip

When you are searching for information, give yourself a moment to consider what kind of resource might hold the information you want.

Is there anyone who would consider this important enough to spend time and effort writing about the subject you want and putting it on the web?

Most pages are compiled by (a) commercial and other official organisations (b) special interest groups (c) enthusiasts.

So, let's say you're looking into the history of surfing (the wet sort that you do in the sea). You'd expect to find companies and government backed tourist authorities, covering equipment, travel and where to stay. They may well add extra background information on their site to entice Net visitors to hang around - possibly including the answers you're looking for.

Similarly, you'd expect to find a fair number of web pages set up by clubs and associations, and not a few personal pages by surfers.

But what if your subject is more difficult. Say, you want to find out about corrupt politicians in the 1990s. Who would want to put up a web page covering the particular sleaze-bags you're digging for dirt on? Probably not commercial companies or government backed tourist authorities!

It may be there's an appropriate pressure group. Or an enthusiast with an axe to grind.

If you're drawing a blank, think of who might feel passionate enough to create a page about your particular subject and use use appropriate terms in your preferred search engines.

But it may be that there isn't one. By no means everything you might want is written up on a web page.

In that case, think laterally. Try to find a Newsgroup that might attract readers who'd know the information you're looking for - searching for related terms using Deja is a good way to start here.

Click here for more information on using Newsgroups and how to post queries on them.

Many of the sections on our links page also have specialist sites that can help in such cases.

2. Edo Jansen writes to recommend ixquick as "the best metaengine of this moment. Tested as best of the world by Zdnet."

3. A clever new variation on human-based research resource, SourceNet is a private research tool that lets journalists post anonymous queries on any topic that will be distributed daily to nearly 10,000 corporate and agency PR professionals.

Journalists worldwide use the service to round out stories, find guests, test new story ideas, or find expert sources for stories in progress. SourceNet say, "It's like having an army of research assistants helping you, for just a minute or two of effort on your part! All queries are completely anonymous; personal contact information is never available to PR people through SourceNet queries."

Free to all working members of the media.

New Links

January 2001

The 'Invisible' Web
When conducting a search on the Net, the standard search engines and directories are always going to be the best place to start. And for most purposes they can be very good indeed. But you should also be aware that there is a great deal of the Web that they are totally unable to search.

This "unsearchable" part of the web is sometimes called the "invisible web" or "deep web". It exists for a number of reasons. Most important is that many excellent databases cannot be searched by the search engines' automatic software ("web spiders"), either because the spiders cannot access the databases, because otherwise ordinary webpages are constructed in ways that interfere with the workings of the spiders for technical reasons.

Frames and dynamic pages interfere with the way that the web spiders access information and return useful addresses for the search engines to use. And the text content of images or Adobe pdf files cannot be examined. In addition, some databases simply won't work with spiders or refuse access.

For example, you can't search on a phone number in Alta Vista and get an answer from Anywho.com. Financial databases, newspaper archives, government information, almost every kind of resource or database is affected by the "invisible web" problem. If you rely solely on search engines, you are limiting your resources to a tiny fraction of what is out there.

The best way to deal with this is to have a good supply of databases which are specific to your subject. These will lead you faster and more surely to the information you need than any search engine, which will on current estimates only cover 1/500th of the 500 billion pages now on the Web.

Many of the links on our links page will return information that search engines cannot find.

Other useful strategies include searching Usenet and other discussion groups, posting queries on discussion groups, and using expert resources.

There are not many tools for researching the Invisible Web, but two of the best include:

www.invisibleweb.com - developed by Intelliseek
www.completeplanet.com - 20,000 approx invisible web databases

Further information on the Invisible Web (and links) can also be found at these two sites:

http://www.searchenginewatch.com/sereport/00/08-deepweb.html

http://websearch.about.com/internet/websearch/library/searchwiz/bl_invisibleweb_apra.htm

Thanks to Steve Hunt to pointing out the Invisible Web issue.

2. Sadly, after Wayne Lutz wrote so enthusiastically about News Ferret...

... it went and died. Indeed all the little Ferrets (Phone Ferret, Email Ferret, etc) are now gone except for the original Web Ferret. You can find it as always at http://www.ferretsoft.com

3. Kip Campbell recommends a business research site - Esearch Central: "A great source for market research reports, white papers, business intelligence including a directory of business report sources and a forum for posting requests for information."

4. On image search engines Trygve Lode writes: "Personally, I'm most impressed by Webshots, and my second-place pick is Ditto."

And going on to more general searchers, "Lately I've gotten a lot of use from AllTheWeb, especially for tracking down copyright violations and unauthorized use of my own work."

Plus: "New, but promising is FindSame (and perhaps also useful for tracking down unauthorized and unattributed use of copyrighted material); worth a look to see how it does as it maps more web content."

Other New Urls

© Charlie Harris 1996-2003

(This FAQ may be copied in whole or in part for non-profit making purposes only, provided you tell me you're doing it, adequate credit is given to those who helped towards it and the home address is given http://www.search-faq.com. In addition, if anyone wishes to post any part anywhere on the Net, you must commit to keeping the posting up-to-date on a monthly basis - there's too much out-of-date stuff on the Net already. Our Disclaimer which disclaims responsibility for everything in the entire universe can be found in the main FAQ)

Back to top

Internet Search FAQ
URLs for a rainy day
Latest Additions