Are Subject Indexes dead or dying?

Well, full-web subject indexes always were a bit of a tilt at windmills. It's not fiscally possible to hire enough people to index the Web, and the Open Directory Project tends to demonstrate that people rapidly weary of trying to do it manually. Proper indexing is hard work!

What tagging and social bookmarking offer is a loose, messy, patchy way of using what people do anyway (that is, bookmark things of interest and give them a mnemonic to remember later) to approximate a subject index. It's not perfect, but let's face it, neither are our subject indexes. And it carries the not-inconsiderable benefit of using users' own language, not language imposed on users by the dictates of a controlled-vocabulary designer.

There's a lot we don't know about how to troll through a folksonomied system for items of interest. I have this problem regularly when I try to grab Creative-Commonsed photos off Flickr to use in slideshows -- I have to work pretty hard at guessing what tag will get me a photo I like. Clustering techniques help (they can both disambiguate homonyms and suggest related terms), and del.icio.us uses them to some extent, but to my mind not enough.

In my dreamworld, there would be a better integration between keyword searches and controlled-vocabulary-based browsing. In my dreamworld, a keyword query would analyze results to see what CV terms come up oftenest, and suggest them to the user as good places to start. (Open WorldCat sorta does this, but not very effectively. NCSU's Endeca catalogue is a little better. No journal database that I know of does it.) But that would require clueful vendors, and when have libraries EVER had those?

Social bookmarking tools generically are more like search engines than subject directories, great for finding but less for directed browsing. They do, however, provide a really easy way of making personal (or institutional) subject directories. So, basically, I don't think they obviate the need for subject directories in the least and, in fact, compliment them well by serving as an easy creation tool.

Yahoo! has hid most of its subject directories. That is telling because, in the beginning, Yahoo! was more directory than it was search.

Tagging really requires taggers though. If you have alot of taggers, then your folksonomy is going to approximate effective browsing. If you don't, you will need some kind of classification system and authoritative is better than a folksonomy where folk =< a critical mass.

I don't know the literature on this, but I wonder if any study has been done regarding how effective tagging is for people who don't tag, but use tags to navigate through the web. One example is that I see "GTD" in del.icio.us. Does your average Josephine know that GTD means "getting things done"? Even more important does Average Joe realize that an "Asian" tag could very likely lead you to porn?

-Ryan

Part of my current job is as a "subject expert" in Ethnic Studies and Psychology. As such, I believe that print indexes are dead and mostly useless. I think that it will be a variety of online tools that replace indexes, including well indexed databases, tagging, subject search engines, etc.

I am also an instruction librarian and in most classes I have to explain WHAT and index IS. Younger students have no point of reference (besides an end of the book index) with which to compare tagging. To them, tagging just is. Just as technology just is.