Thirty Tools for Web Enabled Research (or webwares of scientific method)

The first thing I should points out is that this listing is about tools for doing research through the web, NOT about doing research about the web (although you can surely use these tools to do that kind of research). Still, you should be able to use these tools to do all kind of research, psychology, computer science, technology, ethnography, social dynamics - from start to finish. This will work with pretty much everything that doesn’t require a physical workbench (sorry mechanics, chemists, biologists etc, you can still use some of these tools though). The other thing is that the focus here is mostly on empirical research.

As I was talking with my students yesterday I became a little concerned that they didn’t have as good a sense as I imagined about how to do research. I was thinking of writing some notes or a lecture or something, but then I came up with this list. The idea is that this list is a toolset to guide someone through a web enabled research process from hypothesis to publishing. Before we get into the list, let’s review what research is, from a fuzzy scientific method approach.

• Step one: Find a problem. This is either a practical problem (motivating collaboration is tough!) or a more theoretical problem (can computers think). This isn’t as easy as it sounds, a lot of the work in this step comes from understanding what is and isn’t a problem, building knowledge and understanding the state of the art in the problem domain.
• Step two: Think of an answer (hypothesize!). A hypothesis is a possible state of the world that addresses your problem. It’s the answer to the question. The answer could be something like “lightweight video conferencing motivates collaboration!” Again, this relies a lot on understanding what is going on in the problem domain. There are lots of possible answers but fewer reasonable answers.
• Step three: Test your answer. You were just guessing before. How do you know if your guess is true or false? This part can be really difficult as most things aren’t so cut and dry. A large part of this is -convincing- through scientifically acceptable methods others that your hypothesis is likely (or, of course, unlikely). That means following methods and analysis methods that make sense and make sense to others.
• Step four: Share your results! You don’t want to miss out on the fame and glory and comes with scientific discovery do you? Even if you don’t care about that you should at least care that others can benefit from seeing your work. It can be inspiration, save time, promote discussion, and lots of other good things.

Part One: Problem formation and knowledge building:

Wikipedia - is the well known collaborative encyclopedia that anyone can edit.
For: Getting a good overview of a topic, key figures in the topic or domain, some seminal research, theories, etc. Also comes with (some) references.
Not for: Reference. Wikipedia is neither a prestigious citation, nor do they encourage citations. If you absolutely have to you can always refer to the permenant page (every wikipedia entry has a hard copy that won’t change)
Also: The ultimate research assistant

Open Courseware - is a movement from MIT and beyond to make first class university materials (lectures, course notes, etc.) available to the public.
For: Building some detailed technical knowledge, some (most likely dated) current research, and a large number of figures, theorems, diagrams, and pointers.
Not for: Getting the state of the art in the research field. Open courseware compiles courses, like undergraduate courses (mostly). These are often fantastic resources but they are also lower level than current research in the field. Remember that old statistics book you have in your bookshelf in case you need a formula? This is sort of like that, but you’ll have to become familiar with the material first.

Part Two: Hypothesis formation, more knowledge selection and organization.

Google Alerts - get email updates when search results change - and update feeds in general
For: Keeping up to date on the state of the art what better way than getting new information right as its indexed. Just remember to carefully construct your queries.
Not for: Overly general topics, you’ll end up getting more information than you bargained for.
Also: Google Trends

Google Scholar
For: Seeing the latest published research in a field, google scholar brings you lots of journals, conference proceedings, and workshop papers. (Hint: If a link takes you to a members only subscription login, try searching the paper title in regular Google. Chances are you’ll find a pdf on the author’s web site.)
Not for: Getting a good sense of the role of the research in the field. Google scholar essentially limits the information about where an article belongs in the field to citations. Citations are useful, but newly published work may be incredibly important and under investigation. Citations still tend to be ~ a year (or more) behind.
Also: The Directory of Open Access Journals

Annotation of web pages (lots of options here)
For: I’m not sure, social bookmarking tools are more my scene. From a personal standpoint they seem better suited to individual note taking.
Not for: Passive collaboration sharing, social bookmarking works better.

Citeulike, Connotea, Bibsonomy, Del.icio.us and the social bookmarking crowd.
For: Keeping references of articles and web pages that you want to review or cite later. Also for passive collaboration streams in communities or between colleagues.
Not for: Detailed note taking.

Papers
Ok, ok. This isn’t a web app. It’s a mac app actually but its such a great piece of software.
For: Papers allows you to organize pdfs locally, with annotation, and perform live searches of research repositories.
Not for: Windows users.

Part Three: Hypothesis Testing and Analysis

Survey monkey (and similar survey software) - to build surveys (screen by screen clickthroughs) with questions and data capture.
For: Doing some basic empirical investigation, particularly with regard to demographic details or ethnographic components of a research project. (Think non-observational studies).
Not for: Doing any kind of experimental studies. Surveys are great and they have there place, but you can’t get too fancy with most systems (including surveymonkey). Web based research is a reality but if you need to do anything visual / auditory / or even vaguely computational you’ll either have to roll your own or use some other services.
Also: The Web Survey Toolbox for a survey toolset that can implement some pretty advanced features (through java)

The Mind Canvas and other remote testing and evaluation tools.
For: Conducting remote usability empirical analysis over the web. Though many of these are most suited to usability testing Mind Canvas and the others could be used to do some distributed experimentation with some fairly advanced capabilities.
Not for: Anyone with a thin wallet. Most of these cost a bit and you’ll probably have to pick up the phone.
Also: Ethnio and the Techsmith line of products.

Randomizer - can generate truly random samples easily and quickly!
For: Being all scientific by selecting subjects, conditions, or whatever with truly random numbers.
Not for: Non-random numbers.

Yahoo Pipes - construct feeds and build applications
For: Although it can serve the update feed function (above) it can also serve as a decent toolkit for building simple experimental web applications. The fact that it has content analysis built in means you can do some basic semantic parsing. There is even some interaction capability offered.
Not for: Data capture is pretty limited and there are strong limitations that make it not suited to everything.

Open APIs (Flickr, GoogleMaps, Yahoo Content Analysis, etc.)
For: When you need to do an experiment you might need to go ahead and actually code something. Present some output get some input, capture some data. That sort of thing. Luckily open APIs like these make this easier and easier to do. There are even services that will do this for you but I find them pretty limited.
Not for: The cost of availability is flexibility, the API is not guaranteed to do everything you want.

Swivel & Manyeyes
For: Visualizing your data, letting you do some basic statistical analysis and some pretty advanced visualization (many eyes has more options for this) to help you get a sense of what all the data means. As an added bonus its easy to share this data with the community and your colleagues to get some feedback.
Not for: Running really detailed statistical analysis. Obviously each site supports different levels of visualization and analysis, but you still might need to break out some serious statistical software depending on what you are doing.

Part Four: Collaboration, Publication, and Lecture!

Tiddlywiki (or any wiki really)
For: You can start using this in the note-taking phase to keep your notes and it evolves nicely into a collaboration tool with your colleagues to organize your research ideas. The next logical step is to open it up to your lab, community or the world and share your research ideas and work.
Not for: Targeted documents like papers or presentations.

Google Docs, Zoho as well.
For: Writing a paper. More specifically these tools allow collaboration that lets you write a paper and share it with colleagues. This enables coauthors to work along side you and readers to make inline edits.
Not for: Detailed typesetting as required by some conferences. You also might still want the more detailed and familiar change tracking available in Microsoft Word, its hard to be weaned off of it.

Think-a-ture - Real time collaboration on the web
For: Working on outlining, big idea presentation details, or hammering out the details of a follow up study with your colleagues.
Not for: Detail work.

Scribd - Youtube for documents (Actually I thought the web was the youtube for documents)
For: Sharing your paper or article with typesetting intact, while still supporting some social networking features surrounding your document. Uses flashpaper to accomplish this, pdfs can also serve this purpose.
Not for: Detailed collaboration or serious commenting. Does that whole ‘viral’ thing pretty well though.

Slideshare and S5 and other slide / presentation software.
For: Keeping a backup of your slides in case of sudden laptop failure (SLF), sending out slides to the audience like you promised, sharing a presentation for early feedback or archival purposes.
Not for: Unslide related activities.

That’s all I have right now.
I’m sure I missed something so I’ve enabled comments on this post. It’s amazing what you can accomplish on the web with a set of (mostly) free tools. There are still some gaps, specifically in terms of experimental design and experiment building but hopefully that will change.