Ask Metafilter has a good discussion on how to take notes when reading. The readers have suggestion couple of tools such as paper notes, index card, personal wiki, software and so on. It is interesting to see many people who uses similar tools but different note taking method. For instance:
… 1. In part, I credit the speed with which I finished my dissertation (in English) to limiting the number of books I used–I rarely had more than, say, four or five books out from the library at once. Fewer books to consider simultaneously mean that more material from those books will stick in long-term memory, which cuts out time wasted looking up things over and over again in books you don’t really remember because you read them too quickly the first time. In this case you may find that you don’t need a notetaking “system”–I certainly didn’t have much of one. I just remembered where important bits of text were when I needed them. (But in my case the research and writing phases of my dissertation were simultaneous. I’m not sure that, in English, there’s much need for a dissertation “research phase.”)
2. During my usual reading process I just use index cards for bookmarks–when I’m done with the book, the index card (or cards, on very rare occasions) stay in the back of the book and the book goes on the shelf. Try to limit the notes you make to a single index card–that way you’ll stick to the most important points. Not everything that you think is important at the moment you read it is actually important, or even meaningful–this is doubly the case for much academic prose…
Many people do prefer index card for taking notes when reading – because it is more portable and can use it as bookmark.
How do I take notes on big books – [Ask MetaFilter]