Inter-annotator agreement

Of the four annotators who volunteered for this project, each seemed to have a slightly different pattern of topic marking. Number of topics located per page varied, as did philosophies of where exactly a fading topic ends. However, as each document was segmented only once, this cannot be analysed.

One annotator (not a computational linguist) stated that she began to develop a pattern for locating topic change: Essentially using cue-phrases such as ``so'' and ``anyway'' as a way to add accuracy to her location of topic changes. This is interesting as it recalls strongly some computational methods for topic segmentation, and also highlights the subjectivity and difficulty of detecting topic change by true understanding of the discourse flow.