System parameter experiments

Three parameters of the software system underwent experimentation to determine optimal settings: Pseudosentence length, block (or `pseudoparagraph') size, and trough threshold.

Pseudosentence length refers to the number of words (tokens) in each pseudosentence in the system. As the system compares two overlapping blocks, each containing block_size pseudosentences, effectively the pseudosentence length only controls the size of the overlap between the blocks, and the step size taken by the `rolling window' functionality of the system. For example, a pseudosentence of length 1 has an overlap between the two blocks of only one word, and will step its current window forward a single word at a time.

Block size refers to the number of pseudosentences to compare during each comparison: It affects the total size of the current window (that is, the scope inside which words within the dialogue have any effect) - this window is always $(2 * block\_size) - 1$ pseudosentences in length.

By modifying block size and pseudosentence length, the proportion of overlap between the comparison blocks, the size of the step forwards between each comparison, and the total size of the window under comparison can all be modified.

Trough threshold refers to the sensitivity of the through detection algorithm used to locate potential topic breaks. It dictates a percentage value indicating how deep a trough must be before it is marked. This percentage is a proportion of the total range of the continuous similarity data, from lowest value to highest, across the current document. A trough will be detected if its lowest point has peaks to both sides higher than the specified percentage of the total height of the graph.