Case Studies
Sources of timing error can be many and varied. They can be constant, variable or even transient. There is little doubt that timing errors are present in virtually all computer-based studies to some degree. Only studies that have been calibrated using an oscilloscope and signal generator are likely to be near error free. When the same paradigm is run on different hardware (which includes presentation and response device), there will be a difference in the timing error. Such differences can make replication difficult and can in some cases lead to conditional bias that can affect statistical significance. Even if you make use of a commercial experiment generator, you should self-validate your own timing as a matter of course. In our opinion, the only tool to allow you to do this quickly and easily is the Black Box Toolkit.

To illustrate the point three examples of timing error are given below.

Example 1: How choice of mouse may effect response timing in psychological studies
Using a digital oscilloscope and signal generator, we tested a range of computer mice (across various interfaces), a typical keyboard and commercial response box from PST. Each device was tested independently of the PC. We found a huge variation in timing characteristics. For example, there was some 60 milliseconds difference between the best and worst performing mouse - simply this is the timing error that would be added to each response time if you used this mouse. In addition its timing was highly variable. We then ran a simple E-Prime paradigm that enabled us to make use of each response device in turn. It was found that the prediction made using the signal generator and oscilloscope held for E-Prime. Simply response timings were slowed by the timing characteristics of the particular device. The graph below show the timing error in E-Prime responses when tested using a rig similar to the BBTK.

The upper track is the worst mouse. Note the absolute difference and the variability. The middle track is a standard keyboard. The consistent 10ms track is the best performing mouse. The fastest device is the PST Deluxe Response Box for E-Prime. The key thing to take away is hardware variation can add a significant effect to experimental results regardless of how good the script of paradigm is!

For a full report see: Plant, R. R., Hammond, N. V. and Whitehouse T. (2003), How choice of mouse may effect response timing in psychological studies, Behavior Research Methods, Instruments and Computers 35(2), 276-284.


Example 2: Cross Modal Priming Using Psyscope on a Mac with an external CMU button box
We examined an active researcher's cross-modal priming study in-situ using kit similar to the BBTK. Their paradigm ran on a Mac using Psyscope together with a CMU button box for timing and response measurement.

In this study a human participant's task to press a left or right button when responding to either a left or right star or left or right tone. Stars to tone were either: Concurrent (0ms lag), 50ms lag or 100ms lag. Tones did not have to appear on all trials. We assessed presentation onset and duration accuracy, synchrony between tones and visual stimuli, tone duration and the accuracy of response registration. We also examined whether using the CMU box for timing made any difference over the internal timers.

When bench marking this paradigm we used three fibre optics stuck to the screen of the Mac. Two high performance microphones and one switch closure line to simulate a response. We programmed DSCAR to simulate a response after 300ms after detecting a star. 300ms was about average human performance under that paradigm.

We found the the actual presentation of the stars was very accurate in terms of duration. Audio duration was slightly longer than desired at around 215ms rather than 200ms. The synchronization error between the tone and star was reasonably good at +6ms. However when we looked at the trials where audio was involved there was a consistent +20ms added to the response time measured by Psyscope as compared with trials with no audio. This was a significant systematic conditional bias. In this case the experimenter had the opportunity to correct the response timing post-hoc for trials involving audio as the error was consistent. However before we checked the paradigm this was unknown and adding a significant conditional bias which was effecting the results obtained.


Example 3: Using TFT Screens and Data Projectors Versus CRTs
Using a digital oscilloscope and simple visual presentation paradigm written in E-Prime we evaluated the contribution made to display timing by use of different display devices. We tested a range of TFTs, a data projector and a standard 19" CRT running at 100Hz (10ms screen redraw).

The E-Prime paradigm simply wrote an LPT port out signal for 100ms (upper track), then displayed a bitmap for 100ms and then a black screen for 100ms. We calibrated the paradigm so that the image was displayed for exactly 100ms on the CRT. Then we proceeded to swap through a range of TFT screens with various interface types (DVI or analog D-SUB). Finally we tested a Sanyo data projector.

Typically we found that all TFTs were slower to begin to display the image by around 40+ ms and that they were slower to stop displaying the image. We also noted that images typically were displayed for 20-30ms longer than they should have been. Below we can see a trace for a typical TFT:

  • A: Time from port out signal to start of display
  • B: Warm-up time for display
  • C: Cool down time for for display
  • D: Total display time

    Note: B+C should equal the panels response time in milliseconds.

More crucially we found that the warm-up time for the projector was very slow at around 100ms and that again display time was longer than desired.

This suggests that if display accuracy and synchronicity is crucial to your paradigm then you should either use a CRT or check the timing. For example with the Black Box Toolkit you could use a data projector if you started the visual presentation some 100ms earlier than a sound stimulus and made the display time some 40ms shorter. Then the display would be exactly 100ms and it would be synchronised with the audio presentation.