Using The Black Box ToolKit v3: A worked example


We appreciate that many researchers will not have formally checked the millisecond timing accuracy of their psychology, neuroscience or vision experiments before and that doing so can seem a little daunting. With the easy to use Black Box ToolKit v3 hardware and software checking your own experiments timing accuracy is more straightforward than you might think.

View the BBTK v3 as a virtual human which can respond to any stimulus with millisecond accuracy. Running in this virtual human mode you can program it to respond to patterns of stimuli with a known reaction time you specify. Once running you can leave the BBTK v3 to step through your whole experiment as though a reliably accurate human was sitting in front of your computer. This is done in a noninvasive way using a range of external sensors and response devices so that you can check your own experimental scripts on your own equipment in-situ.

    1. Set-up – Hook up external stimulus sensors (opto-detectors, mics, TTL) and a response device (response pad, robotic key actuator, sounder, TTL)
    2. Test – Use a wizard to select a stimulus pattern to automatically respond to/event mark. Then choose the exact response time, response device and duration
    3. Analyze – Check your stimulus, response and synchronization timings across up to 36 channels. Compare what your experiment recorded with what the BBTK v3 saw

Other models let you event mark stimulus events with TTL triggers in real-time, e.g. for EEG studies, or you could simply monitor events as the BBTK v3 (event marking version) takes over the timing duties from your experiment generator. Whatever mode you use the BBTK v3 in you can be sure that you are measuring presentation and response timing in the physical real world and not just recording times when your experiment generator thinks stimuli are being presented and RTs recorded.

To work out timing error in your experiment simply compare what the BBTK v3 recorded against timings from your experiment. Once you know where the errors are you can tune your experiment generator to produce more accurate millisecond timings. Self-validation of timing accuracy is the future for published research.

Worked example

Shown below is a simple visual stimulus reaction time experiment where we are using a response pad together with a BBTK v3. It is intended to give you a flavor of how the BBTK v3 could work for you and how easy it is to use.

Remember the BBTK v3 could be wired to your own response device in order to trigger that. Plus you could choose to have more monitoring sensors (opto-detectors, mics, TTL inputs etc) and more outputs from the BBTK v3 (sounders, Robotic Key Actuator, TTL event markers/TTL triggers etc).

When carrying out any experiment you should ask yourself:

  • Are you always carrying out the experiments you assume you are?
  • Are you aware of millisecond timing error in your own experiments?
  • Are you confident you can replicate experiments using different hardware and software in another lab?

The key question you should be asking yourself is, "Am I confident in my findings and would I be happy for a researcher in the same field to independently check my experiments?"

Example paradigm

A simple visual stimulus is presented once every second with a blank interval of one second. A human participant's task is normally to press a response button as quickly as possible and then release it.

Questions we should ask

  1. Are visual stimuli presented for one second?
  2. Are interstimulus intervals (ISIs) accurate at one second?
  3. Are response times (RTs) recorded accurately from the first frame of the visual stimulus?
  4. Are response durations accurate?

How we test

A. Opto-detector
An opto-detector, or photodiode, is hooked up to a monitor where a visual event marker will appear (black to white block transition). This will physically monitor for the onset, duration and offset of each visual stimulus.



Opto-detector placed on black/white block

B. Active Switch Closure
An Active Switch Closure lead is plugged into the BBTK USB response pads external button socket 1. This lead could actually trigger your own response device, or one of our Robotic Key Actuators could be used to press a key on a laptop of screen region on an iPad. This signal could also be a TTL output/TTL trigger from the BBTK v3 if required.

C. Select target timings using wizard
Using a wizard enter the target response time (RT) and key down duration. In this case we are using the Digital Stimulus Capture And Response wizard. Target RT is 300mS and button down duration is 100mS. These are our benchmark figures.



Easily enter target timing specifications using a wizard


D. Start experiment
Start the experiment generator and click start on the BBTK v3 software. Let the BBTK respond to each stimulus presentation and step through your experiment as though it was a real human participant.

E. Analyze true real world timing data
Analyze collected timing data using our easy to use PC software. Either look up timings in the Line By Line spreadsheet view or use the moveable cursors to check on individual timings.



The BBTK v3 PC software in action




Easily read off timings such as opto-detector onset, offsets and durations in the spreadsheet view




Use the quick graph to chart any columns in the spreadsheet view




Analyse data in the 20 Channel Logic Analyser

Answers to our original questions

1. Visual stimuli
Visual stimuli are not presented accurately for one second as intended as they vary by up to two refreshes.

2. Interstimulus intervals
Interstimulus intervals (ISIs) are not one second as intended as they vary according to refresh inaccuracies in the monitor used.

3. Response times
By comparing the response times (RTs) recorded by the experiment generator against our target RTs we can see that the true RTs are radically different. This is because the experiment generator times responses from the time it requests the stimulus image be displayed and not when it actually appears in the real world. If you think about it, how can a piece of software know when a visual stimulus appears physically? Remember the target RTs were all 300mS.



Response time error that is due to the hardware alone


4. Response durations
Response durations are not 100mS as intended due to a combination of response times being measured from the wrong start point by the experiment generator, i.e. not the point where the stimulus image appeared in the real world, and because of error introduced by using a standard response device combined with the accuracy and precision issues within the experiment generator.

In this example visual stimuli were not presented as intended and RTs were not recorded correctly. You should note that the figures in this example have been kept deliberately simple. Remember that the variations recorded are purely the result of equipment error as the BBTK v3 consistently fed in RTs of 300mS.

It is likely that your own paradigms will contain much more error than in this simple example!

Software suite

The Black Box ToolKit v3 comes complete with an easy to use software suite that will run on any modern version of Microsoft Windows:

Microsoft Windows XP SP3 Vista SP2 (x32/x64)
Windows 7 SP1 (x32/x64)
Windows 8 (x32/x64)
Windows 8.1 (x32/x64)  
Windows 10 (x32/x64)
Windows 11 (x64)

Once connected via USB you can control the BBTK v3 via intuitive wizards, copy and paste data to Microsoft Excel and analyze millisecond timings in our 20 channel logic analyzer. Up to 36 input and output lines across a range of sensors let you monitor events, generate outputs or TTL event mark/TTL trigger with ease. For an overview of what you could achieve, read a summary of each module below.



The main BBTK v3 interface – analyze timings using the spreadsheets or
use the 20 Channel Logic Analyzer and moveable cursors



Module Which input sensors & output lines can be used (Elite model) What can it do
Digital Stimulus Capture (DSC) INPUT: 4x opto-detectors, 2x mics, 10x TTL trigger inputs, 4-8x response pad buttons Using DSC the BBTK v3 can be your millisecond accurate silent observer. In this mode it simply collects all timing data across a range of sensors enabling you to analyze it. Quickly and easily see what happened in any experiment. Using this module you can easily replace the timing duties of your experiment generator and let the heavy lifting be done accurately and reliably by the BBTK v3. If you wish to collect timing data and also TTL event mark/produce TTL triggers you should consider the mBBTK v3 (event marking version).
Event Generation (EG) OUTPUT: 4x Active Switch Closures, 2x sounders, 10x TTL trigger outputs, 1x Robotic Key Actuator EG lets you generate a series of variable events with known timing characteristics across a range of outputs. For example Active Switch Closures can close a key on your own response device; sounders can trigger your own voce keys; standard 0/+5V TTL signals/TTL triggers generated or a Robotic Key Actuator can press laptop keys or touch screens, e.g. iPads etc. It could be used in finger tapping studies for example.
Digital Stimulus Capture And Response (DSCAR) INPUT: 4x opto-detectors, 2x mics, 10x TTL trigger inputs, 4-8x response pad buttons

OUTPUT: 4x Active Switch Closures, 2x sounders, 10x TTL trigger  outputs, 1x Robotic Key Actuator

DSCAR is at the heart of the BBTK v3 whole ethos. It acts as a virtual human and steps through your experiment recording stimuli and simulating responses with known timing characteristics. In short it is a combination of DSC and EG modules.

With an easy to use wizard select which stimulus patterns on which sensors should trigger a response and then choose a response along with a variable target response time and duration. To check the timing accuracy or your computer and experiment generator compare what the BBTK v3 saw and did with what your system thought. A classic example might be a simple visual reaction time experiment where the BBTK v3 responds to a stimulus after say 300mS and holds a button down for 100mS to represent a button press.
Event Generation Pulse Train (EGPT) OUTPUT: 4x Active Switch Closures, 2x sounders, 10x TTL trigger outputs, 1x Robotic Key Actuator EG lets you generate an unlimited number of fixed events with known timing characteristics across a range of outputs. For example Active Switch Closures can close a key on your own response device; sounders can trigger your own voice keys; standard 0/+5V TTL signals/TTL triggers generated or a Robotic Key Actuator can press laptop keys or touch screens, e.g. iPads etc. It could be used to simulate a regular pulse such as that from an MRI scanner via TTL lines.
Digital Stimulus Response Echo (DSRE) INPUT: 4x opto-detectors, 2x mics, 10x TTL trigger inputs, 4-8x response pad buttons

OUTPUT: 4x Active Switch Closures, 2x sounders, 10x TTL trigger outputs, 1x Robotic Key Actuator

DSRE is very similar to DSCAR the virtual human except that it doesn't record any timing data but simply responds to stimuli with known response characteristics.

With an easy to use wizard select which stimulus patterns on which sensors should trigger a response and then choose a response along with a fixed target response time and duration.
Event Mark (EM) INPUT: 4x opto-detectors, 2x mics, 10x TTL trigger inputs, 4-8x response pad buttons

OUTPUT: 4x Active Switch Closures, 2x sounders, 10x TTL trigger outputs, 1x Robotic Key Actuator

Event mark lets you pair any input sensor to any output line. You can have up to eight real time pairings in use at the same time. Typically this module is used where you need to automatically event mark events in the real world with TTL signals/TTL triggers, e.g. in EEG, fMRI, eye tracking etc.

For example, opto detectors could be paired with a TTL signal so that a temporally yoked output would begin with the input and last until input ceased on that sensor. In an EEG study two TTL output lines could indicate when a visual stimulus was shown and a tone played in the real world. That is, the TTL signals would match real world events to make marking ethnologically valid.

Unlike its sister product, the mBBTK v3 (event marking version), the BBTK v3 records no timing data and is only intended for basic event marking. If you wish to TTL event mark and produce TTL triggers regularity you are advised to consider the mBBTK v3.


Digital Stimulus Capture (DSC)

Using DSC the BBTK v3 can be your millisecond accurate silent observer. In this mode it simply collects all timing data across a range of sensors enabling you to analyze it.

Run DSC and then analyze everything the BBTK v3 saw

 

Event Generation (EG)

EG lets you generate a series of variable events with known timing characteristics across a range of outputs.

Generate a series of fixed of variable events


Digital Stimulus Capture And Response (DSCAR)

DSCAR is at the heart of the BBTK v3 whole ethos. It acts as a virtual human and steps through your experiment recording stimuli and simulating responses with known timing characteristics. In short it is a combination of DSC and EG modules.

Run your virtual human by clicking on LEDs, selecting RTs and durations and then a time limit – What could be easier!


Event Generation Pulse Train (EGPT)

EG lets you generate an unlimited number of fixed events with known timing characteristics across a range of outputs.

Generate an infinite number of events with fixed timing characteristics


Digital Stimulus Response Echo (DSRE)

DSRE is very similar to DSCAR the virtual human bar that it doesn't record any timing data but simply responds to stimuli with known response characteristics.

Respond to an infinite number of stimulus events with fixed timing characteristics

 

Event Mark (EM)

Event mark lets you pair any input sensor to any output line. You can have up to eight real time pairings in use at the same time. Typically this module is used where you need to automatically event mark events in the real world with TTL signals/TTL triggers, e.g. in EEG, fMRI, eye tracking etc.

Note: Unlike its sister product, the mBBTK v3 (event marking version), the BBTK v3 records no timing data and is only intended for basic event marking. If you wish to TTL event mark and produce TTL triggers regularity you are advised to consider the mBBTK v3.

To event mark pair any input sensor to any output modality, e.g. sounder, Robotic Key Actuator, Active Switch Closure or TTL output






Who are we


Founded in 2003 by a team of psychologists, software experts and electronic engineers we are dedicated to improving the millisecond timing accuracy and experimental rigor of researchers in the behavioral and brain sciences.

Get in touch