[lookit-research] Announcement about video stimulus timing on Lookit

[Melissa Kline]

Dear Lookit Researchers,

For any of you who haven't heard from me before, my name is Melissa Kline
Struhl, the new director at Lookit. I'm very happy to have met many of you
on Slack so far, and looking forward to learning more about your research
and experiences on Lookit!

I'm writing with an announcement about study timing on Lookit, especially
regarding studies with video stimuli. In particular, researchers should be
aware that there may be an offset of 150-450 ms (0.15 to 0.45 seconds)
between when the Lookit frame data *reports stimuli to start,* and when
they *actually appear on the participant's browser*.  This was reported by
a group of researchers who tested and reported the issue - my sincerest
thanks to Michaela DeBolt, Lisa Oakes, Aaron Beckner, Christian Harms and
Yi Lin.

Any online study platform will have some degree of both noise and lag that
is dependent on factors outside of Lookit control, but this is a longer lag
than we had been expecting. There is now documentation about this issue in
the Lookit Documentation, and we will add to this page (
https://lookit.readthedocs.io/en/develop/researchers-lag-issues.html) over
time with any new information.

I know that many of you may have questions about whether this affects
current studies or data you've already collected on Lookit - how serious of
an issue this may be depends on a number of factors, including your
experiment design and how you code looking time data.

For questions about specific studies, I'll ask that you post in the
#tech_support channel on Slack, and include a link to your study so that
the rest of the Lookit community can learn from the answers.  If you're not
on Slack or prefer to ask privately, you can also email me directly at
mekline at mit.edu.

This issue is most likely to affect studies that:

- Use very short video stimuli

- Primarily measure the total duration (or absolute timecourse) of looking
time in some context, rather than differences in looking time between
conditions.

- Rely on looking times measured during a short specific window of stimulus
presentation, such as anticipatory looking studies

At this time, there has not been any additional testing on non-video
experiment frames. However, if your study design depends on close timing,
best practice (both now and if we are able to reduce these lag times) would
be to make sure that the critical stimulus onset can be directly measured
from your participant videos, e.g. by including a big change in luminance
that can be seen on a child's face/in the room.

We will be watching this issue closely, so any information you can share
about timing of stimuli on Lookit will be very gratefully received! We are
unfortunately understaffed at the moment, but are planning to hire an
engineer who will be focusing primarily on the experiment runner; this
person will take a close look at the origin of this problem and any steps
we may be able to take to address it.

Finally, I'd like to extend my thanks again to the researchers who both
discovered and extensively tested this lag across a variety of browsers and
internet settings!

Thanks to all of you for your support of Lookit, and I look forward to
hearing from you,

Best,

Melissa


-- 
Melissa Kline Struhl
Lookit Executive Director
Research Scientist, MIT Brain & Cognitive Sciences
mekline at mit.edu
https://lookit.mit.edu/ <http://lookit.mit.edu>
http://www.melissaklinestruhl.com/ <http://www.melissakline.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/lookit-research/attachments/20210916/c1edbe90/attachment.html