Week 1

What are we trying to do here?

The Challenge

We have chosen to be here, to spend our time and energy doing scientific research. Many of us have idealistic motivations and aspirations: we want to make true discoveries, build cumulative knowledge, and translate this knowledge into social goods like medicines, devices, or policies.

Yet in reality, scientists have often failed to live up to these aspirations, because of bad habits and bad institutions. How can we ‘reform’ science? Can we critique science and make it better from within? Can we do it without becoming nihilists, without breaking what works, without destroying public trust?

Readings before Day 1:

Oreskes, N. (2021). Ch. 1 from Why trust science?. Princeton University Press.

Dominus, S. (2017) When the Revolution Came for Amy Cuddy. New York Times Magazine. [PDF]

O’Dea, R (2021) The next 10 years. Keynote talk at AIMOS

Optional extra

de Menard, A.  (2020) What’s Wrong with Social Science and How to Fix It: Reflections After Reading 2578 Papers.

He writes: “Criticizing bad science from an abstract, 10000-foot view is pleasant: you hear about some stuff that doesn’t replicate, some methodologies that seem a bit silly. “They should improve their methods”, “p-hacking is bad”, “we must change the incentives”, you declare Zeuslike from your throne in the clouds, and then go on with your day. But actually diving into the sea of trash that is social science gives you a more tangible perspective, a more visceral revulsion, and perhaps even a sense of Lovecraftian awe at the sheer magnitude of it all: a vast landfill—a great agglomeration of garbage extending as far as the eye can see, effluvious waves crashing and throwing up a foul foam of p=0.049 papers. As you walk up to the diving platform, the deformed attendant hands you a pair of flippers. Noticing your reticence, he gives a subtle nod as if to say: “come on then, jump in”.”

The Tool

The Critical Evaluation

In 1-2 pages, single spaced, answer these questions:

Intake questions:

  • What name would you like us to call you? How do you pronounce it? (e.g., Ashley, ‘ASH-LEE’); If you would like to specify your preferred pronouns, please do so here (e.g., they/she).

  • What is your year and/or position? (e.g., 1st year grad student; postdoc; undergrad)

  • What is your area of focus? (e.g., computational modeling, cognitive neuroscience.

  • At least one thing you’d like the instructors to know about you.

  • Why did you decide to take this class? What are you hoping to get out of this class? 

  • Anything we can do to support your learning in this class? Do you need any special accommodations? If so what are they?

For day 1 discussion, in light of the readings:

  • In general, do you believe the claims of scientists in disciplines about which you have no expertise? Why?

  • In this class we are going to discuss ways that scientists, and scientific institutions, have failed to live up to our aspirations. What is the difference between productive, healthy critique which is necessary for science, versus doubt-mongering intended to both weaken public trust in science and make scientists defensive and closed to sceptical inquiry?