It’s almost a week now since I attended the 2013 Quantified Self Global Conference in San Francisco, and I’m still not sure where to begin with my summary of the event itself. Instead of jumping in with an overview, this time I’ll cover my own session—in which what started out as asking how researchers studying Quantified Self could better connect with each other became an (at times) intense debate about what Quantified Self is, what Quantified Self should be, and what role (if any) academic or institutional research and researchers should have within the Quantified Self community.

At QS Global 2013, I led a breakout session with Jakob Eg Larsen titled simply, “QS Researchers.” Jakob and I had proposed similar sessions for the conference, and were happy to work together after an introductory email from the organizers. We were both well aware of the growing academic interest in Quantified Self, and eager to find ways for members of the expanding QS researcher community to network and collaborate with each other. What would be most useful, and what might other researchers be interested in? I’d tried to start a Google Group mailing list following QS Global 2012, but that hadn’t really gotten off the ground; truth be told, I’m already up to my eyeballs in email, and I suspect many others are in the same position. But what about organizing a pre-conference attached to the Quantified Self Conferences, at which people could present their research and simultaneously get feedback both from other professional researchers and from non-researcher members of QS? What about an informal networking event attached to the QS Conferences? How could we involve researchers who were working in this area, but not able to attend either QS Global or QS Europe? (Was anyone present connected to the Quantified Self Research Network we’d heard about?) What about starting a QS-related journal—what might that look like? And how could we ensure that our efforts to connect with each other as researchers remained open and accessible to the rest of the Quantified Self community, rather than turning into an us/them ivory tower situation?

“Come meet scientists and scholars for whom Quantified Self is a research topic,” the description read. “We will continue the conversation that began at last year’s conference, and welcome new participants.” What I wasn’t expecting was that the session not only came back to that conversation, but also returned to some of the questions that came up in my QS Europe 2013 session about who is and isn’t a part of the Quantified Self community—and sparked an at times intense discussion about what the role of academic researchers is and should be within Quantified Self.

Our breakout was in the last set of sessions at QS Global 2013, and Jakob and I were pleasantly surprised to find ourselves expanding the circle of chairs out, out, and then out again as people kept trickling in; eventually, the ring was as wide as the room itself. As we went around with brief introductions (name, any affiliation, and three word description of research interests), I counted 31 people total: 21 men and 10 women; 26 professional researchers (academic, industry, or other) and 5 non-researchers interested in research (one of whom was an organizer of the QS meetup group in his city)[i]. We then posed our questions: I asked what sorts of opportunities or infrastructures people would find useful in terms of exchanging ideas and exploring collaborations, and Jakob asked how a more formalized academic event might be either attached to or integrated into the QS Conferences. For shorthand I labeled these two possible developments “informal” and “formal” on a pair of giant sticky notes, and we opened the discussion.

And then things went in some directions I hadn’t anticipated.

“It would be nice to maybe bring up the quality of work of people who do QS, and make their work better,” one man said. In summary, a lot of QS projects just didn’t look like Proper Science™ to him—and that wasn’t the only problem. He also suggested that we “document some of these projects, so that people can, you know, not do the same thing over and over again…and god forbid, reference something that happened before.” (A wave of knowing laughter passed through the room: uh-oh.) Another man chimed in talking about the value of citations, and how academics use them to show “an epistemological trail,” thereby establishing the validity of their ideas. Bringing citations into Quantified Self would not only make QS projects more rigorous, but would create the conditions necessary for instituting a system of peer review. He suggested that, next year, the research community should host a breakout session about “why [academics] have conversations the way we have them,” and thereby teach Quantified Self about the value of formal academic systems of knowledge production and dissemination. A third man bemoaned the fact that most QS documentation is in video format, saying, “We don’t want to have to sit and look at a video for 20 minutes. Having something on paper is just much more convenient than that.” Why couldn’t all these people manage to write academic papers instead?

This theme—which I’ll call ‘QS is Bad Science’—came up again and again during our conversation. Put simply, these researchers saw the self-tracking projects of Quantified Self as “science,” but as an inferior kind of science—a homespun, feral science that aspires to produce knowledge for the greater good, but that desperately needs to be educated, disciplined, and restructured in the image of academic/institutional science. A variation on this theme that also came up (although less often) was one I’ll call ‘QS is Great Science’; these researchers felt that n=1 studies have a lot to teach academic/institutional science, if only QSers could learn to speak academic/institutional science’s language so that these beleaguered researchers could have “proof” of what goes on at Quantified Self to bring back for their doubting colleagues. One man in particular pointed out that, for the institution that could turn n=1 into n=“us,” n=1 experiments could revolutionize science by providing an infinite pool of opt-in, unpaid participants for clinical trials. (The implication here: if you harness enthusiasm for n=1 experiments correctly, you can subtract both paying for subject recruitment and paying your research subjects’ time from your study’s budget, and maybe subtract paying for the IRB review, too. In an era of ever-decreasing grant monies, who doesn’t love free labor?)

To me, as a social scientist, this was setting off all sorts of alarm bells—the paternalism alone made me draw connections both to missionaries and to colonialism generally, and that’s never a good thing. (Though it’s bizarre to liken a group of highly privileged, mostly white people to a First Nation, it was clear to me that a number of researchers were viewing Quantified Self either as the “noble savages” who would save starving Science with gifts of barley and corn, or as “savage heathens” who needed new names, new clothes, a new religion, and new jobs serving their “civilized” masters.) This line of discussion also evidenced a profound blindness to the character of Quantified Self itself: in two-and-a-half years, I’ve never once heard a QSer lament not being able to publish their Show&Tell presentation in an academic publication. At the Open Science Summit? Sure. At OSS 2011, there were several conversations about how to get n=1 studies into academic journals—and I do know that there is overlap between QS and OSS. Still, your average QSer isn’t sobbing into her spreadsheets and wishing that she could grapple with arcane citation practices and grueling peer review processes. Quantified Self, at the macro level and on the whole, is not striving for institutional legitimacy. Frankly, QS just doesn’t care.

As often as Quantified Self gets likened to a movement—and as much as quantrepreneurs like to ask, “How can we get more people to track (and in so doing provide us with More Data for our Big Data Whatever)?”—the overall culture of Quantified Self simply is not evangelical. It’s about everyone doing what’s right for him- or her individual self, as she or he defines both “self” and “right.” It’s about adopting what “works” (for you), and ignoring what doesn’t: To thine own self be true. What half the room seemed to be missing, however, is that this principle applies as much to medical advice and to academic research findings as it does to individual Show&Tell presentations. As I noted during my very first QS Researchers breakout session at QS Global 2012, the greater Quantified Self community is open and even welcoming to researchers, but it admits them as equals, not as authorities. This should have been obvious to anyone at QS Global 2013 in particular, as several of the plenaries featured stories of people who had second-guessed their doctors (aka: formal, institutional knowledge), then provided their own second opinions through tracking, and redirected their treatment courses, and ended up the better for it. By now, during the very last session of the conference, why would anyone think that what Quantified Self wants is to become the Mini Me of academic/institutional research?

As it turned out, I wasn’t the only one picking up on this disjuncture. As the one man who had identified himself as an organizer of a QS meetup group said,

Gary [Wolf] in particular is particularly wedded to the concept that this is a community. The point here is actually not data, not science; it’s people sitting together. In fact, you could knock out the talks, and he’d say the event would still be here, that people would come. He wouldn’t care if they’re small conversations, and if nothing goes on paper—I’m sure you’ve noticed the frustration, “Nothing’s on paper!” and “So informal!” and—that’s consciously done that way, to create this character.”

He went on to reiterate the general sentiment toward researchers that I’d picked up at QS Global 2012:

Unless you want to provoke that kind of reaction—do an immunology experiment—you’ve got to find a way to [organize as academics] that doesn’t set it off. I don’t think you want to run a track at the conference, because that’s fragmenting the conference. If you said, “Let’s come a day early and meet, and then go study these animals,” that’s—I think we’re happy to have you study us, in fact, [the conference organizers] will love it, that’s fine, study us—but don’t try to change what we’re doing because [the conference organizers] have got this formula that they like, and we’re all getting something good out of it.

A woman voiced her opposition to the proposed formalization of Quantified Self by referencing the QS “mission statement.” While I don’t know that Quantified Self has a formal mission statement (at least not in the way my old non-profit job did), her point still stands:

You came to the community of QS, which has a mission statement. And the mission statement is contra to what you want to do. It’s about the peers, and it’s about the individual self. Community. But it’s not about a subgroup that says, “Ok, we want to do something else.”

Another man admitted that, yes, as a scientist, he could see some issues with the methodology in some of the QS Show&Tell talks he’d observed—yet he remained adamant that rigorous scientific methodology wasn’t the point. As he said,

I think that the presentations that I’ve heard in the last day and a half have been from people that had some medical issue, they went to the doctor, the doctors couldn’t solve it, they gave up because they were so pissed off, and [they] tracked something, and then came back and told the story about how they discovered what was really wrong with them. And to sit in a room with a bunch of academics saying, “Oh, no, no, no, you gotta do the scientific process”—that failed to help the person—there’s something fundamentally wrong with this conversation.

He went on to say that, yes, he’d seen a number of Show&Tell presentations that seemed to be of “questionable statistical validity,” but also that he’d noticed the presenters “still found meaning and something from that.” Referencing the Q&A portions of the QS Global 2013 plenaries, he said,

Gary [Wolf’s] questions every time are not about, “Hey, did you know you misinterpreted the p-value of this thing?” They’re about, “Did you find tracking this changed your perception of how you did?” It’s so far removed from every scientific conference that I’ve been to that the suggestion that we should actually have a scientific track is completely absurd […] It’s very frustrating to sit in this room right now and hear the conversation that, “This isn’t meeting my needs to meet my tenure track,” when every single one of us has a professional organization to go present at.

That, of course, was the trick right there: This hadn’t become a conversation about what Quantified Self wants, but about what one particular group of researchers wants from Quantified Self. And while some of those desires boiled down to a combination of ill-informed assumptions and epistemological superiority complexes, other desires to formalize QS were more self-interestedly pragmatic: the QS Conferences are insufficiently formal for most academic departments to provide travel funds (such funds are for “delivering papers” at “academic” conferences only), and there’s not much about participating in a QS Conference that earns points with tenure committees. One woman had a slightly different problem: she had been observing the Quantified Self community for a while, and—since her disciplinary convention mandates that she “give back” to the populations she studies—she wanted to do a presentation about her research. Yet every time she approached the organizers about doing a Show&Tell session, she was stymied by the Show&Tell format’s requirement that she tell a personal story. She didn’t have any personal stories, she just had research findings—and as a result, was having a hard time finding a venue to tell QSers about themselves instead of telling them about herself[ii].

The larger pro-formalization camp did, however, raise some interesting points. One man talked about watching one particular Show&Tell presenter repeatedly answer the same detailed questions in informal conversation after his talk. Wouldn’t it be better, he asked, if there was some kind of centralized repository where that man could put his detailed explanation—in whatever format—instead of having to explain the details of his project to every interested individual? Others pointed out that some QSers do want to do more rigorous statistical analysis, but didn’t know how to do that. Shouldn’t we help them? One man insisted, “We ARE part of the community,” and made the case for treating researchers as a “special needs population” within Quantified Self—especially because of the different ways researchers “need” to communicate. (I enjoyed this comment for the subtle slight to academic jargon, though I don’t know how many others in the room read it that way.)

Still another pointed out that researchers have knowledge and skills that might be useful to the Quantified Self community, just like so many of the start-ups present were offering tools that might be useful to the Quantified Self community. Why weren’t academic researchers being given the same amount of space and visibility? (No one pointed out that visibility is often a matter of sponsorship, not “fairness.”) While I don’t think the academic community is entitled to anything from Quantified Self, this comment did highlight something I’ve been thinking about a lot since. Quantified Self is growing rapidly, even “blowing up” (as the kids might say); “quantified self,” as a buzzword, has long since left the barn. And as both the hype and the enthusiasm swell, the scene surrounding Quantified Self is more and more looking like a feeding frenzy: tech sharks of all stripes circling, drunk on the scent of data; academics, unhappy with their lot as scavengers, trying to elbow their way to the table[iii].

Finally, I proposed the following: Back in my pre-grad-school non-profit life, I’d worked at a harm-reduction based human services organization. All of our clients and participants belonged to at least one vulnerable population (if not several), and we were in it for the social justice, so the last thing we wanted to do was come in with a top-down approach and tell our clients and participants what kind of help they needed. Instead, we did needs assessment surveys—studies that asked our clients and participants what kind of help they needed and wanted—and used the results of those studies to plan our programs. What if the research community did a “needs assessment survey” QS-style, in the form of an experiment? What if we did something akin to the QS Conference “office hours,” and sprinkled researchers across a bunch of small tables, and had each person provide a short description of what skills they were offering, and then compared notes after to see which questions had been most common and which sorts of assistance were most in demand? This way we could organize to offer (as volunteers) services and skills that non-researchers in the Quantified Self community are actually interested in, rather than try to implement by force some pre-fabricated notion of what (some) researchers think Quantified Self should be. Someone in the group helpfully named this hypothetical event “Speed Date a Scientist” (which I love), and the idea seemed to go over well. Someone else suggested adding “Speed Date a Data Visualizer” and “Speed Date an Artist” as well (which I also love).

While I’m sure offering scientist speed-dates at QS Global 2014 wouldn’t go far enough for the Formalize QS camp, perhaps it’s a step in a sufficiently similar direction for some of those researchers to want to participate; it would also be a step taken in a way that would be less likely to “provoke an immunological response” from the rest of the Quantified Self community (see also: make the social scientists’ collective skin crawl). After all, what less than half the room seemed to realize was that—subjective entitlement to power notwithstanding—researchers don’t have control within Quantified Self. Researchers don’t have the authority, either as conference organizers (who ultimately determine what the QS Conferences will and will not include) or as “experts.” Quantified Self is about peer-to-peer knowledge transmission, not “expertise”—and while it might be funny (“funny”) to watch a bunch of academic/institutional researchers try to pull rank in a community based on an entirely different epistemic framework, I’d kind of rather not see that happen. (Quantified Self specializes in “soft resistance” [pdf], after all; I imagine the scene rather like watching an attacker (academia) rush at a t’ai chi master (QS), only to end up in an embarrassing heap on the floor as the t’ai chi master steps calmly aside.)

As Quantified Self grows, attracts more interest, and becomes a (somewhat) more diverse group, so too are the people interested in Quantified Self growing and becoming a (somewhat) more diverse group. This was not lost on the room; by the end of the session, we were wondering if “QS researchers” as a subgroup even made any sense. Some researchers wanted access to QSers’ self-tracking data; some researchers wanted to study the gadgets and technologies; some researchers wanted to study the people and the practices. Some researchers identified as self-trackers, and some researchers did not; moreover, there was no consensus as to what the relationship between “QS researchers” and “Quantified Self, broadly conceived” should actually be. Perhaps there were things we researchers could learn from one another, but we also had what seemed to be some insurmountable intellectual differences. Some of us were offended by the lack of scientific rigor in some Quantified Self projects; others of us were offended by the insistence that “scientific rigor” was even necessarily relevant to Quantified Self. (As one social scientist said, “I feel isolated, because I don’t really identify as a positivist.”)

The 4:00 end time for the session came and went, and we were still talking. At 4:15, with half the break before the closing plenaries now past, I asked if anyone had any closing comments. The last remark came from a man who said,

If I may, I do think that part of this conversation, or a role in this conversation, is a clash between old, conservative, academic thinking, and new, liberal, Quantified Self thinking. So just, maybe as a closure for ourselves, think about, ok, what has happened in the last hour during the conversation, and what was part of our own training as an academic, and what is possible for the future, to take with us?

This seemed (to me) to be a gentle way of asking his fellow academics to Check Your Assumptions—and as a quiet murmur of agreement rolled around the room, it seemed that at least some of us had taken his words to heart. I’m still not certain how “QS researchers” will organize, or how many different forms that organizing will take, or which of those forms will ultimately be “successful” (and as defined by whom). What I do know is that Quantified Self is a “sexy” topic right now, which means there will undoubtedly be more and more of us—and as that happens, it seems likely that “QS researchers” will undergo many (if not most) of the growing pains experienced by Quantified Self as a whole.

Whitney Erin Boesel is a sociology PhD student based in Cambridge, MA. She habitually livetweets her QS Conference experiences as @weboesel—except when she’s presenting, because she’s not quite that good. Yet.

“QS researchers” session note images courtesy of the author.