I couldn’t help wondering after this past week’s Metanomics whether Robert Bloomfield is reading the same newspapers I am, whether I misinterpret some of the stuff that got the world into the mess we’re in today, or whether maybe I just have that liberal arts bias or something. According to Bloomfield, disciplines like anthropology need to be, well…they need to be more organized and systematic and scientific….like accounting, say, or experimental accounting, whatever that is.
His commentary followed a wide-ranging interview with Tom Boellstorff and Celia Pearce, two well respected anthropologists, and it seemed to be saying – well, it seemed to say that the entire field of cultural anthropology is bunk because it can’t PROVE anything:
“My impression, as an outsider, is that detailed theory testing in cultural anthropology is so difficult that empiricists often don’t even try. The problem is that anthropologists are walking into existing cultures and then working backwards, trying to figure out whether their theories might explain the culture that has already arisen.”
Intriguingly, this comes at about the same time that I’m reading Panic, by Michael Lewis, which points out that financial panics are often preceded by pronouncements that the “old rules are over” and are followed by endless post-mortems which often end with a collective shrug of the shoulders over what happened – this stuff just isn’t as measurable as you’d like it to be, at least when it comes to finding the reasons for why people behave the way they do.
Lewis points out that much of the current crisis is based on the “provable theory” of Black-Scholes but then, let me echo Bloomfield here and say, I’m not an economist NOR am I an accountant so who knows – maybe the empiricism behind it wasn’t empirical ENOUGH?
I suppose my first point is that the idea that empiricism and ‘provable theory’ (although I can’t help wondering whether there’s a difference between theory and hypothesis, which strike me as two different things) is hardly without its detractors. But in Robert’s world, you’re an intellectual lightweight if you can’t back stuff up with stats, with proof, with empirical evidence: “Whether it’s economics or anthropology, theory is what separates serious research from mere storytelling.”
Don’t Study Worlds, Build Them to Test Culture
Bloomfield echoes the rather frightening memes of economist Edward Castranova’s second book, Exodus to the Virtual World (read my commentary here). In that book, Castranova argued that virtual worlds are both fun, and that they can manipulated to be MORE fun. Similar to Bloomfield, Castranova would like to see virtual worlds as a testbed for academia – setting up controls, tweaking the economies, testing the idea of contagion, say, or setting up alternate banking systems in a virtual world to see, um, whether they’ll collapse I guess…whatever theory they want to test, the idea goes, can be proven in a virtual world.
Absent from these discussions is any reference to the rights of the users. In the hands of academics, these are platforms in which social or economic or accounting theory can be tested. The people using the platforms are just, well…subjects, I guess. And I’m not denying that there are ways to get informed consent, to produce data with consent, but it’s an augmentationist view: these aren’t worlds, these are platforms with controllable inputs and outputs.
Castranova took this one or two frightening steps forward, claiming that if you could increase the “fun quotient” of virtual worlds by manipulating the user’s senses and experience, then there are valuable lessons here in how to make the REST of life more fun. He actually argues that governments should look to how platform owners manipulate virtual world environments in order to “hook” their users and should extrapolate best practices for creating – well, for creating a happy, “fun” society.
I suppose the real world economy is a good example of where this extrapolation of lab theory to the real world can take you no? The last 10 years was one giant test bed for the formulas and theories that said you could ELIMINATE RISK – and those theories were taken out of the arm chairs and into the marketplace.
As the New York Times reported this week, however, academia will probably take a decade to adapt and refine its theories of the economy, although my guess is it will be slower even than that. At least amongst the empiricist set who like numbers and spread sheets and still wish that the world contained only rational actors and controllable variables.
Study Cultures or Create Them
Bloomfield says:
“Virtual Worlds give anthropologists a fascinating new opportunity to actually create cultures. And so I see actually in the backchat Curious Sciurus has said anthropology is not a hard science. I think it can be because Virtual World developers have already been creating cultures, as our guests showed clearly over the last hour. So let’s bring research anthropologists into the mix right up front and use Virtual Worlds as a laboratory to test and refine the predictions of anthropological theory.”
There are so many things wrong with that statement, and I’m not even an anthropologist. The phrase “not a hard science” jumps out at me – as if “hard science” is what we’re all pining for, or what makes something useful.
What I WILL agree with is the concept that anthropologists should be consulted in the development of virtual worlds: as should disciplines like accounting, economics, and every other discipline that touches on PEOPLE – but they should be brought in to shed light on how cultures evolve so that those lessons aren’t lost as new worlds are created. Much as a game designer studies old games to see what mechanics make sense, world developers should consult anthropologists to understand what factors influence a society, but not for the purposes of manipulation and study – but rather for the shared betterment of society.
The notion of using platforms to test hypotheses strikes me as not empirical, but imperial. Human nature is not a hard science, and virtual worlds are not platforms on which to conduct experiments on human nature – they are cultures in their own right, they are societies. If this is true, then why would Bloomfield’s argument be any LESS true if it was applied to actual cultures?
Imagine if he had written the following:
“Tribal cultures give anthropologists a fascinating opportunity to actually create cultures, because modern man can enter these societies and take over the controls and the variables. Our position of power over these cultures gives us a unique opportunity to test anthropological hypotheses.”
Coincidentally, Boellstorff has a piece in the most recent issue of the Journal of Virtual Worlds Research which addresses this issue of methodology and whether research is less valid because of its methodology. In it, he says:
“I have been disappointed to encounter, upon occasion, a methodological partisanship contenting that quantitative methods are the only scientific or rigorous approaches for studying culture in virtual worlds. One way this partisanship manifests itself is via the claim that qualitative methods are “anecdotal.” This profoundly mischaracterizes ethnographic research and fails to consider how quantitative methods using behavioral data and surveys are themselves “anecdotal” (not least because of the term’s etymological meaning of “not yet published”), distillations of complex and meaningful issues not always fully present to consciousness.
To concretize my concerns, it will prove helpful to consider the example of some recent work of the economist Edward Castronova, whose influential research I often cite with great approval in my own. In his article “On the Research Value of Large Games: Natural Experiments in Norrath and Camelot,” Castronova (2006) draws upon large datasets from two online games on develop fascinating insights about interpersonal coordination. Castronova rightly sees in this approach possibilities that have “never before existed in the long history of social thinking” and are “of incredible power and value” (p. 183).
Unfortunately, Castronova predicates this claim of methodological value on methodological partisanship. Contrasting his method “with the methods currently available to social scientists” means, among other things, that “the results are not based on the researcher’s impression after having spent 12 months living with a small subset of one of the populations” (Castronova, 2006, p. 184). He then states that “it should be apparent from the tone” of his argument that he feels his “mode of study is at least as reliable, and quite probably more so, than those that precede it . . . That being the case, a major realignment of social science research methods would seem to be in order” (p. 184).
Tone, indeed! It is extremely important that we interrupt such utterly unnecessary methodological partisanship, which is furthermore at odds with Castronova’s earlier work.
That work typically had a recognizably ethnographic component: at the very least, it did not falsely reduce ethnographic research to the gathering of “impressions.” Nor did it construe its methodological palette in a zero-sum fashion, placing methods on a timeline such that one method can “precede” another. Yet, this placing of differing methods on a timeline is wholly consonant with the implicit narrative of progress that structures Castronova’s partisanship. Given Castronova’s claim to methodological superiority, while asserting that he is discussing culture, it is instructive to recall Strathern’s insight that “culture consists in the way analogies are drawn between things, in the way certain thoughts are used to think others.”
It’s a Whole World Out There
Bloomfield closes his argument in what I’m sure he thought was the happy idea that he could join up the anthropologists with the developers over on places like OpenSim and everyone would win: anthropologists would stop sounding like “storytellers” and would be able to prove stuff, and Robin Gomboy would be, um, manipulating their users to test theories, I guess.
Robert isn’t entirely WRONG – the instinct to use anthropology as a way to understand how virtual cultures are shaped is useful. The idea that the reason to take this approach is so that anthropology can be backed up with “hard science” is a fallacy.
What was kind of odd about his closing was his idea that by doing so, the OpenSim folks might be able to coax anthropologists “out of their arm chairs”:
“Despite a reputation for being free thinkers who challenge old ideas and orthodoxy, academics tend to be a pretty cautious lot and would rather just keeping doing what’s been traditional in their fields. But, for the developers, these tests would be systematic ways to learn how to make a successful World that has the type of culture the developer wants to build. So I see this as the win win kind of relationship that can create a better World in the most literal sense, while helping a cultural anthropologist get tenure.
I understand the risks. I understand the challenges. But, you developers, and, yes, I’m talking to you, Robin Gomboy of Reaction Grid, the pace of industry doesn’t allow you the privilege of patience. So get on the phone to your favorite cultural anthropologist and see if you can get them out of their armchair and into the laboratory: your World.”
Now, I’m all for breaking orthodoxy but this comment ended a show in which he interviewed two people who HAVE broken the orthodoxy, who have been out of their arm chairs and have explored corners of the Grid and participated in the culture, attended art show openings and sat in people’s beach houses and, through observation, have gained insights that you just can’t GET by testing structures and data variables or, for that matter, by simply interviewing their “subjects”.
I’ll have to check Tom’s rez date again, of course, but I’m pretty sure he hasn’t been sitting in any arm chair glancing in at virtual worlds from some ivy-walled tower. Having said that, with some of the theories and hypotheses that did the rounds these past few years, well, maybe an arm chair isn’t such a bad place to be.
Robert Bloomfield is ridiculous. I’m glad that you have the courage to criticize him even though you are sponsoring him.
He’s typical of the narrow-minded specialists that you find in the university now, especially in business, who completely repudiate the humanities and don’t seem to be encumbered by any humanities education themselves.
Perhaps you know the story of C.P. Snow and his essay “The Two Cultures”. At the time he wrote that critique, the pendulum was at another place in the arc, and scientists were feeling under appreciated in the 1950s, and trying to gain access to power in a society dominated by those who were educated in the humanities.
Now, the pendulum has swung all the way over to science, and far more than Snow saw it, to the point of utterly crushing the humanities as non-fact based. I can just hear some of the loons of MMOX and AWGroupies crabbing every five minutes that a claim made by cultural anthropology is “not a falsifiable proposition” or contains a logical fallacy blah blah.
Tom indeed spent a lot of time inworld. He formed a large group of enthusiasts and met with them every week to talk about their experiences. He interviewed avatars endlessly. Tom is all about inworld. There was no ivory tower. And I’m not seeing what is so precious about “Open Sim” here, as if you can’t experiment in Second Life. Huh?
As for Castronova, I find his book sinister, quite frankly. You don’t go far enough in explaining it: what he actually advocates is figuring out how the autonomous nervous system experiences “fight or flight” and “pleasure or pain” work and then hooking people up to virtual worlds and MMORPGs better — and taking the elements of games and incorporating them into real life, like “The Ministry of Fun”. Awful stuff.
What I find duplicitous, too, is that these figures are riding the tide of virtual worlds and social media and all the freedoms and chaos and creative destruction blah blah that they imply, but then inserting very authoritarian agendas on to them.
Take this quote, from an interview from Beth Noveck, who is now in the U.S. government’s Office of Technology, where in fact enthusiasts have just inserted a wiki for civil servants to work:
“When we talk about citizen participation in government, we’re so often talking about tapping into people’s feelings and judgments, says Noveck. Peer-to-Patent [Noveck's crowd-sourcing of patent critiques--PN] works by focusing on fact-based expertise. “The idea of a free-for-all, of putting something up on a wiki,” Noveck says, “is terrible.” But when you architect a system with structured roles and group checks on individuals, it seems like wider civic engagement online is possible.”
http://www.nancyscola.com/articleswritings/better-patents-through-crowdsourcing/
Indeed. It’s like they are inflicting the wiki only to destroy, then laying over that a terrible grid of social controls where there are “group checks on individuals”.
My fierce attacks against tekkies, which you only mildly echo here, are pretty much based on their repudiation and scorn of the humanities, even as they use completely outrageous non-scientific methods themselves.
Bloomfield and other economists/stockbrokers should read The (Mis)Behavior of Markets by Benoit Mandelbrot and Richard L. Hudson (2004). Mandelbrot brought us fractals and wrote as early as 1961 on the non-linearity of finance. The point being that all the math used by brokers and most economists is based on an erroneous assumption.
Also consult Nelson and Winter (1982) An evolutionary theory of economic change. Winter points out errors made by the standard economic theory.
Winter and Mandelbrot do not necessarily agree, except that classic economic theory is flawed and does not describe what empirical evidence shows.
Well, certainly I wouldn’t go so far as to say Bloomfield is ridiculous, and that certainly he isn’t extrapolating to the degree that Castranova does which, I agree, is sinister (and again, I reference back to my original post on his last book).
Bloomfield’s intent was perhaps far less sinister – to use the “language” of code and testable hypotheses to encourage “world developers” to expand their repertoire beyond the code and into discussions of human sociality and culture.
Unfortunately, he couples this with the claim that theory needs data when perhaps a better model is that theories are only testable via hypotheses, and that hypotheses can be tested only through research. Where he REALLY falls down is in assuming that research methods that include observation, which is a significant basis of the humanities, arrive at fundamentally less significant truths than hard facts.
If we were really all rational actors, then maybe we wouldn’t be in this mess we’re in, but the reality is we’re not, which supports the argument that perhaps science leaves off at the humanities rather than the humanities leaving off at science.
The classic definition of science is only the systematic application of knowledge, which means butterfly collecting (as long as you classify your collection’s specimens) is science.
Quantitative people deride “butterfly collection” as not “real science” because it is not predictive. Experiments *require* a theory propose a result before the experiment, so the theory can either become stronger (the expected result) or be thrown into doubt (an unexpected, but more interesting result).
Anthropologists are mostly butterfly collectors. Little experimentation has occurred and the morality of such experimentation is very in doubt. Read the human subjects rules for any university funded research to see how much damage the excesses of the 60s and 70s did to the field. Virtual worlds provide a unique opportunity to create “safer” experiments though, and so I can see how people from the quantitative world would *expect* anthropologists to want do such experiments.
Butterfly collectors aren’t used to running the risk of a theory being shaken by experimentation though, and the idea of building such experiments is tainted by the past, so I doubt one will see many.
Quantitative science does represent a maturation of a science though… chemistry was once “butterfly collecting” (the line between alchemy and chemistry was blurry for a very long time). I have my doubts about anthropology ever crossing that line though because the data set, at the end of the day, is one.
I found this show very disappointing. I made time out of a busy day to attend it because I was looking forward to hearing two highly-intelligent explorers of virtual worlds present their insights. Having sponsored 2 discussions that Tom led inworld, I am well aware of what an excellent experience that can be. Instead, I got way too much Bloomfield and not nearly enough Boellstorff and Pierce.
Bloomfield’s lengthy piece at the end seemed more focused on “hey academics, here’s a new way to get funding” than on any useful research agenda. The last thing we need inworld is a pack of newbie graduate students who believe SL is a platform they can manipulate for their dissertation research.
Thanks for the thoughtful responses. This is not the first time I have been criticized for expressing this type of viewpoint, and it won’t be the last. (Take a look at the responses to my first post on Terra Nova from June 2007.. Here are just a few quick responses.
First, theories in both anthropology and economics purport to impose useful structure on the world as we see it. Empirical research–whether participant observation, econometric analysis or experimentation—gives researchers a way to change those theories for the better. The more convincing the data and method, the more likely it is to change the mind of someone who has already accepted the theory and is using it to structure their own view of the world. Casual interviews and observations, punditry–even fiction and poetry–can provide helpful evidence on ways to see the world. But the softer the methods, the easier it is for everyone, researchers and readers alike, to see the results as supporting whatever they want to and already believe. Harder evidence puts more stress on the theory. Yes, Boellstorff is right that lots of empirical research is not particularly persuasive, and that some questions are not amenable to measurement. But when you see a tool that allows you to test *some* theories in a more rigorous way, I think it is a mistake to reject these methods wholesale.
To apply this to the current financial crisis, I would argue that many risk models were simply not tested enough before being implemented. The theories seemed to make sense, but the experts knew that they were being applied in settings that didn’t exactly match the underlying assumptions. (In particular, market outcomes don’t follow the assumed bell curve perfectly–extremely bad news is slightly overrepresented). However, no one bothered to test whether those rather slight mismatches result in major problems. Since everything seemed to be going well, everyone was happy with storytelling.
Second, my purpose was to point out experimentation as a possible *addition* to the pantheon of anthropological research, not a replacement. Of course participant observation is a helpful method, and I am glad people are doing it–especially as a first pass when theory is very rough. But many of the social sciences that resist experimentation and statistical methods fail to push their theories far enough. This lets people cling to their own pet theories more easily, rather than having to justify them and refine them as people provide counterexamples that aren’t easily wished away.
As an example, I would point out that A. Draken is right–for decades people doubted many of the premises of traditional economics. But no one bought into the alternative–behavioral economics–until repeated experiments showed that it had greater predictive power. Now, behavioral theories abound in economics, for the simple reason that it was so hard to argue with the evidence.
Third, I am surprised (again!) by the resistance of virtual world devotees to the idea that researchers might conduct what are essentially clinical trials for virtual worlds–particularly from Dusan, who deals with drug companies as part of his day job. Drug companies do clinical trials before releasing a drug to the world. Why shouldn’t virtual world makers do the same before they release a world to the world? What better way to learn whether a certain type of orientation or technical feature would reduce griefing or some other undesirable behavior–for, as Dusan puts it, ‘the shared betterment of society’. It might have been nice if we had done some clinical trials with the risk models that turned out to have some rather awful side effects in our financial markets!
What surprises me even more is this: why assume that, because I propose clinical trials based on anthropological theory, that I must think anthropology is bunk, and that I must want to start drugging and vivisecting virtual natives. I wouldn’t have proposed these notions unless I thought anthropological theories could actually help us predict the development of culture in virtual worlds. And hey–if I wanted to experiment of residents of an already existing world, I would have proposed that. But I think that is a terrible idea, in part because it would largely defeat the purpose of a controlled experiment in which the researchers are able to manipulate their variables.
Naturally, this won’t be the last word–maybe we need a Bloomfield-Castronova-Malaby-Boellstorff debate on Metanomics sometime soon.
Rob
p.s. Riven, sorry you got ‘way too much Bloomfield.’ I am still learning how to get out of the way and let my guests have the floor. Maybe next time….
Thanks for stating your concerns concisely and straightforwardly, Robert. My major response is what you are framing as an opposition between anthropological and economic theory is actually not that at all. The limits of experimentation in the *natural sciences* are well-known, as we have plenty of examples of “hard sciences” that do not rely to a great degree on experimentation at all (geology, paleontology, astronomy, to name a few).
The problem here is one of scientific epistemology. It seems that it is often those outside of the natural sciences (I’m sorry to say, most often in economics) who are unaware of the epistemological issues here. I remember hearing a wonderful presentation on river formation from a geologist, for whom the key aim of science was *not* prediction, but rather an understanding of the specific processes at work in a specific site at a specific time. Much of science is this. We do not seek only to predict, for as Maxwell has told us, how regular, how timeless, do we actually think the world is?
What is the difference? What makes some science more particularist and some more generalizable? It is not hard or soft, it is a question of time and change. See my post here on Terra Nova about this issue: http://terranova.blogs.com/terra_nova/2008/05/virtual-general.html. In short, experimental methodologies and “exploratory” ones are actually all on a spectrum, with true replicability (there’s the importance of scientific epistemology to this discussion again) impossible. When we have good reasons to expect that conditions are not in the midst of significant and fundamental change, we can tack closer toward the experimental end of the spectrum in our methods, but when we have many contingencies in play (especially with humans involved, but this is not necessary), then particularist inquiries help us get at the processes in place in specific circumstances.
This is, in a way, the Darwinian legacy in science — a recognition of the limits of our ability to generalize in the midst of a messy and open-ended (not law-driven) world.
Speaking of predictability or generalizability thus excludes a lot of what actual scientists actually do, and comes to stand instead as a bludgeon that some (social) scientists use to denigrate other fields. To hold predictability and testing up as the sine qua non of knowledge production is I am sorry to say, to demonstrate a somewhat impoverished notion of what empirical inquiry can and should be.
This is an interesting discussion, and I think it would be better if it didn’t become a debate as Rob suggest – I don’t see much to gain in partisan ‘my science is harder than your science’ squabble. We’ve got uncharted seas in front of us and an argumentative, greedy, egotistical ship of fools is not good company.
I added “greedy” to the list since empirical science and economics being mentioned made me think of the US congress playing grab-ass while the economy craters. Do any of you really think what happened to the economy was because of a bad set of scientific theory? Because that is what it sounded like. What are the metrics/equations for greed, graft, unaccountability, and then raw panic and fear? The reason why we have “soft” science is so far, humans are not controlled nor fathomed that precisely. Maybe at singularity 1.9
Really, prok, the pendulum is set to science? Maybe only relative to humanities, but with science literacy at large at about the feral neko level, I don’t expect the average avatar checking or understanding the methodology or statistical strength of anything soon, sadly enough.
Next, there seems to be a suspicion and fear of experimentation in virtual worlds, as if the only way to test and understand human behavior is a mad scientist/SciFi scenario. Most social and other scientists do follow codes of ethics and otherwise respect the privacy and safety of their subjects. If you want to be paranoid, there are plenty others watching your buying and other behaviors, and they are not some academics out for a good paper, but blood sucking leeches you find after swimming in the slip streams of media and merchandising.
Dusan, thanks for explaining it so that I could understand that what I thought I heard Bloomfield saying was indeed what Bloomfield was saying.
Although … I really hate to give up the idea that I was just hugely distracted and must surely have misunderstood something.
Pity.
Dusan – thanks for posting this commentary. All these comments are great.
The problem with what Rob did wasn’t just what Rob said, but that he tacked his commentary onto the end of what was supposed to be a conversation, with no advance warning and no opportunity for Celia and I to respond. That’s not dialogue.
There are broader issues here that Rob is not addressing. To claim, for instance, that “the softer the methods, the easier it is for everyone, researchers and readers alike, to see the results as supporting whatever they want to and already believe” is simply incorrect. It is ideology parading as analysis. That statement places all methods on a single scale from “hard” to “soft” without a clear explanation of what that metaphor of softness and hardness references. Methods differ in a range of ways – that is, along a range of axes – and they cannot be lined up like ducks in a row on a single scale from hard to soft.
It’s funny that Rob says the following to you, Dusan: “I am surprised (again!) by the resistance of virtual world devotees to the idea that researchers might conduct what are essentially clinical trials for virtual worlds–particularly from Dusan, who deals with drug companies as part of his day job. Drug companies do clinical trials before releasing a drug to the world. Why shouldn’t virtual world makers do the same before they release a world to the world?”
Perhaps the difference is that human beings aren’t the same thing as drugs? If you take a certain chemical (i.e. drug) with the same structure in the same amount, you can assume the effect is the same. 1 pill of aspirin is interchangeable with another pill of aspirin if they’re made with the same concentration, etc. But people (and many other things in the world) are not so interchangeable.
I think of my work in Indonesia. The issue is not only that it’s not feasible to put one group of Indonesians on one island, and another group on another island, and see which ones end up defining themselves as “gay” in some fashion. The issue is that these Indonesian identities have come into being in historical time, and you can’t set up an experiment to show why these identities have come into being. This whole debate is a great example of why it’s helpful to look beyond virtual worlds and see how many of these issues are relevant to actual-world cultures too.
As I’ve said over and over, there are economists out there doing great work and really contributing to a better understanding of virtual worlds. But the experimental methods that seem to be favored by a good number of economists doing work in virtual worlds are not the only valid method (contra Castronova, and so far as I can tell, Bloomfield as well).
What is of the greatest concern to me is that based on some writings of Castronova, and the way Bloomfield set up our Metanomics event, they aren’t really listening. A particular approach from economics is getting set forth as the most valid method (even as the only legitimate method) for researching virtual worlds. And in this point in time when a vibrant, exciting, diverse community of researchers of virtual worlds is growing and forming, that kind of methodological partisanship is simply not helpful. More debate, more real conversation, is sorely needed.
@Tom,since last June we have closed Metanomics with a short op-ed (usually, but not always, by me). When I write them, I do so after the pre-interview but before the show, and voice them at the very end. This format is pretty common in talk shows, but certainly makes it hard for the op-ed to be a ‘dialogue.’ If you have suggestions for other formats (including dropping this segment of the show, so we have more time for dialogue), I’d love to hear them.
On the substance of Tom’s remarks, let me say (again) that I don’t believe that the only way to do research is the way I do research. Thus, Tom and I are in strong agreement on his concluding point–let’s foster research with a variety of methods, relying on a variety of theories, addressing a variety of topics.
The funny thing is, I thought that is what I did, suggesting one possibility in particular. After our pre-interview discussion, and reading a number of studies by you and Celia (and some others in your journal)–and particularly thinking about our discussion of how small technical features of a virtual world might alter the culture that arises there–I thought to myself “wow, here is a great opportunity to wed some ideas that come a largely non-experimental discipline to experimental methods, and learn something new in a way that can both advance theory and have practical applications for world developers.”
So far, none of the reactions have made me question this original thought. They do, however, make me want to spend some time rethinking how to make this point without implying that cultural anthropologists are not ‘real scientists,’ that methods other than experimentation are invalid, and the like. Maybe anthropologists have heard one too many helpful suggestions like this….
The debate reminds me of one that has been happening in legal research. A handful of lawyers are now conducting statistical research actually trying to model and predict the decisions of judges, criminals and institutions. Justifying this addition to the literature is often taken as disparaging existing legal research methods. Some empiricists might make this argument, I think it is rather silly. But I think the opponents of empirical methods are on even weaker ground.
Great discussion. I take as a given that your intention was a good one, and I said that in my initial post, Rob…the purpose of the ‘Connecting the Dots’ is, I suppose, a chance to kick-start follow-up discussions, so I’m glad those discussions found a home here and elsewhere.
A few comments, I suppose. Rob, clearly it’s useful to review a range of methodologies and disciplines for the study of virtual and actual culture. I can’t help feeling like you’re arguing a particular mind set, and a particular bias, from a place that’s, hmmm, well, I hate to say it – but a little arrogant.
I would have LOVED to hear you say something along the following lines:
“Speaking with Celia and Tom, and having read some of their reviews and articles, I’ve become aware that the “hard science” of accounting and economics would be well served by methodologies that I might have, at one time, considered too “soft”. Because let me be the first to admit: human nature just isn’t as measurable and testable as us folks with our spread sheets and empirical tests might like.
“Take the financial crisis – the reality is, it would have been nice to properly test the theories of risk reduction or how markets move through hype cycles and bubbles. In spite our best efforts over 100s of years, however, we haven’t really come up with a good answer yet, so I won’t hold out a lot of hope that we’ll solve that problem in the near term, not on our own, anyways, and not easily.
“Human nature being what it is, and economists being human – we were perhaps overly anxious to put the models to work when Wall Street came calling – there’s a certain joy in getting work “into the real world”, these things can’t sit in a lab forever, our reasoning goes.
“But even if we HAD been able to empirically test, (and let me state my bias in saying that experimental accounting is the frame from which I view these issues), clearly even well-tested theories end up facing the messy, emotional, and behaviorally erratic fact of human nature. If we’ve learned anything in virtual worlds it’s that emotions can run high, and I believe they can run high in virtual worlds because when the hard science bumps up against those other, immeasurable things like creativity, love, passion, hope and fear – well, the two can sometimes be reconciled, but not always.
“What I’ve learned today is that there’s a wonderful combination of disciplines that have found a common ground – and the common ground are these virtual worlds where there are dreamers and artists, coders and academics, scientists and mathematicians, and people who treat these places as home.
“I’d hope that the tools with which virtual worlds can be built provide fertile opportunities for empirical studies across the disciplines – opportunities to measure social connectivity, to extrapolate data and test points, to study the exact impact of things like the Linden dollar supply or the cost of land. But I’d also hope that those of us in the ‘hard sciences’ can also learn, by virtue of being in a cross-disciplinary world, that there may be virtue and value in getting away from our spread sheets a little and looking thoughtfully at how people like Tom and Celia draw insights that are profound and meaningful, and do so not because they have more rows or columns in their spread sheets, but do so because they bring proven methodologies for extrapolating meaning from observation, from anecdote, from story, from the “mess” of human nature.
“The financial crisis shows us something important: the data points that don’t meet our bell curves are often the most important ones – those are the stories that happen at the edges, the ones that often include human emotions like hope, fear, and greed. Those edge cases, the ones that don’t meet our notions of a “homogeneous mass”, are the ones that hold some of the burning insights that can help us craft new ways forward in recognition of both provable and verifiable data, and our humanity.
“This is highly encouraging – this shared environment, these shared worlds, open the opportunity for us empiricists to sit down and understand how anthropology could very well be a missing piece in how we measure markets and human nature. And, I hope, can be a place where anthropologists can take advantage of the fact that some of the variables are controlled – intentionally or not, as we sometimes see with changes made by platform owners like the Lindens, or like Robin’s experiments in creating a new culture that suits a particular goal.”
“Virtual worlds offer a field in which we can both observe, through the endless data stream, the numbers and formulas and testable hypotheses, but they also offer the chance for folks like myself to embrace the concept that we do not live in intellectual silos anymore, we can not afford to sit in our arm chairs while the world shifts and changes and seemingly tilts on its axis into the future.
“Virtual worlds provide the hope that in this collapsed geography, in these new cultures, we can find clues to our own humanity, whether measured with yard sticks or whether measured in the stories we tell.”
I’m surprised at the reception of Castranova’s book – I found it full of interesting ideas. What is sinister about extrapolating from an obvious trend and trying to find some lessons for the real world?
I agree with some comments re “too much Bloomfield” – I would think most people watch Metanomics for the guests, not for the host (as charming as Robert can be).
@Dusan, this is hardly the first time I have been called arrogant, and unless the call for “less Bloomfield” turns into “no Bloomfield,” I’m sure it won’t be the last.
Virtual worlds are indeed a useful common meeting ground for people with very different perspectives on research, politics and life. I think many of us are here precisely for this type of cross-pollination. If we are lucky, we will be able to move this particular conversation forward, and not get mired in the academic version of ‘culture wars.’
It won’t be easy…
I’d like to append a little comment here about clinical trials. Robert is right, this is an area that we know something about, however he partly misses the mark – in fact, our entire business is based on the fact that the “empirical data” is less than ideal.
The reality of clinical trials is that they bump up against many of the same issues that have been discussed in these comments: there is a bell curve, so to speak….but what’s challenging isn’t the ‘homogeneous mass’ it’s the fact that most people aren’t ‘measurable’ against the criteria of clinical trials.
Most trials are organized to provide clarity of end points: usually mortality and morbidity. As such, they select patients based on criteria that will provide conclusive statistical direction to the results. Yes, this is empirical, and it’s useful, but our work is primarily focused on the fact that the vast majority of patients don’t fit the neat categories studied in trials – and so it takes observation, qualitative approaches, and judgment on behalf of professionals.
The idea that clinical trials can lead to cook book medicine (the application of empirical evidence to algorithms for patient care in all cases) is a fallacy, just as I believe there are few areas that involve human nature where empirical studies will ever be able to create formulas or algorithms to map out what people do, how they think, or their state of wellness.
To take a simple example: most clinical trials study heart drugs by recruiting patients who have heart disease. Simple enough – gives a nice “clean” end point. But the people who are EXCLUDED from the trials are more like what a physician SEES: the patients who also have diabetes, obesity, depression, or a host of other concomitant conditions.
As a result, physicians use the empirical evidence as a base line, but the reality of practice is they need to use anecdotes, personal experience (i.e. “stories”) to make decisions against those pesky patients who don’t conform to the data.
Finally, one of the challenges with clinical data is that much of it doesn’t include measurement of quality of life data…a significant oversight, especially in areas like oncology where there are marginal differences between mortality and morbidity results, and the impact of lower side effects can’t be quantified by simple “spread sheets” – how a patient feels about their treatment, the quality of their life on one drug over another – not usually measured, and even when it is there is dispute over the measurement scales.
Because of all these factors, the leading thinkers in the field of medicine, anyway, recognize the combination of evidence with clinical judgment and have created useful models that integrate empirical with qualitative methodologies.
I don’t disagree that the empirical data is the foundation piece in the case of medicine for testing efficacy and safety (let’s not talk about how even the approval of something so based on “hard science” can be impacted by human nature, politics and greed, OK?).
So you’re right – but please don’t hold clinical trials up as an example of a field that is able to provide a role model for an ‘empirical approach’ because the truth is far removed from that reality.
@Iyan – Are we talking about the same book?
Castranova proposes that we take the lessons from how the users of virtual worlds can be manipulated through sensory programming and apply that lesson to manipulating users in the “real world” through similar approaches.
Look – it happens already. Stores program their environments with music, smell, sight, lights to try to get you to BUY more. Castranova takes this further, and proposes turning work into a programmed game where we remain productive because the “system” has been constructed with these findings in mind: that individuals can be manipulated to feel like they’re having more fun, even when it’s as boring as farming herbs in Warcraft for hours on end, or completing a “kill quest” that takes hours and hours of endless grinding.
A few select quotes from Castranova:
“Clearly, it is a challenge to make people happy through manipulation of the social order. Yet this is precisely the challenge that virtual-world designers have faced and mastered.”
“There is also a distinct relationship between motivational systems and human emotions. For one thing, emotions come later…stimuli that engage the motivational system more powerfully will have a greater effect on the emotional system….In other words, the game is designed to be so enjoyable and fun that you lose yourself in it.”
“This is clearly a fast-moving field, but the point is, joys, and probably fun too, can be obtained and even increased when a person’s environment is correctly designed.”
You don’t find anything, um, disturbing in Castranova’s proposal that we “manipulate the social order?”
I parallel this with the following quote:
“The intent is to change a mind so that its owner becomes a living puppet – a human robot – without the atrocity being visible from the outside. The aim is to create a mechanism in flesh and blood, with new beliefs and new thought processes inserted into a captive body. What that amounts to is (a race) always amenable to orders, like an insect to its instincts. (Edward Hunter, Brainwashing)”
@Dusan
I think that you are, in effect, frightened by some phrases – and do not see the forest for the trees. The crux of Castranova’s position, as I understood it, was:
* there will come a time when virtual worlds will be in direct competition with real worlds (how many children play in the playgrounds today versus when we were young?)
* for the real world to compete, it has to change – because, right now, it is mostly no fun at all.
He then tried to show that some concepts we take for granted are proved false by the virtual environments – basically, how we perceive things like reward, fun, equality etc, and tried to apply the lessons learned by the virtual world designers in the real world.
An aside: It’s only in virtual worlds that there is an actual evolution of environments, because there is competition and things can change quickly – that’s why they are such great experimentation tools.
What’s wrong with shaping the environment? Do you have the same complaints against Friedman’s book “Hot, flat and crowded”, for example? It does the same thing – points out what works, what doesn’t and what should be done to ensure a sustainable future.
Do you think it is not shaped now? It definitely is – by the corporations, mass media, marketing departments, and some governments (although their influence wanes). What do you prefer – an environment focused on unbridled consumerism (which is, I think, what we have now), or one focused on the happiness of individuals?
Mind you – I come from an ex-communist country, so I am very familiar with the effects of over-shaping the environment. However, the current eco-political situation is a bad advertisement for the laissez-faire approach to it, too. There must be a middle way – and Castranova only tries to point out some potential paths.
celia a “well respected anthropologist?” who says?
You all are so FULL of it:0) At least her punditry on Interactive Design has over 20 years experience in the field to back up her views.
Celia Pearce/Biography
Celia Pearce is a game designer, author, researcher, teacher, curator and artist, specializing in multiplayer gaming and virtual worlds, independent, art, and alternative game genres, as well as games and gender. She began designing interactive attractions and exhibitions in 1983, and has held academic appointments since 1998. Her game designs include the award-winning virtual reality attraction Virtual Adventures (for Iwerks and Evans & Sutherland) and the Purple Moon Friendship Adventure Cards for Girls. She received her Ph.D. in 2006 from SMARTLab Centre, then at Central Saint Martins College of Art and Design, University of the Arts London. She currently is Assistant Professor of Digital Media in the School of Literature, Communication and Culture at Georgia Tech, where she also directs the Experimental Game Lab and the Emergent Game Group. She is the author or co-author of numerous papers and book chapters, as well as The Interactive Book (Macmillan 1997) and the forthcoming Communities of Play: Emergent Cultures in Multiplayer Games and Virtual Worlds (MIT 2009). She has also curated new media, virtual reality, and game exhibitions and is currently Festival Chair for IndieCade, an international independent games festival and showcase series. She is a co-founder of the Ludica women’s game collective.
It’s hard not to appreciate the timing of this article in The New Republic, by Leon Wieseltier:
“In Tough Times, the Humanities Must Justify Their Worth.” So The New York Times announced this week, in a report that made a grim country feel grimmer. “Previous economic downturns have often led to decreased enrollment in the disciplines loosely grouped under the term ‘humanities’–which generally include languages, literature, the arts, history, cultural studies, philosophy, and religion. Many in the field worry that in this current crisis those areas will be hit hardest.” The complaint against the humanities is that they are impractical. This is true. They will not change the world. They will change only the experience, and the understanding, and the evaluation, of the world. Since interpretation is the distinctively human activity, instruction in the traditions of interpretation should hardly be controversial–except in a society that mistakes practice for a philosophy. It is worth remembering, then, that the crisis in which we find ourselves was the work of practical men. The securitization of mortgages was not conceived by a head in the clouds. No poet cost anybody their house. No historian cost anybody their job. Not even the most pampered of professors ever squandered $87,000 of someone else’s money on a little rug. The creativity of bankers is a luxury that we can no longer afford. But now I read about “defending the virtues of the liberal arts in a money-driven world,” as the Times says. I would have thought that in these times the perspective of money would be ashamed to show itself. “
[...] Taking aim at Congressmen, corporations, and a hypothetical flood of future users, Linden Lab announced today that it was planning to force a large chunk of its customer base in Second Life to relocate, creating an ‘adult-oriented’ continent and tagging its databases so that users can be better assured that they can toggle all that smutty adult-type stuff on and off based on how puritanical they feel. The move opens the door to the merger of the Teen Grid with Second Life, a move hinted at by Philip Rosedale on a recent episode of Metanomics, and to the kind of mass cultural and economic change that maybe only the anthropologists and accountants would love. [...]
So, for what it’s worth, my take on it, having worked in online community for the last 12 years or so, and having come to academic cultural anthropology from the pragmatic perspective of doing my job better…this cat’s left the station long ago.
Briefly, cultural manipulation occurs in every online world, and, indeed, is part of what the consumer is buying. Is there sufficient impetus for building casual ties? Is there reason enough to bond into larger groups? If not, we need to tweak it, or the churn rate goes up, and our world fails: people stay (to a large extent) because of other people, and tweaking the social dynamics is just one of the tools a game designer, or community manager, uses as part of their work. I’d certainly agree that most don’t see it that way (yet)… but by definition the community manager’s job is to “manage” a more or less small virtual culture, castigating miscreants and celebrating “approved” community leaders.
I differ from Dr. Castronova in some respects, and I need to read his book, but my take is that we’re already having a tremendous impact on people’s unconscious awareness of the plasticity of culture. If I can find a plethora of differing yet consistently meaningful (to me, in whatever terms, and there’s a whole other discussion there) social identities with a few mouse-clicks – and Second Life in itself is a darned credible example of this – clearly my cultural identity soon becomes… “which culture?” Ie., the myth of the monolithic culture is doomed (unless you’re talking in the sense of a post-techno convergence culture, ala Henry Jenkins, which I think is much more reasonable, but also much more aware of itself as an artifact), and the technology is returning to us the sense of a participatory culture (as in the sense of “participant observer”, and yes, irony intended).
We’re always involved in shaping our culture, to some extent or other – supporting the trends or cultural leaders that suit us, or becoming them – synthetic worlds give us the chance to seek out specific cultural aspects as part of the consumer choice, again returning the control of culture to the individual…
Most every design choice in a virtual world or MMO (and to a lesser extent, I’d argue the same applies to any online community) impacts the culture of the community that will (presumably) come to be participants in that world. Culture may be in some sense sacrosanct when we’re talking the evils of imperialism, but online it’s already basically a consumer commodity. Which I find really exciting… and yes, should fuel all sorts of fun learnings about how culture operates…
A couple quick clarifications in regard to cdz’s statement above:
“celia a ‘well respected anthropologist?’ who says? You all are so FULL of it:0) At least her punditry on Interactive Design has over 20 years experience in the field to back up her views.”
Two issues here. At the Metanomics discussion that’s being debated here, Bloomfield incorrectly referred to Celia as a “cultural anthropologist.” She is not and has never identified as such – that was Rob’s error. She does, however, conduct ethnographic research.
Second issue: your phrase “You all are so FULL of it” makes it sound to me like you are also implying she is not a well-respected ethnographer. So if that’s what you intended by saying “who says,” then I SAY she’s a well-respected ethnographer. Since I’ve read her book manuscript and many other writings, I should know. And since I have a PhD in anthropology from Stanford and am an Associate Professor of anthropology at UC Irvine and am Editor-in-Chief of American Anthropologist, the flagship journal of the American Anthropological Association, I sorta get to say what counts as good ethnography. Her work counts as good ethnography. So there.
At the risk of sounding repetitive: those of us interested in building a strong and diverse research community should stand up to all forms of dismissal or methodological partisanship. I know from my work as an editor that not all research is of the same quality, but I also know that there’s a lot of diverse work out there that deserves respect and deserves a hearing. No research will be everyone’s cup of tea, but we can learn from a wide range of research methods.
A quick tag on Ron Meiners’s comment-it’s absolutely true that cultures are always in a state of change and that change is shaped by “internal” and “external” forces, and questions of ethical research, design, and governance of virtual worlds should take this into account. A good and important point.
[...] On which point I have some issues, but we’ll leave that to the accountants and anthropologists to sort out. [...]
I’m going to defend my use of the term “ridiculous” quite plainly: because it is so. Prof. Robert Bloomfield came on to Terra Nova and said he wanted to find virtual worlds to do prototyping and studies with students in his accounting classes at his university. And I was one of the ones who said, oh, don’t go to those closed war games like Ted Castronova plays, come to Second Life where possibilities are open and unlimited. He at first held this at bay, but then *did* come to SL.
Did Beyers Sellers make any prototypes or models or run them? Did he use SL to make any kind of experiment in business or accounting? Did he perform any research? Did he create any learning experiences for students?
No. None of the above. Beyers did no such thing.
Instead, he created….um…a talk show. He invited people to come on a talk show, let them gab, and he filmed them and put it on TV, and after awhile, even put himself on as a jazz lounge act. Er, where’s the scholarship? Where’s the “idea of the university”? Here he’s bagging on Tom for not having some rigorous methodology, but Tom carefully created a group, held meetings with people and discussed directly with them very specific topics and solicited their input, took notes, did other interviews, and went through the discipline of writing a book about this field exploration. His meetings and group went on for at least a year or more, hundreds of subjects and discussions. There was direct interaction, without the distraction of all the “production” of the Metanomics shtick. Tom talked with people and researched and got input in a rigorous manner, even if he didn’t do a “scientific survey” (there are so many silly ’sociological surveys’ in SL that are skewed from the start that it’s hard to believe they get to be called “science” whereas what Tom did would be questioned.)
So it really is silly. I’m all for having talk shows and guests and chat and jazz. It’s a great way to explore topics. But…to then go around banging on people who actually perform more serious research as not having any methodology, when you haven’t done anything even remotely resembling some sort of academic model, is just absurd.
What’s truly awful about the challenging of legitimate experts like Tom with the sillyness of the Web 2.0 cult of the amateur as Andrew Keen calls it is that he’d have to come on here and keep explaining his credentials.
What needs explaining is why this accountant from Cornell came into SL and didn’t do any accounting or business models, but did a talk show on TV.
[...] to test anthropological theory using virtual worlds as a laboratory. The 20-odd comments on Dusan Writer’s post after the show suggest that there is a lot more to talk about. And talk about it we will — Tom [...]
lol.
“So these two ameobas walk into a petri dish….”
- walk?
[...] Writer published some critical remarks on his blog, Dusan Writer’s Metaverse. I’ll mention a few points here, adding my first reactions on his [...]
Prokofy,
Metanomics is not my research program. But as the proponents of qualitative research have been emphasizing, it is unwise to try to conduct experiments in virtual worlds without understanding them. Metanomics has been very helpful in that respect.
Here is a paper that conveys where my research program is heading. Could I have written this without running Metanomics? Maybe. But then I wouldn’t have learned what I needed for some of my other projects that are not yet ready for the public eye.
I’ve only just now had time to read this thread, but I highly recommend anyone with a sustaining interest in this issue have a listen at our follow-up discussion which happened today. It should be archived at the Metanomics Web site in the next couple of days.
We had a fascinating conversation today, in which some very salient points were addressed, with the aid of our guest, Thomas Malaby, and our guest moderator, Roland Legrand. Although the dialog covered a lot of ground, it sadly concluded on a somewhat contentious note that brought us full circle back to where we started. I’ll look forward to hear the further engagement…
And by the way, Tom is absolutely correct that I have never referred to myself as an anthropologist and unfortunately, I was erroneously referred to that way again today. I am a games researcher who uses anthropological methods in my work.
For anyone interested, I have a blog post about yesterday’s session here .
[...] wrote a lengthy post, which elicited lengthy replies, and then we held a special Metanomics community forum, and the [...]
…there is one other possibility that is being neglected…. that virtual worlds can be contexts for qualitatively researching issues that are ethically or practically impossible in off-line contexts…such as sexuality or violence… such qualitative methodologies are able to produce understanding that is simply not feasible quantitatively precisely because of the potential disjunctures between on-line/off-line worlds… so “ethnographic experimentation” in virtual worlds need not just be read as favoring quantitative over qualitative research