The polls got it wrong, again... The end of 'social science'? Time to stop predicting and start listening...

This is clearly the end of the line for the hierarchy of social 'science' in its current form. They used to call it 'armchair' anthropology, but contemporary social science has been spearheaded for years by people sittting in fancy swivel seats in metallic offices with brightly polished windows who stare at computers all day and don't listen. The result is that economists failed to predict the 2008 crash and have had no idea what to say about its consequences: in gross they have offered advice that is barely more successful than the Chinese astrologers of a long gone era. Psychologists have produced 'laboratory' studies in the thousands whose 'results' can't be replicated--in many cases because they chose the narrowest possible social catchment, college students. Pollsters have consistently got the results of the most decisive recent political events wrong. In the UK, the best predictions about the recent British election and the Brexit referendum were wrong. And now we know that, in the U.S., the long statistical worm pointing to a Clinton victory was an illusion too: they might as well have had pulpo paul running the polling. These are the most highly paid social scientific servants of business and government and yet their methods and key concepts often appear to be little better than phlogiston. 

Here is what a commentator wrote this morning about the polling trend that had Clinton with an 80% or more likelihood of victory:

"The polls were wrong. And because we are obsessed with predicting opinions rather than listening to them, we didn’t see it coming. So, the world woke up believing that Republican candidate Donald Trump had a 15% chance of winning based on polling predictions – roughly the same chance of rolling a six if you have two dice. Despite those odds, the next US president will be Donald Trump."

The one person who got this scenario right was Donald Trump. He predicted 'Brexit plus' and that is what happened. He saw there was fertile ground for victory by appealing to people whose social lives and institutions have been atomised over the last thirty years and he was right. Of course, he wasn't talking to or for anything like a single social grouping, but the truth is that social scientists don't know who he was really talking to and listening to, because they haven't been listening themselves; social scientists don't know what is going on at all.

Malinowski recognised this kind of phenomenon in the 1930s which is why he became so closely involved in the mass observation studies of that era. Mass observation may be largely forgotten now, but the principle was clear--if you want to know what is going on then you need to stop paying attention to the most superficial elements of culture and look at and listen to real patterns of action. The standard Malinowskian mantra is usually reproduced in the phrase 'what people say and what they do is not the same.' The bulk of social science seems obsessed almost to the point of mania with 'what people say'.

Views: 520

Reply to This

Replies to This Discussion

Hear, hear. 

The examples are timely and dramatic. The arguments are familiar to ethnographers who work in corporate or other organizational contexts. The small success of Business Anthropology becoming a recognized thing during the last three decades was, I once heard John Sherry say, the result of the failure of the standard sociological surveys and focus groups combined with ideas from social psychology that were, and in many contexts, still remain the standard approaches of market and management consulting research to come up with the insights that corporations and other organizations need to drive innovation. 

A point to remember, however, is that the numbers cannot be ignored. They have to be present and used smartly to ground persuasive stories. 

And  one more, even supported by ethnographic research, most product or other innovations will fail. Effective insights tend to be more accidental flashes of inspiration than results of methodical plodding. More than ever this comment from Victor Turner that I frequently cite seems relevant.

"In moving from experience of social life to conceptualization and intellectual history, I follow the path of anthropologists almost everywhere. Although we take theories into the field with us, these become relevant only if and when they illuminate social reality. Moreover, we tend to find very frequently that it is not a theorist's whole system which so illuminates, but his scattered ideas, his flashes of insight taken out of systemic context and applied to scattered data. Such ideas have a virtue of their own and may generate new hypotmheses. They even show how scattered facts may be systematically connected! Randomly distributed through some monstrous logical system, they resemble nourishing raisins in a cellular mass of inedible dough. The intuitions, not the tissue of logic connecting them, are what tend to survive in the field experience."
"Social Dramas and Ritual Metaphors." In Victor Turner, ed., Dramas, Fields and Metaphors: Symbolic Action in Human Society, Cornell University Press, 1974, p. 23.

P.S. With a tip of the hat to Kim Schueftan, an old friend.

Morning After To-Do List:
1. Take over the Democratic Party and return it to the people. They have failed us miserably. 
2. Fire all pundits, predictors, pollsters and anyone else in the media who had a narrative they wouldn't let go of and refused to listen to or acknowledge what was really going on. Those same bloviators will now tell us we must "heal the divide" and "come together." They will pull more hooey like that out of their ass in the days to come. Turn them off. 
3. Any Democratic member of Congress who didn't wake up this morning ready to fight, resist and obstruct in the way Republicans did against President Obama every day for eight full years must step out of the way and let those of us who know the score lead the way in stopping the meanness and the madness that's about to begin. 
4. Everyone must stop saying they are "stunned" and "shocked". What you mean to say is that you were in a bubble and weren't paying attention to your fellow Americans and their despair. YEARS of being neglected by both parties, the anger and the need for revenge against the system only grew. Along came a TV star they liked whose plan was to destroy both parties and tell them all "You're fired!" Trump's victory is no surprise. He was never a joke. Treating him as one only strengthened him. He is both a creature and a creation of the media and the media will never own that. 
5. You must say this sentence to everyone you meet today: "HILLARY CLINTON WON THE POPULAR VOTE!" The MAJORITY of our fellow Americans preferred Hillary Clinton over Donald Trump. Period. Fact. If you woke up this morning thinking you live in an effed-up country, you don't. The majority of your fellow Americans wanted Hillary, not Trump. The only reason he's president is because of an arcane, insane 18th-century idea called the Electoral College. Until we change that, we'll continue to have presidents we didn't elect and didn't want. You live in a country where a majority of its citizens have said they believe there's climate change, they believe women should be paid the same as men, they want a debt-free college education, they don't want us invading countries, they want a raise in the minimum wage and they want a single-payer true universal health care system. None of that has changed. We live in a country where the majority agree with the "liberal" position. We just lack the liberal leadership to make that happen (see: #1 above).

Let's try to get this all done by noon today. 
-- Michael Moore

 

    As Huon notes, a big loser in the presidential election was applied social science:  exit polls, tracking polls, regular polls, focus groups, all parts of an elaborate and expensive quantitative enterprise that got it wrong

    Intriguingly, Michael Moore (whom John quotes) got it right, and back in the summer in a piece in Huffington Post.  He wrote that Trump was going to win – with the disclaimer that it was not too late to stop him.  And Moore, a high school dropout, based his prediction, not on statistically precise surveys, but on his experience of living life as an ordinary American.  He grew up in Flint Michigan (the city of poison water), saw the factories close, knows lots of folks who were victims of all that.  And this morning on the NBC talk show, “Morning Joe” he talked about being a fan of all those reality shows intellectuals love to diss:  America’s Top Model, Bachelorette, Dancing with the Stars, and, tellingly, Celebrity Apprentice.   He knew what people were thinking, knew why Bernie pulled off a major upset win over Hillary in Michigan, and knew why Trump won that former democrat stronghold. 

    Moore’s take on things reminds me a lot of Studs Terkel’s classic Working, which consists of dozens of transcribed interviews with, yes, working people from all walks of life.  I’ll hazard a guess that Terkel could have called the election as well as Moore. 

    And Huon’s citing Malinowski made me wonder how that ancestor would have proceeded with fieldwork in 2016 America, and what prediction he might have made about the election.  I think it likely he would have steered clear of the artificial survey techniques and hung around bars and coffee shops talking to folks. 

    American anthropologists are in shock over the election (take a look at Savage Minds),  but they should consider whether their own professional identity isn’t closer to that of the failed pollsters than to those of Terkel and Malinowski.  Anthro books and journal articles have become so arcane and esoteric that they float way above the ground most Americans tread.  Then when reality smacks them in the face, they head for the safe spaces and crying rooms now filled to capacity on university campuses. 

What prediction Malinowski may have made about highly unlikely event like Trump effect?

 _ To study it, but not to speculate about Trump unconsciousness. As professional applied mathematician of the 1920s he had limited choice, his slighly "naive" functional approach ( inspired by Banach ) could be changed in the 21 century.May be.

Huon, 

I m glad reading your opinion and description about social sciences today, styles of chairs, psycologists, and overall how the anthropologists are obsessed with predictions...the capitalisme mania of to buy everything, even the future, it is inisde the discipline with this repetition of cristal ball ..so the anthropologists have lost the practice of listening and looking, categorize, etc. etc and as well try to explain a long list of things, never ending... and always terrible..and the world of course it is in bad moment but with the predictions...we are not going to help very much to chnage this if we put our force on it...Is it not a kind of hiden under the bed face to a difficult situation?

.I m very close Slow Anthropology and in one hand I m very quantic and quechua..future is past and present, it is not a line...Sometimes I dont go to many places because the debats are around the predictions, common places, frustration, and there arent voices...

Anthropology has limitations and if we dont consider them we could be inside of a megalomania sympton rooteed  in our limitations

It is time to re-see the time and the aims in our work and how we are as well object of all these problematics that affect this period of the history. To see with scales could help as well.

regards, 

C

Huon, you know I think that social science is a big mistake, especially when used in policy or for prediction. The UK Treasury uses some 50 equations of a certain type to 'model' the whole national economy and make predictions about it. I have a friend who is a materials scientist -- he uses 27 of the same equations to measure experimentally small changes in metals and the exercise is entirely retrospective. He knows even more precisely than you and I what a con and farce prediction is for something like a country's economy or a national election. Poll is a Germanic word for head, counting heads -- but they counted in a town or village square to find out which side had won, not to predict something. Science as a whole extrapolates from consistencies revealed by the past; it is lousy at guessing the future.

If science doesn't work for figuring out the movement of society, we could fall back on the method of the humanities to learn what is going on, focusing on particular persons, places and events in search of wider truths. Some branches of the study of iterature, history, ethnography, case law, philosophy, social psychology rely on a humanities approach, but it is not enough. They are often not in a position to 'listen' to people live. The ethnographic revolution in anthropology is something to be proud of and to practice. But we should value literature, history, the work of some disciplines close to our interests and even statistics too. Mass Observation may have been great -- I have never examined it, but how often was it carried out? Anthropological ethnographers more often get things right than disciplines that use other methods, but how often do they carry out fieldwork in a lifetime? Their job consumes more time in teaching, writing and administration. Journalists often listen on a shorter-term and more routine basis.

In our age, people are interested in elections before they happen. The TV news feeds this impulse, but shallowly. Polls are the stuff of liberal commentators' columns, but they get it wildly wrong as well. I don't know what they lack more urgently, ethnographic method or a historical perspective. But polls and retrospective counts are a useful means of finding out some of what has already happened. Thus the voter abstention rate for the 2016 election (45% -- it will be several weeks before all the results can be assembled and analyzed definitively) was the highest in 20 years, but in the large swing states (Florida, Pennsylvania, N.Carolina, Michigan), where both candidates campaigned vigorously and which Trump won, it was lower than in 2012. This is impossible to predict by normal polling methods and would not be revealed by interviewing a few individuals or living with a handful of families around the country; and it was decisive this time.

I didn't make a bet this time -- I was too uncertain about the key variables and didn't have a positive conviction about the candidate, beyond not wanting either to be elected, but I thought that Hillary would win at low odds. In 2012, I and everyone else thought Obama would get re-elected easily, again at an unfavourable price. In 2008 I made £5000 (£1000 at 5-1) by betting on Obama to win by a largish margin, when the liberal commentators were fretting about McCain/Palin and the rednecks between the two nominating conventions. I have been betting on US presidential elections since 1960, usually with success or not betting at all. Timing and the prejudices of who you bet with are important. Otherwise I bet on big hunches --that Carter was a Rockefeller Republican in drag, for example, whose energy price hikes had killed jobs in the Rust Belt, so Reagan would win there and with Nixon's Southern strategy. I made bets with lefty academics who couldn't stand him before he was nominated as a candidate.

In 2008, Obama used new techniques of campaigning whose effects were not picked up by the polls, but more than that, after the 2006 mid-term elections, it was clear to me that enough Americans would not elect another Republic president this time. The main trend in 20th century elections has been for small margins in the popular vote to generate large swings in electoral college seats, but some recent elections had been close. The constitution makes sure that these elections are not popularity contests. All candidates know that. So I made my bet on the electoral college spread, higher than the polls and commentators predicted (he was black, young and inexperienced, the country wasn't ready etc). I didn't talk to anyone -- I wasn't even in the country at the time. But you tell me how ethnographic methods could predict the voter turnout and national outcome by intensive fieldwork in places as unlike as Vermont and Texas.

In January 1917, Lenin made a speech in Zurich where he confessed that he did not expect a revolution in his lifetime. He returned to Russia in April and was immediately impressed by the self-organized soviets on the streets: I realised that hitherto, he said, I was just a bourgeois politician with extremist rhetoric. In July he argued vigorously against the Bolsheviks joining actively in the mass insurrection. In September he was the only member of the leadership to urge going for the revolution now -- the party as a whole was dead against it. In October the Russian revolution was a done deal. Lenin sent a letter to the party explaining in general terms why his attitude had changed so radically in two months. It is published online as 'Marxism and insurrection'. What he didn't say was that, although the czar had abdicated in March, in the period July-September, millions of soldiers had returned home from the Eastern front of their own accord and the war minister, Kerensky, had occupied St Petersburg with the army. The ruling class was by now demoralised and significant sections of the population were literally up in arms.

The Trinidadian revolutionary and writer, CLR James, on his way to the US in 1938, stopped off in Mexico City to tell Trotsky that he had the 'negro question' all wrong and should delegate it to him. He had just published a book, based on 150 years of black uprisings on both sides of the Atlantic, where he predicted that Africans would win emancipation from colonial empire soon; no-one else believed that then, but he was right. His main contemporary source (he had not yet visited Africa) was George Padmore's (another Trinidadian he had grown up with) oral and written knowledge gained when he was Stalin's commissar for Blacks around the world in the Kremlin. James asked Trotsky why Lenin was right and the party wrong in September 1917, expecting him to say that he was a great politician, psychologist or whatever. Instead, he replied "Lenin always watched the people closely".  This is good for us, but he didn't do it by small-scale fieldwork: he did it by observing and analyzing the workers' strikes, the soviets, the mass exodus of soldiers and what they did with their weapons when they got back home around the country -- through the lens of is own historical vision and politics.

In this US election I read books like JD Vance's Hillbilly Elegy and Thomas Frank's Listen Liberal, critiques from a conservative realist and a disillusioned Democrat. I learned more from them than from news and commentary, but I still read the latter. Thomas Piketty (of mega-book fame) said early on in Le Monde that Bernie Saunders had shown the Democrats the way to the White House -- it wouldn't be him, but someone younger and "less white" later. The Democratic National Committee used all manner of dirty tricks to stop Bernie. And now they have lost their chance, because Trump, who overlapped with Bernie considerably on trade, jobs, the ruling elite and Hillary, came up with a better strategy. The youth who flocked to Bernie were less likely to vote, especially if they were told constantly that Clinton was going to win, and the white working class were unlikely to vote for a 74-year old socialist or Clinton for that matter. Trump is 70, but his looks and macho attitude diverted attention from that.

Trump was a better intuitive politican than his opponents could grasp. I doubt if he listened to anybody, but he knew that talking with conviction from his gut in front of crowds could work, when everyone else's eye was on the polls, the debates and corporate donations. The mainstream media delighted in telling the world that he had hardly any campaign funds left, a minute figure compared with Hillary's (I call her this, not because I am a closet misogynist, but to distinguish her from Bill). Hillary's unpopularity rating was almost as bad as Trump's. Many Americans, but few outsiders, by now have grasped in outline why and how Trump won where it counts -- in the electoral college. The number of retrospective books to come will be legion. We can't guess right now how much his presidency will be an extension of the campaign or different. Wait and see before you emigrate to Canada, Europe or Australia. They are allin pretty bad shape too and will be worse when WW3 breaks out. Trump's election will destabilize political elites around the world, not least the Chinese communist party. He will be belligerent, but I might put some money of Europe launching that war, as they did the other two.

Hegel thought that a dialectic of sameness-in-difference moves history. The political and intellectual culture of Western politicians, media and academics can't generate the methods needed to get it right, but they hang on out of sheer institutional inertia. Rousseau, Hegel, Marx and Engels, Lenin and James were open to dialectical methods on a scale adequate to understanding big events and social trends. I make no bones of asserting that ethnographic methods belong in the first category, not the second. They have a value, for sure, but they don't help us to develop a joined up picture of the world we live in.

I am with Keith here. A broader perspective is essential. Serendipitously I have been invited to teach Business Anthropology and Research Methodology in Taiwan next spring. Reading bits and pieces of what Taiwanese anthropologists have written in Chinese, I notice the absence of what I have labeled "ethnographic involution," the contemporary tendency to focus more and more effort on smaller and smaller places. My Taiwanese colleagues are strong believers in the fine detail and thick description than ethnography produces. At the same time, however, they are strong advocates of comparative method and teaching that requires reading ethnography from widely different parts of the world, to fertilize thinking about the details that their ethnographic research reveals.

Keith's argument for a broader historical perspective also resonates strongly with developments in research on China, where thanks to relationships created through area studies programs, anthropologists and historians examine similar topics from different perspectives, thus enriching each other's research. A lot can be learned about such classic topics as kinship and marriage, popular religion, and peasant and local market economies when the anthropologist can think about her field notes in a context informed by several thousand years of Chinese history.

There is one feature that seems obvious to me and I mention it as a side note because it has been missed in all the wailing and gnashing of teeth about Brexit and Trump--and this is because the self-elected commentators on our political system tend not to have very much sense of humour--which is this: huge numbers of voters for both these events made their vote with a shark smile at the sheer absurdity and carnival of bringing these results about. There is an atavistic, burlesque, element to all this which suits voters fine. They were well aware that in neither of these cases did the people involved-- Farage, Boris Johnson,... Trump --mean even a fraction of what they were saying; it was the grotesque comedy of the outcome that was attractive--not simply 'épater la bourgeoisie', but raucous laughter accompanied by the disclaimer 'just a prank bro!'. This does not fit into the contemporary political playbook especially in an era when politics is played out in terms of 'trigger warnings' and 'micro-aggresssions', in other words the high-minded pursuit of 'correct language', but that is how it is; the most successful politicians of the current era are comedians, more accurately clowns or buffoons in the literal sense. This grotesque element marks an important difference in the sameness that Keith pointed to that needs to be considered: is this history repeating itself as farce, or something quite new?

http://www.independent.co.uk/#gallery?CMP=ILC-refresh

 

Huon:  . . . huge numbers of voters for both these events [Brexit and Trump] made their vote with a shark smile at the sheer absurdity and carnival of bringing these results about. There is an atavistic, burlesque, element to all this which suits voters fine.

 

    This is an extremely important perspective to add to the social-cultural autopsies of the two Events now underway, which range from glib and often biased media commentaries to Keith’s deeply reasoned big picture interpretation deploying Hegel, Marx, Lenin, James & Co. 

    Lots of Americans walked into the voting booth and relished delivering a loud F. U. to the whole kettle of stinking fish that assaulted their very sense of being – not only to the graft and corruption of the Clinton Crime Family, but to the general paralysis of the society around them.  This sentiment was behind much of the error in the telephone and exit polls.  A down-home Amuricun would pick up the phone and find a stranger on the other end who didn’t tawlk laik folks at all, an uppity, pushy guy wanting to know how he was going to vote.  And they either hung up or, just for fun and with that shark smile, lied to him.  Screw with me and I’ll screw with you.  Ditto for exit polls, where bright-eyed, earnest young people pestered voters wanting to know how they’d voted. 

    These and other factors figure into the “secret Trumper” phenomenon that has been under-discussed: out of embarrassment, desire to avoid conflict with friends, family and coworkers, or just plain stubbornness, a great many people (we now know) did not acknowledge their support for Trump. 

    Following Huon, we should give a lot of weight to people’s taste for “the sheer absurdity and carnival,” not only in elections but in social life generally.  Aristophanes, Petronius, Bakhtin (who made much of the “carnivalesque” in literature) showed the way for modern joker-commentators like Tom Wolfe and George Carlin (both of whom, I’d wager, could outperform the pollsters). 

   Behind all this may be the looming question of why and if we should expect things to make sense.  Obviously the legion of pollsters did; they thought they could figure it out and tell us all about it.  They were wrong.  Hey, maybe at the heart of human existence is entrenched ambivalence and, yes, absurdity.  Either way, with Trump in office we’re in for Mr. Toad’s wild ride, to give another nod to Aristophanes. 

Lee, I agree that Huon has offered us an important perspective. Now what are we going todo with it? Besides, that is, adding more rants to the thousands that already clutter social media?

The dialecticians will tell us that what the bubbling up of comedy, in particular blunt satire, burlesque, surrealism, clowning, points to is a phase transition in the social state. But is this the right answer? Keith has placed his bet on revolution, but it is not clear what that means. Most of the revolutions of the last century were catastrophes causing the deaths of millions of people and the demoralisation and terrorisation of countless others. Darkness at Noon. In contrast, most of the good things in public life have been introduced by incremental change and negotiation; in Britain the old age pension introduced by the Liberals in 1908, Keynes' fiscal expansion policies, the National Health Service, free education etc. These were the results of boring stolid institutionalism responding to the principle of social betterment in the face of organised social pressure. This was all held together by a dutiful middle class until the mid-1970s when they too finally got bored with the programme, especially at the loss of social status it involved, and they gave the reins to Thatcher and the Hayekians. They looked on as the post-WWI nation state was shredded and they handed over the problem of social morality to computers.

I might as well break the self-imposed rule for this thread and offer a gross simplification and a prediction: at every stage it was the middle class who decided the outcome. They are the grouping who have the most cultural / educational capital, yet still have to work for a living; in other words they depend on the continued existence of the social institutions that their grouping set up in the first place over time. Short of an unprecedented catastrophe it will be the middle class who have to sort out this new situation. This time round the middle class is internationalised to an unprecedented degree; Yannis Varoufakis has recently given a model for a middle class transition (albeit this is a volte face from his previous view of Brexit as a punch in the nose for the German-led EU autocracy):

trumps-victory-comes-with-a-silver-lining-for-the-worlds-progressives

Can the spectre of this nationalist international [i.e. the new global alliance of nationalist-populists] be absorbed or defeated by the global establishment? It takes a great deal of faith to think that it can, in view of the establishment’s deep denial and persistent coordination failures. Is there an alternative? I think so: a progressive international that resists the narrative of isolationism and promotes inclusive humanist internationalism in place of the neoliberal establishment’s defence of the rights of capital to globalise.

In Europe this movement already exists. Founded in Berlin last February, the Democracy in Europe Movement 2025 (DiEM25) is attempting to achieve that which an earlier generation of Europeans failed to do in 1930. We want to reach out to democrats across borders and political party lines asking them to unite to keep borders and hearts open while planning sensible economic policies that allow the West to re-embrace the notion of shared prosperity, without the destructive “growth” of the past.

But Europe is clearly not enough. DiEM25 is encouraging progressives in the US, who supported Bernie Sanders and Jill Stein, in Canada and in Latin America to band together into a Democracy in the Americas Movement. We are also seeking progressives in the Middle East, especially those shedding their blood against Isis, against tyranny, and against the West’s puppet regimes, to build a Democracy in the Middle East Movement.

Trump’s triumph comes with a silver lining. It demonstrates that we are at a crossroads when change is inevitable, not just possible. But to ensure that it is not the type of change that humanity suffered from in the 1930s, we need movements to spring out and to forge a progressive international to press passion and reason back into the service of humanism.

(I wonder if it was out of a residual sense of humour that the proponents insist on calling their movement by an incomprehensible google-password style acronym?)

The immediate reaction from Savage Minds seems to be that academics should open their journals and model responsible argumentation and knowledge production for the public. To Lee's point about professional identity, this passage leaves little doubt where some anthropologists stand, "The real challenge now is to bring the quality of debate that is inside our journals out into the world, to model critical intelligence intelligence and deep erudition." Ha! I would love to introduce the rust belt to reviewer number one.

Rex's post is remarkable for both its patronizing tone and utter lack of introspection. Lawrence Cremin, the historian of American education, got it right when he quipped (I paraphrase), in most countries they have revolutions, in America we organize a course. Rex's response is hand in glove with responses to economic equality that hold the poor should act more like the rich. It is simply the the culture of poverty argument transposed onto the education system. This has been the elite response to social ills in America for well over a century now. And it is (still) a reprehensible response.

On the other hand, it might be productive to ask: Where is anthropology now? The election results weren't surprising if you have been reading Sarah Kendzior's work. She has been writing all year about the role of conspiratorial thinking (what could be more carnivalesque than conspiracy?) in the Trump campaign. But, not unlike Gillian Tett's warnings on finance, her warnings fell on deaf ears.

There is more to anthropology than ethnography, or good journalism. This was an election cycle accompanied by an enormous amount of what used to be called folklore. The writing was on the walls, and the baseball caps, at every Trump rally. It was there to see, and, dare I say, to learn from. A forward step at this point would be simply paying attention to phenomena easily written off as a product of ignorance, rather than offering (threatening?) to bring the academic journal to the rust belt.

Reply to Discussion

RSS

Translate

OAC Press

@OpenAnthCoop

Events

© 2017   Created by Keith Hart.   Powered by

Badges  |  Report an Issue  |  Terms of Service