Dealing with variables that depend on each other.

This article attempts to follow the effect of the recession on the games industry using a graph. It seems that industry growth has been in decline since a peak in Sept 07, and that since March sales have been shrinking dramatically. I think it's actually an interesting example of how particular graphing tools are only useful in particular situations.



The graph plots year-on-year change in sales figures. Year-on-year figures are useful, because they allow you to normalise for the variation across the year. Suppose you're running a games company. Games traditionally sell very poorly in the summer and rocket off the shelves around Christmas. If you plot a simple graph of sales figures over the year, it really doesn't tell you much about how your business have been doing, because the variation you see is almost entirely down to the expected seasonal peaks and troughs. By comparing this year's figures to last year's figures, however, you can see whether your company had better summer.

My issue with the graph is that it plots these figuers over several years, and therefore the data points for different years are actually dependent upon each other. If you have an unusually good month this year, the same month next year will look bad by comparison. The chart tells us that Sept 07 was an incredible 80% better than Sept 06. In turn, Sept 08 looks pretty bad, because of the huge sales the year before. In fact, Sept 08 could've been a pretty good month.

A more informative way to represent this data would be as a series of individual year-on-year graphs, and highlight that the areas above the zero line are periods of growth. With a bit of cropping, I get this:



Sales were actually up fairly consistently for the year Sept 07 to Sept 08. Contrary to the downward slope, this was a growth period, it's just that growth in the summer wasn't quite as impressive as the growth in the summer, and January was more or less static. Here's some colour, and with the scale for decline extended to match the scale for growth:



Now we have to look seperately at Sept 08 to Sept 09, and remember that we're comparing it to the year before. It's telling us that sales in the winter of 2008 were even better than they were in the winter 2007, but summer sales have dropped by comparison.



The trouble with sticking these two charts next to each other was that a single graph was presenting two different sets of figures as though they related to the same thing. Each of the two charts above uses the previous year as its baseline, so you can't really do that. Presenting them as seperate charts, and applying colour coding to indicate the importance of positive and negative values and not just the movement of the line, clarifies this.

To really grasp what's going on over the whole period, we have to remove the dependence between the two charts, and give them the same baseline. To do this I'll take Sept 06 to Sept 07 as our starting year and plot the change compared to that year throughout.



For most of the year, sales in Sept 08 to Sept 09 were actually better than those in Sept 06 to Sept 07. The growth in the summer of 2008 concealed this in the original graph, implying that sales this year had fallen far below 2007 levels. Unwittingly building in this dependence between the different parts of the line made a big difference to how the results were presented.

Given that the years 2006 and 2007 were both huge success stories, selling at that rate is objectively impressive. It's been pointed out to me that my sales analysis is rather naive, though. Games publishers have investors, and they expect to see the publisher sell games at an ever faster rate with each passing year as an indication that the company is growing. Even if the company's still selling well, that growth has to be there. Correspondingly a year-on-year shrink isn't good news, even if it's a shrink to sales volumes which were once unprecedented.

UK Government sends message to its advisors: You Don't Matter

The UK Government has just sacked the head of its drugs advisory body, for claiming that his organisation's work had been "devalued" in favour of the government's existing policies. It's a claim that's hard to argue against, given that the government soundly ignored this body's advice in the past whenever it conflicted with their pre-established policy stance (e.g. this and this). By simply using science as a way to promote its existing agenda, and not as a way of making actual decisions, the government had been sending a clear message to its advisors. "Your decisions and expertise do not actually matter. You are here to service us." This firing merely underlines that truth.

Given that this is the government behind the "dodgy dossier", such evidence- and reality-averse approaches to running the country are hardly surprising. However I have little reason to conclude that any of the other parties would act any differently. Is there a party out there willing to act on what the cold, hard facts tell it, or must I resign myself to living in a country where decisions are made on the basis of preconception, supposition, instinct, hysteria, and the opinions of whoever can best whisper in the MPs' ears?

Update: The article has since been expanded. I find this quote especially telling:

I cannot have public confusion between scientific advice and policy and have therefore lost confidence in your ability to advise me as Chair of the ACMD. I would therefore ask you to step down from the Council with immediate effect.

The letter asking for him to step down:

It is important that the government's messages on drugs are clear and as an advisor you do nothing to undermine the public understanding of them.

Clearly, if there's a mismatch between policy and the evidence, it should be kept out of public sight. So much for the ACMD's independence.

Here's to you, helpful bookseller

I actually found this anecdote on "Not Always Right" kind of charming. Chemistry gets a bad reputation because of the connotations of "chemical", and they not only managed to explain what chemistry is in a simple way, but they convinced someone who was deeply opposed to the idea! So here's to you, whoever you are. I've stuck a printout of that page up above my desk.

The man at the reins of the caveman at the reins of the Soil Association

I've been following the response to the recent systematic review of the health benefits and nutritional content of organic food with some interest. It's left me with an odd sense of despair which I'm trying to figure out. It's not so much the Soil Association's piss-poor retort¹, but the sense that neither statement has any bearing on the public's feelings on the issue, or policy.

The Soil Association's willful misrepresentation of a powerful scientific tool (systematic review) is odious, but I don't think that it's going to adversely affect the public's understanding of systematic review. Anyone who blithely swallows their implied argument of "cherry picking" does so because it's a pleasing rationalisation of their existing bias, not because it makes any logical sense, so I doubt they will form some general conclusion that systematic reviews are a poor tool. Likewise, I doubt many who accept the report's findings will do so because of an understanding of its methodological rigour. This is an example of "confirmation bias", the tendency for us to favour findings which match our preconceived notions, and to dismiss those that contradict them.

I've recently finished reading Dan Gardner's "Risk" ², which draws extensively on evolutionary psychology and the structure of the mind to discuss how we rationally and irrationally interact with risks. At the heart of the book is the notion that our brains run on two "systems", essentially the instinctive and the rational. The animal-like and instinctive "system 1", crafted by evolution, calls its shots using various hard-coded and unconscious "heuristics". The rational "system 2", the conscious human, applies flexible logic, imagination, reason, and emotion.

Unfortunately for the human, the caveman brain gets to call the shots by default. Speed is the issue. If the sleek modern mind scratches its chin as we're approached by a tiger, heedless of the caveman's instinctive cry to flee, we're hosed. So the caveman gets first dibs, and it's up to the modern mind to attempt to interpret correct that response if it gets the time. Often, its override doesn't work very well, especially if the caveman's in an imposing emotional state. The caveman's decision can stand unopposed, with the modern man only stepping in to provide a post-hoc rationalisation. This is bad enough when I've elected to buy Legends Springer, but it's a serious concern when the caveman winds up making public policy decisions.

The depressing - and hopefully inaccurate - conclusion is that our rational minds are cludgy patches to a piece of software that is meant to roam the savanna, a piece of software as cold and remorseless as any nightmarish android.³ I've concluded that this idea was floating around in my mind after I finished Gardner's book, and that what bothers me about the Soil Association's reaction, and everyone else's for that matter, is that it's such a stunning reminder of what irrational beings we actually are. If we're really just puny minds at the reigns of mighty cave men, what hope is there? What power exists that could "change our minds" for more than a lifespan?

I'm finishing this blog entry a few days after I started it, so my emotional state isn't quite the same. That last question is no longer rhetorical. To what extent can our modern mind over-rule the ancient one? What can we, as individuals or communities, do to take a more rational approach? Can politics move beyond using science as another rhetorical club when it's in its favour and discrediting it when it's not? I have no reason to believe so, but I hope that our rational minds are up to the job.

¹Goldacre's links once again providing a fun counterpoint to his article titles.
²Amazon link.
³In the extreme case, we have this result, which suggets the conscious mind's freedom is an illusion, and the unconscious brain has already made our decisions for us! However the decision in question - pressing a button when the time seems "right" - is probably instinctive anyway.

This is going to make my job so much easier

Dubious research says little about videogames

"Video game conditioning spills over into real life", New Scientist says. I can't say I'm convinced.

My main objection is that the conditioning they're talking about had nothing to do with the virtual environment in question. In the experiment (abstract here*), subjects were hooked up to a bicycling video game. Apparently they were informed that it was to test a new sports drink delivery system, so I would imagine the scenario is one involving exercise bikes which control the speed at which the player travels. (Details would be appreciated if anyone can read the article.)

Whenever their own team passed by, the players were given a mouthful of sweet drink via the hose; whenever the opponents overtook them, they were given a mouthful of salty tea. Both teams had easily distinguishable logos on the back of their shirts. The researchers then called the participants back in for an fMRI session, and looked at how they responded to the presence of the opposing team's logo. It turns out, they didn't like sitting near it (on a towel) in the waiting room. The response to the logo was correlated with activity in some brain regions on the fMRI.

They conclude that "stimulus in the virtual environment can acquire motivational properties that persist and modify behavior in the real world". Well, I call bullshit. The main stimulus here wasn't in the virtual environment. They were being fed unpleasant drinks whenever the opposing logo was on screen, and pleasant drinks whenever their own team's logo appeared. That's more than enough to condition the subjects against the opposing logo.

Imagine a "Clockwork Orange" scenario where you were tied to a chair watching "Jaws" and I pinched you in the neck every time Roy Schneider spoke, and gave you delicious cake whenever the shark killed someone. Would I be able to reasonably conclude that the movie had conditioned you to hate Schneider's voice and love seeing people get mauled?

Let's speculate. What if they gave the players a positive taste when they were overtaken by opponents, and a negative one when they were overtaken by their own team? Without altering the "virtual world", I'd be willing to bet they'd get entirely the opposite response to logos. This would quite easily invalidate the idea that "stimulus in the virtual environment" was responsible. This test, while obvious, was not performed.

Furthermore, they could've just as easily sat the subjects down in front of the game, sans straws, and performed the same experiment. Failure in a game is often accompanied by a negative visual and audio stimulus, however even this is probably not necessary for conditioning, as being passed by the "other team" is supposed to illicit a negative response in itself (you are losing). This would be much closer to an actual video game experience**, and any conditioning shown would purely be the "video game conditioning" the New Scientist article speaks of.

The act of conditioning their subjects with drinks is so unnecessary to the experiment's goals, and puts such a massive hole in the research, that I have to wonder what the hell they were thinking.

They find an interesting correlation between the subject's self-perceived immersion in the game and their susceptibility to the conditioning, which is much more valid a conclusion in my opinion. I'm curious as to whether one's involvement in a virtual world makes one necessarily vulnerable to being conditioned, in comparison to a neutral situation, or watching TV, or reading a book. Perhaps one's degree of involvement in any fiction- or non-fiction media increases susceptiblity to conditioning. However that research simply has not been done here.

A better title for the study would be "real-life conditioning spills into real life". Sadly, that's not nearly as headline-grabbing.

*New Scientist provide the article's DOI but I couldn't find a way to actually use it on the journal website, which was irritating. It's The Journal of Neuroscience, January 28, 2009, 29(4):1046-1051.

**I don't know about you, but I don't chug salt every time Mario pops his clogs

Rookie footballer to be turned into one-man "clone" of greatest living footballers

So, I'm reading the Daily Record in the barber shop, and I come across this (which admittedly loses some of its impact with the omission of the print version's diagrams). Arton Baleci is being (self-)trained in the techniques of eleven elite footballers, given state-of-the-art visual acuity training, and subjected to a regime of neuro-linguistic programming (that's a whole other rant), to create a "clone" combining the capabilities of the greatest football players of our time. Does that sound familiar to anyone? What dark powers could be bankrolling his training?

("The Beautiful Aim", the website for Baleci's project. My advice is that if anybody offers him anything involving nanomachines, he should decline politely.)