Tuesday, Jul 28, 2009
One for Paul!
Love money: How job losses affect house prices
"According to my nifty spreadsheet, the correlation coefficient in my example was -0.81.
In other words, when unemployment rose, house prices dropped 81% of the time, and vice versa. This is quite a strongly negative relationship." Not exactly thesis standard but hey...
Posted by techieman @ 02:22 PM (1830 views) Add Comment
51 Comments
- If you do not have an admin password leave the password field blank.
- If you would like to request a password allowing you to add comments and blog news articles without needing each one approved manually, send an e-mail to the webmaster.
- Your email address is required so we can verify that the comment is genuine. It will not be posted anywhere on the site, will be stored confidentially by us and never given out to any third party.
- Please note that any viewpoints published here as comments are user's views and not the views of HousePriceCrash.co.uk.
- Please adhere to the Guidelines
1. 51ck-6-51x said...
Incorrect mathematics.
A correlation coefficient of -0.81 does not imply that 81% of the time the two moved together. Furthermore if this guy is working out the correlation coefficient using that lemma then there is a stronger correlation coefficient.
Furthermore correlations between financial data series are [ notoriously? well no! ] unstable. It's always worth analysing different time period separately to assess the general usefulness, and to test correlation coefficients for extreme values, and abrupt changes too - often where such things break down.
[ Also ( but less severely I think ) correlation is a linear dependence measure and hence one should consider the two series distributions - or rather their joint distribution should be normal ( Note: that is not to say that both distributions need be normal. ) ]
A reliance on an assumption of such constant correlations was ( in my opinion at least ) a big factor in the making of the credit crisis! ( Think CDO, CDO^2... )
What I'd like to know is what is the covariance of these data series? Anyone got the data?
2. luckyjim said...
Incorrect mathematics and incorrect logic.
Here's the same logic in action.
By brother lives up north, I live in the south. My life expectancy is nearly ten years longer than his based on ONS figures. I keep telling him to move down here so that he can live longer. He wouldn't even have to give up his cigarettes.
3. mark wadsworth said...
From the way the article was written, I think his -0.81 correlation is mathematically accurate, i.e. he used a spreadsheet function.
More to the point, he then lagged unemployment figure/changes by 18 months and got to a correlation of -0.84.
4. luckyjim said...
mark
Buy a house. House prices always* go up.
*(0.84 correlation to truth)
5. timmy t said...
Is it me or is this badly written? Two quotes below...
"A correlation coefficient of +1 shows that two sets of data are perfectly correlated. In other words, they rise or fall together at exactly the same rate."
"According to my nifty spreadsheet, the correlation coefficient in my example was -0.81. In other words, when unemployment rose, house prices dropped 81% of the time..."
Which is it? The extent to which the rates of change correlate or the extent to which they move in the same direction?
6. another alan said...
Isn't it that when unemployment rises there is a 0.81 fall in house prices (dependent upon the units used) on average, and nothing at all about probabilitites.
7. techieman said...
I KNEW this would get you all going!
The point is ... is the point valid that there is a positive corelation between unemployment and house prices GENERALLY?
Lets leave the nuances (coefficients) to the clever guys like 666. Personally i think there is a relationship but a lagging one as the journo tries to extrapolate. But maybe thats just curved fitted to correlate to my prefernce and/or common sense?
8. techieman said...
another alan - whether flawed or not i think he is saying that if a unemployment falls by 50%, house prices fall by 81% of that 50%.
Isnt that an option delta 666? In the same way as an option delta, that % must move up and down though.
9. another alan said...
Yes, dependent on the units (and it is on average) and it has nothing to do with probabilities at all. This still my read on it.
10. techieman said...
Silly me!
"another alan - whether flawed or not i think he is saying that if a unemployment falls by 50%, house prices fall by 81% of that 50%. "
I of course meant unemployment RISES by 50%, HP falls by 81% of 50% or 40.5%. Sorry.
11. another alan said...
I agree with you! And knew that your comment had nothing to do with probabilities unlike what the extract from the article appears to be suggesting above.
12. refusetobuy said...
I did a similar thing and got a correlation of -0.3. Wonder if he looked at changes in HP & Unemployment or just absolute levels
13. techieman said...
dear oh dear a bad afternoon for me - no hes saying that the falls are 81% of the time when unemployment increases - no quantative analysis. So when unemployment falls then 81% of the time HPs go up and when unemployment rises 81% of the time HPs fall.
But it doesnt say what percentage rise in unemployment will produce what percentage fall in HPs. So if we had a 50% uincrease in unemployment that wont produce - of itself - a 50% fall in HPs because we would still have say 80% of the working age population in work or x % of the overall population. That (% of population in work and able to support a mortgage in various strata) must be the key figure...
14. refusetobuy said...
Since he thinks it's a lagging indicator, why didn't he lag his data. i.e. compare July house price change to June unemployment. pfff
15. another alan said...
With the info presented, I think it is difficult to read too much into it. (Perhaps too simplistic to tell us much.)
16. brickormortis said...
I konw that peopl ewithout jobs can't get or pay their mortgages as well as people who have jobs. That, I now, is a correlation. I agree, though, that the analysis is flawed but there would be some use in examining the gradient of the regression ine of y on x where the dependent variable, y, is house prices. This might suggest the expecetd fall in house prices, lets say, per 10,000 jobs lost which, given teh 0.81 cc might be worthwhile (if indeed teh relationship is known to be linear).
17. refusetobuy said...
We need flashmans black box :-)
18. mystie010 said...
Some of the comments really made me smile because just like we criticise articles in the Daily Express for example about rising house prices; there are a lot of comments that are criticising Cliff D'arcy because he sold to rent. Just as we sometimes say well they have a vested interest so that's why they wrote what they did. Some people in the comments section have questioned his impartiality as an author as apparently he wants house prices to tank due to his own sold to rent position. It's almost like there are two very distinct camps in the UK right now. Those with property and those who would like to buy property. It will be interesting to see which way prices actually do go because I get the feeling that there is a huge amount of resistance to any price drops. That's just my observation for what it is worth.
19. Adrian said...
If you read the comments for the original article it is only Ainsworld who correctly interprets the meaning of the correlation co-efficient - "From a correlation of -0.81 you could say that 66% of the variation in house prices can be explained by unemployment (66% is 0.81 squared), not 81%"
20. flashman said...
I agree that in strictly academic terms, his correlation coefficient is worthless. However, I don’t think that, in this case, we should get too hung up on non-degenerate random variables and Boolean functions. In an ideal world I would like to see a robust, verifiable model without arbitrary assumptions but we absolutely need to make an assumptive intervention because there is no suitable data set available and almost nothing of this nature can be verified absolutely with the ‘approved’ application of pure data. A Mark Wadsworth says, “ there has to be a number” so we’ve got to do whatever it takes to get it (groping in the dark doesn’t help us make decisions).
His correlation number is quite useful if we are prepared to intervene with the leap of faith assumption that the effect of unemployment on house prices in non-linear and lagging. By non-linear, I mean that as unemployment gets to higher levels, the effect it has on house prices become exponentially/disproportionately higher. By lagging I mean that redundancy payment, savings and disbelief are known to keep people going for a while. These assumptions certainly gel with what we are witnessing and thinking (witnessing that thus far unemployment has hardly made a dent whilst intuitively thinking that it must eventually have a big effect)
To answer techieman’s question - This guy’s correlation coefficient is very useful if you believe that it is reasonable to fortify it with the ‘non-linear’ and ‘lagging’ assumptions. If you don’t believe these assumptions are reasonable then the correlation coefficient is worthless because as 666 says, his method is supect and too many causal correlations and variables have been discounted.
Personally, I am quite happy with the assumptions, so I think the correlation is very useful
21. flashman said...
refusetobuy: How on earth did you get –0.3? I though they were supposed to be doing a good job on the latest generation of risk interns :)
22. quiet guy said...
An extract from the Nationwide May 2005 House Price Index (http://www.nationwide.co.uk/hpi/historical/MPR0505.pdf):
"The biggest threat to the housing market is the labour market. Unemployment was the biggest factor affecting the market in the last cycle, having a two-fold impact. First, on borrowers’ ability to meet their mortgage payments, and secondly on the levels of distressed sales. Both of these, but particularly the latter, caused house prices to fall sharply in the early 1990s. The labour market is now quite
different. Unemployment levels in the UK are now at their lowest levels since the 1970s and the numbers in employment continue to rise – employment has grown by more than 180,000 in the last year. While economic growth in 2005 is expected to be much lower than in 2004, the consensus opinion is that there will be only a very slight increase in the rate of unemployment."
At the time, I didn't (want to) believe it but it does appears that unemployment is the required ingredient to force sales. Not nice but with hindsight, it makes sense.
23. quiet guy said...
@mystie010
"It will be interesting to see which way prices actually do go because I get the feeling that there is a huge amount of resistance to any price drops."
Yes, so far the reluctance to drop prices has been very strong. I recall discussing this with someone who bought at the top of the last bubble; they just hung on grimly for a decade until the next boom, unable to move for years.
24. timmy t said...
Flashman - can you explain what the guy is trying to say please? I think Techieman got it right in post 13 (Wrong in 8) - do you agree? In which case, all he's saying is that as unemployment is rising, there is an 81% chance of house prices falling. Why is this so useful?
25. flashman said...
timmy t: yes techieman got it wrong in 8 and I think you'd better ask him to explain what he means in 13. Its not for me to say.
This guy is saying that there is a strong correlation between unemployment and house prices. Specifically he is saying that when unemployment increases, house prices almost always fall. He has quantified 'almost always' as 81% of the time. Don't get too hung up on his numbers because there will always be a debate on how these things should be calculated and he has, out of necessity, over simplified his equations.
His article title suggests that it has not happened this time but then he goes on to say that it is logical that there is a delayed effect between the two. In fact he says that when he introduced some measure of this delay into his sums the probability of rising unemployment causing falling house prices increased from 81% to 84%.
For what its worth, I think the correlation between thee two things is so strong and obvious that it can be considered the 'master' correlation and it should therefore be taken seriously. My mob (economists and analysts) always over complicate things and common sense is usually more important than blind maths. Other factors like interest rates are at play here but this is not nearly as strong as the unemployment factor and should therefore, eventually be overcome. Also, as I said earlier, when unemployment gets to a more serious number it will have a proportionately higher effect. This is what I meant by the effect not being linear. My best guess is that when unemployment reaches about 2.75 million, we will see an acceleration in house price falls
26. mark wadsworth said...
The correlation coefficient of -0.84 does not mean that prices fall 84% of the time in the month 18 months after unemployment rose, it is far more subtle than that (it's got to do with averages of square roots of means and all sorts of stuff like that).
But Flashman agrees with me "There has to be a number" (even if we don't know what it is).
And as Mystie010 points out that we are in two camps - those who think/hope that prices will rise and those who think/hope that prices will fall (and I am firmly in the latter camp!).
So all this correlation does is give us (who are in the latter camp) the best vague idea of the timing - house prices will [probably] turn up again 18 months after unemployment starts dropping. And given that unemployment only starts falling a year or so after the economy picks up again (which it will - but that might be months or even one or two years away), that all gives us the long, flat trough in house prices as seen in the early/mid 1990s.
PS, I might be a rabid property bear now, but in the late 1990s I went round shouting Buy! Buy! and did so myself (house and four buy-to-lets, if you must ask).
27. timmy t said...
Thanks Flash - Makes perfect sense. And on the basis that if you compare Halifax and Nationwide you'd be forgiven for thinking that house prices don't even correlate with house prices, I think you're right in saying that common sense is as important as any maths!
28. paul said...
For me, this is one of those odd stats for which the precise numbers have little meaning.
A little bit like when wholesale oil prices soar, but there's no change on the forecourt prices is it right to conclude that they are disconnected? Only if you have learning difficulties.
One inevitably follows the other. It doesn't matter if you don't see it yet - it's in the pipeline. If unemployment is rising there is only one trajectory for house prices.
29. Mathematici said...
The value of -0.81 means very little without knowing the size of the dataset being considered. The "81% of the time" quote is absolute nonsense. If you know the size of the data set, it is then possible to gauge the confidence, expressed as a percentage, with which you can make the statement "these two variables are negatively correlated, and these values did not occur by natural variation".
If the data set were just 2 pieces of data, -0.81 would mean absolutely nothing.
If there are over 100 pieces of data, then -0.81 means you can be more than 99% confident ofthe two variables being negatively correlated. There are significance tables to help decide the confidence levels.
So be wary of journalists using maths they don't understand...
30. mark wadsworth said...
@ Mathematici, he said he used quarterly for the last 26 years, so that's over 100.
31. vindicated said...
My brain has officially melted...... wibble wibble.......
32. tenyearstogetmymoneyback said...
quiet guy @ 23 wrote "I recall discussing this with someone who bought at the top of the last bubble; they just hung on grimly for a decade until the next boom, unable to move for years."
Been there, done that (the clue is in the user name).
One of the biggest shocks was in the early 1990s when one of our software guys was refused redundancy insurance,
When he applied as the insurer didn't like the look of the company (Siemens). I also remember doing the calculation that (including depreciation) my house was costing me about £1300 a month when I was earning about £1200 a month.
What is interesting this time is how everyone expects a quick fix and for things to be back to "normal". Back in the 1990s
price drops just dragged on for years and years until everyone forgot that houses could go up in price and all the "smart" money
headed towards Technology Shares.
33. tenyearstogetmymoneyback said...
Just thought of something else for the mathematicians to try and correlate
Corporate Instability.
Back in about 1996 when it looked as if house prices had bottomed out, and after our division in the company (Siemens)
I mentioned in the last post had made a profit for the first time in years, they put us up for sale.
I certainly didn't want to be moving house then when I didn't know who I would be working for next year.
Looking at the current situation I wonder how many people working for Lloyds, Halifax, Honda, British Airways etc etc etc
are thinking "now would be a good time to buy a more expensive house" ?
34. flashman said...
mathematici: In my business we have young analysts fresh from college and some older working guys. The younger guys are often a bit prissy about their maths but tend to be useless because they get hung up on purity and robustness. The older guys have the experience to intervene when it becomes apparent that sticking to the text book will not produce anything useful. Experiences mathematicians know how to avoid the blind alleys that prissy mathematicians get stuck in. They have to do this because in the real world we need answers and guidance. It's what the approximation theory part of applied mathematics is all about.
As an aside, it is unfair to blame these guys for the CDO type mess because traders and bankers gleefully misused their work to suit themselves
35. refusetobuy said...
Appologies if this doesn't work
36. refusetobuy said...
Ooops!
That's my source data. Whatever I do, I can't get anywhere near -.8 correlation
37. refusetobuy said...
38. refusetobuy said...
Thats slightly better formatting.
Sorry for taking up all the page.
At least this gives a datasource for us to use.
39. 51ck-6-51x said...
Wow, I thought I felt the cogs turning while I was away from my machine!
Flash
- You quite correct in your observation that in the world of financial forecasting there is little room for thinking about every hard and fast mathematical rule as you'll never make a trade. A very trader-centric attitude, but true when it comes down to it.
I would add, however, that it is probably well worth letting those prissy analysts think about these rules and when they may become important. As you pointed out the relationship may be non-linear, albeit indiscriminable for most of the range - it is thinking about at what point models starts to break down and what happens in these relatively rare cases that separates you from the crowd; that is when you can really shine ( or even just not lose when others are ).
refusetobuy
- You need two data series to attain a correlation.
40. 51ck-6-51x said...
41. 51ck-6-51x said...
^^ Go on, hover that mouse over the image; you know you want to. ^^
42. 51ck-6-51x said...
refusetobuy
Ah - you dohave two data series!
43. letthemfall said...
I may as well add my two penneth since you're all enjoying the sums so much. Correlation is just a measure of the strength of association between two variables; the number itself does not measure a quantitative relationship between them. Yes, one can object to its use on theoretical grounds here, but since it is only telling us what is pretty obvious anyway, there's not really much to object to.
44. 51ck-6-51x said...
refusetobuy
I calculate your sample correlation as
-0.23427125954678
However your house prices are in nominal rather than inflation adjusted, that could make a huge difference.
45. 51ck-6-51x said...
letthemfall
- Not sure if you are being ultra-smart...
Correlation is a numerical measure of linear dependence of two random variables lying in the continuous range [-1,1] . It is, therefore, quantitative.
I don't think anyone is objecting to the use of the measure in such analysis. I for one am not, but rather I would always warn of two caveats which are all too often taken for granted as non-existent, but may be pertinent:
i) Real correlation may change over time and it may or may not be a function of time; and
ii) The joint distribution of the set of data series may not be normal.
(i) is hard to take into account, since if correlation is not constant there is no instantaneous measure to analyse - this is why I'm not sure if you are being ultra-smart! Mind you this does not matter too much as long as we bear it in mind - one can measure the correlation over many short periods and observe how it changes.
Ignoring (ii) can be very dangerous though, and is often the source of mal-reporting on correllations.
( there is of course (iii) - correlation does not imply causation, which is mentioned in the article and is remarked upon in the comic strip I poste above )
46. letthemfall said...
51ck
Correlation produces a (unitless) number, yes, but doesn't quantify how one vbl changes in relation to the other (as in regression). That's what I meant by not quantitative.
I agree with your 3 points, but with regard to normality, most observed data is non-normal, although a large enough data set overcomes this problem in many cases. I find that many users of statistics tend to exercise themselves over particular aspects, notably statistical significance, when often enough a graph or two tells you all you need to know without recourse to the fancy stuff. With the housing figures I don't see much point doing statistical calcs, partly for the reasons you say, and mainly because the conclusions are clear anyway.
I could be being infra-smart here!
47. flashman said...
666: I was once a prissy analyst but I now have a foot in both camps. I am not really a trader (in the pure sense) so it is not the need to make at trade that has changed me. Being ridiculed by fat bond traders back in the day is partly responsible and age and cynicism did the rest.
48. timmy t said...
So in summary:
IT IS VERY LIKELY THAT AN INCREASE IN UNEMPLOYMENT WILL CAUSE HOUSE PRICES TO DECLINE.
Glad we're all agreed on that then.
49. 51ck-6-51x said...
letthemfall
- You are correct that the correlation and regression are different. You are also correct in your implication that regression analysis is a useful tool here, especially once causal reasoning has been ascribed successfully.
However, correlation does quantify how one random variable changes in relation to another, and is quantitative; it just does not quantify the dependence ( like regression ).
Furthermore looking at graphs may be useful as a starting point, but is notoriously confusing. The human mind will perform regression analysis of it's own, without verifying dependence and skewed by a-priori judgement - this is dangerous and is why technical analysis can be dangerous ( it can be useful, but one has to design the rules independently of trading decisions and follow those rules to the letter so as not to be fooled by one's privative noodle! )
The conclusions are indeed clear, but I think this thread quickly moved beyond the case in point.
50. 51ck-6-51x said...
letthemfall
- This may help ( ? )
Formally, the square of the correlation coefficient is the proportion of the sum of squares of the dependant variable which is accounted for by a sum of squares residual regression.
51. 51ck-6-51x said...
timmy t
:o)
You are so right. The keyword in your post is CAUSE. Unemployment will CAUSE house prices to fall!