The Discrepancy

[scottbeard]

2 minutes ago, sell2rent said:

95% CI of +-1.5% would involve the total number as well as a percentage of the sample, it would also rely as people have already said on the first 10% not being biased (if the next tranche shows similar the precision would increase), and also relies on all of it being a normal distribution which it probably isn't. It might not be invalid, but it is prematurely drawing a conclusion. It may be making an interesting idea to watch to firm up later. Not a statistician, but had to calculate 95% confidence intervals for a publication 25 years ago, did the calculations myself and got it through peer review.

Perfectly put.

And moreover, it is absolutely NOT proof of any kind of mistake or cover up.

It's the same frustration as the people who do their head in over the average age of COVID deaths (82) being same as the average age of all deaths (also 82) and therefore (mis)concluding that COVID was a nothingburger.  Of course if I went out and poisoned a hundred 82-year-olds the average age of death amongst them would ALSO be 82, and yet no-one would argue those killings were just natural/normal/unpreventable.  That 82 is not an irrelevant figure, but is only part of the picture.  Same with this raw data here.

[fellow]

12 minutes ago, sell2rent said:

95% CI of +-1.5% would involve the total number as well as a percentage of the sample, it would also rely as people have already said on the first 10% not being biased (if the next tranche shows similar the precision would increase), and also relies on all of it being a normal distribution which it probably isn't. It might not be invalid, but it is prematurely drawing a conclusion. It may be making an interesting idea to watch to firm up later. Not a statistician, but had to calculate 95% confidence intervals for a publication 25 years ago, did the calculations myself and got it through peer review.

You mean like the LR themselves draw a premature conclusion every month? The LR's own HPI reports are also comparing 10% of data from 2023 with a much bigger data set from 2022 so why are people criticizing Charlie for looking at the non manipulated average from the raw data? How do we know the LR's overly complicated algorithms aren't skewing their average sale price upwards, especially when the mix of buyers has likely changed significantly compared to previous years, which could result in some of their model's assumptions being inaccurate?

[sell2rent]

13 minutes ago, fellow said:

You mean like the LR themselves draw a premature conclusion every month? The LR's own HPI reports are also comparing 10% of data from 2023 with a much bigger data set from 2022 so why are people criticizing Charlie for looking at the non manipulated average from the raw data? How do we know the LR's overly complicated algorithms aren't skewing their average sale price upwards, especially when the mix of buyers has likely changed significantly compared to previous years, which could result in some of their model's assumptions being inaccurate?

You may well have a good point there, but my first question to which I've not sought the answer from the source, is are they doing the same thing with the same method? It would be unexpected for the highest quality official statistics to have intentional or accidental methodological flaws, and therefore would require good quality evidence to prove that point. This is not the same position as blind acceptance of official statistics or ruling out that there is an unrecognised methodological flaw, but Alex and Charlie could simply be cherry picking to suit a narrative where they would dismiss similar assertions from thin data if they went against it. It's natural, there is a middle ground I hope between skepticism and conspiracy, not that I'm accusing you of either, you make an interesting point.

[scottbeard]

32 minutes ago, fellow said:

You mean like the LR themselves draw a premature conclusion every month? The LR's own HPI reports are also comparing 10% of data from 2023 with a much bigger data set from 2022 so why are people criticizing Charlie for looking at the non manipulated average from the raw data? How do we know the LR's overly complicated algorithms aren't skewing their average sale price upwards, especially when the mix of buyers has likely changed significantly compared to previous years, which could result in some of their model's assumptions being inaccurate?

I believe that is incorrect.  The land registry wait until 3 months or so has passed, so yes they are comparing incomplete data with complete data from a year ago BUT it will be a lot more than 10%, and also they weight the results to an average property mix rather than use it unadjusted so that if (say) just a lot of expensive houses happened to sell in one month it doesn't skew the overall conclusion

10 minutes ago, sell2rent said:

You may well have a good point there, but my first question to which I've not sought the answer from the source, is are they doing the same thing with the same method? It would be unexpected for the highest quality official statistics to have intentional or accidental methodological flaws, and therefore would require good quality evidence to prove that point. This is not the same position as blind acceptance of official statistics or ruling out that there is an unrecognised methodological flaw, but Alex and Charlie could simply be cherry picking to suit a narrative where they would dismiss similar assertions from thin data if they went against it. It's natural, there is a middle ground I hope between skepticism and conspiracy, not that I'm accusing you of either, you make an interesting point.

 No, basically, as above.  Though it is true that Land Registry themselves do publish figures on incomplete data, which is why they get revised later.

[Smith]

If the drops in non-adjusted figures are as dramatic as Charlie and Alex are claiming, then surely we could go back a few months and we should see the bulk of the percentage falls.

[Armus]

25 minutes ago, Smith said:

If the drops in non-adjusted figures are as dramatic as Charlie and Alex are claiming, then surely we could go back a few months and we should see the bulk of the percentage falls.

I think LR publish the data, I just can’t be bothered to go back and check.

[Armus]

@smithy looks like Housing Stig has done what you’re asking. See 07:50 in this video. To me the correlation seems strong so I think they’re full of shit.

 

[Smith]

Interesting. I'd like to see raw numbers compared between, say, June '22 and June '23, to see whether a proportion of those huge drops is evident. I may try to find these numbers over the weekend, if I get time.

[APOLOGIES for not quoting anyone in my replies; this is still impossible to do on Android as ads cover the pop up.]

[Armus]

12 minutes ago, Smith said:

Interesting. I'd like to see raw numbers compared between, say, June '22 and June '23, to see whether a proportion of those huge drops is evident. I may try to find these numbers over the weekend, if I get time.

[APOLOGIES for not quoting anyone in my replies; this is still impossible to do on Android as ads cover the pop up.]

You can get the data here: https://landregistry.data.gov.uk

The article Charlie has posted. They’ve only gone back to May 2022 though: https://mhwc.co.uk/blog/house-price-discrepancy/

[A.steve]

8 hours ago, scottbeard said:

Of course if I went out and poisoned a hundred 82-year-olds the average age of death amongst them would ALSO be 82, and yet no-one would argue those killings were just natural/normal/unpreventable.

It's interesting what different people think they can get away with - isn't it?

[bearishonhouses]

As we all know, LR figures are subject to revision. The revisions for the house price index seem to be only for 12 months. (It is possible, the revision period is longer, but there were no revisions more than 0.005% 13 months out.) 

col (1) is month of interest
col (2) is LR house price index ('HPI') for England and Wales as reported in December 2022
col (3) is HPI for this month as reported in August 2023
col (4) is percentage change in index  revision /initial value)

Unsurprisingly, the biggest revision between December 2022 and August 2023 in percentage terms is for October 2022; the earlier months had already been revised between initial computation and December 2022. As it happens, over this period, the index was being revised downwards as more data were processed.  However, I do not doubt that the statisticians at LR monitor the direction of revisions very carefully. If they find that revisions are, on balance, downwards, I'll wager they would  adjust their initial reported estimate to take into account the anticipated direction of what would be otherwise be a revision.   

[Smith]

10 hours ago, bearishonhouses said:

As we all know, LR figures are subject to revision. The revisions for the house price index seem to be only for 12 months. (It is possible, the revision period is longer, but there were no revisions more than 0.005% 13 months out.) 

col (1) is month of interest
col (2) is LR house price index ('HPI') for England and Wales as reported in December 2022
col (3) is HPI for this month as reported in August 2023
col (4) is percentage change in index  revision /initial value)

Unsurprisingly, the biggest revision between December 2022 and August 2023 in percentage terms is for October 2022; the earlier months had already been revised between initial computation and December 2022. As it happens, over this period, the index was being revised downwards as more data were processed.  However, I do not doubt that the statisticians at LR monitor the direction of revisions very carefully. If they find that revisions are, on balance, downwards, I'll wager they would  adjust their initial reported estimate to take into account the anticipated direction of what would be otherwise be a revision.   

Thanks very much for taking the time to dig out these figures and do the calculations!

If I'm reading it correctly then your analysis shows that the final published values (based on all transactions) for HPI do not historically vary by more than 2% from the initial published values (based on initial 10% or so of transactions).

This means that criticisms of Charlie and Alex's methodology based on the fact that they're using the initial published values for September 2023 are invalid. As they say in the video, even if the final published values vary by 2% this barely alters the magnitude of the price drops they've uncovered.

For example, the raw figures show that sale prices of flats and terraced houses have dropped on average by 19.6% in a year. Once the full data set is available, based on what you've found, the final figure is extremely unlikely to end up being less than a 17% drop or more than a 22% drop.

This is the point Alex is making in the video when he talks about confidence limits. He's saying that whilst the limited number of data points for September 2023 makes it impossible to be totally accurate, the magnitude of what they've found is not likely to change. 

[Armus]

51 minutes ago, Smith said:

Thanks very much for taking the time to dig out these figures and do the calculations!

If I'm reading it correctly then your analysis shows that the final published values (based on all transactions) for HPI do not historically vary by more than 2% from the initial published values (based on initial 10% or so of transactions).

This means that criticisms of Charlie and Alex's methodology based on the fact that they're using the initial published values for September 2023 are invalid.

No, LR / ONS is not always based on 10%, it's based on how many transactions have been processed up to the time they publish then adjusted via their algorithm.  They're not just taking 10% of raw data and saying that's representative. @bearishonhouses is comparing the index to later revisions to the index, not the index (or the housing values suggested by the index) to the raw data, which is what Charlie and Alex are doing.

Edit - There might be something here where we can get the information we're discussing: https://landregistry.data.gov.uk/app/ukhpi/browse?from=2017-01-01&location=http%3A%2F%2Flandregistry.data.gov.uk%2Fid%2Fregion%2Funited-kingdom&to=2023-08-01&lang=en

Edited November 4, 2023 by Armus

[Smith]

5 minutes ago, Armus said:

No, LR / ONS is not always based on 10%, it's based on how many transactions have been processed up to the time they publish then adjusted via their algorithm.  They're not just taking 10% of raw data and saying that's representative. @bearishonhouses is comparing the index to later revisions to the index, not the index (or the housing values suggested by the index) to the raw data.

Edit - There might be something here where we can get the information we're discussing: https://landregistry.data.gov.uk/app/ukhpi/browse?from=2017-01-01&location=http%3A%2F%2Flandregistry.data.gov.uk%2Fid%2Fregion%2Funited-kingdom&to=2023-08-01&lang=en

Hang on, please don't confuse this issue as it will make people who don't understand the simple stats here suspicious again. No one is suggesting that @bearishonhouses is comparing the index to raw data. That's not what they said and not what I said. Please go back and reread so you can edit your post.

[Armus]

3 minutes ago, Smith said:

Hang on, please don't confuse this issue as it will make people who don't understand the simple stats here suspicious again. No one is suggesting that @bearishonhouses is comparing the index to raw data. That's not what they said and not what I said. Please go back and reread so you can edit your post.

You are saying that Charlie and Alex's methodology isn't flawed based on bearishonhouses post, which I don't think is correct.

[Smith]

26 minutes ago, Armus said:

You are saying that Charlie and Alex's methodology isn't flawed based on bearishonhouses post, which I don't think is correct.

I'm saying that, based on bearishonhouses calculations, it's legitimate to base year-on-year calculations on the initial September 2023 release, because this isn't likely to change materially once all the figures are in. From your post above it doesn't sound like you've understood this, which is just going to cause confusion so we'll end up talking about what you don't understand, rather than the main issue. Just like we are now. 

[70PC]

26 minutes ago, Armus said:

You are saying that Charlie and Alex's methodology isn't flawed based on bearishonhouses post, which I don't think is correct.

How can a flawless number be possible? There are so many variables. Charlie and Alex's video challenges the validity of the smoothing formula being used. They have subjected it to 3rd party scrutiny by the BBC which is good practice. I don't know the true state of the market but I have deep reseservations about Nationwide's recent claims. The housing market is swamped with big price reductions in the counties west of London.

[Armus]

1 hour ago, Smith said:

I'm saying that, based on bearishonhouses calculations, it's legitimate to base year-on-year calculations on the initial September 2023 release, because this isn't likely to change materially once all the figures are in. From your post above it doesn't sound like you've understood this, which is just going to cause confusion so we'll end up talking about what you don't understand, rather than the main issue. Just like we are now. 

Setting aside whether you think I’ve understood (pffft), if there is a strong correlation between the index values and the raw data then what’s the significance of what Charlie and Alex think they’ve found?

@70PC It’s fair to challenge the validity of the smoothing via algorithm/model but we’re too early into the dip to say how accurate or not it is. 
 

The BBC are completely irrelevant. Charlie and Alex should have discussed the “issue” directly with the LR. They could validate it with a university statistics professor, what expertise do the BBC have?

[mynamehere]

It's very trumpian isn't it, when the numbers are not going your way, cry fake news and turn on a firehose of confusion.

 

What's left is vague sense you can't trust anyone. Vote Charlie!

It doesn't seem like many on here even have a clear idea about which aspect of the news they are supposed to be decrying as fake

If agreed prices have really fallen 20%, there are remarkably few reports of happy purchasers on here or reddit

 

 

[Smith]

23 minutes ago, Armus said:

Setting aside whether you think I’ve understood (pffft), if there is a strong correlation between the index values and the raw data then what’s the significance of what Charlie and Alex think they’ve found?

What makes you think there's a strong correlation between the index values and the raw data?

[kzb]

On 03/11/2023 at 08:40, 14stFlyer said:

Works if the sample is random or “representative” of the full population.  In this case, we are looking at the first 10% of transactions to be processed. Not random.  This said, they could still be representative.  The point I am making is that this is not an appropriate example on which to impose your 95% confidence interval. 

It also depends on the numbers.  To say that you are always 95% confident that you are within 1.5% with a 10% sample is incorrect.

It must be based on Poisson statistics, and that depends on the actual numbers, not a percentage.

[Armus]

27 minutes ago, Smith said:

What makes you think there's a strong correlation between the index values and the raw data?

Alex's own graph (https://mhwc.co.uk/blog/house-price-discrepancy/😞

 

But hang on, we're going round in circles here, let's see if we can agree some things (correct these as you see fit):

1. Alex and Charlie have compared house price raw data from September 2022 to an incomplete set of raw data September 2023 (I think that's what they've done) (https://mhwc.co.uk/blog/house-prices-fall-faster-than-reported/)

2.. They think that because the incomplete September 2023 data is 10%-ish of the September 2022 data that means the September 2023 data is representative (the confidence level discussion) of the picture of what prices are doing in that month

3. They point out that the LR / ONS index say prices rose 0.2% in August 2023 (https://www.gov.uk/government/statistics/uk-house-price-index-for-august-2023/uk-house-price-index-summary-august-2023) but their comparison of September 22 to September 23 raw data shows some big percentage falls

4. They think this indicates a cover up (same link as point 1 above)

5. They think the LR / ONS "smoothing" of data through whatever mechanism they use is flawed, especially during market turbulence (I'm not disagreeing with the second part of that necessarily)

6. The index price is not the same thing as the raw data price

 

Edited November 4, 2023 by Armus
Correct list numbers

[70PC]

9 minutes ago, Armus said:

Setting aside whether you think I’ve understood (pffft), if there is a strong correlation between the index values and the raw data then what’s the significance of what Charlie and Alex think they’ve found?

@70PC It’s fair to challenge the validity of the smoothing via algorithm/model but we’re too early into the dip to say how accurate or not it is. 
The BBC are completely irrelevant. Charlie and Alex should have discussed the “issue” directly with the LR. They could validate it with a university statistics professor, what expertise do the BBC have?

"It’s fair to challenge the validity of the smoothing via algorithm/model but we’re too early into the dip to say how accurate or not it is."

Information published in near real time, should be what it says. It cannot be exact for obvious reasons. All markets go up and down but the methods used to monitor these should align as closely as possible to the true picture. Charlie and Alex are saying that the analysis being employed to kill the noise is flawed.     

"The BBC are completely irrelevant. Charlie and Alex should have discussed the “issue” directly with the LR. They could validate it with a university statistics professor, what expertise do the BBC have?"

University professors are very mixed bag. Some are brilliant, many are not and reach elevated positions through a process of 'dead man's boots'. The BBC has weakenesses but on balance they are a better are a better option for this. They employ people with financial knowhow in a more competitive environment and the consequences of them getting it wrong is generally more serious. 

[Smith]

The two lines are bound to reasonable correlated because they're closely related: one being an adjusted version of the other. But crucially you can see from the graph that the gap between the two lines is not constant. What Charlie and Alex have found is that over time the gap between raw and published data is getting larger. This means prices do not appear to be falling when you look at the adjusted figures, but they do if you look at the raw data.

I'm not sure anyone is claiming this is a "cover up" exactly, more that whatever they're doing to adjust the raw data is leading to over reporting.

Edited November 4, 2023 by Smith

[Armus]

4 minutes ago, Smith said:

I'm not sure anyone is claiming this is a "cover up" exactly, more that whatever they're doing to adjust the raw data is leading to over reporting.

I’m not going to debate with you if you’re not going to bother to read my answers to the questions you’re asking:

[/url]

A 0.7842 correlation is the definition of a strong correlation. http://www.dmstat1.com/res/TheCorrelationCoefficientDefined.html
 

As I asked you before if the index and raw data is “bound” to be correlated then what is it that Charlie and Alex think they’ve found?