TheCountOfNowhere Posted May 21, 2018 Share Posted May 21, 2018 Hi All, In case anyone's missed it "GettingBored" has done a brilliant replacement for property bee, called Property Log for the Chrome browser. Deserves a thread all of it's own I think. You can get it here https://chrome.google.com/webstore/detail/property-log/jccihedpilhidcbkconacnalppdeecno They've managed to import most of the property lions database ( work in progress for the missing entries ) so they have a great starting point for detecting falls. Big pat on the back for GB, top man ( or woman ). Many thanks from me. The more people that use it the better the data will get, so get browsing. Quote Link to comment Share on other sites More sharing options...
crooksook Posted May 21, 2018 Share Posted May 21, 2018 Propertylog looks very nice - in particular the presentation is very clean. However I was wondering - do you guys know there's a seemingly decent plugin already for Chrome (property tracker) that's been running for a couple of years? And if so, I was wondering how propertylog plans to differentiate itself? Quote Link to comment Share on other sites More sharing options...
GettingBored Posted May 21, 2018 Share Posted May 21, 2018 6 minutes ago, crooksook said: Propertylog looks very nice - in particular the presentation is very clean. However I was wondering - do you guys know there's a seemingly decent plugin already for Chrome (property tracker) that's been running for a couple of years? And if so, I was wondering how propertylog plans to differentiate itself? Yep, know all about Property Tracker but was never a fan of the layout (having to click the icon to see what the drops are etc) and there were some occasional bugs with it. Last time I used it you had to manually refresh the page every time you browsed through the search pages to see the updated drops. A large number of us loved Property Bee and it's simple design which showed you the drop data at a glance. Property Log has the support of a number of very active forum users who come from a development background and are in this for the long haul to the bottom. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 21 minutes ago, crooksook said: Propertylog looks very nice - in particular the presentation is very clean. However I was wondering - do you guys know there's a seemingly decent plugin already for Chrome (property tracker) that's been running for a couple of years? And if so, I was wondering how propertylog plans to differentiate itself? We all loved property bee, great tool, this looks like a great replacement. PB was so good I never bothered with property tracker. Says it all to me, but horses for courses, PB did exactly what I needed. Now property lion is a different paradigm and is throwing up some interesting results. Doesn't matter what tool you use, only an idiot would buy a house right now. Quote Link to comment Share on other sites More sharing options...
crooksook Posted May 21, 2018 Share Posted May 21, 2018 Thanks guys. Makes perfect sense. And propertylog looks much cleaner indeed. Also (as a quantitative-y developer myself) - I always wondered why neither of these tools (property bee or property tracker) ever had a separate web presence where they displayed aggregate area stats on the collected data (e.g. average drop, time to sell, possibly joined with LR data in a smart way to do a asking vs sold index). I kind of assumed it was because it would be too obviously against the T&Cs - but maybe it was also that they never got around to it. Hoping that's what property lion will do. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 9 minutes ago, crooksook said: Thanks guys. Makes perfect sense. And propertylog looks much cleaner indeed. Also (as a quantitative-y developer myself) - I always wondered why neither of these tools (property bee or property tracker) ever had a separate web presence where they displayed aggregate area stats on the collected data (e.g. average drop, time to sell, possibly joined with LR data in a smart way to do a asking vs sold index). I kind of assumed it was because it would be too obviously against the T&Cs - but maybe it was also that they never got around to it. Hoping that's what property lion will do. I dont remember agreeing to any T&Cs. It's publicly available data, if anyone wants to impose T&S they need to add a password and get you to agree to them. If someone wants to show me a legal case that says otherwise I'd be very interested. Quote Link to comment Share on other sites More sharing options...
GettingBored Posted May 21, 2018 Share Posted May 21, 2018 Britain's cheapest house (until they correct the price): http://www.rightmove.co.uk/property-for-sale/property-71726801.html Quote Link to comment Share on other sites More sharing options...
CanAffordWontPay Posted May 21, 2018 Share Posted May 21, 2018 1 minute ago, GettingBored said: Britain's cheapest house (until they correct the price): http://www.rightmove.co.uk/property-for-sale/property-71726801.html Great extension GB. I've switched over from the other chrome one. Is it possible for the extension to show the total value a house has been reduced by as well as a percentage? Quote Link to comment Share on other sites More sharing options...
leonardratso Posted May 21, 2018 Share Posted May 21, 2018 14 minutes ago, GettingBored said: Britain's cheapest house (until they correct the price): http://www.rightmove.co.uk/property-for-sale/property-71726801.html still too much, might be ok if they can grow some spuds in that pig pen. Quote Link to comment Share on other sites More sharing options...
CanAffordWontPay Posted May 21, 2018 Share Posted May 21, 2018 eg something like this? Quote Link to comment Share on other sites More sharing options...
Henrik Posted May 21, 2018 Share Posted May 21, 2018 I'd be interested in mining the data set a bit @GettingBored - any chance of a peek? E.g. I'd like to do the asking price vs eventual selling price (probably still too early, given that there's only data from the 4th of April), but eventually it should be possible to find out. Also interested in building some super simple linear regressions to predict eventual selling price (I'm doing a data analytics course at the moment...) Quote Link to comment Share on other sites More sharing options...
crooksook Posted May 21, 2018 Share Posted May 21, 2018 1 hour ago, TheCountOfNowhere said: I dont remember agreeing to any T&Cs. It's publicly available data, if anyone wants to impose T&S they need to add a password and get you to agree to them. If someone wants to show me a legal case that says otherwise I'd be very interested. The LR data is clearly publicly available. You'd need to parse RMs T&Cs carefully to be sure screenscraped data (even done on a per-browser basis) wasn't in violation: http://www.rightmove.co.uk/this-site/terms-of-use.html In particular the following prevented me from writing a scraper and doing it myself: You must not use or attempt to use any automated program (including, without limitation, any spider or other web crawler) to access our system or this Site. You must not use any scraping technology on the Site. Any such use or attempted use of an automated program shall be a misuse of our system and this Site. Obtaining access to any part of our system or this Site by means of any such automated programs is strictly unauthorised. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 7 minutes ago, crooksook said: The LR data is clearly publicly available. You'd need to parse RMs T&Cs carefully to be sure screenscraped data (even done on a per-browser basis) wasn't in violation: http://www.rightmove.co.uk/this-site/terms-of-use.html In particular the following prevented me from writing a scraper and doing it myself: You must not use or attempt to use any automated program (including, without limitation, any spider or other web crawler) to access our system or this Site. You must not use any scraping technology on the Site. Any such use or attempted use of an automated program shall be a misuse of our system and this Site. Obtaining access to any part of our system or this Site by means of any such automated programs is strictly unauthorised. That's like you printing some T&S on your missues backside saying no one's allowed to sh*g her, no one else has agreed to it. Quote Link to comment Share on other sites More sharing options...
crooksook Posted May 21, 2018 Share Posted May 21, 2018 5 minutes ago, TheCountOfNowhere said: That's like you printing some T&S on your missues backside saying no one's allowed to sh*g her, no one else has agreed to it. The issue here is if a company decides to fire you a cease-and-desist, it's going to be expensive and stressful to try to prove that point. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 (edited) 11 minutes ago, crooksook said: The issue here is if a company decides to fire you a cease-and-desist, it's going to be expensive and stressful to try to prove that point. Thats fine. I am a wealthy man. I will stand my ground. Show me where I agreed to any T&Cs and then I will desist. Didnt even know RM had any T&Cs, had a look on their main page and couldnt see any mention. They can easily enforce their rights by having a password protected system. I refuse to be bullied by anyone. can they stop you writing down the listing prices on a bit of paper and sharing it with your friends, I doubt it . Perhaps they should start looking at the what could be illegal advertising from estate agents and sorting out their house price index. Anytime I've posted them dodgy listing they are more than happy to help and I have found RM to be a great website. I wonder how expensive for them it will be to pursue someone in India Edited May 21, 2018 by TheCountOfNowhere Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 Has a look cant even see any T&Cs on their website. Nothing like adding in small print somewhere random to try and trick people. Seems like something an estate agent would do. Quote Link to comment Share on other sites More sharing options...
GettingBored Posted May 21, 2018 Share Posted May 21, 2018 35 minutes ago, CanAffordWontPay said: Great extension GB. I've switched over from the other chrome one. Is it possible for the extension to show the total value a house has been reduced by as well as a percentage? Yes this is possible and something I have already taken into account behind the scenes. The one issue(?) we have is that we only have price drops from the beginning of April onwards meaning that it could skew some results. What I could do is add the option in the extension to turn this functionality on or off (off would be default until we have a lot more price drops). 1 minute ago, crooksook said: The issue here is if a company decides to fire you a cease-and-desist, it's going to be expensive and stressful to try to prove that point. Why do you have to be so negative all the time? What the Count does with Rightmove is up to him. Perhaps instead of trying to find a flaw in everything why don't you contribute to the community in a way that actually helps us. Quote Link to comment Share on other sites More sharing options...
crooksook Posted May 21, 2018 Share Posted May 21, 2018 1 minute ago, GettingBored said: Yes this is possible and something I have already taken into account behind the scenes. The one issue(?) we have is that we only have price drops from the beginning of April onwards meaning that it could skew some results. What I could do is add the option in the extension to turn this functionality on or off (off would be default until we have a lot more price drops). Why do you have to be so negative all the time? What the Count does with Rightmove is up to him. Perhaps instead of trying to find a flaw in everything why don't you contribute to the community in a way that actually helps us. Honestly - why do I have to be so negative? Web sites get shut down all the time for screenscraping. I'd love to see this - but if you guys put a whole bunch of effort into building it and it's super useful, but you've not considered how to mitigate this kind of risk - who wins? Please don't shoot the messenger. Quote Link to comment Share on other sites More sharing options...
GettingBored Posted May 21, 2018 Share Posted May 21, 2018 5 minutes ago, crooksook said: Honestly - why do I have to be so negative? Web sites get shut down all the time for screenscraping. I'd love to see this - but if you guys put a whole bunch of effort into building it and it's super useful, but you've not considered how to mitigate this kind of risk - who wins? Please don't shoot the messenger. What the Property Log extension is doing is no different to what Property Bee has been doing for the last ~10 years and Property Tracker for the last ~8. The extension is not generating any extra load on their servers (which dedicated scraping would do) as it's a capturing an id and a price from a page they were looking at already. Yes, there is an argument for it falling into a grey area but why do you feel the need to highlight this to the world? Let it drop, please. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 2 minutes ago, crooksook said: Honestly - why do I have to be so negative? Web sites get shut down all the time for screenscraping. I'd love to see this - but if you guys put a whole bunch of effort into building it and it's super useful, but you've not considered how to mitigate this kind of risk - who wins? Please don't shoot the messenger. Examples ? https://parissmith.co.uk/blog/beware-spiders-web-crawling-screen-scraping-legal-position/ As explained above there is no specific law against scraping or using publicly available information which has been obtained through the use of scraping techniques, however, the owner of the website may have a claim against the user if the scraping and subsequent use of the information infringes the website owner’s intellectual property rights, or if the user is in breach of any terms and conditions of website use. Furthermore, it is possible for website owners to prohibit companies from scraping information from their sites through the use of contractual restrictions. If the user agrees to a website’s terms of use which includes a restriction on scraping and using the publicly available information on their website but the user decides to go ahead and scrape and use that information, the website owner may be able to claim against the user for breach of contract. In summary, there is no specific piece of legislation which restricts the use of a web crawler to gather information. The website owners, however, may have legal rights against the company under intellectual property law and contract law. Each case will turn on its own facts though and this is very much dependent upon what information is scraped from the websites. Companies should beware of contractual provisions which they have agreed to in respect of a website’s terms of use – these may prohibit the user from taking and using the data off the site. If the data being scraped includes personal data, then compliance with data protection law must also be borne in mind. The only way to be truly certain that the rights of a website owner have not been infringed is to obtain their express consent to the screen scraping and subsequent use of the information. Property Lion is directing traffic to Rightmoves website and it's a non commercial tool, RM would do well to embrace it and get people looking for the discounts. Similarly this new tools is not saving the data away for commerical purposes and only to enhance the rightnmoves website. If you can find some T&Cs from rightmove that try and prohiobit that and where anyone's signed up for it and who's profitting from the data or using it for a seperate website then I'd really like to have a look and pass it onto my lawyer. Thanks Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 2 minutes ago, GettingBored said: What the Property Log extension is doing is no different to what Property Bee has been doing for the last ~10 years and Property Tracker for the last ~8. The extension is not generating any extra load on their servers (which dedicated scraping would do) as it's a capturing an id and a price from a page they were looking at already. Yes, there is an argument for it falling into a grey area but why do you feel the need to highlight this to the world? Let it drop, please. It's worth considering, I had wondered if Property lion morphed into a seperate website then it would be an issue, but we've give up on that with this new tool and all Property Lion is doing is directing people to the right move web site and trying to pick up lion estate agents, all for non commercial reasons. Quote Link to comment Share on other sites More sharing options...
crooksook Posted May 21, 2018 Share Posted May 21, 2018 1 minute ago, TheCountOfNowhere said: Examples ? https://parissmith.co.uk/blog/beware-spiders-web-crawling-screen-scraping-legal-position/ As explained above there is no specific law against scraping or using publicly available information which has been obtained through the use of scraping techniques, however, the owner of the website may have a claim against the user if the scraping and subsequent use of the information infringes the website owner’s intellectual property rights, or if the user is in breach of any terms and conditions of website use. Furthermore, it is possible for website owners to prohibit companies from scraping information from their sites through the use of contractual restrictions. If the user agrees to a website’s terms of use which includes a restriction on scraping and using the publicly available information on their website but the user decides to go ahead and scrape and use that information, the website owner may be able to claim against the user for breach of contract. In summary, there is no specific piece of legislation which restricts the use of a web crawler to gather information. The website owners, however, may have legal rights against the company under intellectual property law and contract law. Each case will turn on its own facts though and this is very much dependent upon what information is scraped from the websites. Companies should beware of contractual provisions which they have agreed to in respect of a website’s terms of use – these may prohibit the user from taking and using the data off the site. If the data being scraped includes personal data, then compliance with data protection law must also be borne in mind. The only way to be truly certain that the rights of a website owner have not been infringed is to obtain their express consent to the screen scraping and subsequent use of the information. Property Lion is directing traffic to Rightmoves website and it's a non commercial tool, RM would do well to embrace it and get people looking for the discounts. Similarly this new tools is not saving the data away for commerical purposes and only to enhance the rightnmoves website. If you can find some T&Cs from rightmove that try and prohiobit that and where anyone's signed up for it and who's profitting from the data or using it for a seperate website then I'd really like to have a look and pass it onto my lawyer. Thanks Fine. I'm going to stop talking about this. Final word: I really do wish you success so hope you are putting some serious thought into avoiding unwanted attention. I've not seen Property Lion yet, but if it involves a website it'll be a bigger target than a browser plugin, and more centralised, so I think your risk is substantially higher. Quote Link to comment Share on other sites More sharing options...
CanAffordWontPay Posted May 21, 2018 Share Posted May 21, 2018 35 minutes ago, GettingBored said: Yes this is possible and something I have already taken into account behind the scenes. The one issue(?) we have is that we only have price drops from the beginning of April onwards meaning that it could skew some results. What I could do is add the option in the extension to turn this functionality on or off (off would be default until we have a lot more price drops). Having the option to turn it on/off would probably be best. Glad to hear it's somethign you're already taking into consideration/working on. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 7 minutes ago, crooksook said: Fine. I'm going to stop talking about this. Final word: I really do wish you success so hope you are putting some serious thought into avoiding unwanted attention. I've not seen Property Lion yet, but if it involves a website it'll be a bigger target than a browser plugin, and more centralised, so I think your risk is substantially higher. I understand your concern and fair enough raising the issue, it's something I've considered. Property Lion couldnt be any more obvious to RM, it tweets them dodgy listings that it find. I've just sent them this " Property Lion @UkPropertyLion 34s34 seconds ago I just want to remind people that Uk Property Lion is a non-commercial tool that's only purpose is to bring @rightmove price reductions and the insane asking prices to the public's attention and flag up erroneous listings ( honest mistakes!!! ). RM is a great website. " The lion sends then several tweets a day point out dodgy listings. I've they've missed it and/or are pissed off by it, they've had plenty opportunity to send a PM asking for the tool to stop. The website ideas is now dead thanks to Property Log. Property Log is the same as Property bee which was actually bought by a company and RM never said a peep. Thanks for your concern. Did you find any T&C that we might be breaking, I would genuinely be interested. Quote Link to comment Share on other sites More sharing options...
TheCountOfNowhere Posted May 21, 2018 Author Share Posted May 21, 2018 17 minutes ago, crooksook said: Fine. I'm going to stop talking about this. If you have information that helps us then please share it,. I appreciate your concern. Thanks Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.