Post

I'm having so much fun with this dataset. For some reason I'm doing a lot of analysis of fast food places. Did you know Istanbul has the highest density of fast food restaurants, with Burger King being #1?M

At their scale I imagine they won't particularly care unless it's costing them tens of thousands of dollars extra per month - plus I expect most people to download the files once rather than constantly scanning them in-place, just to get better performance

Nice! For those using ClickHouse and want to read all rows from all the parquet files at once, the following query will give you a glimpse on how to achieve that (see the usage of /* at the end of the S3 path):

Because it can run queries directly against Parquet files over HTTP without needing to download the whole file - it uses HTTP range requests to fetch just the bits of the file it needs to answer the question

FourSquare is in the business of data syndication - to Apple, Alexa, etc and to anyone else that needs it. So this dataset is only like a point-in-time export.

If they did I think that would defeat the purpose of their business. Businesses have to pay them to update this database and those that need the data have to pay too.