I got the same email. Despite my hobby project (a random food picker) having been broken and not used for years (because of yelp API updates), they also told me my usage was higher than other developers...
The email also arrived in my spam folder, so I was lucky to even see it. Once I got back to them they did increase the cutoff by a few days but it has since been stopped.
They never answered my question about the discrepancy between these prices and the prices on their website. Both would be too high to make Restaurants low sales sustainable.
That said, I'll say again that they were perfectly in their right to start charging for their API. They just should not have done so with 4-days notice and with such a threatening email.
$10 per request at the lowest tier of enhanced?! The only business case I can think of for this is if you save all the data in your own database then serve requests for your own service out of your own cache. I guess if the Yelp data doesn't change very often, then you just choose to stay 6-months behind on the data or something...
Interesting and an excellent read. The death spiral at Travelocity (pre-Expedia acquisition) worked differently and was partially self-induced.
Some background is required first. Travelocity was the first online travel agency (OTA). I believe they started around 1996.
Through most of its life at that point all, I mean 95%+, of its success came from only two factors: best marketing in industry and growth of the internet. This is severely problematic because marketing only gets you so far in business. Conversely Expedia had really shitty marketing in comparison and yet came to dominate the industry because they were extremely aggressive at growing their supplier relations.
You cannot EVER rely on growth of media adoption, like mom and pop coming online, because once that stops you have no fuel left in the tank. Reliance on growth of a media platform is like a gravy train that you did nothing to build and returns amazing wealth if you are in the right place at the right time, but once it stagnates its like your train derails and everybody dies. That is because everybody expects growth to continue, except you did nothing to earn the growth and now have no answers and nothing to show for it. This was around later 2008 when I joined the company and became unavoidably obvious to everybody over the next year.
So, at that point what do you do? You competitors are far out pacing you by ignoring fun stuff, marketing, technology, and all the other bullshit that technology people look to. Instead they are focusing on core business principles and eating your golden goose while laughing at you. So, what do you do?
In the case of Travelocity all the executives leave. New executives come into trying to figure out what to do. Like every great web business they focus more aggressively on marketing and advertising. This was Travelocity's death spiral.
You have to understand that people DO NOT like advertisements. Really, I know its surprising, but when your site becomes littered with advertisements everywhere and all kinds of hidden telemetry people will leave and never come back. Your wonderful palace has become a trailer park.
The business loves advertising. Revenue from advertisements is immediate. That is really significant. In e-commerce there is a massive lag between each stage of profit, revenue, and sales because you have to account for the cost of operations, sales, and inventory. The more expensive the product the longer the lag and that lag really complicates projections. So advertisements are like cocaine, because they immediately return profit that requires no effort while rotting your health slowly until you are a hollow skeleton.
To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
My learning from this is that a business that earns profit from selling something directly should not fuck up conversion or go out of its way to make customers hate them. When I put that way it sounds obvious, but web business get that wrong all the time because they get distracted by shiny things.
>To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
So in the travel industry you have limited lines of business, primarily: air, hotels, and car rentals. Air is super low margin because its set directly by the supplier plus some arbitrary service fee, typically $7 per ticket. Air is high volume though and is the primary draw for OTA customers, so it cannot be ignored. Rental cars are super low priority because they are both low margin and low volume.
The real interest in the travel agency business is on hotel inventory. This is how Expedia came to dominate the industry, because they aggressively hired geographically focused hotel relationship personnel and Travelocity wasn't keeping up. So Expedia had much greater inventory from various different properties and stronger relationships with those properties for their business prioritization. Hotels, typically hate dealing with OTAs, because OTAs are a transparent barrier between the hotel and the guest staying at the hotel, but the OTAs bring people to the properties. The name of the game for hotels is "heads in beds" and while up-selling is beneficial if you get the "heads in beds" part wrong you cannot exist.
The margin on hotel properties is much higher and highly variable. Hotels in many cases will even eat the cost of airfare to get heads in beds if the pricing is right. That provides a huge opportunity for a marriage between smart hotel systems and OTAs because the hotels are doing the smarter inventory management and the OTAs are supplying the airfare volume and customers to populate that smart inventory management. The hotel data systems, Property Management Systems (PMS), were at that time much smarter than the inventory availability OTAs had, because they had to do much more to account for seasonal volume planning and various customer demands.
Where Travelocity was strong was in writing algorithms to account for these considerations and offer volume pricing discounts the competition could not and also air + hotel package pricing at massive discounts the competition could not compete with. In parallel Travelocity was slowly building up a corporate travel business to take advantage of that volume pricing intelligence and at one point had both WalMart and Lockheed-Martin as customers.
Again, both of those were amazing and could have really helped retake lost market, but conquering competition is a fragile slow process while advertising is immediate.
> they also told me my usage was higher than other developers...
I'm curious to what they compare this. The non-existent free plan where everything higher than 0 is unusual. Or a large mass of dead accounts who have really zero to none usage. Or the actually paying account who are now on their enterprise plan. In the later case, this might indicate some real problem if their usage is low enough that smaller projects are already exceptional.
Author here. I actually discovered something in their terms of use today while looking up something else that someone had asked me in this thread:
"2. CHANGES
Yelp reserves the right to modify the API Agreement at any time. If Yelp reasonably determines that a modification may materially and adversely impact You, Yelp will provide email notice to you using the email address you provided during registration no less than ten (10) days prior to the material adverse modification taking affect. IF YOU DO NOT WISH TO BE BOUND TO ANY NEW OR MODIFIED TERMS, YOU MUST TERMINATE THE API AGREEMENT BY CEASING USE OF THE API WITHIN TEN (10) DAYS OF RECEIPT OF SUCH NOTICE."
So, the 4-days would have violated their original notification terms of use anyway. I think they realized this after the fact of sending all of us the threatening emails and this is why they ultimately closed my API key after 10-days as described in my original post.
Anyway, just an interesting aside that the poorly written, inaccurate original email that I shared in my original post was even inaccurate to their own policies.
"YOU MUST TERMINATE" rather than be held to the original terms you initially signed up for ... because Yelp RESERVES the right.
We have all now been reminded that RESERVED RIGHTS can and will be used against you maliciously. And if you want protection from such malice, there needs to be a RIGHTS of the USER clause.
This should be handled and enforced by government statutes.
(It is an amazing world we live in where rights of the user have to be expressed.)
Seen this story play out so many times. I audited a company years ago that claimed to have excellent, personal almost, relationship with Google and all needed paperwork to use their platform as a core of their business. They went bust 6 months after because of "unexpected" change of Google's product strategy.
Real lesson here is to avoid single points of failure, regardless if it's API, people or partners. Ask yourself a question if there's a single entity that can kill your business and remove that reliance.
If you're depending on the continued existence of any Google product beyond GMail, Google Cloud and advertising you should really consider fallback positions. The existence of Youtube is also guaranteed, but the existence of any particular API or service that you might depend on? Not so much. Hell they'd probably consider killing GMail if they didn't need the accounts to tie advertising impressions to individuals.
That's a ton of money, (and I've heard that they paid Apple even more last year) but it's kind of understandable. Apple customers don't usually have android phones snitching on them, are less likely to use chrome, and tend to have their devices and data in the Apple ecosystem. Google has fewer ways to spy on Apple user's lives and push ads at them. Search, gmail, and youtube probably give them the best opportunities. I wouldn't doubt that Apple users are getting as frustrated with Google's search engine as everyone else is though.
Google used to need Search in order to learn about the intimate details of our lives. It's how they knew what we were interested in, what our medical problems were, what we were learning, and what we were thinking about.
Now they have chrome giving google people's entire browsing history, android devices collecting realtime data on what people are doing offline, where they are and who they are with. Google also has products like fitbit, nest, and gmail that gather still more data for google. Google doesn't need Search to spy on us anymore, so they haven't invested in keeping it useful.
In fact, it's better for google if you can't find what you want and have to make multiple searches for information because it gives them more chances to throw ads at you, and the harder it is for people to find websites using Search the more sites might feel like they have no choice but to pay Google to keep them at the top of the search results.
People who find Search increasingly useless though are turning to alternatives and for many people AI could end up replacing google's Search product as the first thing they turn to.
I would not count on Search staying around forever.
They need search for different reasons: it's their biggest moat against Meta by being the start page of the internet for a great many number of people. A whole lot of browsing sessions start with search and end with search, and this way Google can claim attribution for virtually every traffic a website gets outside of direct hits.
This keeps people developing content to please Google and running ads on Google platforms to get more traffic.
Yup. Search is gradually (for about 15 years now) being transitioned from a search feature to a self-serve ad-requesting platform. “Show up, type a term, and we will select 10 ads for you.”
> for many people AI could end up replacing google's Search product
The quality of Google search lately has felt pretty meh, especially for generic topics that get hits for crummy blogspam. Google’s own AI results at the top are pretty much the thing keeping me there atm aside from inertia. My personal browser is on DDG which is fine just a tad slow comparatively
It's not as if they can't/don't also push ads at people in gmail, in youtube, Google maps, on android devices (phones, TVs, chromebooks, etc), in chrome, and they still have AdMob and AdSense and the Google Display Network
That is only half-true. On the one hand, I agree and I would never build a serious business on another single entity. But on the other hand, OP built an app that ran successfully for 10 years, generating some revenue.
Most products have an expected expiration date and you can provide a useful service (that generates revenue) by building on other platforms, even if it won't last forever.
By now I missed more opportunities by having the mindset of "not relying on other people's APIs" than real changes/shutdowns would warrant this kind of caution.
It's so sad that we're at such a devoid barren state.
There was so much hype & excitement in web2 age, about APIs & intertwingularity. Humanity was going to keep improving & iterating & crafting ever neater ideas.
Now it's all bureaucracy & lawyers arrayed to defend the cloud-keeps, moats & walls.
The user & agency of the world has been steady dimished, eroded away by ever advancing legal IP Protectionism.
That's not going to work in practice as money and time are limited. Choose your partners wisely but don't obsess over the scenario of them going bankrupt.
I built a service around helping podcasters automatically convert their audio podcast into a YouTube channel. I went through tons of review with Google in order to get access to the YouTube API and make sure everything I was doing was in compliance with their terms - literally months of back and forth. I had been testing in my development and staging environments against their API for 6+ months. I launched in production, got a few videos uploaded to YouTube, and they disabled my API key. I spent months emailing them and never got anything more than the same boilerplate copy/pasted answer. I could have pivoted or something, but I just shut it down and moved on. Lesson learned.
I worked for a very early billpay company where you could pay your bills online to vendors, even if the vendor didn't support it. We used API's where we could, but where we couldn't...
We had a whole team dedicated to keeping up the changes vendors would make to their websites that we scraped for info. The team was called, of course, "Scrape and P(r)ay".
If you build your scraper to find data on the page based on the shape of the data itself instead of the structure of the page then it will be resilient to most changes that don't materially change what data is displayed on the page.
So, prefer regex over css selectors, and css selectors over xpath, where possible. And don't select based on nesting or position if possible.
Depends on your development and per-action cost. And on the possible latency. It also changes your whole stack from "send a request" to "emulate each step in a browser while taking screenshots at (hopefully) the right event/delay" - that's a huge difference.
That's because Booking was also committing some type of misrepresentation and taking revenue away from Ryanair through their browser automation. Even then, the infraction was sooo bad that they got a $5k fine.
> In November 2022 the U.S. District Court for the Northern District of California ruled that hiQ had breached LinkedIn's User Agreement and a settlement agreement was reached between the two parties.
Reminder that the earlier ruling was overturned, it is no longer clear whether scraping is legal or not.
My wife hates it when 1099 her for services rendered, especially since she refuses to bill against the PO I set up for her. The kids at least accepted NET15 payment terms. Although the oldest said if I short pay him for lawn care again, he’s going to take away my early pay discount.
How does that work, though? Setting up a company has an initial cost and then recurrent costs (accountant, etc). Are the benefits that high for the average Joe?
It's hard to answer without specifics, even if you're not doing anything neferious there are a lot of benefits to putting an entity between yourself and your customers. It depends on where you live and what your business is of course.
My first Saas was built around Facebook, Twitter and Reddits API.
Facebook broke like monthly and required random updates, sometimes documented, sometimes not. Zero support.
Twitter worker flawlessly until I hit some limits and there was no way to increase them because I didn't hit some other limits. No way they would talk to me.
Reddit just worked until I gave up.
What I learned is to never again do any business based on someone else.
A good lesson, but a more nuanced one is to do sure diligence. There are companies I would never build my business around (Google, Oracle, etc.) for a variety of reasons. There are ones I know I can trust (at least until they change). There are many where I need a contract.
Change is also a constant, and there are plenty of good companies give bad, and a few in reverse. So due diligence is an ongoing process.
That dramatically raises the price of SaaS, so I use it much less than mainstream industry practice, but much more than zero.
> What I learned is to never again do any business based on someone else.
That's a little dramatic, don't you think?
Supply chains are a core of many (most?) businesses out there. They depend on others to get their business done. No supply -> no business.
What you probably meant is not building a business based SOLELY on irreplaceable business relationships. Unfortunately in a lot of cases involving the tech giants and little startups, there are no alternatives.
More or less, however I really try to not build on anyone else ever again.
I can't find a single dependency in my serious money generating chain right now that isn't replaceable with a similar product within a few minutes or hours if necessary.
My most recent project didn't even last a full year as there was no replacement that worked for me for ChatGPT in that moment. It was built to die, a gamble essentially. I wouldn't waste my time for things like this if I didn't had the end calculated in already.
Same experience, slightly different. It made me realise why some parts of an MBA (e.g. strategic place/partnership/spring boarding from a brand) actually have value I never saw before for tech companies. There are essentially king makers in certain fields - either by their apathy at first-movers, or their choice on who they allow to continue.
Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul. Reddit, Twitter, and company all got big because they had open APIs and people were able to use them extensively for really creative things.
I agree, Google has swallowed up the video streaming market unfortunately.
I keep thinking back to how Vine was basically TikTok, and they threw it away.
Take it from someone who watched with interest, Reddit and Twitter did not get big due to open APIs. They got their first big steps before they even had APIs... In the first year of twitters launch it was in the news in the UK at 300k users in press releases about how it was a "new form of communication" etc... 2008 was actually when the original set of API-laden websites began to fade because with the recession, we could not have nice things be free so much. Many API services fell into disrepair in 2009, certainly the peak of APIs for the UK was around 2008. Check out Tom Scotts video on this for a picture of the open API internet many people thought would occur (and which did exist in some places until FAANG began to productise and dominate and use accounts across products which do not incentivize APIs). Yes it is true open network graphs/facebook etc had some API access, but the APIs before were more single purpose and numerous; facebook lets you explore their network and to some extent data, but that's not the same as the useful APIs that became restricted, paid or were cut in the aftermath of the recession.
Those startups are their competitors. You don’t have to pay millions of dollars to acquire a competitor that was never started because of your inconsistent API policy.
Google makes enough money that losing several currently non-existent revenue streams that are theoretically $10 million apiece isn’t hurting them. It’s hurting their users.
Google is all about ads - why would they give a shit about the users?
>Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul.
Google made $300B last year. "billions over the long haul" is a lot of money, an unimaginable amount even, to you and I. But to Google?
First off - how much of revenue usage last year was on fines etc. That is to say profit they could potentially have had.
Second there is lots of revenue you use for "business" because you don't want to pay taxes on the profit and anyway it is giving you something you like to have.
How much of Google's revenue is used to do things for top executives and people with power in the company that is really something they (the executives) should be taxed on but is instead a business expense of google (cars, transport, 'working' vacations, security, super cool chefs preparing meals at the company...) Hard to say really because if you knew the answer it would actually be something they were taxed on. But it's not 0 - sure probably not a billion, but a couple hundred million splashed around wouldn't surprise me.
How much of Google's revenue usage is for wages and other forms of payout to executives etc. (stock) that does not get counted as profit but of course it is amounts those people want to have.
Google made $73.795B profit last year, and expensed slightly over 262B - some portion of which the people who run Google no doubt personally thought of the way we would consider profit in our day to day existence, and another portion of which were fines for things they did in getting the rest of the money.
Obviously the fines, which are the first thing mentioned, is relevant for how much the shareholders got.
Literally the other stuff is relevant for how remunerative the people who actually run the companies, day-to-day, feel those companies are (put in day-to-day because I got the feeling you might give me a lecture about the shareholders actually running the company)
Personally I think you have a decent point since in practice employees of GOOG are shareholders too. Although I don’t know if their interests matter to the corporate entity because I don’t know what percentage is held by insiders.
I don't think that's nearly as universal an assumption as you're claiming. Like--I made $X,000 dollars last year, but I also have a mortgage and like to eat food.
The assumtion things were great wasn’t mine. I live in a HCL area and am not able to save much so in this sense the ‘profit’ of my work is close to zero. That is not great at all…
I mean, they "make" both. "Make" does not imply profit. If you say a company "made" a revenue figure, then you mean it in the revenue sense. It's contextual.
100% contextual. In every business where I've been an executive, or had access to the budget, we talk "made" as gross revenue and EBIDA as a stand in gross profit and specifically call out net profit in internal meetings. For example, "We made 10 million with an EBIDA of 1 million and net of 200,000.00." Using EBIDA to talk to potential investors and as a guiding metric if we didn't have a well established gross profit formula that followed GAAP.
It's a testimate to the health of our free market that the company throwing away millions, and billions over the long haul, is the overwhelmingly dominant market leader.
Attempted to build a similar thing a few years ago when living in Tokyo.
Provide a selection of restaurants within a 1km radius and automagically provide 3 recommendations based on my preferences.
We had multiple API providers available, Tabelog, Gurunavi and Hotpepper, all required a paid developer license. We still needed to use the Google maps API to get the user's current location though.
This was also just when Google Maps suddenly raised their API pricing. After spending a couple of weekends building a working prototype, we stopped as we couldn't justify the cost of paying multiple API providers for basically 3 guys looking to save 10 mins deciding where to get lunch.
Also, this kind of app is a common theme if you frequent meetups in Tokyo. There's always at least 1 person that has built such a thing.
> We still needed to use the Google maps API to get the user's current location though.
I’m sure as somebody who actually built a prototype, you understand the situation better than me, but I’m curious: why would just getting the user’s lat/long require anything you’d pay for? Wouldn’t the OS or browser provide that to you?
Oops. I meant that we were using the maps/places API to get lat/long of the restaurants using the street address from the other API providers, then using that to calculate a 'N mins walk away'.
We were just testing the abilities of the various APIs. Google was the only that provided travelling time estimates.
I was thinking about how to solve this given that one of the primary problems is that of fast, global content distribution. I like the idea of paying people in crypto as part of a ledger transaction to host and serve content, like bittorrent with a crypto payment. Unfortunately I can't also think of a way to prevent such a system from being abused to distribute harmful media such as CP. I guess it's not like this isn't a problem with BitTorrent today though.
Regulation might here - something like minimum mandatory 3 months notice for shutting down the API keys. Considering the average age of our politicians, I doubt they’ll understand what “API” is, much less be willing to take on giant tech corporations even if they did understand the problems
Yeah, what’s popular is actually to crap all over the first amendment by mandating private companies to allow whatever each politician’s special definition of “free speech” is (but of course probably also not allowing whatever the politician’s definition of “dangerous misinformation” is).
As for the actual anticompetitive power of big tech, absolute crickets.
Competitive markets are by far more the exception than the rule. It's just not how capitalism works (because pretty much any competitive market is ripe for “consolidation”, which increases aggregate shareholders value by reducing competitive pressure).
If you want competitive market in a capitalist economy, then you need very active state enforcement.
The markets are absolutely competitive. Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
They are now hemorrhaging search to OpenAI that popped into public existence just recently.
To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
If you wanna see lack of competition, look at government granted monopolies on utilities. Guaranteed but capped rates means you reduce investment right to $0 and cut costs as much as possible since there is no other way to make money. That “state enforcement” you are calling for is how you end up with PG&E and scenarios like all insurances companies pulling out of the state.
> Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
Phew, here I was stressing out about lack of competition, but you helped me relax.
Google has to invest into Youtube lest it loses out to Twitch (Amazon), Netflix ($35bn per year), HBO Max (Warner Bros, $50bn per year), (Amazon) Prime, Plus (Disney, $90bn per year).
My faith in humanity is restored now that there are alternatives.
That's not competition, that's an oligopoly and they are mostly focus on their own niche for which there isn't more than a minority competitor (there's no competitor to Youtube on for non-live video, Twitch is in a monopolistic competition situation for game streaming, so is netflix on TV series, and Disney and HBO each have a monopoly on their IP).
Claiming that this is a competitive market is a joke.
> To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
Please tell me how many “product advancement” in Google search or YouTube over the past decade… From a consumer's perspective all that happened was enshitification and despite being owned by a search engine company, YouTube never managed to ship a functional search on their platform.
> If you wanna see lack of competition, look at government granted monopolies on utilities.
Utilities are “natural monopolies” though, and as such they should be state owned. Making a natural monopoly owned for profit is a recipe for rent seeking, and that's why it was promoted …
> That “state enforcement” you are calling for
No, the state enforcement I'm calling for is proper enforcement of antitrust laws, forbidding consolidation through M&A and disbanding companies megacorps. That is to say, what existed in US's golden age.
>Competitive markets are by far more the exception than the rule it's just not how capitalism works
Empirically this absolutely isn't the case; the majority of listed companies have fairly low margins, especially non-tech companies, which can be trivially seen from their financial statements. A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
> A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
Not necessarily: if prices are elastic then even a monopoly can aim for low profit margin (in percentage) in order to increase profit. What matters is how much total profit are being made, margin only matters when measuring risk.
Also, corporations are social structures, and low competition also encourage complacency in the corporate structure itself, which drives costs up and reduces profit margin.
Do you know if there is any open source tools in this field that a programmer can work with/automate, I have mainly textual and graphic content, a little bit of video, but thinking to branch out to podcasts and music soon, unfortunately the other products supporting text seem pricey and geared to big teams.
on edit: obviously I can write my own, but I am hoping for a project that has already done a bunch of things that I can extend for my needs, as writing my own would be at the point where it would be more cost benefit to purchase.
I believe I am entirely in compliance with YouTube's terms and policies, but I was flagged for some bullshit as well and got the boilerplate responses. So what I decided to do was hide it for logged out users so it looked like that specific feature was entirely removed to the dudes in India that review this stuff... One year later and my fingers are crossed we don't get another audit.
We went through a three-week Facebook API shutdown due to a clear glitch - Meta support couldn't go outside the script, or escalate in any way beyond "a supervisor will email you". Only resolved when our CEO found some VP at Facebook on LinkedIn and got them to escalate it internally.
(The dev community thread is full of people still impacted, so I think they literally just edited our app's flags directly.)
Even tried to invoke GDPR's rights to be exempt from automatic decision making, but their privacy questions email address responds with "nope fuck off" to those.
Basically any hope of solving a problem with a Meta property is to know employees who can escalate your issue. Sometimes your friend at Meta also needs to argue with the bug owner to not close it frivolously.
Within the customer service industry, they'd claim a great "deflection rate". That's a metric that all of these large companies hold around any kind of help/support channel that may involve humans because people are a cost. It's often covered by some kind of satisfaction metric based off post-issue surveys, but fundamentally, if you just go away that's success.
There are often complaints here about what amounts to bean counting affecting other aspects of business. Customer service at larger scale is costly so attracts a section of very analytical leaders. They don't get, or ever prioritize, the human elements. It's only when satisfaction numbers are bad, or another exec outside of customer service takes action, that things improve.
The scale part kills the customer. You can have great support at one size. Once you grow then leadership, structure, and the culture change. These analytical leaders don't want to carry over the culture and structure because it comes at a cost. It really needs force and support from outside of customer support leadership to maintain it.
Had this issue with Ubisoft recently. Installed Trackmania through Steam for free, then had to create a Ubisoft account, then had to buy the Trackmania annual pass for $20. I guess I followed the wrong flow or something, but I ended up with the purchase on my Ubisoft account, but I guess their system also auto-creates a Ubisoft account for Steam accounts? So the purchase on the Ubisoft account ended up on the account not tied to Steam.
Emailed support about it on 13/07/2024.
Got back a reply on 23/07/2024:
> Our Support Team is currently experiencing high case volumes, so we are reaching out with a message to check-in and make sure you still need help!
> If you've found a solution to your problem already, there's nothing you need to do. Just ignore this message and your case will automatically close in 2 days.
Didn't see the reply until the 3rd day.
> As we have not heard from you in the last 2 days your case has been closed automatically.
I think people, even on HN, grossly underestimate how much customer service costs at any major company that works with the public.
Most calls to customer service are easily self-serviceable. My brother used to work for X-Box Live support, and he said over half his calls were for password resets, something people could easily do on their X-Box.
But then there's bad actors. Another significant chunk of his calls were from people complaining about being banned, and he'd see the reports of their spamming racial slurs.
Not to mention how many people try to abuse customer support in attempts to take over someone else's account.
Yeah...customer service is expensive. Even if you automate as much as you can, but offer an appeal process, that appeal process with be absolutely abused by bad actors, so that will end up automated as well.
I am curious if you would be interested to retry the idea? I might have an in with the YouTube team. I feel like it's a shame to let this go. Would you be open to chat? Please reach out r@pehul.com
No. The default API quota is not large enough to upload videos. You would have to contact Google, explain your use case, and jump through several hoops to get the quota increased... which is a huge process.
Yeah. Uploading legit videos is non-trivial. And if you ever upload the same video twice (which you think you might do during testing, right?), it's a violation of their terms and they disable your access.
I think Yelp is shutting off their API for the same reason as Reddit: to ensure that AI training makes money for them. It sucks that you’re a drive-by casualty of that, and if I’d bought this app, then - same as Apollo iOS - I would not request a refund.
YouTube already uses Widevine for movies. they will never use it for regular videos, because it will kill their website. widevine is a resource killer, for both client and server.
> To wit, there is exactly one business strategy: When you are small, be nice. When you are big, pull that ladder up behind you.
I'd add a "in venture capital and big capital" after "business strategy". When you don't have VCs or the stonk market breathing down your neck all the time, incentives massively change.
IMHO, there is only one solution left... once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust that has a clear mandate to run its companies in a way beneficial to society at large. That way the government doesn't have to spend taxpayer money on countless r&d experiments, VC investors have a perspective to payout, and the world gets kept from utter bullshit like API games.
Well if small capital doesn't pay good, I have no incentive to run a small business at all!
> once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust
Interesting, haven't heard that one. I'll say maybe. I'd rather have Facebook not exist at all than have it be an *official* arm of the NSA
> the government doesn't have to spend taxpayer money on countless r&d experiments
If you expect the government to make a tender offer on the largest market-cap companies, they sure as heck _are_ paying for r&d indirectly. And a lot more!
I think it's quite different. It involves money, but QE is all about monetary supply and interest rates alone. Buying entire companies comes with a lot more implications, like decision-making power over the organization. You want that for API keys, but do you want that across the board? Seems like regulation would be much cheaper.
Thanks for replying, today I corrected a misunderstanding i had about QE. I had previously believed they were buying and selling stocks (I misunderstood their usage of "Securities")
Why? The problem at the core is the mantra of the WSB et al groupthink: stonk only go up.
Of course when WSB makes jokes about it and catapults a junk stock like GME to ridiculous heights, squeezing shortsellers dry, it's pretty funny to watch. But it gets problematic when just about everyone's pension is tied to "stonks go up" as well. That leads to all sorts of perverse incentives.
Just look at Boeing and how far they've fallen. Twice in recent years, literally and not just proverbially, with hundreds dead as the inevitable consequence of stonk market games.
I think the GP had a visceral reaction to your intentional misspelling of "stock" as "stonk".
I tend to agree, though I don't think I'd go so far as to think someone saying "stonk" has "lost all credibility"... but to me, it's like Republicans deliberately mispronouncing Kamala Harris' first name, because they can, and because their base eats it up. It's a cheap shot, and it feels childish, like you don't have a real argument, so instead you act like an elementary school kid on the playground calling the other kid names.
> It's a cheap shot, and it feels childish, like you don't have a real argument, so instead you act like an elementary school kid on the playground calling the other kid names.
This has been one [unintended?] consequence of AI promulgation. A direct disincentive toward the kind of open access that so used to be common and provided a lot of low-hanging fruit for independent developers trying to increase interoperability within their favorite niches.
So now not only is AI filling the web with garbage that poisons future model development[1], it provides incentive to further close and wall off access to (user-provided!) data.
Yes. We're entering a new era of web applications, whether we want to or not. Companies used to be able to gate their data behind UIs, because the search results had more value than the raw data.
Things are starting to flip. Increasingly, people would rather have access to the raw dataset to pipe into an LLM. The question, as always, is how to control this access and charge people accordingly.
I think you're right. API strategies are being reviewed because AI training data.
That said... I think API policies are also vestiges of a defunct idea set that prevailed circa 2009.
At that time, "twitter is a protocol." APIs and openness were going to be the new worldwideweb. Social media was going to be an ecosystem of independent apps and services with complex interactions mitigated by protocols.
YC even did an "rfs" recruiting startups to build off the twitter API.
That didn't happen, but APIs intended to enable it were already in the wild.
As API access becomes strategic again (because AI), "why do we even have this?" is likely to come up.
I think the general lack of willingness to help in Big Tech is very problematic. You can almost never get through those thick-skulled reps that email you out of the blue...
The problem is that by having a thick-skulled rep walk in and send a few e-mails, Yelp has already lost more money on API customer acquisition than the app developer was willing to pay. That's why these APIs had free tiers: they covered these kinds of micro-usages that would be far too cheap for a sales rep to cover.
The reason why those free tiers went away is that AI came along. Not so much that the AI scrapers were abusing Yelp free tier[0], but that they could. And once companies realized how much money was floating around in selling data access, non-abusive free tier users went from "a cool goodwill gesture" to "freeloading parasites".
David Kopec and Restauraunts got steamrolled in a case of technological gentrification. If you're selling data access for $TOO_CHEAP_TO_METER/call to a random indie, Apple, Google, and/or Microsoft will use that as a comparable for why Yelp should charge peanuts. Or they'll just acquihire him. They need him and his app to go away because he is inconvenient to the long-term valuation plan of Yelp, an old guard Web 2.0 business that never quite became sovereign.
[0] Though, to be clear, AI scrapers are absolutely abusive in general.
A couple years ago, I was confused about why my Apple TV remote was behaving weirdly, and a few taps in the support app and I had a real person call me and fix my issue within minutes.
I was looking for a completely random Molex adapter from the early 1980's for a hobby project. After reaching out to the company they had someone call me and attempt to help. They didn't find the part (since it hasn't existed for 30 years) but they did give me some good leads.
Amazon’s AWS has good customer service. I get good customer service at the Apple Store, and I also got good customer service when Microsoft had retail stores. I have used Azure’s customer service and it was OK. Amazon’s AWS is know for outstanding customer service. My main point is some big tech companies have terrible service. Some don’t.
I meant, if you have the ability to initiate a chargeback (pay money, and have the ability to take it back out of their control for bad service), then they will probably provide customer service.
I'd like to see how someone can see a directly employed Google support agent / repair technician in person for support like you can with the Genius Bar.
Additionally, as far as I know, the kind of support you get calling AppleCare is unmatched by anyone else.
> You can almost never get through those thick-skulled reps that email you out of the blue...
He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
> He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
> What more would you want the rep to do?
Not send a threatening, inaccurate email with a 4-day (1 business day Friday->Monday) deadline in the first place?
Also for the record, this was not a sales manager. This was someone in "growth" that sent the email. I guess maybe that just means "sales."
> Your API usage is higher than lots of other Yelp Fusion developers
You're right it wasn't just inaccurate it was outright deceptive, trying to frame this as something specific to OP that he is responsible for rather than a change in their business applying to all API users.
Employ a shred of critical thinking and realize this app probably drives more value to Yelp than the cost to run the API, and flag it for an exception.
Customer service or sales people like that are usually at the lowest point on the totem pole. It’s likely that they have zero leeway, either you’re gonna pay exorbitantly or you’re out. It’s an institutional problem.
In that case it's not customer service but a customer firewall. If the CS employees are not empowered to induce solutions even if they benefit both the customers and the company then they are literally useless.
And yes, many companies have "customer service" like that designed to waste your time until you go away. That doesn't mean it's the only possible way.
But Yelp is not big tech, certainly not by market cap. It could be old tech, in the way that a 2010-era data API may have been a growth hack intended to create an app ecosystem. But 10 years later, there’s no app ecosystem and the company can’t afford to support a (stagnant) lone indie developer. Especially in the current era of LLM scrapers.
Author here—seems they noticed that people were unhappy. Just got this email from Yelp (August 1, 2024):
> Earlier this month, we sent you an email about your Yelp Fusion API usage. That email gave developers until July 23rd to contact us if they want to continue using Yelp’s data for use in their app. We realize you might need more time and are extending your free access for an additional 90 days starting today. Your access should be available now.
> We’re sorry for any inconvenience or frustration this abbreviated transition might have caused. Please respond to this email or contact us at api@yelp.com if you have any questions.
> First of all, I was not “trialing” the Yelp API. I had been using it for a decade and had official permission from Yelp to create Restaurants.
This looks like a Face/Off situation to me.
(Spoiler alert for a 1997 action movie: in Face/Off a cop surgically exchanges faces with an imprisoned villain in order to go undercover in their organization, but the villain then murders everyone who knew about the swap and steals the cop's life.)
Somebody at Yelp in 2014 knew that you had been approved to build this app. That person almost certainly no longer worked at Yelp ten years later, so the institutional knowledge of that agreement had likely been lost.
I absolutely wouldn't expect there to be notes against an API key.
Heck, in many companies I have encountered that they don't even know who is using API keys they've issued to third parties, especially over generations of systems. I am impressed that after getting email approval for an app a decade ago, Yelp still had contact details enough to let the O/P know!
None of that negates the issue of 4-days notice and Yelp shooting themselves in the foot by removing access to an app which redirects people to their website. But someone in executive management made a decision on the basis that they wanted revenue from the API overhead, wanted a slice of other people's pie and that's that.
To the O/Ps point about subscription, there was a UK food app which gave you restaurant food safety scores taken from the food safety agency "Scores on The Doors" it's gone now but I paid a few £ a year for it and never had an issue. If the Yelp pricing is sustainable at that level it's not a bad idea to pay for API access, Yelp has to pay for their servers and a portion of the API calls won't be converted to traffic. Who knows if Yelp still has a sustainable business? Maybe that API thing doesn't pay for itself? I don't know.
Maybe not for an API key, but for an account, you would think they would have some method of keeping notes. I've never worked for a company that had a concept of accounts, and teams of people that interfaced with accounts, without having a method of keeping notes about an account. That's just a basic necessity, especially when the accounts exchange money with the company, there's usually accounts that pay more than others that require special handling, which is usually documented in an account notes screen of some kind.
On the Yelp developer site you have an app profile where there is the name and description of the app. There is also a separate field for the URL of the app website, which I did have filled in. So, they have that at a minimum. If they didn't retain the records of the screenshots/communications that led to the original approval and raise of the API daily limit then that is poor recording keeping but understandable in a corporate environment after 10 years.
The API key usually has a contact email. After a year that contact email probably doesn’t work anymore, certainly for free tier.
Which underlines what I concluded long ago: the best and most durable form of identity on the internet is rooted in the ability to pay money. Any identity that is free to create is doomed.
Oh I completely agree - 4 days notice is _never_ OK. I get uncomfortable with 30 days notice because I've had the occasional vacation that long!
Ideally they'd have a notes field against developer apps and a robust process for recording this kind of thing - but I've worked for companies, so it doesn't surprise me at all that there's no good mechanism in place for that.
In real life, sometimes months isn't enough. On August 5, UPS will complete their API transition to OAuth based authorization after many months of process. Many of my customers haven't responded to my attempts to warn them about the change. I've resigned myself at this point to August 5 being a crazy day.
Yeah, you'd think so. But as many of us have learned, many companies can't and won't be considerate. Sorry that you got hit with it, but at least it's a quick and clean break.
Don't sweat the refunds too much (unless someone is being really rude about it). Apple certainly won't.
Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
> Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
Right, the biggest thing is a failure of communication. There should have been emails months ago, not 4-days (1 business day Friday->Monday) before.
There should have been an email months ago saying they were converting their API to be paid. Not a 4-day threat to shutoff your API key or pay-up. As I mentioned in my post, I was not aware that they were converting to a paid program and had received no prior emails.
Yes, I got that, but why should they bother? Your understanding is that you were business partners, while in fact, you were just lucky enough to get access to their service for some years.
If you had an actual contract, it would be a different story. Your product would still be not more than some UI to some walled garden, and nothing particularly valuable, but at least you would be officially permitted to do so.
So you basically sold something to Apple users that you don't actually own, and that is completely out of your control, and that was always in danger to just stop working suddenly.
And, on the other hand, I'm sure you've implemented a (for a hobby) decent app, and maybe you actually brought a handful of new visitors to them (which wouldn't have just used their website if your app wouldn't exist). But do you really think that at Yelp they are sitting there and just think how they can return the favor?
They basically think "well, that funny dude has now played enough". You never were a partner for them. Just what people call " useful idi*t" (sorry, but that's just the term ^^) at best. They haven't talked about you at their christmas party!
If you're up front and reasonable, you're more likely to convert developers to pay for the new API. If you do it last minute, not only can't developers make the switch in time, you've lost their trust, making it harder to convince developers to bother investing their time and resources.
Their business is completely independent from a funny guy who writes some iPhone app. They haven't lost anything in that story. If they need a new iPhone app, I'm sure they will find developers for that.
My first thoughts while reading the story was that their champion at Yelp quit and whoever's left didn't know/care enough about them. 10 years ago, Yelp would still be scrappy start up and there was a motivated employees willing to go outside the painted lines. Now, it's just a faceless person following orders.
or in the departed. Martin Sheen is the only one who knew dicaprio was undercover. With him dead the protection goes with it. Only a real contract would CYA. When a company goes from friendly to unfriendly they're happy they were 'scrappy' not to write everything down.
I’m actually surprised the API access lasted for so long. The person that provisioned it is probably long gone from Yelp. Similarly, the company’s values and priorities have changed as well.
Also, I’m guessing this new API pricing policy is they way of combating data scraping to train AI.
If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions, or the service in question was built poorly from the start.
There is little room to blame behavior like this on bad actors and not bad decision making. 10s of thousands of companies have managed to figure out combatting API abuse without shooting themselves in the foot.
> If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions
You are entirely right, yet I see this train of thought happen so very often in different companies. Some of these people are a bit saner than others and actually do listen to others explaining why it is an idiotic train of thought. Others really are just see numbers in isolation and somehow refuse to look at them in context.
Depending on what kind of approvals they gave him 10 years ago, it MIGHT be possible that doing this violates a contract. It sounds like they had some kind of understanding when they gave him access that he had some sort of informal approval. Even if nothing was signed, that still forms a contract. Even if there was a formal contract or terms of service (TOS) agreement, there are certain restrictions around when and how a company can change their TOS. In particular, there's often requirements about how much advance notice has to be given if the terms of service change.
It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
> It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
Yes, the monetary stakes are too small for it to be worth it for me to pursue. I could probably dig up some old emails from 10+ years ago but it just wouldn't be worth it. Exposing this kind of bad behavior (4-days notice!) is enough.
What's needed to form a contract is an offer, acceptance, and consideration - if an offer was made and accepted and something of value was exchanged, and there wasn't confusion about the terms (after 10 years, there wouldn't be), that's good enough for a contract to have legal force.
Not forever, no. But usually with any sort of contract, when one side wants to break or alter the contract they have to give the other side adequate notice. In certain circumstances a contract can eliminate such a requirement, and the definition of “adequate” would vary a lot based on specifics, but that is very much a possibility in this case.
It sounds like he requested API access in order to make a native Mac application for Yelp. The specifics matter a lot here, but “if you develop a native Mac application for Yelp, we’ll give you free API access” sounds a lot like consideration. That could be completely false based on exactly how that went down, of course.
> Developing an app to use someone's free API is not an exchange of consideration.
I would never pursue anything further over this app that had about $2,000 of revenue over 10 years. That said, for the record there was consideration.
I had to go through an official approval process that required providing evidence of the app's functionality and a few emails back and forth. I think I actually sent them a prototype of it. And based on that process they decided how many daily API calls to give me. A normal free user did not receive 25,000 API calls per day. I believe if you didn't go through approval back then you got something like 1,000 per month. So there was a consideration process on their part and a determination to green light my use case.
You could argue that the indefinite API access was in exchange for writing the app (service for service), but if this were me, I probably wouldn't bother. Maybe I'd write an adversarial-interoperability backend to replace the API, or open-source the app to allow other interested parties to do so. Or maybe I'd just say "It was a nice run", and let it die.
If you care enough to send a polite email, you could say that Yelp has a prior agreement, you'd appreciate them not reneging, and if they do, you'd appreciate compensation for your labour (minus, of course, the money you made from the software). Probably won't go anywhere, but…
granting API access in excess of the free tier would most likely constitute something of value, but yeah - probably wouldn't bother, it would be expensive to pursue and not worth it.
Officially, yes, something of value. In practice, just something.
> To my mind the acquiring and delivering of the [used chocolate bar] wrappers was
certainly part of the consideration in these cases, and I see no good reason
for drawing a distinction between these and other cases. — Lord Reid
> It is said that when received the wrappers are of no
value to Nestlé’s. This I would have thought irrelevant. A contracting
party can stipulate for what consideration he chooses. A peppercorn does
not cease to be good consideration if it is established that the promisee
does not like pepper and will throw away the corn. — Lord Somervell of Harrow
Oh thank you for that clarification. That's very interesting, but of course it is not worth it to me to pursue for the reasons mentioned in the post and the comments here on HN.
Yeah I think the only option in which this would make sense for you to peruse legal action is in small claims court for however much you had to pay in refunds. It’s fairly easy to do that pro se (representing yourself). And often companies are motivated to negotiate if they are sued in small claims, because just sending a lawyer to represent them would cost as much, if not more than just paying the damages. Obviously this depends a lot of jurisdiction and the specifics.
But at that point it’s not a question of cost, it’s a question of how much of your time and headache you want to spend. So yeah, probably not worth it
I would be upset too, but this sort of sounds like someone did him a favor 10 years ago, management changed, and now management wants him to pay. This sucks, but near infinite free api calls sounds unsustainable
We received the exact same letter from Yelp. Our usage is significantly higher and when we talked to them, the prices they quoted were ridiculously high (thousands of dollars a month).
Exactly, we're not. We're closing our products. They are perfectly welcome to charge too high a price for us. They just should've communicated it better.
We have a analytics SaaS which displays, among many other pieces of data, ratings from different providers, including Yelp.
I assumed that Yelp had been doing this all along and we ran above some predetermined limit, but the email we received was identical to yours. So I'm thinking that Yelp is trying its best to monetize all API users
> a analytics SaaS which displays, among many other pieces of data,
Does yelp (or others) allow the restaurant to modify the menu/offer coupons in a dynamic way? (i.e. change multiple times per hour, update with in minutes)
A 25,000 calls per day limit for an API sounds absurdly low. Any idea if the API calls actually would have used much in the way of backend resources, or do you reckon it was more just mostly a database lookup?
Honestly, making people pay for an app that only uses a public API you’re not paying for, and no form of fallback is asking for trouble. This is not a responsible way to do business and I hope people reading this thread will understand that.
Since ~15 years, people got the idea that all the WhatsApps, Instagrams or Twitters around them aren't companies, but kind of public infrastructure. They all knew that it's a wrong assumption, but it's sooo convenient, and at some point, people around you force you in that direction, even if you don't like all those walled gardens. And once this wrong philosophy was established, people started escalating it to some smaller shops, I guess. Those are also the guys that complain when some 3rd party Reddit or Youtube hobby clients must disappear. They just have no understanding at all about some basic mechanics of human interaction. The biggest problem: There are entire generations of people nowadays with that misunderstanding.
but the author pointed out that it drove traffic to Yelp, and that almost certainly seems to be true.
In fact, little integration plays like this app might have been the only thing keeping Yelp sort-of alive after Google got into the game.
Killing off your bizdev partners seems incredibly short-sighted and foolish. (Also, feeding the reviews into an AI... for what, exactly? To train a model on how to write reviews? Or perhaps to detect fake reviews -- actually, that was an issue on Yelp even before AI, so it seems like it wouldn't be the best training content.)
I think Apple has been developing a yelp clone internally; and have been slowly releasing bits and pieces.
Apple has added rating systems for destinations. Amenities available at location (ie, “accepts Apple Pay”, dog/cat friendly). Maybe in the next decade Yelp integration will be phased out completely
They've already got something in Maps that switches locations from Yelp reviews to the Apple thumbs up/down system once they hit some critical mass of data, and the same for location photos.
Was this a move to try and get some profit from AI companies trying to train on their data? With the API providing limited results, I’d think it would be of limited use. I always found Apple’s use of it in Maps frustrating, because seeing POI details basically required I also have Yelp installed.
If this move was AI driven, like with Reddit and others, I’m starting to dislike AI more and more… at least the rent seeking end of it, which seems to be slowing killing the open internet.
SaaS can provide “open core” or better yet simply sell a hosted version of their fully open source code. If the provider fails to provide, you can fall back to self hosting.
The API equivalent would be open sourcing the data. This is the OpenStreetMap model. If the API provider fails to provide, you can fallback to the underlying data.
That's asking too much. SaaS should give an option for you to export all your own data in a simple, parseable format (like JSON). That's about it. They don't owe anyone their source code, and they don't owe any ethically sourced data (such as employees researching and manually entering).
API access needs better terms. Like guaranteed access for X years at $Y price with Z days notice if there's a change, where Z > 3 months or so.
This was the hope behind the StackExchange data dumps, that the community at large could always take their contributions elsewhere if the service jumped the shark.
Well, before the SE organization tried to kill the data exports off in an attempt to commercialize it towards AI companies, but thats a whole other issue.
That would probably as big or bigger project than the entire rest of the app itself, and since it is such a single developer and not a super profitable app it likely makes no sense to do this.
SerpAPI is a convenient wrapper API for scraping various sites. I assume they've vetted all the legality of things. They have a YouTube API: https://serpapi.com/yelp-search-api
Still they have a boilerplate ToS with some glaring mistakes:
> These Terms of Service and any separate agreements whereby we provide you Services shall be governed by and construed in accordance with the laws of 5540 N Lamar Blvd #12, Austin, TX, 78756, United States. (sic)
In that case, the court found 3Taps was criminally guilty for scraping publicly available craigslist data while logged out because 3Taps knew their use was not authorized.
This person has just received an email from Yelp telling them their free usage is not authorized, so circumventing that may well be illegal, now that they've been given that sort of communication, even if it might be questionably legal for other serpapi users.
The courts are altering their views on scrapping. This [0] is a good paper that explored the last 20 years of rulings (although it hasn't been updated with the most recent cases).
It is in Illinois. Their Computer Tampering law specifically makes violation of a web sites ToS punishable by up to 5 years in prison. Probably other states have similar.
Typically for copyright infringement. They'll sue you for the maximum legal damages possible per copy, multiplied by the number of times your bot loaded a page, probably in the trillions of dollars.
I'm happy to write code to let users circumvent the restrictions put in place by assholes, as long as it doesn't end up getting them in legal trouble I'm fine with it. If companies like google don't want people to try and screw them over, I suggest they offer an olive branch and stop being unethical jerks.
Coprporations (or other organiztions or individuals) can make things available to the public without completely ceding control over how they are used.
Imagine a landowner who allows public access to hikers using defined trails but no overnight camping. That's legal and just has to be posted. If you don't like it, don't use it.
Not sure why this is getting downvoted, because it's a reasonable question - the answer being that such scraping requires a lot more time and technical expertise to engineer than a simple API call. Also devolves into a cat and mouse game between your application's backend and whatever proxy they put in front of yelp like cloudflare, which you'll probably lose or will be prohibitively expensive.
It was never different. Just, with the advent of iPhone, FB, WhatsApp, Youtube, etc, a lot of people basically decided to ignore this fact of life. So good and for so long that they completely forgot it. They forgot it good enough that even their childs got no education in that regard. And every some days, someone gets a lesson. ;)
Why should Yelp not throw you under the bus for money?
The problem was that (impersonal) you stopped understanding basic mechanisms of human interaction. For no real reason.
> It seemed they were in fact encouraging me to finish the app and release it.
That was quite the assumption. They gave you access to something for free, not encouragement. I do feel bad for OP, but they weren't paying for the API, and should not have had any assumption that it would last forever because there was no contract or terms or anything. This is the risk we take by building our house on someone else's foundation.
The gist I got from my communications with them 10 years ago was that they were encouraging me to finish it.
I agree with you there was no expectation of it being free forever. I never said there was. What was unreasonable was the 4-days notice that it was coming to an end. That's just not enough time.
That statement is an outright lie too. Not only are they operating at less than 1% of what the free plan offered, but the email doesn't tell them the honest truth: That the free plan is reaching sunset, and they want all API users to switch to a paid plan.
This isn't unreasonable, but they absolutely should provide more than 1 business day notice.
Of course it's generic and perhaps not a technically correct statement. And now? Why should they spend time on writing you nicer emails? What was your value to them? Zero? Or is that too optimistic actually?
> if you utilize a third-party API for the core of your app, you are at their whim
More than that, if you aren't paying for use of that third-party API, the people who run it will not care about you, and will think nothing of shutting you down.
I think Yelp handled this poorly, and Restaurants was probably a net positive for their business -- a positive that they were getting for near-zero cost. It's a shame that companies are so short-sighted like this.
But ultimately if you build on top of someone else's platform, with no backups and no alternatives, it's not really truly your app.
Enshittification is at an all time high, and API access is being tightened and monetized in response to many industry factors including "freeloading" AI trainers.
Cory Doctorow is right, if you want to disrupt, or make any improvement to an existing large platform, adversarial interoperability(that is, reverse engineering) is the only way forward and has to be explicitly legalized in cases where it's a tool for progress.
My previous statement is arrogant, as it assumes developers are entitled to take any data they want and profit from it. It also puts services in a situation where harmful crawling like what is performed by some new AI actors with no experience is an expected thing. This is of course wrong, but I want to argue that had Yelp and other actors not wanted such a future, they shouldn't have tightened free access to their proper APIs where they have the ability to set ground rules and have the ability to talk to their users.
Big companies are amazingly bad at keeping track of things internally - a promise in an e-mail is easily forgotten 10 years later. But why should the user be punished for Yelps lack of control?
I'm sorry to read that. Unfortunately, it isn't an uncommon story.
> if you utilize a third-party API for the core of your app, you are at their whim.
That's the money quote, there. I avoid using third-party APIs like the plague. I have written backend aggregators and facias, to avoid having to use the API.
I Just. Will. Not. embed an opaque codeball into my app. I'm a cranky old bastard, I know, but I sleep well at night.
Open-source libraries aren’t opaque. Third-party web APIs are. The latter should only be used if they are non-critical, easily replaceable, or contractually bound to sufficient assurances.
Amen. Doing the wrong thing is easy. Doing the right thing is hard. These are thermodynamically-mandated rules that cannot ever be circumvented by cleverness or money or hard work.
yelp had so much potential but they pissed it all away. Google brain raped them [1,2]. Then they never recovered from it.
Then there is ongoing issues between merchants and yelp [3]
Yelp used to be a great place to find some decent place to eat in a new city. But the platform
has gotten stale. Reviews are less reliable. Star rating often not useful.
Probably what happened was 1 support rep helped you out but did it through some undocumented backdoor to unblock you and not a formal contract. Later a completely unrelated set of employees are tasked with figuring out who the biggest API users are and to either cut them loose or get them to start paying.
Yes, this was surely somewhat tied to employee turnover and poor record keeping. But I was almost certainly not one of the "biggest API users."
The thing is, regardless of the turnover or situation, 4-days (actually 1-business day Friday->Monday) is not a reasonable timeframe to threaten to shutoff someone's API key who hasn't violated any terms of service. They have the right to do it, but it doesn't make you want to work with them in the future.
> The other thing this taught me is the danger of an up-front paid model for apps that depend on ongoing access to third-party services. If users were continually paying for the app, paying for Yelp’s APIs would not be as much of an issue. And I wouldn’t feel as guilty about the app being discontinued since if the fees were charged on a monthly basis, they would just end at the same time the app ceased to exist instead of facing an expectation upon purchase of “forever access.” On the other hand, how would you charge a monthly fee for an app that people are only willing to spend less than $5 for upfront?
There are apps that I like and would like to purchase, but paying e.g. 24 euros per year feels like too much. So I stick with the free version...
A hard lesson I learned after several times getting burned in my time long ago doing indie stuff is “never depend on a single company without a backup plan”.
Sometimes this can be pretty difficult to even see. If you were a Mac shareware dev depending on VersionTracker and MacUpdate to drive downloads, with no actual marketing budget, you might not have noticed that you were implicitly relying on Apple not creating an App Store which would turn the entire ecosystem upside down.
This is one of the reasons that it’s so tough to make a business work at a small scale. In some ways, you have more flexibility than a big company, but the lack of capital means that certain events that a larger company would shrug off can totally upend everything you’re doing.
I have seen many notices like this in the past few months. My guess is anbody of signficant size whose reason for existing includes curating user-generated content is trying to ensure that gen-AI ingestors don’t swallow up their data through unpaid API access or scraping. (Maybe a podcast or article popular with CxO and IT directors on the topic??)
Which they seemingly already have done anyway. Another unintended side-effect of the borderline illegal and generally immoral “AI” companies efforts to get as many data sets as possible.
It seems to be mostly a US phenomenon. As I pointed out in the post almost all of Restaurants sales were to the US. The directory is very up-to-date with active reviews everywhere that I've travelled in this country.
I got the same email regarding my hobby website "The Wheel of Lunch" (https://wheelof.com/lunch). The site makes no money and sends traffic to Yelp. The cheapest plan offered was $230 a month! I can't afford that.
I have stopped calling the Yelp API for local listings and put up a notice on the site. It was fun while it lasted!
We've been bitten by going the "free 3rd-party <x>" route so many times, so I sympathise with the problem here.
Off-topic, but kudos to OP for still engaging with the threads. This certainly wasn't a dump and run post. They've probably spent more time answering questions here than they did developing the app!
Yes, for sure. Some people explain it away as "well if you buy Wrangler jeans from Walmart and there's something wrong with them you ask Walmart for a refund." So if you buy X's app from Apple you ask Apple for a refund. That kind of makes sense. But if Wrangler does a recall, they have a way of getting that across to their customers.
I'd say, no you had no shared customers. Yelp had the customers, and you wrote some app that uses Yelp. For no real reason you assumed that you could be part of them (as with Facebook earlier). Sure you had some kind of informal permission at first. And then they revoked it. That's their right to do, right? Maybe you should not write Facebook or Yelp apps, if you are neither Facebook nor Yelp nor asked by them to write clients for their walled gardens. Or if you really want, set up an actual license agreement with them. Those networks are walled gardens and they don't want to cooperate with you that way. They are just not those kinds of shops which you can cuddle with. If you want to create something that has an own value, create something more than just another Faceyelptwitterddit frontend.
The good thing: You took your lesson. Maybe this time it was sufficient to actually learn sth.
Person who paid for my app also later uses the Yelp website via a link for a specific review from my app. Also a customer.
Therefore shared customers.
> Sure you had some kind of informal permission at first.
I don't know why you would define it as informal. They had an official review process for deciding what apps to grant higher daily API limits. My app went through that review process 10 years ago including considering its functionality, screenshots, and I believe I even sent them a prototype back then. We had a few emails back and forth to confirm my intentions to only develop for the Mac and what I was building.
I'm sure that got lost in the corporate shuffle over 10 years but I clearly had their permission to build the app and in fact given the relatively high API limit they gave (25000 per day versus the free at that time 10 years ago I think being 1000 per month), arguably blessing.
You created this account solely to respond on this post and all of your replies are defending an ostensibly horrible business practice, which is to give your API users less than one business day of notice to switch to a different tier.
No matter how you cut it, that is simply terrible developer relations/customer service and a terrible way to run a business.
have you thought about writing your own API instead of using the de facto API? it may involve screen scraping but it also will be a lot more reliable and there's no limits, especially when using proxies....
I've written thousands of tools that scrape websites and never used the apis for this reason, you can never trust the API, either because of the reliability, cost, limits imposed, etc... Nobody wants you pulling data from their site anyway, so you're back to scraping anyway, its better to start out there then to have to end up there years later for some other reason...
That was a long blog post to tell us that they are no longer entitled to valuable free API access. It was a mistake to think it would last. This story has been repeated so many times here. As soon as the author started making money (they conveniently excluded their sales revs), they should have negotiated a written contract for access for a fee. It could be cut of app revs or pure API charges. Also, I am sure some smart managers looked at all their free API giveaways and decided there we no longer useful to their business model.
The author disclosed his sales revenues here: https://news.ycombinator.com/item?id=41124971 . He sold 467 copies of the software after 10 years, for a total revenue of around $2000. He never exceeded 200 API calls per day. I don't think it would have been worth Yelp's time to negotiate for a percentage of that.
Drama aside, the guy signed up for the free Yelp API 10 years ago, which has since been discontinued. He was offered the option to switch to a paid API, which he chose not to consider.
Yes, Yelp was a bit clumsy in handling this, but discontinuing the free API after 10 years is totally within their rights. The developer didn't even bother getting their pricing proposal, which might have been totally reasonable (or not), considering his app is paid.
He's didn't argue it wasn't within their rights. He called them "quite rude", which seems hard to deny.
No matter the rate Yelp set, the apps economics no longer make sense. The existing customers, already paid, and he has no way to transition them to a subscription.
He mentions 100 API calls per day. If Yelp offered him a rate of a fraction of a cent per call, it might be a negligible expense that could be offset by past or future sales.
If the app has outlived its lifecycle, the end of the free API might be a signal to retire it. Blaming Yelp and making a drama out of it seems a bit much. Suppose they had given him 30 days instead of 4; would his decision really be any different?
> Suppose they had given him 30 days instead of 4; would his decision really be any different?
Possibly not, but the point of OP's complaint is that Yelp was rude, handled it poorly, and gave him an unreasonably short deadline. No one is arguing that Yelp doesn't have the right to discontinue free API access, or that OP's business model was a good and sustainable one.
But agreed: if OP could have gotten a rate that would have cost, say, 10 cents per day (or even more, like 50 cents or a dollar a day), maybe that would have been ok. And maybe he could have changed the pricing on the app for future purchasers to a subscription model, some small token amount like $1/mo or even $5/year.
But also consider it's pretty crappy to give someone such a short amount of time to make the decision as to whether or not that new business model would work, and if it's worth it to put more development effort into the app to enable that new pricing scheme.
Well, don't read the comment that you reply on. ;)
The only interesting part of the story is to what extent others actually learn sth from it (that lesson should be trivial, but nowadays it obviously is all but common sense). And I guess: Not very much. They will 'give a fuck' here how bad the world and all is, and then continue in their naive way tomorrow.
There's a tangible difference between shitting on a person who chooses to share an experience, and shitting on a person who is shitting on a person who shared an experience. If you choose to read irony or hypocrisy in it, that's your own issues with context. ;)
> He mentions 100 API calls per day. If Yelp offered him a rate of a fraction of a cent per call, it might be a negligible expense that could be offset by past or future sales.
Except, elsewhere in this thread:
> the prices they quoted were ridiculously high (thousands of dollars a month).
No problem. He can just dust off his business contract with Yelp and have the courts set them straight. No contract? Then why would one expect the world to cater to their whim. Yelp promised nothing and he got exactly what they promised.
I guess you didn't read my blog post because I addressed everything you wrote. The issue is not that it went paid, it's the 4-days notice. They are perfectly in their right to start charging for their API, they just can't give us 4-days notice.
Well they can do whatever timeframe they want of course. And I can write about how rude it is. 4-days (really 1-business day in the original email Friday->Monday) is not a reasonable timeframe within which to threaten to cutoff an app with real users.
Just killing your app is an emotional, not a rational, decision. You have a human responding to your emails, so here's a business way to handle this situation:
Dear Yelp,
Your decision to discontinue the free API was unexpected, and it’s difficult for me to switch to the new one within the given very short 4-day timeframe. Not only is this not enough time to estimate how your new API pricing will affect my business model, but it also requires some engineering work to switch my app to the new API.
Given my 10-year history of working with Yelp, I would appreciate it if you could send me your new pricing proposal ASAP and also give me some time to consider it. If accepted, I would need additional time to implement it.
It's just not worth it. I have a full-time job and many other projects and apps I am supporting. I don't want to work with a company that provides long-term API users 4-days notice about a major change and threatens to cut you off in that time period. As you saw in the blog post I did write back to them and did not get any kind of response indicating flexibility (did you read my whole post?).
Also, the money is very very low stakes. This app sells dozens of copies a year. Not hundreds or thousands. It's just not worth it financially. It sold 467 copies over 10 years. People who used it loved it, but it's not a money maker.
> I don't want to work with a company that provides long-term API users 4-days notice about a major change and threatens to cut you off in that time period.
If you don't want to deal with the slightest inconvenience don't run a business and don't take money from customers. You owe it to your users to care at least a bit.
> If you don't want to deal with the slightest inconvenience don't run a business and don't take money from customers. You owe it to your users to care at least a bit.
I did care "at least a bit" which is why I kept updating the app for 10-years despite it not making almost any money. How many indie apps survive that long? But based on the pricing they quoted me it would be a money-losing venture to continue (see slide 3 base monthly fee $229 from the deck they sent me):
https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
And we have to decide what we work on when we are just one person. If it's money-losing and they don't treat you well it might not make sense to keep doing it.
That said, as I expressed in the blog post I do feel really bad for any of the users that bought the app and I want all of them to get a refund from Apple as explained in the post. They can use these directions:
https://support.apple.com/en-us/118223
That's when you take matters into your own hands and extract the API key out of one of their official apps. At least that's what I would've done.
Though I'm not sure how legal that would've been if done in a paid app. It feels like a serious difference between just providing a better UX for someone else's service through adversarial interoperability for free, and profiting off of it.
I sold 467 copies over 10 years for less than $5 per copy. It's about $2000 in revenue. Surely not a good use of my time. But I liked the app and so did its few users. We should build things we want to use ourselves right? I think it was a much nicer and faster frontend to the Yelp restaurant directory than the Yelp website.
Your other question about the cost is answered in the blog post and in several other comments I have left in this thread. In short, when I started the app 10 years ago there was no paid API (perhaps there was some kind of "enterprise" version but I'm not sure). The point of my post is how poorly they handled the transition from free->paid.
Yes, definitely based on other people in this thread getting the same letter. It was an inaccurate and threatening form letter bizarrely sent from a personal email address of a Yelp employee with another personal email address CCed.
But how was the API key stored? If it's stored inside the binary, without an intermediate server managed by OP, someone could've disassembled the binary and starting using the API key independently for their own gain, triggering abnormal usage.
Say, one of our Windows products uses a web API for TTS, and we've built an intermediate web server which stores the key and manages rate limits per user so that there was no abuse (the app uses our own auth).
Yelp has an API dashboard where you can see how many API calls have been made. I can tell you it was never exceeding 200 calls per day and often below 100.
The difference, I think, is that the former are platforms to run applications while the latter are communication platforms. There used to be a time when the phone company could press charges if you connected your "app" (a tone-generating machine) to their "API" (the phone line) to "do things the official app didn't support" (place long distance calls for free). It will always be beneficial to someone who owns a communication channel to assume as much control of the pipe as possible. If Twitter could jack directly into your brain stem and refuse to work until you install its client on your wetware, it would.
Not surprising at all. Yelp, Paypal and their ilk deserve all of their earned derision for, amazingly, being simultaneously user, customer, and developer hostile.
Related to the earlier "old & grumpy" 3rd party API post, I've seen far too many, otherwise, outstanding businesses held hostage and summarily executed by either sudden un-explained usage-tier/throttling policies without economic recompense or the outright deactivation of API keys w/o notice or explanation.
When will people realize that using any big tech company API is a recipe for disaster? These companies, and their revolving door of employees, could care less about you, your app, or your users.
Same as the Reddit stunt versus the Apollo app dev, except that his app had a big enough audience you'd think they'd have figured something out.
Nope. The paid app meant money from users to the app dev instead of from advertisers to the site.
What's strange is Reddit didn't seem to do the math on how little they should have charged the app dev for API access if all they wanted was to offset revenue from that user base. Perhaps the fear was much as with TV streaming: they know advertisers want audiences willing to pay, not only the audiences seeking free.
> "People still use yelp? I thought it was widely known that they suppress bad reviews for money, and suppress good reviews if you don't pay."
A judge said that it wasn't extortion for yelp to hide good reviews until a company pays yelp, and that it wasn't extortion for yelp to remove bad reviews for companies that do pay them money though so I guess we're supposed to act like that's acceptable behavior and that their reviews can be considered trustworthy
Question-- with the yelp api now being paywalled-- and obviously you'd have to write code to handle scraping-- but can you legally scrape yelp to do this?
Does the linkedin scraping lawsuit permit scraping yelp?
I wrote a review for a grocery store turned food service shovel basically said "this will be the first store that charges for air." Guess what? I use their bathroom every other day,for free. I think it's been months since I bought anything, and I help myself to utensils.
Every customer will eventually turn anything they can use for free, into a profitable feature for themselves.
There was a book called the 1 minute manager: it was followed by a book called the 59 second employee.
Never rely on an unscupulus third party, or if a third party becomes unscrupulous... Hunt them down.
I disliked Yelp, and held it in deep distain, then while working for a service company, Yelp called. "For a fee we can remove all negative reviews." This is the definition of ensh*tification. So... I told him I would get back to him. I called a few numbers at Yelp and was basically able to socially engineer a vast list of their revinue growth supervisors.
I made a list of every business in that city and the next, and related my experence, and how worthless a Yelp review was. Almost all the business responded to poor reviews with a 'i bet you are a Yelp employee drumming up money for fake reviews.' after a few weeks, I gathered up a lot of these, and sent a three page letter of them in a package with 50 of the Yelp peoples's names in it. I let that stew for a few days, and then called the idiot back. He said "we don't care." They don't. They do not care in the least. Google, OAth, and Facebook do not either. Hurt them in their advertising review and then they will listen.
i use to be that guy telling people not to put their eggs in other peoples basket. Some laughed at me for a decade, almost convincingly and then...
Of course my stuff also breaks because you cant really do anything anymore without trusting anything even against better judgement.
I think we will eventually get expensive quality terms of service for those who think it might be fun if things work forever, like html documents and megalithic structures.
For a site that caters to a startup and entrepreneurial crowd, it's hilarious the number of comments here that amount to "tough cookies, bud" and "Yelp can do whatever they want, and because they can, you should just shut up."
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
> which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
Not everyone. Business that build on top of other company's APIs will arrange contracts with their API providers. Those contracts generally include warning periods for changes or discontinuation and penalties for early termination.
The key here is that it was a free API with no contract or guarantees. Four days is short notice and frustrating, but it wouldn't have really changed the trajectory of his business if they had given him 180 days. If he didn't intend to pay for the API, he couldn't really sell an app that was going to stop working in a few months.
So I know we're supposed to be angry about the 4 days thing. It's not good, obviously. However, I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
> So I know we're supposed to be angry about the 4 days thing. It's not good, obviously. However, I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
As I said in the post and comments here if it made financial sense and they gave me a more reasonable deadline with a less threatening email I would be willing to pay for the API. In this case it didn't make financial sense, so you're right at the current API prices it wouldn't make sense even with 6 months-notice.
That said, 6-months (your suggested time period) is a much better grace period for our shared users (users of Restaurants who use it as a frontend and continue to read more reviews at Yelp.com) and much more likely to make me convert to a paid API customer if it had made financial sense.
> and much more likely to make me convert to a paid API customer if it had made financial sense
I don't understand. Are you saying that even if it did make financial sense, you would have voluntarily shut the app down in protest of the 4-day notice period? Even though the sales rep pointed you toward the free trial option to continue using the API beyond the 4 days while you decided?
I know you're angry and want us all to be angry at Yelp too, but I have a difficult time believing that anyone would choose to destroy a profitable application out of protest just to stick it to the company about a short notice period.
> I don't understand. Are you saying that even if it did make financial sense, you would have voluntarily shut the app down in protest of the 4-day notice period? Even though the sales rep pointed you toward the free trial option to continue using the API beyond the 4 days while you decided?
An app that sold 467 copies over 10 years at less than $5 a copy is not worth the trouble of dealing with a company that gives you Friday->Monday ultimatums. Obviously, if it were a big source of my income I would have to seriously consider it. But luckily, it's not. I discussed this in the "Development Ends" section of the post. Here is the pricing deck they sent me:
https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
It seems to indicate a $229 base monthly price on the third slide for my use case.
> I know you're angry and want us all to be angry at Yelp too, but I have a difficult time believing that anyone would choose to destroy a profitable application out of protest just to stick it to the company.
I'm sorry you're reading so much anger in my post. I thought my blog post was pretty balanced. The worst I called them is "quite rude" (I think it's hard to read their emails otherwise) and spent the first half of it describing my app. I never expressed much emotion in my post, and frankly as mentioned above it really doesn't matter in the scheme of things for my life. What I do want is Yelp to change the way it treats developers. Perhaps if someone there reads this it will cause a tiny reflection on their part. I also hope the experience here expressed in the "Lessons Learned" section of the post is useful to other indie developers.
Both your neutral tone and the fact that you want Yelp and other big companies to treat developers better were very clear!
That's what's crazy to me about all these comments. What does it say that so many developers have glossed over this simple ask for more considerate and respectful treatment for THEMSELVES? What does it say that the knee jerk response is fatalism to whatever big tech does?
Thank you. It's actually just a couple users who have posted multiple negative comments in this thread like gp. Not sure what nerve my blog post hit with them but they are free to not like it! It seems like they're more upset about my blog post than I am about the actual situation.
>> I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
This is kind of the salient point.
Either you test on the free API and plan on paying for access slightly before its ready to go live, or you try the "free lunch" approach and see if you can get one by the tendy and see how long you can go before you get shut down and have to pony up the money.
Either way. they should've had the cost of the API in their budget.
We should all know by now. . . . nobody rides for free.
> Either way. they should've had the cost of the API in their budget.
There was no cost to the API ten years ago. I submitted a prototype of my app to their developer program, described its functionality and exchanged a few emails back and forth with someone in developer relations. They specifically approved it and decided how many API calls to give me. A paid API didn't exist back then to my knowledge (perhaps there was some kind of enterprise API but I don't know?). The point of the post is how badly Yelp handled the transition from free to paid. They are perfectly in their right to transition to paid. But they should've handled the emails and transition better.
As mentioned in the post I developed the app on a whim. But after 10 years, it had a few users, although not many. They and I should have received more than a few days notice from Yelp that the API was going to become unsustainable (see my comments elsewhere in this thread).
I hear you, but this story keeps happening over and over and over. The reality is once these companies have you and your product by the balls, they will start squeezing. You can pay money to reduce the pressure, or leave and not be squeezed. I would argue using an unpaid API takes you into the unknown with considerable risk.
Even if you pay, most likely you have a contract that effectively gives you close to no power because it's full of conditions favoring the service provider and trying to use the little power you have will be expensive because laywers and courts get involved.
Almost everyone is working on projects that “use APIs” in some general sense, sure. But I don’t think it’s the case that all or even most people are working on a project that entirely depends on a single third party’s API and is useless without it.
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users.
Or, they completely get it but they work for large platforms that leverage API access commercially or strategically, so their response to unruly peasants is to figuratively chop their heads off.
Thanks—yeah I actually think they mostly just didn't read the whole post since I addressed this in detail in the last two bold sections "Development Ends" and "Lessons Learned."
> They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
This has been known for like 20 years now. We all know that if you're relying on someone else's API that's a massive risk to your business. What more is there to say at this point? What sympathy is there to give when the inevitable happens?
I think this is the giant internet trick... do anything to scale, then when you reach critical mass (or someone wants OKRs on their status report), pull out the rug.
To be fair there wasn’t much scale. We’re talking about an app that sold 467 copies over its 10 year span. But it was an app I really liked, the people who paid for it really liked, and just drove traffic to Yelp, so they should’ve liked too. It was also a nice portfolio piece for me.
Wow. less than 47 copies per year on average.. Yelp really cut off their nose to spite their face there.
Even leaving out that it was with only a grotesquely unprofessional four day notice, this is ridiculous. It's not like Yelp was suffering a DDoS from all of those API calls.
Yeah, given the numbers it doesn't make sense for Yelp to kill it. It's not like a few hundred api calls are costing yelp any significant amount of money.
I think Yelp needs to just die at this point. Reviews are gamed, usability is worse and worse. Not sure anyone still uses or trusts Yelp. They’re just in inertia from their better times.
Is there a more open reviews platform not associated with a company? OpenReviews sort of thing or perhaps an open POI database associated with OSM, with reviews as attributes?
Somewhat silly we’re still relying on big companies to collect this data when they then gate it. The timing is right with LLMs that can make maintaining and rating reviews less burdensome on human labor (“moderation” broadly speaking). It’s just text blobs and possibly images linked to people and places.
I’m not sure that this is something that scales. Online reviews seems to attract only the worst in mass media (e.g., gamed reviews, complaint box for influencers who don’t get “VIP” treatment).
The traditional restaurant reviewer filled this role much better. His role as a trusted arbiter was closer to what people want from a site like Yelp, but it’s very labor intensive and obviously doesn’t scale.
I'm honestly not sure how big a problem that was. If anything, critics tend towards the the new and unusual because they're bored. The problem is much more that no one is willing to pay for them.
I love the idea but anything like that that got successful would be such a target for gaming that the people working on it would have to focus more on anti-bot work than the core product. And whack-a-mole with bots is just not a fun hobby.
Only ways I could see it working at scale are Wikipedia model, which is labor intensive for a large number of people, or some kind of web of trust / reputation system, which is complicated and attackable.
My thoughts as well. I can envision a platform where you can review anything from specific make and model of a product to a restaurant to anything else. I think it could be immensely useful.
But it would quickly become a victim of its own success as people start using it and the database gets big enough to be useful. bots and spammers would swarm the platform to try to game it. That would require a constant cat and mouse game that would not be fun. Without significant ongoing attention, the site is going to become worthless. In order for a site like this that is labor intensive and not fun to exist, it would need to be monetized at least to the extent it can pay salaries. That's the point at which I don't see any realistic solution. In order to maintain neutrality, it would need to be supported by users, meaning users would need to pay. I don't see very many people paying for something like this. And even if some would, there would have to be some benefit or reason for paid users to pay, and that seems like it would open the door for more cat and mouse games and abuse.
The Wikipedia model is close to what I had in mind. One way (there are others) identity can be solved for using Stripe Identity or something where you same for a gov credential to identity proof. It costs you $1.50 per request, but is no match for bots. Assuming you bind passkeys or other strong auth factors at account provisioning, and there are only so many high quality reviewers, you’ve got a reasonable limit to effort required.
High quality people want to be helpful, they just need a platform where curation and distribution takes place. See: here. I’m just tired of seeing startups and other companies build and kill these knowledge and community based third spaces, and the intangible value inherent to them.
(customer identity is a component of my work in fintech as an infosec lead, I get paid to squash bots and fraud)
I have to imagine that a big part of any solution to review gaming and bot spam is data diversity. eg: whether the submitter has been within X meters of the thing they're reviewing in the last Y days.
Google seems to have done a good job with review spam on Google Maps, better than their job with SEO spam and malicious search results, and deep+diverse datasets are effectively Alphabet's (not just Google's) entire business.
Yelp would have a very hard time getting to know as much about review submitters, an open data platform would have a hard time gathering even what Yelp already collects.
Where "reliable sources" tends to mean traditional media. Which makes it both more reliable (for many purposes) and much more limited. For restaurants, I do tend to trust newspaper restaurant critics more but there's also far less of that especially at a small-scale non-urban local level.
i don't think newspaper restaurant critics count as 'reliable sources' in wikipedia with respect to the questions people are interested in here, such as whether the chicken is overcooked. i am pretty sure that if you try to put an assertion like that into a wikipedia article about a restaurant, it will be reverted as unverifiable pov pushing, regardless of how many newspaper restaurant critics you can cite in support of it. you can probably cite them for things like when the restaurant opened and why it moved to its current location, as well as use them for evidence of notability
I don't really expect something in Wikipedia to get into the nuances of particular dish quality. But, in general, I would expect that--to the degree Wikipedia gets into the quality of a particular restaurant--a New York Times review would carry a lot of weight. In general though I agree that Wikipedia is probably not the place for restaurant reviews even to the degree they're sources from well-known restaurant critics.
this is generally limited to things that wikipedia editors of vastly opposing viewpoints can agree on, like whether and when the restaurant was awarded a michelin star
like, imagine a vegan, a devout hindu, a muslim, a carnivore, a neighborhood kid, and a foodie editing an article about a texas steakhouse that is known for its barbecued pork. whatever text any of these people finds unfairly biased is going to get deleted from the article, and what's left is what they can agree on
this process produces astoundingly reliable articles on contentious topics like the armenian genocide, but the results are predictably of poor literary quality, and generally not very useful for things like guessing which restaurant you would prefer to eat at
I really hate that they force you to install the app to be able to view reviews that people have freely contributed. This is especially a problem on iOS where you cannot change your user agent so that it will serve you the desktop version.
I mean, as at least one other comment noted, anything like Yelp is going to be gamed above and beyond any particularly questionable processes of Yelp itself.
Zagat worked pretty well for "foodies" back in a day when raters had to physically mail in hardcopy questionnaires and people had to buy guidebooks in stores. I'm not sure how you duplicate that today absent physical pay-walls on both sides which I can't really see working.
I do think the various rating sites--Yelp, TripAdvisor, Google--"work" in the sense that they're probably better than picking at random. And you can't really call the local newspaper's food critic on the phone (which is what my dad used to do). So I'm not sure what the alternative really is. Yes, there's some word of mouth but that's pretty random.
And, as I wrote elsewhere, if you travel a lot you need some criteria and Yelp (along with other very imperfect Internet raters) is probably better than eenie-meenie-miney-mo.
Yep I specifically keep Google Maps around to find restaurants even though I far prefer the audible navigation from Apple Maps nowadays once I actually want to drive there.
i do the same. Plus Apple Maps handles audio and lock screen much better. Apple must be calling a private API to manage lock screen during driving. When I drive with Google Maps, the screen locks and blocks navigation
I've noticed that Apple Maps incorporated their own rating system some time ago (I want to say within the last one or two years); it's simple, just asking you to give a thumbs up or thumbs down on certain businesses, sometimes with a bit of granularity (e.g., rate the atmosphere, food, value, and service separately). There appears to be some threshold for how many ratings they have when they switch from displaying Yelp reviews to their own stuff, although it's not clear what the heuristic is.
Anyway, tl;dr: I think they're working on replacing Yelp.
Especially since there were apparently complaints that Yelp was doing pay-to-play with good reviews for a while, which diminishes the truthiness value of any reviews:
(Anecdotally, I also seem to be seeing this on Google Maps now! It seems like highly rated but local restaurants don't even show up on the map at all until I zoom in literally to the building -- perhaps because the owners don't pay for ads? Crazy if true... and Bing maps seem to not have all the restaurants or ratings, and you can't seem to filter based on rating either, which seems like a massive Bing fail. Maybe the review apps were always destined to crumble under a business model that encourages dishonesty on both sides..)
Hang on, they didn't give his app 4 days - they gave him 4 days to respond with some screenshots, and if he didn't respond, they would shut down his access. They didn't say they'll shut it down in 4 days regardless.
I mean, don't take this as me defending Yelp - they're scumbags, and deserve any hate coming their way - but I don't think that the headline is an accurate description of what happened.
I did respond with the required information and they still shut down my access (10-days post though, not 4-days in the end). But regardless, no, I don't think sending you a request on a Friday for information due on a Monday or shutting down all of your access is reasonable. As I mentioned in my post, what if I had been on vacation? 1-business day or shut you down is just not a reasonable time frame to make anyone want to work with you in the future.
Which seems to indicate a base price of $229 per month. I have no idea why that doesn't line up with the pricing on their website. But the fact that they shared it with me indicated to me that my use case fell under some kind of "enterprise" usage. Regardless, I would not continue to work with them after 4-days notice and the threatening email. It's too small and app for it to be worth it to me.
I did ask in one of my emails about the pricing discrepancy and got no reply.
A dollar a day is $30/month, and at $1.99/sale that means OP would need to sell 15 apps ONLY to break even, which given the comments in the blog might be close to reality. This is pennies for both sides, but the way that Yelp distastefully contacted OP is probably worth just shutting the whole thing, it's not like OP was making bank on the app to begin with.
Throw on taxes, administrative overheads, etc, they are probably looking at 30-45 sales per month. Which is likely not realistic.
On top of that, this is a continuous payment. Even if I was looking at 5-10x rate of return, I would be very hesitant as that's the rate-of-return today while the sales are forever.
I've been wondering how realistic microsubscriptions are... Say $1-2 dollars a month per user to maintain an app, perhaps limited to just power users, would support a lot of infrastructure.
I got the same email. Despite my hobby project (a random food picker) having been broken and not used for years (because of yelp API updates), they also told me my usage was higher than other developers...
The email also arrived in my spam folder, so I was lucky to even see it. Once I got back to them they did increase the cutoff by a few days but it has since been stopped.
Their new prices seemed insane to me.
“they also told me my usage was higher than other developers...”
Next you’ll tell me that they’re NOT actually experiencing higher-than-normal call volumes!
"We've understaffed our call center" just doesn't have the same ring to it.
The prices in the deck they shared with me are on slide 3 if anyone is interested: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
They never answered my question about the discrepancy between these prices and the prices on their website. Both would be too high to make Restaurants low sales sustainable.
That said, I'll say again that they were perfectly in their right to start charging for their API. They just should not have done so with 4-days notice and with such a threatening email.
$10 per request at the lowest tier of enhanced?! The only business case I can think of for this is if you save all the data in your own database then serve requests for your own service out of your own cache. I guess if the Yelp data doesn't change very often, then you just choose to stay 6-months behind on the data or something...
No, it's $10 per thousand on average (you get 1,000 calls per day) with a $299/month minimum commitment by my reading.
> Their new prices seemed insane to me.
I call this the "going out of business sale".
The death spiral.
https://steveshuconsulting.com/2006/03/the_pricing_dea/
Interesting and an excellent read. The death spiral at Travelocity (pre-Expedia acquisition) worked differently and was partially self-induced.
Some background is required first. Travelocity was the first online travel agency (OTA). I believe they started around 1996.
Through most of its life at that point all, I mean 95%+, of its success came from only two factors: best marketing in industry and growth of the internet. This is severely problematic because marketing only gets you so far in business. Conversely Expedia had really shitty marketing in comparison and yet came to dominate the industry because they were extremely aggressive at growing their supplier relations.
You cannot EVER rely on growth of media adoption, like mom and pop coming online, because once that stops you have no fuel left in the tank. Reliance on growth of a media platform is like a gravy train that you did nothing to build and returns amazing wealth if you are in the right place at the right time, but once it stagnates its like your train derails and everybody dies. That is because everybody expects growth to continue, except you did nothing to earn the growth and now have no answers and nothing to show for it. This was around later 2008 when I joined the company and became unavoidably obvious to everybody over the next year.
So, at that point what do you do? You competitors are far out pacing you by ignoring fun stuff, marketing, technology, and all the other bullshit that technology people look to. Instead they are focusing on core business principles and eating your golden goose while laughing at you. So, what do you do?
In the case of Travelocity all the executives leave. New executives come into trying to figure out what to do. Like every great web business they focus more aggressively on marketing and advertising. This was Travelocity's death spiral.
You have to understand that people DO NOT like advertisements. Really, I know its surprising, but when your site becomes littered with advertisements everywhere and all kinds of hidden telemetry people will leave and never come back. Your wonderful palace has become a trailer park.
The business loves advertising. Revenue from advertisements is immediate. That is really significant. In e-commerce there is a massive lag between each stage of profit, revenue, and sales because you have to account for the cost of operations, sales, and inventory. The more expensive the product the longer the lag and that lag really complicates projections. So advertisements are like cocaine, because they immediately return profit that requires no effort while rotting your health slowly until you are a hollow skeleton.
To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
My learning from this is that a business that earns profit from selling something directly should not fuck up conversion or go out of its way to make customers hate them. When I put that way it sounds obvious, but web business get that wrong all the time because they get distracted by shiny things.
>To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
I am curious what these amazing things were.
So in the travel industry you have limited lines of business, primarily: air, hotels, and car rentals. Air is super low margin because its set directly by the supplier plus some arbitrary service fee, typically $7 per ticket. Air is high volume though and is the primary draw for OTA customers, so it cannot be ignored. Rental cars are super low priority because they are both low margin and low volume.
The real interest in the travel agency business is on hotel inventory. This is how Expedia came to dominate the industry, because they aggressively hired geographically focused hotel relationship personnel and Travelocity wasn't keeping up. So Expedia had much greater inventory from various different properties and stronger relationships with those properties for their business prioritization. Hotels, typically hate dealing with OTAs, because OTAs are a transparent barrier between the hotel and the guest staying at the hotel, but the OTAs bring people to the properties. The name of the game for hotels is "heads in beds" and while up-selling is beneficial if you get the "heads in beds" part wrong you cannot exist.
The margin on hotel properties is much higher and highly variable. Hotels in many cases will even eat the cost of airfare to get heads in beds if the pricing is right. That provides a huge opportunity for a marriage between smart hotel systems and OTAs because the hotels are doing the smarter inventory management and the OTAs are supplying the airfare volume and customers to populate that smart inventory management. The hotel data systems, Property Management Systems (PMS), were at that time much smarter than the inventory availability OTAs had, because they had to do much more to account for seasonal volume planning and various customer demands.
Where Travelocity was strong was in writing algorithms to account for these considerations and offer volume pricing discounts the competition could not and also air + hotel package pricing at massive discounts the competition could not compete with. In parallel Travelocity was slowly building up a corporate travel business to take advantage of that volume pricing intelligence and at one point had both WalMart and Lockheed-Martin as customers.
Again, both of those were amazing and could have really helped retake lost market, but conquering competition is a fragile slow process while advertising is immediate.
Thank you for taking the time to give a comprehensive and interesting answer!
Thank you for this comment. Very interesting! And very applicable to everything we are doing again.
Thank you for the link, nicely explained!
Or they want to get the traffic back directly and overprice on purpose.
Doesn’t Google own yelp?
No, it is a publicly listed company:
https://www.macrotrends.net/stocks/charts/YELP/yelp/net-inco...
Been going sideways for a decade now though:
https://www.macrotrends.net/stocks/charts/YELP/yelp/market-c...
Doesn't seem like they do. They attempted to acquire them back in 2009 but it didn't work out.
> they also told me my usage was higher than other developers...
I'm curious to what they compare this. The non-existent free plan where everything higher than 0 is unusual. Or a large mass of dead accounts who have really zero to none usage. Or the actually paying account who are now on their enterprise plan. In the later case, this might indicate some real problem if their usage is low enough that smaller projects are already exceptional.
Author here. I actually discovered something in their terms of use today while looking up something else that someone had asked me in this thread:
"2. CHANGES Yelp reserves the right to modify the API Agreement at any time. If Yelp reasonably determines that a modification may materially and adversely impact You, Yelp will provide email notice to you using the email address you provided during registration no less than ten (10) days prior to the material adverse modification taking affect. IF YOU DO NOT WISH TO BE BOUND TO ANY NEW OR MODIFIED TERMS, YOU MUST TERMINATE THE API AGREEMENT BY CEASING USE OF THE API WITHIN TEN (10) DAYS OF RECEIPT OF SUCH NOTICE."
So, the 4-days would have violated their original notification terms of use anyway. I think they realized this after the fact of sending all of us the threatening emails and this is why they ultimately closed my API key after 10-days as described in my original post.
Anyway, just an interesting aside that the poorly written, inaccurate original email that I shared in my original post was even inaccurate to their own policies.
"YOU MUST TERMINATE" rather than be held to the original terms you initially signed up for ... because Yelp RESERVES the right.
We have all now been reminded that RESERVED RIGHTS can and will be used against you maliciously. And if you want protection from such malice, there needs to be a RIGHTS of the USER clause.
This should be handled and enforced by government statutes.
(It is an amazing world we live in where rights of the user have to be expressed.)
10 days.
How very generous of them.
Seen this story play out so many times. I audited a company years ago that claimed to have excellent, personal almost, relationship with Google and all needed paperwork to use their platform as a core of their business. They went bust 6 months after because of "unexpected" change of Google's product strategy.
Real lesson here is to avoid single points of failure, regardless if it's API, people or partners. Ask yourself a question if there's a single entity that can kill your business and remove that reliance.
If you're depending on the continued existence of any Google product beyond GMail, Google Cloud and advertising you should really consider fallback positions. The existence of Youtube is also guaranteed, but the existence of any particular API or service that you might depend on? Not so much. Hell they'd probably consider killing GMail if they didn't need the accounts to tie advertising impressions to individuals.
Not even search makes the cut?
Search is already dying. Google doesn't need it.
Well, they pay $25B+ per year to be the default search engine on various platforms, so that seems unlikely in the near term.
(https://www.cnbc.com/amp/2023/10/27/google-paid-26-billion-i...)
That's a ton of money, (and I've heard that they paid Apple even more last year) but it's kind of understandable. Apple customers don't usually have android phones snitching on them, are less likely to use chrome, and tend to have their devices and data in the Apple ecosystem. Google has fewer ways to spy on Apple user's lives and push ads at them. Search, gmail, and youtube probably give them the best opportunities. I wouldn't doubt that Apple users are getting as frustrated with Google's search engine as everyone else is though.
Do you have a source for this? I suspect search makes at least 75% of Google’s profits.
see https://news.ycombinator.com/item?id=30347719
Google used to need Search in order to learn about the intimate details of our lives. It's how they knew what we were interested in, what our medical problems were, what we were learning, and what we were thinking about.
Now they have chrome giving google people's entire browsing history, android devices collecting realtime data on what people are doing offline, where they are and who they are with. Google also has products like fitbit, nest, and gmail that gather still more data for google. Google doesn't need Search to spy on us anymore, so they haven't invested in keeping it useful.
In fact, it's better for google if you can't find what you want and have to make multiple searches for information because it gives them more chances to throw ads at you, and the harder it is for people to find websites using Search the more sites might feel like they have no choice but to pay Google to keep them at the top of the search results.
People who find Search increasingly useless though are turning to alternatives and for many people AI could end up replacing google's Search product as the first thing they turn to.
I would not count on Search staying around forever.
They need search for different reasons: it's their biggest moat against Meta by being the start page of the internet for a great many number of people. A whole lot of browsing sessions start with search and end with search, and this way Google can claim attribution for virtually every traffic a website gets outside of direct hits.
This keeps people developing content to please Google and running ads on Google platforms to get more traffic.
Yup. Search is gradually (for about 15 years now) being transitioned from a search feature to a self-serve ad-requesting platform. “Show up, type a term, and we will select 10 ads for you.”
> for many people AI could end up replacing google's Search product
The quality of Google search lately has felt pretty meh, especially for generic topics that get hits for crummy blogspam. Google’s own AI results at the top are pretty much the thing keeping me there atm aside from inertia. My personal browser is on DDG which is fine just a tad slow comparatively
Umm, they still need to sell Search ads right?
It's not as if they can't/don't also push ads at people in gmail, in youtube, Google maps, on android devices (phones, TVs, chromebooks, etc), in chrome, and they still have AdMob and AdSense and the Google Display Network
They do, I'm not sure what that other comment was on about.
Given how management is behaving lately, I wouldn't rule out any "shooting yourself in the foot" moves by them
That is only half-true. On the one hand, I agree and I would never build a serious business on another single entity. But on the other hand, OP built an app that ran successfully for 10 years, generating some revenue.
Most products have an expected expiration date and you can provide a useful service (that generates revenue) by building on other platforms, even if it won't last forever.
By now I missed more opportunities by having the mindset of "not relying on other people's APIs" than real changes/shutdowns would warrant this kind of caution.
It's so sad that we're at such a devoid barren state.
There was so much hype & excitement in web2 age, about APIs & intertwingularity. Humanity was going to keep improving & iterating & crafting ever neater ideas.
Now it's all bureaucracy & lawyers arrayed to defend the cloud-keeps, moats & walls.
The user & agency of the world has been steady dimished, eroded away by ever advancing legal IP Protectionism.
That's not going to work in practice as money and time are limited. Choose your partners wisely but don't obsess over the scenario of them going bankrupt.
I built a service around helping podcasters automatically convert their audio podcast into a YouTube channel. I went through tons of review with Google in order to get access to the YouTube API and make sure everything I was doing was in compliance with their terms - literally months of back and forth. I had been testing in my development and staging environments against their API for 6+ months. I launched in production, got a few videos uploaded to YouTube, and they disabled my API key. I spent months emailing them and never got anything more than the same boilerplate copy/pasted answer. I could have pivoted or something, but I just shut it down and moved on. Lesson learned.
I used to not bother with APIs and instead use selenium to drive the websites. It neatly avoided all of the politics around API access.
HTTP is an API of sorts…
> HTTP is an API of sorts…
True, but it is an API that they can't easily deprecate on a whim.
But they can quickly change the "structure" of that API.
I worked for a very early billpay company where you could pay your bills online to vendors, even if the vendor didn't support it. We used API's where we could, but where we couldn't...
We had a whole team dedicated to keeping up the changes vendors would make to their websites that we scraped for info. The team was called, of course, "Scrape and P(r)ay".
I believe this is where smarter tools like Kadoa [0] can be of help. It detects data structure changes for existing workflows, and adapts to them.
[0] https://www.kadoa.com/
A small problem compared to not having an API key and being stonewalled as to why.
If you build your scraper to find data on the page based on the shape of the data itself instead of the structure of the page then it will be resilient to most changes that don't materially change what data is displayed on the page.
So, prefer regex over css selectors, and css selectors over xpath, where possible. And don't select based on nesting or position if possible.
Isn't that similar to deleting your API key except you can at least fix the structure of the selenium one.
You're one CAPTCHA added to the flow from this access being effectively deprecated.
In the AI era, is this as true?
Depends on your development and per-action cost. And on the possible latency. It also changes your whole stack from "send a request" to "emulate each step in a browser while taking screenshots at (hopefully) the right event/delay" - that's a huge difference.
That may be a CFAA violation, a felony in the United States. See Ryanair DAC v. Booking Holdings, Inc.
That's because Booking was also committing some type of misrepresentation and taking revenue away from Ryanair through their browser automation. Even then, the infraction was sooo bad that they got a $5k fine.
I think the hiQ Labs vs LinkedIn case is a better representation that scraping is generally allowed: https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
> In November 2022 the U.S. District Court for the Northern District of California ruled that hiQ had breached LinkedIn's User Agreement and a settlement agreement was reached between the two parties.
Reminder that the earlier ruling was overturned, it is no longer clear whether scraping is legal or not.
When it comes to CFAA violations, corporations get a $5K fine, while individuals get hounded to suicide.
Start a company then do my nefarious work under that, got it!
You should do ABSOLUTELY EVERYTHING in your life under a company. And I mean EVERYTHING
My wife hates it when 1099 her for services rendered, especially since she refuses to bill against the PO I set up for her. The kids at least accepted NET15 payment terms. Although the oldest said if I short pay him for lawn care again, he’s going to take away my early pay discount.
Have you considered short paying him again through the same entity? After all, that's one use case of limited liability ;)
Kafka is smiling appreciatively at your approach to marriage
corporations are more people than people.
Unironically, yes.
How does that work, though? Setting up a company has an initial cost and then recurrent costs (accountant, etc). Are the benefits that high for the average Joe?
It's hard to answer without specifics, even if you're not doing anything neferious there are a lot of benefits to putting an entity between yourself and your customers. It depends on where you live and what your business is of course.
Just say you train an AI on it and it's all fair game.
Good thing I am not an American, don't live in the US and have no intentions of visiting then.
(I gave you an upvote, I don't think your comment deserve to be nuked for bringing up a possible legal risk)
Very Probably No way they could find me.
This literally made me lol. Not sure if it’s true. I might be true. But come on!
My first Saas was built around Facebook, Twitter and Reddits API.
Facebook broke like monthly and required random updates, sometimes documented, sometimes not. Zero support.
Twitter worker flawlessly until I hit some limits and there was no way to increase them because I didn't hit some other limits. No way they would talk to me.
Reddit just worked until I gave up.
What I learned is to never again do any business based on someone else.
A good lesson, but a more nuanced one is to do sure diligence. There are companies I would never build my business around (Google, Oracle, etc.) for a variety of reasons. There are ones I know I can trust (at least until they change). There are many where I need a contract.
Change is also a constant, and there are plenty of good companies give bad, and a few in reverse. So due diligence is an ongoing process.
That dramatically raises the price of SaaS, so I use it much less than mainstream industry practice, but much more than zero.
> What I learned is to never again do any business based on someone else.
That's a little dramatic, don't you think?
Supply chains are a core of many (most?) businesses out there. They depend on others to get their business done. No supply -> no business.
What you probably meant is not building a business based SOLELY on irreplaceable business relationships. Unfortunately in a lot of cases involving the tech giants and little startups, there are no alternatives.
More or less, however I really try to not build on anyone else ever again.
I can't find a single dependency in my serious money generating chain right now that isn't replaceable with a similar product within a few minutes or hours if necessary.
My most recent project didn't even last a full year as there was no replacement that worked for me for ChatGPT in that moment. It was built to die, a gamble essentially. I wouldn't waste my time for things like this if I didn't had the end calculated in already.
Same experience, slightly different. It made me realise why some parts of an MBA (e.g. strategic place/partnership/spring boarding from a brand) actually have value I never saw before for tech companies. There are essentially king makers in certain fields - either by their apathy at first-movers, or their choice on who they allow to continue.
>Lesson learned.
And that lesson is don't base your business model around playing in someone else's waled garden.
Precisely that.
Funnily, if I understood correctly, he already had that lesson with Facebook. Let's see what comes next...
Their loss of additional content, traffic and ad revenue.
The impact on google losing $10m of revenue is exactly equal to the impact of John Doe losing $10m of revenue
In a competitive market John Doe would go to one of googles competitors. We don't have a competitive market. We have winner takes all market.
Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul. Reddit, Twitter, and company all got big because they had open APIs and people were able to use them extensively for really creative things.
I agree, Google has swallowed up the video streaming market unfortunately.
I keep thinking back to how Vine was basically TikTok, and they threw it away.
Take it from someone who watched with interest, Reddit and Twitter did not get big due to open APIs. They got their first big steps before they even had APIs... In the first year of twitters launch it was in the news in the UK at 300k users in press releases about how it was a "new form of communication" etc... 2008 was actually when the original set of API-laden websites began to fade because with the recession, we could not have nice things be free so much. Many API services fell into disrepair in 2009, certainly the peak of APIs for the UK was around 2008. Check out Tom Scotts video on this for a picture of the open API internet many people thought would occur (and which did exist in some places until FAANG began to productise and dominate and use accounts across products which do not incentivize APIs). Yes it is true open network graphs/facebook etc had some API access, but the APIs before were more single purpose and numerous; facebook lets you explore their network and to some extent data, but that's not the same as the useful APIs that became restricted, paid or were cut in the aftermath of the recession.
Those startups are their competitors. You don’t have to pay millions of dollars to acquire a competitor that was never started because of your inconsistent API policy.
Google makes enough money that losing several currently non-existent revenue streams that are theoretically $10 million apiece isn’t hurting them. It’s hurting their users.
Google is all about ads - why would they give a shit about the users?
>Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul.
Google made $300B last year. "billions over the long haul" is a lot of money, an unimaginable amount even, to you and I. But to Google?
> Google made $300B last year.
No, Google made $73.795B last year. Revenue is not profit, net income is profit and what you ‘make’ as a company.
Revenue is not profit.
First off - how much of revenue usage last year was on fines etc. That is to say profit they could potentially have had.
Second there is lots of revenue you use for "business" because you don't want to pay taxes on the profit and anyway it is giving you something you like to have.
How much of Google's revenue is used to do things for top executives and people with power in the company that is really something they (the executives) should be taxed on but is instead a business expense of google (cars, transport, 'working' vacations, security, super cool chefs preparing meals at the company...) Hard to say really because if you knew the answer it would actually be something they were taxed on. But it's not 0 - sure probably not a billion, but a couple hundred million splashed around wouldn't surprise me.
How much of Google's revenue usage is for wages and other forms of payout to executives etc. (stock) that does not get counted as profit but of course it is amounts those people want to have.
Google made $73.795B profit last year, and expensed slightly over 262B - some portion of which the people who run Google no doubt personally thought of the way we would consider profit in our day to day existence, and another portion of which were fines for things they did in getting the rest of the money.
Literally none of that is relevant for much the corporate entity made (I.e. what the shareholders got).
Obviously the fines, which are the first thing mentioned, is relevant for how much the shareholders got.
Literally the other stuff is relevant for how remunerative the people who actually run the companies, day-to-day, feel those companies are (put in day-to-day because I got the feeling you might give me a lecture about the shareholders actually running the company)
Personally I think you have a decent point since in practice employees of GOOG are shareholders too. Although I don’t know if their interests matter to the corporate entity because I don’t know what percentage is held by insiders.
I don't think that's nearly as universal an assumption as you're claiming. Like--I made $X,000 dollars last year, but I also have a mortgage and like to eat food.
Mortgage accumulates equity. Bad example
Mortgage interest payments don't
Your assets might inflate in value, but that's nothing to do with the mortgage
By that logic I make what I can save, lol
Well yes.
Companies get to write off expenses of existing
People don't
right, which if you follow that assumption you can see that things are not as great as you thought.
The assumtion things were great wasn’t mine. I live in a HCL area and am not able to save much so in this sense the ‘profit’ of my work is close to zero. That is not great at all…
[dead]
Even if we subscribe to this interpretation, the overall point stands.
Revenue isn't profit, but your wasted business expenditures are properly measured against your revenue. That's what they're spent from.
I mean, they "make" both. "Make" does not imply profit. If you say a company "made" a revenue figure, then you mean it in the revenue sense. It's contextual.
That’s like saying a trading company made a trillion dollars because you counted all of the securities sales but not the costs.
No, it’s not contextual. “Company X made $N billion dollars” only means one thing, net income.
100% contextual. In every business where I've been an executive, or had access to the budget, we talk "made" as gross revenue and EBIDA as a stand in gross profit and specifically call out net profit in internal meetings. For example, "We made 10 million with an EBIDA of 1 million and net of 200,000.00." Using EBIDA to talk to potential investors and as a guiding metric if we didn't have a well established gross profit formula that followed GAAP.
Sorry bro it's contextual.
This is a bizarre nitpick.
1. I know it's revenue - the GP says The impact on google losing $10m of revenue
2. Do you think the 10m that Google would have booked from API usage would be booked as pure profit?
It's a testimate to the health of our free market that the company throwing away millions, and billions over the long haul, is the overwhelmingly dominant market leader.
Attempted to build a similar thing a few years ago when living in Tokyo.
Provide a selection of restaurants within a 1km radius and automagically provide 3 recommendations based on my preferences.
We had multiple API providers available, Tabelog, Gurunavi and Hotpepper, all required a paid developer license. We still needed to use the Google maps API to get the user's current location though.
This was also just when Google Maps suddenly raised their API pricing. After spending a couple of weekends building a working prototype, we stopped as we couldn't justify the cost of paying multiple API providers for basically 3 guys looking to save 10 mins deciding where to get lunch.
Also, this kind of app is a common theme if you frequent meetups in Tokyo. There's always at least 1 person that has built such a thing.
> We still needed to use the Google maps API to get the user's current location though.
I’m sure as somebody who actually built a prototype, you understand the situation better than me, but I’m curious: why would just getting the user’s lat/long require anything you’d pay for? Wouldn’t the OS or browser provide that to you?
Oops. I meant that we were using the maps/places API to get lat/long of the restaurants using the street address from the other API providers, then using that to calculate a 'N mins walk away'.
We were just testing the abilities of the various APIs. Google was the only that provided travelling time estimates.
I was thinking about how to solve this given that one of the primary problems is that of fast, global content distribution. I like the idea of paying people in crypto as part of a ledger transaction to host and serve content, like bittorrent with a crypto payment. Unfortunately I can't also think of a way to prevent such a system from being abused to distribute harmful media such as CP. I guess it's not like this isn't a problem with BitTorrent today though.
Regulation might here - something like minimum mandatory 3 months notice for shutting down the API keys. Considering the average age of our politicians, I doubt they’ll understand what “API” is, much less be willing to take on giant tech corporations even if they did understand the problems
> […] much less be willing to take on giant tech corporations […]
I’d argue that, on the contrary, it seems to be all the rage lately.
Not really. There is a big difference between making noise and actually doing something about the power of big tech.
Yeah, what’s popular is actually to crap all over the first amendment by mandating private companies to allow whatever each politician’s special definition of “free speech” is (but of course probably also not allowing whatever the politician’s definition of “dangerous misinformation” is).
As for the actual anticompetitive power of big tech, absolute crickets.
And yet: https://news.ycombinator.com/item?id=41164240
It isn't much, but I'd like to see that as a start.
Competitive markets are by far more the exception than the rule. It's just not how capitalism works (because pretty much any competitive market is ripe for “consolidation”, which increases aggregate shareholders value by reducing competitive pressure).
If you want competitive market in a capitalist economy, then you need very active state enforcement.
The markets are absolutely competitive. Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
They are now hemorrhaging search to OpenAI that popped into public existence just recently.
To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
If you wanna see lack of competition, look at government granted monopolies on utilities. Guaranteed but capped rates means you reduce investment right to $0 and cut costs as much as possible since there is no other way to make money. That “state enforcement” you are calling for is how you end up with PG&E and scenarios like all insurances companies pulling out of the state.
> Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
Phew, here I was stressing out about lack of competition, but you helped me relax.
Google has to invest into Youtube lest it loses out to Twitch (Amazon), Netflix ($35bn per year), HBO Max (Warner Bros, $50bn per year), (Amazon) Prime, Plus (Disney, $90bn per year).
My faith in humanity is restored now that there are alternatives.
From other MegaCorps.
That’s competition, don’t shift the goalposts.
I picked well-known names as examples. There are thousands and thousands of niche providers.
That's not competition, that's an oligopoly and they are mostly focus on their own niche for which there isn't more than a minority competitor (there's no competitor to Youtube on for non-live video, Twitch is in a monopolistic competition situation for game streaming, so is netflix on TV series, and Disney and HBO each have a monopoly on their IP).
Claiming that this is a competitive market is a joke.
> To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
Please tell me how many “product advancement” in Google search or YouTube over the past decade… From a consumer's perspective all that happened was enshitification and despite being owned by a search engine company, YouTube never managed to ship a functional search on their platform.
> If you wanna see lack of competition, look at government granted monopolies on utilities.
Utilities are “natural monopolies” though, and as such they should be state owned. Making a natural monopoly owned for profit is a recipe for rent seeking, and that's why it was promoted …
> That “state enforcement” you are calling for
No, the state enforcement I'm calling for is proper enforcement of antitrust laws, forbidding consolidation through M&A and disbanding companies megacorps. That is to say, what existed in US's golden age.
>Competitive markets are by far more the exception than the rule it's just not how capitalism works
Empirically this absolutely isn't the case; the majority of listed companies have fairly low margins, especially non-tech companies, which can be trivially seen from their financial statements. A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
> A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
Not necessarily: if prices are elastic then even a monopoly can aim for low profit margin (in percentage) in order to increase profit. What matters is how much total profit are being made, margin only matters when measuring risk.
Also, corporations are social structures, and low competition also encourage complacency in the corporate structure itself, which drives costs up and reduces profit margin.
In addition, a low profit margin may increase lifetime profits by deterring potential competitors.
[flagged]
Might there be other reasons to report lower profits?
>The impact on google losing $10m of revenue is exactly equal to the impact of John Doe losing $10m of revenue
no, it's not, see declining marginal utility of wealth
https://www.economicshelp.org/blog/12309/concepts/diminishin...
https://www.investopedia.com/terms/l/lawofdiminishingutility...
Cool idea, I was almost about to do the same app/service. I didn't know it could end like that. (Naive, I know.)
Have you seen https://repurpose.io? They existed before I started working on my service and they do the same thing.
Do you know if there is any open source tools in this field that a programmer can work with/automate, I have mainly textual and graphic content, a little bit of video, but thinking to branch out to podcasts and music soon, unfortunately the other products supporting text seem pricey and geared to big teams.
on edit: obviously I can write my own, but I am hoping for a project that has already done a bunch of things that I can extend for my needs, as writing my own would be at the point where it would be more cost benefit to purchase.
I run a webapp that uses the YouTube API too.
I believe I am entirely in compliance with YouTube's terms and policies, but I was flagged for some bullshit as well and got the boilerplate responses. So what I decided to do was hide it for logged out users so it looked like that specific feature was entirely removed to the dudes in India that review this stuff... One year later and my fingers are crossed we don't get another audit.
We went through a three-week Facebook API shutdown due to a clear glitch - Meta support couldn't go outside the script, or escalate in any way beyond "a supervisor will email you". Only resolved when our CEO found some VP at Facebook on LinkedIn and got them to escalate it internally.
(The dev community thread is full of people still impacted, so I think they literally just edited our app's flags directly.)
Even tried to invoke GDPR's rights to be exempt from automatic decision making, but their privacy questions email address responds with "nope fuck off" to those.
Basically any hope of solving a problem with a Meta property is to know employees who can escalate your issue. Sometimes your friend at Meta also needs to argue with the bug owner to not close it frivolously.
It's the same with Google.
Within the customer service industry, they'd claim a great "deflection rate". That's a metric that all of these large companies hold around any kind of help/support channel that may involve humans because people are a cost. It's often covered by some kind of satisfaction metric based off post-issue surveys, but fundamentally, if you just go away that's success.
There are often complaints here about what amounts to bean counting affecting other aspects of business. Customer service at larger scale is costly so attracts a section of very analytical leaders. They don't get, or ever prioritize, the human elements. It's only when satisfaction numbers are bad, or another exec outside of customer service takes action, that things improve.
The scale part kills the customer. You can have great support at one size. Once you grow then leadership, structure, and the culture change. These analytical leaders don't want to carry over the culture and structure because it comes at a cost. It really needs force and support from outside of customer support leadership to maintain it.
Had this issue with Ubisoft recently. Installed Trackmania through Steam for free, then had to create a Ubisoft account, then had to buy the Trackmania annual pass for $20. I guess I followed the wrong flow or something, but I ended up with the purchase on my Ubisoft account, but I guess their system also auto-creates a Ubisoft account for Steam accounts? So the purchase on the Ubisoft account ended up on the account not tied to Steam.
Emailed support about it on 13/07/2024.
Got back a reply on 23/07/2024:
> Our Support Team is currently experiencing high case volumes, so we are reaching out with a message to check-in and make sure you still need help!
> If you've found a solution to your problem already, there's nothing you need to do. Just ignore this message and your case will automatically close in 2 days.
Didn't see the reply until the 3rd day.
> As we have not heard from you in the last 2 days your case has been closed automatically.
Excellent customer experience.
> Customer service at larger scale is costly
I think people, even on HN, grossly underestimate how much customer service costs at any major company that works with the public.
Most calls to customer service are easily self-serviceable. My brother used to work for X-Box Live support, and he said over half his calls were for password resets, something people could easily do on their X-Box.
But then there's bad actors. Another significant chunk of his calls were from people complaining about being banned, and he'd see the reports of their spamming racial slurs.
Not to mention how many people try to abuse customer support in attempts to take over someone else's account.
Yeah...customer service is expensive. Even if you automate as much as you can, but offer an appeal process, that appeal process with be absolutely abused by bad actors, so that will end up automated as well.
I've heard that serving them process is another way to talk to a human being.
My sympathy is veeery limited. But once you have an app that does the exact opposite, please tell me!!
And, just in that particular case: I'm veery okay if that would be a cloud service.
YouTube to podcast is way easier than the other way around. I'm sure solutions already exist.
Are you based in EU? If yes, then I would a complaint letter to the competition bureau. And you can CC Google offices in your country.
I am curious if you would be interested to retry the idea? I might have an in with the YouTube team. I feel like it's a shame to let this go. Would you be open to chat? Please reach out r@pehul.com
If only we weren't living in a world run by abusive mafia... o_O
open source your code ?
Couldn't you have bypassed them and used `yt-dlp` or something similar?
Why get Google/YouTube's permission at all about this?
I was creating YouTube channels and uploading videos to them on behalf of my users. That requires using the YouTube API.
Can you let the user bring their own API key?
No. The default API quota is not large enough to upload videos. You would have to contact Google, explain your use case, and jump through several hoops to get the quota increased... which is a huge process.
I once tried using a Python library to upload videos with an API key. All videos uploaded got forced private due to "spam".
Yeah. Uploading legit videos is non-trivial. And if you ever upload the same video twice (which you think you might do during testing, right?), it's a violation of their terms and they disable your access.
reread: The project was going in the other direction, From a pure audio source to a Youtube video.
I think Yelp is shutting off their API for the same reason as Reddit: to ensure that AI training makes money for them. It sucks that you’re a drive-by casualty of that, and if I’d bought this app, then - same as Apollo iOS - I would not request a refund.
I think nobody wants an API. The same reason why YouTube plays cat-and-mouse with yt-dl and longs for the sweet nectar of Widevine DRM
To wit, there is exactly one business strategy: When you are small, be nice. When you are big, pull that ladder up behind you.
YouTube already uses Widevine for movies. they will never use it for regular videos, because it will kill their website. widevine is a resource killer, for both client and server.
It's just more of Cory Doctrow's observation of Enshittification.
Or, perhaps more simply, The Oatmeal's Reaching People on the Internet: https://theoatmeal.com/comics/reaching_people
> To wit, there is exactly one business strategy: When you are small, be nice. When you are big, pull that ladder up behind you.
I'd add a "in venture capital and big capital" after "business strategy". When you don't have VCs or the stonk market breathing down your neck all the time, incentives massively change.
IMHO, there is only one solution left... once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust that has a clear mandate to run its companies in a way beneficial to society at large. That way the government doesn't have to spend taxpayer money on countless r&d experiments, VC investors have a perspective to payout, and the world gets kept from utter bullshit like API games.
Well if small capital doesn't pay good, I have no incentive to run a small business at all!
> once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust
Interesting, haven't heard that one. I'll say maybe. I'd rather have Facebook not exist at all than have it be an *official* arm of the NSA
> the government doesn't have to spend taxpayer money on countless r&d experiments
If you expect the government to make a tender offer on the largest market-cap companies, they sure as heck _are_ paying for r&d indirectly. And a lot more!
Isn't that kind of what they do with QE?
I think it's quite different. It involves money, but QE is all about monetary supply and interest rates alone. Buying entire companies comes with a lot more implications, like decision-making power over the organization. You want that for API keys, but do you want that across the board? Seems like regulation would be much cheaper.
Thanks for replying, today I corrected a misunderstanding i had about QE. I had previously believed they were buying and selling stocks (I misunderstood their usage of "Securities")
[flagged]
Why? The problem at the core is the mantra of the WSB et al groupthink: stonk only go up.
Of course when WSB makes jokes about it and catapults a junk stock like GME to ridiculous heights, squeezing shortsellers dry, it's pretty funny to watch. But it gets problematic when just about everyone's pension is tied to "stonks go up" as well. That leads to all sorts of perverse incentives.
Just look at Boeing and how far they've fallen. Twice in recent years, literally and not just proverbially, with hundreds dead as the inevitable consequence of stonk market games.
I think the GP had a visceral reaction to your intentional misspelling of "stock" as "stonk".
I tend to agree, though I don't think I'd go so far as to think someone saying "stonk" has "lost all credibility"... but to me, it's like Republicans deliberately mispronouncing Kamala Harris' first name, because they can, and because their base eats it up. It's a cheap shot, and it feels childish, like you don't have a real argument, so instead you act like an elementary school kid on the playground calling the other kid names.
> It's a cheap shot, and it feels childish, like you don't have a real argument, so instead you act like an elementary school kid on the playground calling the other kid names.
Exactly all of this.
Minus one out of ten credibility then. Torched is quite an overreaction.
This has been one [unintended?] consequence of AI promulgation. A direct disincentive toward the kind of open access that so used to be common and provided a lot of low-hanging fruit for independent developers trying to increase interoperability within their favorite niches.
So now not only is AI filling the web with garbage that poisons future model development[1], it provides incentive to further close and wall off access to (user-provided!) data.
1. https://www.nature.com/articles/s41586-024-07566-y
Yes. We're entering a new era of web applications, whether we want to or not. Companies used to be able to gate their data behind UIs, because the search results had more value than the raw data.
Things are starting to flip. Increasingly, people would rather have access to the raw dataset to pipe into an LLM. The question, as always, is how to control this access and charge people accordingly.
Reminds me of this video about the old, open web:
https://www.youtube.com/watch?v=BxV14h0kFs0
I was just wondering why - for multiple screens of text - nobody came and just posted some pointless youtube link.
But this case seems to be funny at least (acc to your text): A YT link about the 'old, open web'. YMMD!
I think you're right. API strategies are being reviewed because AI training data.
That said... I think API policies are also vestiges of a defunct idea set that prevailed circa 2009.
At that time, "twitter is a protocol." APIs and openness were going to be the new worldwideweb. Social media was going to be an ecosystem of independent apps and services with complex interactions mitigated by protocols.
YC even did an "rfs" recruiting startups to build off the twitter API.
That didn't happen, but APIs intended to enable it were already in the wild.
As API access becomes strategic again (because AI), "why do we even have this?" is likely to come up.
Why would anyone ever want to train an AI off Yelp reviews? That sounds horrifying.
Local recommendations is a big category of LLM questions that they're mostly bad at today that they can sell ads for in the future.
I think the general lack of willingness to help in Big Tech is very problematic. You can almost never get through those thick-skulled reps that email you out of the blue...
The problem is that by having a thick-skulled rep walk in and send a few e-mails, Yelp has already lost more money on API customer acquisition than the app developer was willing to pay. That's why these APIs had free tiers: they covered these kinds of micro-usages that would be far too cheap for a sales rep to cover.
The reason why those free tiers went away is that AI came along. Not so much that the AI scrapers were abusing Yelp free tier[0], but that they could. And once companies realized how much money was floating around in selling data access, non-abusive free tier users went from "a cool goodwill gesture" to "freeloading parasites".
David Kopec and Restauraunts got steamrolled in a case of technological gentrification. If you're selling data access for $TOO_CHEAP_TO_METER/call to a random indie, Apple, Google, and/or Microsoft will use that as a comparable for why Yelp should charge peanuts. Or they'll just acquihire him. They need him and his app to go away because he is inconvenient to the long-term valuation plan of Yelp, an old guard Web 2.0 business that never quite became sovereign.
[0] Though, to be clear, AI scrapers are absolutely abusive in general.
One of the things I most admire and also loathe about Big Tech is how much they normalized nonexistent customer service.
A couple years ago, I was confused about why my Apple TV remote was behaving weirdly, and a few taps in the support app and I had a real person call me and fix my issue within minutes.
I was looking for a completely random Molex adapter from the early 1980's for a hobby project. After reaching out to the company they had someone call me and attempt to help. They didn't find the part (since it hasn't existed for 30 years) but they did give me some good leads.
Amazon’s AWS has good customer service. I get good customer service at the Apple Store, and I also got good customer service when Microsoft had retail stores. I have used Azure’s customer service and it was OK. Amazon’s AWS is know for outstanding customer service. My main point is some big tech companies have terrible service. Some don’t.
github support is nil in my experiences
Par for the course for any product or service owned by Microsoft.
on the other hand, gitlab support has impressed me a few times with the speed of the response and the knowledge level of the support techs.
Maybe if you can initiate a chargeback, you get customer service.
I have never had to initiate a chargeback to get customer service.
I meant, if you have the ability to initiate a chargeback (pay money, and have the ability to take it back out of their control for bad service), then they will probably provide customer service.
Hey now, don't lump Apple into this.
I'd like to see how someone can see a directly employed Google support agent / repair technician in person for support like you can with the Genius Bar.
Additionally, as far as I know, the kind of support you get calling AppleCare is unmatched by anyone else.
> You can almost never get through those thick-skulled reps that email you out of the blue...
He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
What more would you want the rep to do?
> He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
> What more would you want the rep to do?
Not send a threatening, inaccurate email with a 4-day (1 business day Friday->Monday) deadline in the first place?
Also for the record, this was not a sales manager. This was someone in "growth" that sent the email. I guess maybe that just means "sales."
The email wasn't inaccurate: They really were going to shut down his API key and then they did.
> Your API usage is higher than lots of other Yelp Fusion developers
You're right it wasn't just inaccurate it was outright deceptive, trying to frame this as something specific to OP that he is responsible for rather than a change in their business applying to all API users.
> What more would you want the rep to do?
Employ a shred of critical thinking and realize this app probably drives more value to Yelp than the cost to run the API, and flag it for an exception.
Customer service or sales people like that are usually at the lowest point on the totem pole. It’s likely that they have zero leeway, either you’re gonna pay exorbitantly or you’re out. It’s an institutional problem.
In that case it's not customer service but a customer firewall. If the CS employees are not empowered to induce solutions even if they benefit both the customers and the company then they are literally useless.
And yes, many companies have "customer service" like that designed to waste your time until you go away. That doesn't mean it's the only possible way.
Completely agree with you.
This is a strategy, like quiet firing, to make people go away.
Rather than being up front, this is an underhanded way of making folks leave.
But Yelp is not big tech, certainly not by market cap. It could be old tech, in the way that a 2010-era data API may have been a growth hack intended to create an app ecosystem. But 10 years later, there’s no app ecosystem and the company can’t afford to support a (stagnant) lone indie developer. Especially in the current era of LLM scrapers.
And I'd put Apple App Store Review team (specifically, the appeals/rejection team) at the top of this list
Author here—seems they noticed that people were unhappy. Just got this email from Yelp (August 1, 2024):
> Earlier this month, we sent you an email about your Yelp Fusion API usage. That email gave developers until July 23rd to contact us if they want to continue using Yelp’s data for use in their app. We realize you might need more time and are extending your free access for an additional 90 days starting today. Your access should be available now.
> We’re sorry for any inconvenience or frustration this abbreviated transition might have caused. Please respond to this email or contact us at api@yelp.com if you have any questions.
> First of all, I was not “trialing” the Yelp API. I had been using it for a decade and had official permission from Yelp to create Restaurants.
This looks like a Face/Off situation to me.
(Spoiler alert for a 1997 action movie: in Face/Off a cop surgically exchanges faces with an imprisoned villain in order to go undercover in their organization, but the villain then murders everyone who knew about the swap and steals the cop's life.)
Somebody at Yelp in 2014 knew that you had been approved to build this app. That person almost certainly no longer worked at Yelp ten years later, so the institutional knowledge of that agreement had likely been lost.
That's fair, but you'd think their admin interface would show how long an app had been active for and any notes about the app from the beginning.
Regardless, even if it had been active for 10 months not 10 years, 4-days notice is unacceptable.
I absolutely wouldn't expect there to be notes against an API key.
Heck, in many companies I have encountered that they don't even know who is using API keys they've issued to third parties, especially over generations of systems. I am impressed that after getting email approval for an app a decade ago, Yelp still had contact details enough to let the O/P know!
None of that negates the issue of 4-days notice and Yelp shooting themselves in the foot by removing access to an app which redirects people to their website. But someone in executive management made a decision on the basis that they wanted revenue from the API overhead, wanted a slice of other people's pie and that's that.
To the O/Ps point about subscription, there was a UK food app which gave you restaurant food safety scores taken from the food safety agency "Scores on The Doors" it's gone now but I paid a few £ a year for it and never had an issue. If the Yelp pricing is sustainable at that level it's not a bad idea to pay for API access, Yelp has to pay for their servers and a portion of the API calls won't be converted to traffic. Who knows if Yelp still has a sustainable business? Maybe that API thing doesn't pay for itself? I don't know.
Maybe not for an API key, but for an account, you would think they would have some method of keeping notes. I've never worked for a company that had a concept of accounts, and teams of people that interfaced with accounts, without having a method of keeping notes about an account. That's just a basic necessity, especially when the accounts exchange money with the company, there's usually accounts that pay more than others that require special handling, which is usually documented in an account notes screen of some kind.
On the Yelp developer site you have an app profile where there is the name and description of the app. There is also a separate field for the URL of the app website, which I did have filled in. So, they have that at a minimum. If they didn't retain the records of the screenshots/communications that led to the original approval and raise of the API daily limit then that is poor recording keeping but understandable in a corporate environment after 10 years.
The API key usually has a contact email. After a year that contact email probably doesn’t work anymore, certainly for free tier.
Which underlines what I concluded long ago: the best and most durable form of identity on the internet is rooted in the ability to pay money. Any identity that is free to create is doomed.
Oh I completely agree - 4 days notice is _never_ OK. I get uncomfortable with 30 days notice because I've had the occasional vacation that long!
Ideally they'd have a notes field against developer apps and a robust process for recording this kind of thing - but I've worked for companies, so it doesn't surprise me at all that there's no good mechanism in place for that.
> I get uncomfortable with 30 days
In real life, sometimes months isn't enough. On August 5, UPS will complete their API transition to OAuth based authorization after many months of process. Many of my customers haven't responded to my attempts to warn them about the change. I've resigned myself at this point to August 5 being a crazy day.
> I get uncomfortable with 30 days notice because I've had the occasional vacation that long!
Four weeks is the statutory minimum annual leave entitlement here in New Zealand. Most people I know get more.
Yeah, you'd think so. But as many of us have learned, many companies can't and won't be considerate. Sorry that you got hit with it, but at least it's a quick and clean break.
Don't sweat the refunds too much (unless someone is being really rude about it). Apple certainly won't.
Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
> Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
Right, the biggest thing is a failure of communication. There should have been emails months ago, not 4-days (1 business day Friday->Monday) before.
why? for what?
There should have been an email months ago saying they were converting their API to be paid. Not a 4-day threat to shutoff your API key or pay-up. As I mentioned in my post, I was not aware that they were converting to a paid program and had received no prior emails.
Yes, I got that, but why should they bother? Your understanding is that you were business partners, while in fact, you were just lucky enough to get access to their service for some years.
If you had an actual contract, it would be a different story. Your product would still be not more than some UI to some walled garden, and nothing particularly valuable, but at least you would be officially permitted to do so.
So you basically sold something to Apple users that you don't actually own, and that is completely out of your control, and that was always in danger to just stop working suddenly.
And, on the other hand, I'm sure you've implemented a (for a hobby) decent app, and maybe you actually brought a handful of new visitors to them (which wouldn't have just used their website if your app wouldn't exist). But do you really think that at Yelp they are sitting there and just think how they can return the favor?
They basically think "well, that funny dude has now played enough". You never were a partner for them. Just what people call " useful idi*t" (sorry, but that's just the term ^^) at best. They haven't talked about you at their christmas party!
If you're up front and reasonable, you're more likely to convert developers to pay for the new API. If you do it last minute, not only can't developers make the switch in time, you've lost their trust, making it harder to convince developers to bother investing their time and resources.
It goes both ways.
Btw, he had a contract.
Their business is completely independent from a funny guy who writes some iPhone app. They haven't lost anything in that story. If they need a new iPhone app, I'm sure they will find developers for that.
> Btw, he had a contract.
Then fine, go to the court with it, end of story.
There was no contract and nothing. Even instant revocation would be okay. Why not?
Usually it would've been a 2 day notice. The great magnanimous Yelp noticed it was a weekend and decided to give the author another 2 days.
/s
John Travolta and Nic Cage at their finest.
My first thoughts while reading the story was that their champion at Yelp quit and whoever's left didn't know/care enough about them. 10 years ago, Yelp would still be scrappy start up and there was a motivated employees willing to go outside the painted lines. Now, it's just a faceless person following orders.
> 10 years ago, Yelp would still be scrappy start up
Yelp was founded 20 years ago :-)
or in the departed. Martin Sheen is the only one who knew dicaprio was undercover. With him dead the protection goes with it. Only a real contract would CYA. When a company goes from friendly to unfriendly they're happy they were 'scrappy' not to write everything down.
I’m actually surprised the API access lasted for so long. The person that provisioned it is probably long gone from Yelp. Similarly, the company’s values and priorities have changed as well.
Also, I’m guessing this new API pricing policy is they way of combating data scraping to train AI.
If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions, or the service in question was built poorly from the start.
There is little room to blame behavior like this on bad actors and not bad decision making. 10s of thousands of companies have managed to figure out combatting API abuse without shooting themselves in the foot.
> If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions
You are entirely right, yet I see this train of thought happen so very often in different companies. Some of these people are a bit saner than others and actually do listen to others explaining why it is an idiotic train of thought. Others really are just see numbers in isolation and somehow refuse to look at them in context.
How much money should yelp spend to help Restaurants continue to work?
Depending on what kind of approvals they gave him 10 years ago, it MIGHT be possible that doing this violates a contract. It sounds like they had some kind of understanding when they gave him access that he had some sort of informal approval. Even if nothing was signed, that still forms a contract. Even if there was a formal contract or terms of service (TOS) agreement, there are certain restrictions around when and how a company can change their TOS. In particular, there's often requirements about how much advance notice has to be given if the terms of service change.
It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
> It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
Yes, the monetary stakes are too small for it to be worth it for me to pursue. I could probably dig up some old emails from 10+ years ago but it just wouldn't be worth it. Exposing this kind of bad behavior (4-days notice!) is enough.
I would be amazed if it’s somehow possible for a company to implicitly commit themselves to giving you API access forever without a formal contract.
What's needed to form a contract is an offer, acceptance, and consideration - if an offer was made and accepted and something of value was exchanged, and there wasn't confusion about the terms (after 10 years, there wouldn't be), that's good enough for a contract to have legal force.
Not forever, no. But usually with any sort of contract, when one side wants to break or alter the contract they have to give the other side adequate notice. In certain circumstances a contract can eliminate such a requirement, and the definition of “adequate” would vary a lot based on specifics, but that is very much a possibility in this case.
> Depending on what kind of approvals they gave him 10 years ago, it MIGHT be possible that doing this violates a contract.
What contract? He never entered into a contract or even exchanged consideration with Yelp for the API as far as I can tell.
Getting a green light via e-mail to use a free service is not a binding contract and does not come with any obligations.
It sounds like he requested API access in order to make a native Mac application for Yelp. The specifics matter a lot here, but “if you develop a native Mac application for Yelp, we’ll give you free API access” sounds a lot like consideration. That could be completely false based on exactly how that went down, of course.
> but “if you develop a native Mac application for Yelp, we’ll give you free API access” sounds a lot like consideration.
He was using a free API that anyone could sign up for.
They did not exchange anything with him. Developing an app to use someone's free API is not an exchange of consideration.
> Developing an app to use someone's free API is not an exchange of consideration.
I would never pursue anything further over this app that had about $2,000 of revenue over 10 years. That said, for the record there was consideration.
I had to go through an official approval process that required providing evidence of the app's functionality and a few emails back and forth. I think I actually sent them a prototype of it. And based on that process they decided how many daily API calls to give me. A normal free user did not receive 25,000 API calls per day. I believe if you didn't go through approval back then you got something like 1,000 per month. So there was a consideration process on their part and a determination to green light my use case.
Ah, that's not what consideration means in contract law. Consideration means something is exchanged for something else. (See https://en.wikipedia.org/wiki/Consideration_in_English_law)
You could argue that the indefinite API access was in exchange for writing the app (service for service), but if this were me, I probably wouldn't bother. Maybe I'd write an adversarial-interoperability backend to replace the API, or open-source the app to allow other interested parties to do so. Or maybe I'd just say "It was a nice run", and let it die.
If you care enough to send a polite email, you could say that Yelp has a prior agreement, you'd appreciate them not reneging, and if they do, you'd appreciate compensation for your labour (minus, of course, the money you made from the software). Probably won't go anywhere, but…
something of value*
granting API access in excess of the free tier would most likely constitute something of value, but yeah - probably wouldn't bother, it would be expensive to pursue and not worth it.
Officially, yes, something of value. In practice, just something.
> To my mind the acquiring and delivering of the [used chocolate bar] wrappers was certainly part of the consideration in these cases, and I see no good reason for drawing a distinction between these and other cases. — Lord Reid
> It is said that when received the wrappers are of no value to Nestlé’s. This I would have thought irrelevant. A contracting party can stipulate for what consideration he chooses. A peppercorn does not cease to be good consideration if it is established that the promisee does not like pepper and will throw away the corn. — Lord Somervell of Harrow
https://www.bailii.org/uk/cases/UKHL/1959/1.html
Oh thank you for that clarification. That's very interesting, but of course it is not worth it to me to pursue for the reasons mentioned in the post and the comments here on HN.
Yeah I think the only option in which this would make sense for you to peruse legal action is in small claims court for however much you had to pay in refunds. It’s fairly easy to do that pro se (representing yourself). And often companies are motivated to negotiate if they are sued in small claims, because just sending a lawyer to represent them would cost as much, if not more than just paying the damages. Obviously this depends a lot of jurisdiction and the specifics.
But at that point it’s not a question of cost, it’s a question of how much of your time and headache you want to spend. So yeah, probably not worth it
I would be upset too, but this sort of sounds like someone did him a favor 10 years ago, management changed, and now management wants him to pay. This sucks, but near infinite free api calls sounds unsustainable
Well the thing is they gave me 25000 per day, but the actual use was less than 100. The 25000 number is almost a red herring to the rest of the story.
It’s understandable they went to paid. What’s not understandable is the 4 days notice and tone of the email.
We received the exact same letter from Yelp. Our usage is significantly higher and when we talked to them, the prices they quoted were ridiculously high (thousands of dollars a month).
If the price is too high for you, here is my idea for you: don't use their product. ;)
Exactly, we're not. We're closing our products. They are perfectly welcome to charge too high a price for us. They just should've communicated it better.
Sorry to hear that. What was your product?
We have a analytics SaaS which displays, among many other pieces of data, ratings from different providers, including Yelp.
I assumed that Yelp had been doing this all along and we ran above some predetermined limit, but the email we received was identical to yours. So I'm thinking that Yelp is trying its best to monetize all API users
> a analytics SaaS which displays, among many other pieces of data,
Does yelp (or others) allow the restaurant to modify the menu/offer coupons in a dynamic way? (i.e. change multiple times per hour, update with in minutes)
thanks
A 25,000 calls per day limit for an API sounds absurdly low. Any idea if the API calls actually would have used much in the way of backend resources, or do you reckon it was more just mostly a database lookup?
API rate-limits have always been a cash grab and a way to discourage efficient automated use in favor of getting a human to "engage" instead.
I would be surprised if it were much more than a database lookup.
Yeah, it's probably that simple. 25k calls may only be a few seconds worth of actual database runtime too, if that.
Honestly, making people pay for an app that only uses a public API you’re not paying for, and no form of fallback is asking for trouble. This is not a responsible way to do business and I hope people reading this thread will understand that.
Since ~15 years, people got the idea that all the WhatsApps, Instagrams or Twitters around them aren't companies, but kind of public infrastructure. They all knew that it's a wrong assumption, but it's sooo convenient, and at some point, people around you force you in that direction, even if you don't like all those walled gardens. And once this wrong philosophy was established, people started escalating it to some smaller shops, I guess. Those are also the guys that complain when some 3rd party Reddit or Youtube hobby clients must disappear. They just have no understanding at all about some basic mechanics of human interaction. The biggest problem: There are entire generations of people nowadays with that misunderstanding.
> near infinite free api calls
but the author pointed out that it drove traffic to Yelp, and that almost certainly seems to be true.
In fact, little integration plays like this app might have been the only thing keeping Yelp sort-of alive after Google got into the game.
Killing off your bizdev partners seems incredibly short-sighted and foolish. (Also, feeding the reviews into an AI... for what, exactly? To train a model on how to write reviews? Or perhaps to detect fake reviews -- actually, that was an issue on Yelp even before AI, so it seems like it wouldn't be the best training content.)
I think Yelp marketing is on an incredible shortsighted and profit-losing path.
I too used the macos App.
Looking forward to Apple Map dumping Yelp, because that combo doesn't work for me and I do not want Yelp cluttering my Apple map.
I think Apple has been developing a yelp clone internally; and have been slowly releasing bits and pieces.
Apple has added rating systems for destinations. Amenities available at location (ie, “accepts Apple Pay”, dog/cat friendly). Maybe in the next decade Yelp integration will be phased out completely
They've already got something in Maps that switches locations from Yelp reviews to the Apple thumbs up/down system once they hit some critical mass of data, and the same for location photos.
It's not even popular outside of the US, I'm not sure how they can grow with that.
If I look at my closest city in the UK at restaurants, nothing has been reviewed recently (2018, 2019) and it's mostly from US visitors coming here.
Was this a move to try and get some profit from AI companies trying to train on their data? With the API providing limited results, I’d think it would be of limited use. I always found Apple’s use of it in Maps frustrating, because seeing POI details basically required I also have Yelp installed.
If this move was AI driven, like with Reddit and others, I’m starting to dislike AI more and more… at least the rent seeking end of it, which seems to be slowing killing the open internet.
My son, it is time for you to embrace the good news of our lord and savior web scraping.
You reminded me of this "meme" about APIs vs scraping: https://i.imgur.com/koPo3M0.png
"his business is profitable" is a little too close to home.
We need to move towards “zero trust” for APIs.
SaaS can provide “open core” or better yet simply sell a hosted version of their fully open source code. If the provider fails to provide, you can fall back to self hosting.
The API equivalent would be open sourcing the data. This is the OpenStreetMap model. If the API provider fails to provide, you can fallback to the underlying data.
That's asking too much. SaaS should give an option for you to export all your own data in a simple, parseable format (like JSON). That's about it. They don't owe anyone their source code, and they don't owe any ethically sourced data (such as employees researching and manually entering).
API access needs better terms. Like guaranteed access for X years at $Y price with Z days notice if there's a change, where Z > 3 months or so.
This was the hope behind the StackExchange data dumps, that the community at large could always take their contributions elsewhere if the service jumped the shark.
Well, before the SE organization tried to kill the data exports off in an attempt to commercialize it towards AI companies, but thats a whole other issue.
If the api was as basic as you say, can you replace it with some screen scraping on yelp’s site?
Not only would that violate their terms of service, but the monetary stakes are also too low for it to be worth it to me even if it didn't.
That would probably as big or bigger project than the entire rest of the app itself, and since it is such a single developer and not a super profitable app it likely makes no sense to do this.
Probably not without violating other terms of service.
It should be ok as long as you do it logged out.
SerpAPI is a convenient wrapper API for scraping various sites. I assume they've vetted all the legality of things. They have a YouTube API: https://serpapi.com/yelp-search-api
> I assume they've vetted all the legality of things.
They claim they did: https://serpapi.com/blog/scraping-public-pages-legality/
Still they have a boilerplate ToS with some glaring mistakes:
> These Terms of Service and any separate agreements whereby we provide you Services shall be governed by and construed in accordance with the laws of 5540 N Lamar Blvd #12, Austin, TX, 78756, United States. (sic)
It's not quite so clear-cut. We even have a historical precedent here:
https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc.
In that case, the court found 3Taps was criminally guilty for scraping publicly available craigslist data while logged out because 3Taps knew their use was not authorized.
This person has just received an email from Yelp telling them their free usage is not authorized, so circumventing that may well be illegal, now that they've been given that sort of communication, even if it might be questionably legal for other serpapi users.
The courts are altering their views on scrapping. This [0] is a good paper that explored the last 20 years of rulings (although it hasn't been updated with the most recent cases).
[0] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3221625
I don't care about Yelp's terms of service, do you?
You will when a process server hands you a notice.
A notice of what? It's not illegal to violate someone's random wishlist. It's not like you've signed a contract with them.
It is in Illinois. Their Computer Tampering law specifically makes violation of a web sites ToS punishable by up to 5 years in prison. Probably other states have similar.
https://njal.la/ looks interesting.
Typically for copyright infringement. They'll sue you for the maximum legal damages possible per copy, multiplied by the number of times your bot loaded a page, probably in the trillions of dollars.
So have the bot act on behalf of the user, using workers run on the user's machine. That's fair use.
Are you in the habit of releasing software that causes users to violate the ToS of services?
I'm happy to write code to let users circumvent the restrictions put in place by assholes, as long as it doesn't end up getting them in legal trouble I'm fine with it. If companies like google don't want people to try and screw them over, I suggest they offer an olive branch and stop being unethical jerks.
Yelp can afford lawyers who will try to convince a judge otherwise. Can you afford lawyers to argue your case?
It's so cute watching people play pretend lawyer on the internet.
Corporate policy is not the law. If you didn't sign a contract with them they should have no legal power. The DMCA was a mistake.
Coprporations (or other organiztions or individuals) can make things available to the public without completely ceding control over how they are used.
Imagine a landowner who allows public access to hikers using defined trails but no overnight camping. That's legal and just has to be posted. If you don't like it, don't use it.
Have you seen the results of the hiQ Labs v LinkedIn case? https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
Not sure why this is getting downvoted, because it's a reasonable question - the answer being that such scraping requires a lot more time and technical expertise to engineer than a simple API call. Also devolves into a cat and mouse game between your application's backend and whatever proxy they put in front of yelp like cloudflare, which you'll probably lose or will be prohibitively expensive.
Sorry to hear how Yelp treated the OP...
Given how many stories of greed and throwing people under the bus for money I hear nowadays, we might already be living in a dystopia.
It was never different. Just, with the advent of iPhone, FB, WhatsApp, Youtube, etc, a lot of people basically decided to ignore this fact of life. So good and for so long that they completely forgot it. They forgot it good enough that even their childs got no education in that regard. And every some days, someone gets a lesson. ;)
Why should Yelp not throw you under the bus for money?
The problem was that (impersonal) you stopped understanding basic mechanisms of human interaction. For no real reason.
> It seemed they were in fact encouraging me to finish the app and release it.
That was quite the assumption. They gave you access to something for free, not encouragement. I do feel bad for OP, but they weren't paying for the API, and should not have had any assumption that it would last forever because there was no contract or terms or anything. This is the risk we take by building our house on someone else's foundation.
The gist I got from my communications with them 10 years ago was that they were encouraging me to finish it.
I agree with you there was no expectation of it being free forever. I never said there was. What was unreasonable was the 4-days notice that it was coming to an end. That's just not enough time.
I am in the same boat, sadly.
I am particularly disappointed by the generic "Your API usage is higher than lots of other Yelp Fusion developers" statement.
My giveback service has a tiny user-base and find it hard to believe my API usage level can be higher than average.
https://try-something-new.web.app was built a couple years ago.
That statement is an outright lie too. Not only are they operating at less than 1% of what the free plan offered, but the email doesn't tell them the honest truth: That the free plan is reaching sunset, and they want all API users to switch to a paid plan.
This isn't unreasonable, but they absolutely should provide more than 1 business day notice.
Absolutely. You wrote it better than I did myself in the original post.
Perhaps theere are a lot of inactive API users with zero usage so the statement is technically true ;)
Textbook half truths. :-|
Of course it's generic and perhaps not a technically correct statement. And now? Why should they spend time on writing you nicer emails? What was your value to them? Zero? Or is that too optimistic actually?
If I knew about this app before, I would've definitely bought it! The Yelp site and app are incredibly slow and tedious to use
Thank you. Yup that was the main thing people liked about it. It was really fast and had no ads.
> if you utilize a third-party API for the core of your app, you are at their whim
More than that, if you aren't paying for use of that third-party API, the people who run it will not care about you, and will think nothing of shutting you down.
I think Yelp handled this poorly, and Restaurants was probably a net positive for their business -- a positive that they were getting for near-zero cost. It's a shame that companies are so short-sighted like this.
But ultimately if you build on top of someone else's platform, with no backups and no alternatives, it's not really truly your app.
Enshittification is at an all time high, and API access is being tightened and monetized in response to many industry factors including "freeloading" AI trainers.
Cory Doctorow is right, if you want to disrupt, or make any improvement to an existing large platform, adversarial interoperability(that is, reverse engineering) is the only way forward and has to be explicitly legalized in cases where it's a tool for progress.
My previous statement is arrogant, as it assumes developers are entitled to take any data they want and profit from it. It also puts services in a situation where harmful crawling like what is performed by some new AI actors with no experience is an expected thing. This is of course wrong, but I want to argue that had Yelp and other actors not wanted such a future, they shouldn't have tightened free access to their proper APIs where they have the ability to set ground rules and have the ability to talk to their users.
Big companies are amazingly bad at keeping track of things internally - a promise in an e-mail is easily forgotten 10 years later. But why should the user be punished for Yelps lack of control?
I'm sorry to read that. Unfortunately, it isn't an uncommon story.
> if you utilize a third-party API for the core of your app, you are at their whim.
That's the money quote, there. I avoid using third-party APIs like the plague. I have written backend aggregators and facias, to avoid having to use the API.
I Just. Will. Not. embed an opaque codeball into my app. I'm a cranky old bastard, I know, but I sleep well at night.
"Opaque codeball" is a very large category. Are you saying you won't use any 3rd-party libraries at all?
Open-source libraries aren’t opaque. Third-party web APIs are. The latter should only be used if they are non-critical, easily replaceable, or contractually bound to sufficient assurances.
Almost none, but “at all” is also a very large category.
Amen. Doing the wrong thing is easy. Doing the right thing is hard. These are thermodynamically-mandated rules that cannot ever be circumvented by cleverness or money or hard work.
Therefore, doing the easy thing is wrong.
Usually. If you are doing the easy thing, so can your competitors. Your competition usually can't do the hard things.
Not necessarily. p=>q does not mean q=>p, after all.
It does when !p=>!q.
!p=>!q is exactly q=>p, and p=>q has nothing to do with that.
I feel like you should read the original comment again.
yelp had so much potential but they pissed it all away. Google brain raped them [1,2]. Then they never recovered from it.
Then there is ongoing issues between merchants and yelp [3]
Yelp used to be a great place to find some decent place to eat in a new city. But the platform has gotten stale. Reviews are less reliable. Star rating often not useful.
[1] https://www.theguardian.com/technology/2015/mar/20/google-il...
[2] https://techcrunch.com/2015/02/06/google-takes-on-yelp-elite...
[3] https://en.m.wikipedia.org/wiki/Yelp
Yelp suffers from the classic issue of not having a way to monetize that isn't a conflict of interest with their core business model.
Google search has the same issue.
Probably what happened was 1 support rep helped you out but did it through some undocumented backdoor to unblock you and not a formal contract. Later a completely unrelated set of employees are tasked with figuring out who the biggest API users are and to either cut them loose or get them to start paying.
Yes, this was surely somewhat tied to employee turnover and poor record keeping. But I was almost certainly not one of the "biggest API users."
The thing is, regardless of the turnover or situation, 4-days (actually 1-business day Friday->Monday) is not a reasonable timeframe to threaten to shutoff someone's API key who hasn't violated any terms of service. They have the right to do it, but it doesn't make you want to work with them in the future.
Indeed, 4 days does seem too aggressive. You are probably a victim of automation:
There's yellow squiggly underneath userShouldPay(apiKey) that reads:
userShouldPay(apiKey) always evaluates to True.
This feels important:
> The other thing this taught me is the danger of an up-front paid model for apps that depend on ongoing access to third-party services. If users were continually paying for the app, paying for Yelp’s APIs would not be as much of an issue. And I wouldn’t feel as guilty about the app being discontinued since if the fees were charged on a monthly basis, they would just end at the same time the app ceased to exist instead of facing an expectation upon purchase of “forever access.” On the other hand, how would you charge a monthly fee for an app that people are only willing to spend less than $5 for upfront?
There are apps that I like and would like to purchase, but paying e.g. 24 euros per year feels like too much. So I stick with the free version...
A hard lesson I learned after several times getting burned in my time long ago doing indie stuff is “never depend on a single company without a backup plan”.
Sometimes this can be pretty difficult to even see. If you were a Mac shareware dev depending on VersionTracker and MacUpdate to drive downloads, with no actual marketing budget, you might not have noticed that you were implicitly relying on Apple not creating an App Store which would turn the entire ecosystem upside down.
This is one of the reasons that it’s so tough to make a business work at a small scale. In some ways, you have more flexibility than a big company, but the lack of capital means that certain events that a larger company would shrug off can totally upend everything you’re doing.
> is that if you utilize a third-party API for the core of your app, you are at their whim.
Well, yes. I think people figured that out more than 10 years ago
And business models can and do change
And by checking the pricing page, Yelp's commercial API is $15 per 1000 API calls per month. Which sounds ok?
Right, they never answered my questions about the discrepancy between that and the pricing deck they specifically sent me for my app. See slide 3 with a $229 per month base price: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
The economics of Restaurants actually mean it wouldn't be profitable either way though. That's how low the sales are.
They're welcome to start charging for their API whatever they want, but they should give more than 4-days (Friday->Monday, 1 business day) notice.
I have seen many notices like this in the past few months. My guess is anbody of signficant size whose reason for existing includes curating user-generated content is trying to ensure that gen-AI ingestors don’t swallow up their data through unpaid API access or scraping. (Maybe a podcast or article popular with CxO and IT directors on the topic??)
Which they seemingly already have done anyway. Another unintended side-effect of the borderline illegal and generally immoral “AI” companies efforts to get as many data sets as possible.
Is Yelp even still relevant these days?
At least where I am in Europe I find it incredibly outdated, showing restaurants that have closed a long long time ago and none of the new ones.
It seems to be mostly a US phenomenon. As I pointed out in the post almost all of Restaurants sales were to the US. The directory is very up-to-date with active reviews everywhere that I've travelled in this country.
I got the same email regarding my hobby website "The Wheel of Lunch" (https://wheelof.com/lunch). The site makes no money and sends traffic to Yelp. The cheapest plan offered was $230 a month! I can't afford that.
I have stopped calling the Yelp API for local listings and put up a notice on the site. It was fun while it lasted!
Sorry to hear that. Remember your site from 15 years ago when I was an intern trying to figure out where to eat.
We've been bitten by going the "free 3rd-party <x>" route so many times, so I sympathise with the problem here.
Off-topic, but kudos to OP for still engaging with the threads. This certainly wasn't a dump and run post. They've probably spent more time answering questions here than they did developing the app!
> But due to the way that the Mac App Store works we don’t have our customers’ email nor any way to directly refund them.
> if you utilize a third-party API for the core of your app, you are at their whim.
I think you might also want to revisit your relationship with Apple Incorporated.
Yes, for sure. Some people explain it away as "well if you buy Wrangler jeans from Walmart and there's something wrong with them you ask Walmart for a refund." So if you buy X's app from Apple you ask Apple for a refund. That kind of makes sense. But if Wrangler does a recall, they have a way of getting that across to their customers.
I'd say, no you had no shared customers. Yelp had the customers, and you wrote some app that uses Yelp. For no real reason you assumed that you could be part of them (as with Facebook earlier). Sure you had some kind of informal permission at first. And then they revoked it. That's their right to do, right? Maybe you should not write Facebook or Yelp apps, if you are neither Facebook nor Yelp nor asked by them to write clients for their walled gardens. Or if you really want, set up an actual license agreement with them. Those networks are walled gardens and they don't want to cooperate with you that way. They are just not those kinds of shops which you can cuddle with. If you want to create something that has an own value, create something more than just another Faceyelptwitterddit frontend.
The good thing: You took your lesson. Maybe this time it was sufficient to actually learn sth.
> I'd say, no you had no shared customers.
Person pays for my app. That was a customer.
Person who paid for my app also later uses the Yelp website via a link for a specific review from my app. Also a customer.
Therefore shared customers.
> Sure you had some kind of informal permission at first.
I don't know why you would define it as informal. They had an official review process for deciding what apps to grant higher daily API limits. My app went through that review process 10 years ago including considering its functionality, screenshots, and I believe I even sent them a prototype back then. We had a few emails back and forth to confirm my intentions to only develop for the Mac and what I was building.
I'm sure that got lost in the corporate shuffle over 10 years but I clearly had their permission to build the app and in fact given the relatively high API limit they gave (25000 per day versus the free at that time 10 years ago I think being 1000 per month), arguably blessing.
If it wasn't informal, then fine! Then better take your contracts and bring them to the court, instead of fishing for sympathy in some web forums.
Do you work for Yelp?
You created this account solely to respond on this post and all of your replies are defending an ostensibly horrible business practice, which is to give your API users less than one business day of notice to switch to a different tier.
No matter how you cut it, that is simply terrible developer relations/customer service and a terrible way to run a business.
have you thought about writing your own API instead of using the de facto API? it may involve screen scraping but it also will be a lot more reliable and there's no limits, especially when using proxies....
I've written thousands of tools that scrape websites and never used the apis for this reason, you can never trust the API, either because of the reliability, cost, limits imposed, etc... Nobody wants you pulling data from their site anyway, so you're back to scraping anyway, its better to start out there then to have to end up there years later for some other reason...
That’s trash. I’ll delete yelp over that, bad customer or dev relations sucks.
What is the precise meaning of 'dev relations' here? What is the actual relation? They haven't ordered him to dev sth.
It seems that you still can spin the Wheel of Lunch, but there are only generic proposals like Pizza, Ramen, Thai Food, and so on.
That was a long blog post to tell us that they are no longer entitled to valuable free API access. It was a mistake to think it would last. This story has been repeated so many times here. As soon as the author started making money (they conveniently excluded their sales revs), they should have negotiated a written contract for access for a fee. It could be cut of app revs or pure API charges. Also, I am sure some smart managers looked at all their free API giveaways and decided there we no longer useful to their business model.
The author disclosed his sales revenues here: https://news.ycombinator.com/item?id=41124971 . He sold 467 copies of the software after 10 years, for a total revenue of around $2000. He never exceeded 200 API calls per day. I don't think it would have been worth Yelp's time to negotiate for a percentage of that.
Yes, that's why they haven't. ;)
Building atop centralized platforms, you run this risk. In a decentralized environment, you’d be able to keep a fork running just as it was.
Control your own future, or you’ll have no future.
Reminds me of something Reddit did just recently...
I think when the AI scraping funding models go away, all these APIs will magically open up again.
Drama aside, the guy signed up for the free Yelp API 10 years ago, which has since been discontinued. He was offered the option to switch to a paid API, which he chose not to consider.
Yes, Yelp was a bit clumsy in handling this, but discontinuing the free API after 10 years is totally within their rights. The developer didn't even bother getting their pricing proposal, which might have been totally reasonable (or not), considering his app is paid.
He's didn't argue it wasn't within their rights. He called them "quite rude", which seems hard to deny.
No matter the rate Yelp set, the apps economics no longer make sense. The existing customers, already paid, and he has no way to transition them to a subscription.
He mentions 100 API calls per day. If Yelp offered him a rate of a fraction of a cent per call, it might be a negligible expense that could be offset by past or future sales.
If the app has outlived its lifecycle, the end of the free API might be a signal to retire it. Blaming Yelp and making a drama out of it seems a bit much. Suppose they had given him 30 days instead of 4; would his decision really be any different?
> Suppose they had given him 30 days instead of 4; would his decision really be any different?
Possibly not, but the point of OP's complaint is that Yelp was rude, handled it poorly, and gave him an unreasonably short deadline. No one is arguing that Yelp doesn't have the right to discontinue free API access, or that OP's business model was a good and sustainable one.
But agreed: if OP could have gotten a rate that would have cost, say, 10 cents per day (or even more, like 50 cents or a dollar a day), maybe that would have been ok. And maybe he could have changed the pricing on the app for future purchasers to a subscription model, some small token amount like $1/mo or even $5/year.
But also consider it's pretty crappy to give someone such a short amount of time to make the decision as to whether or not that new business model would work, and if it's worth it to put more development effort into the app to enable that new pricing scheme.
> Blaming Yelp and making a drama out of it seems a bit much.
Don't read it if you don't like it. Some of us actually give a fuck how badly companies are treating people, even if you don't.
Those of us who do, never expected much from a company like yelp to start with.
Well, don't read the comment that you reply on. ;)
The only interesting part of the story is to what extent others actually learn sth from it (that lesson should be trivial, but nowadays it obviously is all but common sense). And I guess: Not very much. They will 'give a fuck' here how bad the world and all is, and then continue in their naive way tomorrow.
There's a tangible difference between shitting on a person who chooses to share an experience, and shitting on a person who is shitting on a person who shared an experience. If you choose to read irony or hypocrisy in it, that's your own issues with context. ;)
> He mentions 100 API calls per day. If Yelp offered him a rate of a fraction of a cent per call, it might be a negligible expense that could be offset by past or future sales.
Except, elsewhere in this thread:
> the prices they quoted were ridiculously high (thousands of dollars a month).
I may be reading it incorrectly, but it looks like $9.99/1000 API calls.
https://docs.developer.yelp.com/page/start-your-free-trial
No problem. He can just dust off his business contract with Yelp and have the courts set them straight. No contract? Then why would one expect the world to cater to their whim. Yelp promised nothing and he got exactly what they promised.
He hasn't promised not to scrape their site using even more of their resources...
Used to be that, coupled with competition with competing services, was the main reason sites offered APIs.
> Yelp promised nothing
According to the article, Yelp promised him 25,000 API calls per day:
> In fact, without me specifically asking for it, they provided a 25,000 per day API call limit
They altered the deal as was most certainly declared their prerogative in the original agreement.
[flagged]
I guess you didn't read my blog post because I addressed everything you wrote. The issue is not that it went paid, it's the 4-days notice. They are perfectly in their right to start charging for their API, they just can't give us 4-days notice.
Well they can do whatever timeframe they want of course. And I can write about how rude it is. 4-days (really 1-business day in the original email Friday->Monday) is not a reasonable timeframe within which to threaten to cutoff an app with real users.
Just killing your app is an emotional, not a rational, decision. You have a human responding to your emails, so here's a business way to handle this situation:
Dear Yelp,
Your decision to discontinue the free API was unexpected, and it’s difficult for me to switch to the new one within the given very short 4-day timeframe. Not only is this not enough time to estimate how your new API pricing will affect my business model, but it also requires some engineering work to switch my app to the new API.
Given my 10-year history of working with Yelp, I would appreciate it if you could send me your new pricing proposal ASAP and also give me some time to consider it. If accepted, I would need additional time to implement it.
Thank you.
It's just not worth it. I have a full-time job and many other projects and apps I am supporting. I don't want to work with a company that provides long-term API users 4-days notice about a major change and threatens to cut you off in that time period. As you saw in the blog post I did write back to them and did not get any kind of response indicating flexibility (did you read my whole post?).
Also, the money is very very low stakes. This app sells dozens of copies a year. Not hundreds or thousands. It's just not worth it financially. It sold 467 copies over 10 years. People who used it loved it, but it's not a money maker.
I had the exact same experience you did, and feel the exact same way.
> I don't want to work with a company that provides long-term API users 4-days notice about a major change and threatens to cut you off in that time period.
If you don't want to deal with the slightest inconvenience don't run a business and don't take money from customers. You owe it to your users to care at least a bit.
> If you don't want to deal with the slightest inconvenience don't run a business and don't take money from customers. You owe it to your users to care at least a bit.
I did care "at least a bit" which is why I kept updating the app for 10-years despite it not making almost any money. How many indie apps survive that long? But based on the pricing they quoted me it would be a money-losing venture to continue (see slide 3 base monthly fee $229 from the deck they sent me): https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
And we have to decide what we work on when we are just one person. If it's money-losing and they don't treat you well it might not make sense to keep doing it.
That said, as I expressed in the blog post I do feel really bad for any of the users that bought the app and I want all of them to get a refund from Apple as explained in the post. They can use these directions: https://support.apple.com/en-us/118223
> You owe it to your users to care at least a bit.
Christ on a bike, he is giving people refund and you act like he does not care at all.
The yelp guy even preemptively offered a solution, telling him to sign up for a free trial if he needed some extra time.
Give them your credit card? No he did the right thing.
> they just can't give us 4-days notice.
Unfortunately, if you're not a paying customer with a contract they can discontinue free service whenever they want.
Frustrating? Absolutely.
Right I don't mean legally. I mean in terms of making people want to continue to work with them.
That's when you take matters into your own hands and extract the API key out of one of their official apps. At least that's what I would've done.
Though I'm not sure how legal that would've been if done in a paid app. It feels like a serious difference between just providing a better UX for someone else's service through adversarial interoperability for free, and profiting off of it.
Digital sharecropping. I don't know what the answer is, but it's sad to see it unfolding!
Curious how much money the app developer made, and how much they paid for Yelp API access.
I sold 467 copies over 10 years for less than $5 per copy. It's about $2000 in revenue. Surely not a good use of my time. But I liked the app and so did its few users. We should build things we want to use ourselves right? I think it was a much nicer and faster frontend to the Yelp restaurant directory than the Yelp website.
Your other question about the cost is answered in the blog post and in several other comments I have left in this thread. In short, when I started the app 10 years ago there was no paid API (perhaps there was some kind of "enterprise" version but I'm not sure). The point of my post is how poorly they handled the transition from free->paid.
467 users seems like it would not trigger abnormal API usage.
Sounds like Yelp sent a form letter.
Yes, definitely based on other people in this thread getting the same letter. It was an inaccurate and threatening form letter bizarrely sent from a personal email address of a Yelp employee with another personal email address CCed.
But how was the API key stored? If it's stored inside the binary, without an intermediate server managed by OP, someone could've disassembled the binary and starting using the API key independently for their own gain, triggering abnormal usage. Say, one of our Windows products uses a web API for TTS, and we've built an intermediate web server which stores the key and manages rate limits per user so that there was no abuse (the app uses our own auth).
Yelp has an API dashboard where you can see how many API calls have been made. I can tell you it was never exceeding 200 calls per day and often below 100.
Yahoo used to have a decent restaurants API. I assume that's dead now.
If building an app on a platform does not increase the networks effects of that platform, this will always be the outcome.
Good platforms get more valuable with more apps: iOS, Windows, etc
Bad platforms don't get better with more apps: reddit, netflix, twitter. So they always end up killing the API.
The difference, I think, is that the former are platforms to run applications while the latter are communication platforms. There used to be a time when the phone company could press charges if you connected your "app" (a tone-generating machine) to their "API" (the phone line) to "do things the official app didn't support" (place long distance calls for free). It will always be beneficial to someone who owns a communication channel to assume as much control of the pipe as possible. If Twitter could jack directly into your brain stem and refuse to work until you install its client on your wetware, it would.
Real scumbag move from Yelp, sorry to read this man
Not surprising at all. Yelp, Paypal and their ilk deserve all of their earned derision for, amazingly, being simultaneously user, customer, and developer hostile.
Related to the earlier "old & grumpy" 3rd party API post, I've seen far too many, otherwise, outstanding businesses held hostage and summarily executed by either sudden un-explained usage-tier/throttling policies without economic recompense or the outright deactivation of API keys w/o notice or explanation.
Yelp is a scumbag company, I'm surprised it's still in business as it's reviled by business owners and TripAdvisor crushes it in its segment.
When will people realize that using any big tech company API is a recipe for disaster? These companies, and their revolving door of employees, could care less about you, your app, or your users.
Dollars to donut stores, what's likely going on with the switch to expensive API is this:
https://business.yelp.com/products/yelp-ads/
Same as the Reddit stunt versus the Apollo app dev, except that his app had a big enough audience you'd think they'd have figured something out.
Nope. The paid app meant money from users to the app dev instead of from advertisers to the site.
What's strange is Reddit didn't seem to do the math on how little they should have charged the app dev for API access if all they wanted was to offset revenue from that user base. Perhaps the fear was much as with TV streaming: they know advertisers want audiences willing to pay, not only the audiences seeking free.
Could pivot to customer provided api keys :\
This really sucks for OP, but my first thought on seeing yelp was:
"People still use yelp? I thought it was widely known that they suppress bad reviews for money, and suppress good reviews if you don't pay."
Yelp's path to monetization has always been kind of scummy IMO.
> "People still use yelp? I thought it was widely known that they suppress bad reviews for money, and suppress good reviews if you don't pay."
A judge said that it wasn't extortion for yelp to hide good reviews until a company pays yelp, and that it wasn't extortion for yelp to remove bad reviews for companies that do pay them money though so I guess we're supposed to act like that's acceptable behavior and that their reviews can be considered trustworthy
Have you considered building your own restaurant database? You could add features to the app to allow users to submit and update entries.
This is just pure boneheadedness across the Yelp management chain, all the way down to the IC level.
- IC should have recognized the site was driving traffic to Yelp and flagged to management.
- Management should have realized that some API usage is beneficial to Yelp overall, and crafted a plan around this.
Just pure insanity. If I were a VP I'd fire everyone involved for a lack of basic critical thinking skills.
Question-- with the yelp api now being paywalled-- and obviously you'd have to write code to handle scraping-- but can you legally scrape yelp to do this?
Does the linkedin scraping lawsuit permit scraping yelp?
The big takeaway here is to never rely on a third party, they will inevitibly fuck you over.
Every business will eventually turn anything that they can into a profitable feature for them.
I wrote a review for a grocery store turned food service shovel basically said "this will be the first store that charges for air." Guess what? I use their bathroom every other day,for free. I think it's been months since I bought anything, and I help myself to utensils.
Every customer will eventually turn anything they can use for free, into a profitable feature for themselves.
There was a book called the 1 minute manager: it was followed by a book called the 59 second employee.
Never rely on an unscupulus third party, or if a third party becomes unscrupulous... Hunt them down.
I disliked Yelp, and held it in deep distain, then while working for a service company, Yelp called. "For a fee we can remove all negative reviews." This is the definition of ensh*tification. So... I told him I would get back to him. I called a few numbers at Yelp and was basically able to socially engineer a vast list of their revinue growth supervisors.
I made a list of every business in that city and the next, and related my experence, and how worthless a Yelp review was. Almost all the business responded to poor reviews with a 'i bet you are a Yelp employee drumming up money for fake reviews.' after a few weeks, I gathered up a lot of these, and sent a three page letter of them in a package with 50 of the Yelp peoples's names in it. I let that stew for a few days, and then called the idiot back. He said "we don't care." They don't. They do not care in the least. Google, OAth, and Facebook do not either. Hurt them in their advertising review and then they will listen.
i use to be that guy telling people not to put their eggs in other peoples basket. Some laughed at me for a decade, almost convincingly and then...
Of course my stuff also breaks because you cant really do anything anymore without trusting anything even against better judgement.
I think we will eventually get expensive quality terms of service for those who think it might be fun if things work forever, like html documents and megalithic structures.
For a site that caters to a startup and entrepreneurial crowd, it's hilarious the number of comments here that amount to "tough cookies, bud" and "Yelp can do whatever they want, and because they can, you should just shut up."
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
> which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
Not everyone. Business that build on top of other company's APIs will arrange contracts with their API providers. Those contracts generally include warning periods for changes or discontinuation and penalties for early termination.
The key here is that it was a free API with no contract or guarantees. Four days is short notice and frustrating, but it wouldn't have really changed the trajectory of his business if they had given him 180 days. If he didn't intend to pay for the API, he couldn't really sell an app that was going to stop working in a few months.
So I know we're supposed to be angry about the 4 days thing. It's not good, obviously. However, I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
> So I know we're supposed to be angry about the 4 days thing. It's not good, obviously. However, I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
As I said in the post and comments here if it made financial sense and they gave me a more reasonable deadline with a less threatening email I would be willing to pay for the API. In this case it didn't make financial sense, so you're right at the current API prices it wouldn't make sense even with 6 months-notice.
That said, 6-months (your suggested time period) is a much better grace period for our shared users (users of Restaurants who use it as a frontend and continue to read more reviews at Yelp.com) and much more likely to make me convert to a paid API customer if it had made financial sense.
> and much more likely to make me convert to a paid API customer if it had made financial sense
I don't understand. Are you saying that even if it did make financial sense, you would have voluntarily shut the app down in protest of the 4-day notice period? Even though the sales rep pointed you toward the free trial option to continue using the API beyond the 4 days while you decided?
I know you're angry and want us all to be angry at Yelp too, but I have a difficult time believing that anyone would choose to destroy a profitable application out of protest just to stick it to the company about a short notice period.
> I don't understand. Are you saying that even if it did make financial sense, you would have voluntarily shut the app down in protest of the 4-day notice period? Even though the sales rep pointed you toward the free trial option to continue using the API beyond the 4 days while you decided?
An app that sold 467 copies over 10 years at less than $5 a copy is not worth the trouble of dealing with a company that gives you Friday->Monday ultimatums. Obviously, if it were a big source of my income I would have to seriously consider it. But luckily, it's not. I discussed this in the "Development Ends" section of the post. Here is the pricing deck they sent me: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
It seems to indicate a $229 base monthly price on the third slide for my use case.
> I know you're angry and want us all to be angry at Yelp too, but I have a difficult time believing that anyone would choose to destroy a profitable application out of protest just to stick it to the company.
I'm sorry you're reading so much anger in my post. I thought my blog post was pretty balanced. The worst I called them is "quite rude" (I think it's hard to read their emails otherwise) and spent the first half of it describing my app. I never expressed much emotion in my post, and frankly as mentioned above it really doesn't matter in the scheme of things for my life. What I do want is Yelp to change the way it treats developers. Perhaps if someone there reads this it will cause a tiny reflection on their part. I also hope the experience here expressed in the "Lessons Learned" section of the post is useful to other indie developers.
Both your neutral tone and the fact that you want Yelp and other big companies to treat developers better were very clear!
That's what's crazy to me about all these comments. What does it say that so many developers have glossed over this simple ask for more considerate and respectful treatment for THEMSELVES? What does it say that the knee jerk response is fatalism to whatever big tech does?
Thank you. It's actually just a couple users who have posted multiple negative comments in this thread like gp. Not sure what nerve my blog post hit with them but they are free to not like it! It seems like they're more upset about my blog post than I am about the actual situation.
True, but that developer had zero chance to get Yelp to sign any such contract.
Just as I have zero chance of getting Apple to sign that they won't remove my app if they feel like it.
>> I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
This is kind of the salient point.
Either you test on the free API and plan on paying for access slightly before its ready to go live, or you try the "free lunch" approach and see if you can get one by the tendy and see how long you can go before you get shut down and have to pony up the money.
Either way. they should've had the cost of the API in their budget.
We should all know by now. . . . nobody rides for free.
> Either way. they should've had the cost of the API in their budget.
There was no cost to the API ten years ago. I submitted a prototype of my app to their developer program, described its functionality and exchanged a few emails back and forth with someone in developer relations. They specifically approved it and decided how many API calls to give me. A paid API didn't exist back then to my knowledge (perhaps there was some kind of enterprise API but I don't know?). The point of the post is how badly Yelp handled the transition from free to paid. They are perfectly in their right to transition to paid. But they should've handled the emails and transition better.
As mentioned in the post I developed the app on a whim. But after 10 years, it had a few users, although not many. They and I should have received more than a few days notice from Yelp that the API was going to become unsustainable (see my comments elsewhere in this thread).
I hear you, but this story keeps happening over and over and over. The reality is once these companies have you and your product by the balls, they will start squeezing. You can pay money to reduce the pressure, or leave and not be squeezed. I would argue using an unpaid API takes you into the unknown with considerable risk.
> That should be concerning to anyone working on projects that use APIs
Well, free APIs anyway. If you are paying for API access, you hopefully have a contract which gives you power.
Even if you pay, most likely you have a contract that effectively gives you close to no power because it's full of conditions favoring the service provider and trying to use the little power you have will be expensive because laywers and courts get involved.
Almost everyone is working on projects that “use APIs” in some general sense, sure. But I don’t think it’s the case that all or even most people are working on a project that entirely depends on a single third party’s API and is useless without it.
Please don't sneer, including at the rest of the community.
https://news.ycombinator.com/newsguidelines.html
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users.
Or, they completely get it but they work for large platforms that leverage API access commercially or strategically, so their response to unruly peasants is to figuratively chop their heads off.
Thanks—yeah I actually think they mostly just didn't read the whole post since I addressed this in detail in the last two bold sections "Development Ends" and "Lessons Learned."
> They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
This has been known for like 20 years now. We all know that if you're relying on someone else's API that's a massive risk to your business. What more is there to say at this point? What sympathy is there to give when the inevitable happens?
[dead]
I think this is the giant internet trick... do anything to scale, then when you reach critical mass (or someone wants OKRs on their status report), pull out the rug.
To be fair there wasn’t much scale. We’re talking about an app that sold 467 copies over its 10 year span. But it was an app I really liked, the people who paid for it really liked, and just drove traffic to Yelp, so they should’ve liked too. It was also a nice portfolio piece for me.
Wow. less than 47 copies per year on average.. Yelp really cut off their nose to spite their face there.
Even leaving out that it was with only a grotesquely unprofessional four day notice, this is ridiculous. It's not like Yelp was suffering a DDoS from all of those API calls.
lol, I was talking about yelp. I'm pretty sure when they made the API, it was actually laying out the rug. Let everyone build on top of it.
Yeah, given the numbers it doesn't make sense for Yelp to kill it. It's not like a few hundred api calls are costing yelp any significant amount of money.
I'd say, given the numbers it doesn't make sense for Yelp to bother. Just stop it and send the kids an email.
I think Yelp needs to just die at this point. Reviews are gamed, usability is worse and worse. Not sure anyone still uses or trusts Yelp. They’re just in inertia from their better times.
Is there a more open reviews platform not associated with a company? OpenReviews sort of thing or perhaps an open POI database associated with OSM, with reviews as attributes?
Somewhat silly we’re still relying on big companies to collect this data when they then gate it. The timing is right with LLMs that can make maintaining and rating reviews less burdensome on human labor (“moderation” broadly speaking). It’s just text blobs and possibly images linked to people and places.
I’m not sure that this is something that scales. Online reviews seems to attract only the worst in mass media (e.g., gamed reviews, complaint box for influencers who don’t get “VIP” treatment).
The traditional restaurant reviewer filled this role much better. His role as a trusted arbiter was closer to what people want from a site like Yelp, but it’s very labor intensive and obviously doesn’t scale.
But then reviews are heavily biased by the taste of a tiny reviewer pool, which prevents breakout surprises and new ideas etc.
I'm honestly not sure how big a problem that was. If anything, critics tend towards the the new and unusual because they're bored. The problem is much more that no one is willing to pay for them.
I love the idea but anything like that that got successful would be such a target for gaming that the people working on it would have to focus more on anti-bot work than the core product. And whack-a-mole with bots is just not a fun hobby.
Only ways I could see it working at scale are Wikipedia model, which is labor intensive for a large number of people, or some kind of web of trust / reputation system, which is complicated and attackable.
My thoughts as well. I can envision a platform where you can review anything from specific make and model of a product to a restaurant to anything else. I think it could be immensely useful.
But it would quickly become a victim of its own success as people start using it and the database gets big enough to be useful. bots and spammers would swarm the platform to try to game it. That would require a constant cat and mouse game that would not be fun. Without significant ongoing attention, the site is going to become worthless. In order for a site like this that is labor intensive and not fun to exist, it would need to be monetized at least to the extent it can pay salaries. That's the point at which I don't see any realistic solution. In order to maintain neutrality, it would need to be supported by users, meaning users would need to pay. I don't see very many people paying for something like this. And even if some would, there would have to be some benefit or reason for paid users to pay, and that seems like it would open the door for more cat and mouse games and abuse.
Just a hard problem I think.
The Wikipedia model is close to what I had in mind. One way (there are others) identity can be solved for using Stripe Identity or something where you same for a gov credential to identity proof. It costs you $1.50 per request, but is no match for bots. Assuming you bind passkeys or other strong auth factors at account provisioning, and there are only so many high quality reviewers, you’ve got a reasonable limit to effort required.
High quality people want to be helpful, they just need a platform where curation and distribution takes place. See: here. I’m just tired of seeing startups and other companies build and kill these knowledge and community based third spaces, and the intangible value inherent to them.
(customer identity is a component of my work in fintech as an infosec lead, I get paid to squash bots and fraud)
I have to imagine that a big part of any solution to review gaming and bot spam is data diversity. eg: whether the submitter has been within X meters of the thing they're reviewing in the last Y days.
Google seems to have done a good job with review spam on Google Maps, better than their job with SEO spam and malicious search results, and deep+diverse datasets are effectively Alphabet's (not just Google's) entire business.
Yelp would have a very hard time getting to know as much about review submitters, an open data platform would have a hard time gathering even what Yelp already collects.
wikipedia depends on citations to reliable sources and has a policy prohibiting original research. this is why
Where "reliable sources" tends to mean traditional media. Which makes it both more reliable (for many purposes) and much more limited. For restaurants, I do tend to trust newspaper restaurant critics more but there's also far less of that especially at a small-scale non-urban local level.
i don't think newspaper restaurant critics count as 'reliable sources' in wikipedia with respect to the questions people are interested in here, such as whether the chicken is overcooked. i am pretty sure that if you try to put an assertion like that into a wikipedia article about a restaurant, it will be reverted as unverifiable pov pushing, regardless of how many newspaper restaurant critics you can cite in support of it. you can probably cite them for things like when the restaurant opened and why it moved to its current location, as well as use them for evidence of notability
I don't really expect something in Wikipedia to get into the nuances of particular dish quality. But, in general, I would expect that--to the degree Wikipedia gets into the quality of a particular restaurant--a New York Times review would carry a lot of weight. In general though I agree that Wikipedia is probably not the place for restaurant reviews even to the degree they're sources from well-known restaurant critics.
this is generally limited to things that wikipedia editors of vastly opposing viewpoints can agree on, like whether and when the restaurant was awarded a michelin star
like, imagine a vegan, a devout hindu, a muslim, a carnivore, a neighborhood kid, and a foodie editing an article about a texas steakhouse that is known for its barbecued pork. whatever text any of these people finds unfairly biased is going to get deleted from the article, and what's left is what they can agree on
this process produces astoundingly reliable articles on contentious topics like the armenian genocide, but the results are predictably of poor literary quality, and generally not very useful for things like guessing which restaurant you would prefer to eat at
https://lib.reviews/ is one I've used but none of the open options seem to be able to gain significant traction.
I have never in it's lifetime used Yelp. It has been plagued with shit like this from the beginning.
Same thing, never used it. In fact, the most attention I ever give to the site stem from a Vice series titled "One Star Reviews"
I really hate that they force you to install the app to be able to view reviews that people have freely contributed. This is especially a problem on iOS where you cannot change your user agent so that it will serve you the desktop version.
I mean, as at least one other comment noted, anything like Yelp is going to be gamed above and beyond any particularly questionable processes of Yelp itself.
Zagat worked pretty well for "foodies" back in a day when raters had to physically mail in hardcopy questionnaires and people had to buy guidebooks in stores. I'm not sure how you duplicate that today absent physical pay-walls on both sides which I can't really see working.
I do think the various rating sites--Yelp, TripAdvisor, Google--"work" in the sense that they're probably better than picking at random. And you can't really call the local newspaper's food critic on the phone (which is what my dad used to do). So I'm not sure what the alternative really is. Yes, there's some word of mouth but that's pretty random.
>Not sure anyone still uses or trusts Yelp.
The results clearly indicate it is.
Remember the world is far more vast than just Hacker News.
And, as I wrote elsewhere, if you travel a lot you need some criteria and Yelp (along with other very imperfect Internet raters) is probably better than eenie-meenie-miney-mo.
Yelp sucks. I wish Apple Maps would drop it like a hot potato.
Apple Maps Yelp integration is irritating. Any click on a photo prompts Yelp install. I wish they had a "Don't Show Me Again" option
They don’t care what you wish for, you’re the product.
Yep I specifically keep Google Maps around to find restaurants even though I far prefer the audible navigation from Apple Maps nowadays once I actually want to drive there.
i do the same. Plus Apple Maps handles audio and lock screen much better. Apple must be calling a private API to manage lock screen during driving. When I drive with Google Maps, the screen locks and blocks navigation
I've noticed that Apple Maps incorporated their own rating system some time ago (I want to say within the last one or two years); it's simple, just asking you to give a thumbs up or thumbs down on certain businesses, sometimes with a bit of granularity (e.g., rate the atmosphere, food, value, and service separately). There appears to be some threshold for how many ratings they have when they switch from displaying Yelp reviews to their own stuff, although it's not clear what the heuristic is.
Anyway, tl;dr: I think they're working on replacing Yelp.
Their website should be a primary case study for normalizing ux dark patterns
Especially since there were apparently complaints that Yelp was doing pay-to-play with good reviews for a while, which diminishes the truthiness value of any reviews:
https://www.dailydot.com/via/yelp-extortion-lawsuit/
https://cutthroatmarketing.com/heres-why-you-shouldnt-advert...
https://www.reddit.com/r/smallbusiness/comments/h9ohs6/has_a...
(Anecdotally, I also seem to be seeing this on Google Maps now! It seems like highly rated but local restaurants don't even show up on the map at all until I zoom in literally to the building -- perhaps because the owners don't pay for ads? Crazy if true... and Bing maps seem to not have all the restaurants or ratings, and you can't seem to filter based on rating either, which seems like a massive Bing fail. Maybe the review apps were always destined to crumble under a business model that encourages dishonesty on both sides..)
[flagged]
Surely nothing like this will happen to the folks that are using LLMs at the core of their app. /s
Who could have predicted that a company with a scammy business model would have such shady behavior when it comes to changing API pricing models?
/s
Hang on, they didn't give his app 4 days - they gave him 4 days to respond with some screenshots, and if he didn't respond, they would shut down his access. They didn't say they'll shut it down in 4 days regardless.
I mean, don't take this as me defending Yelp - they're scumbags, and deserve any hate coming their way - but I don't think that the headline is an accurate description of what happened.
I did respond with the required information and they still shut down my access (10-days post though, not 4-days in the end). But regardless, no, I don't think sending you a request on a Friday for information due on a Monday or shutting down all of your access is reasonable. As I mentioned in my post, what if I had been on vacation? 1-business day or shut you down is just not a reasonable time frame to make anyone want to work with you in the future.
That's a very fair point.
[flagged]
Surprised the guy never even inquired as to how much the API access to support 100 API calls per day would cost.
After 2 seconds of idle research, I have found that it would cost less than a dollar a day.
https://docs.developer.yelp.com/page/start-your-free-trial
It's just not worth working with a partner that will shut you off at 4-days notice.
In addition though, they shared the following pricing deck with me: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
Which seems to indicate a base price of $229 per month. I have no idea why that doesn't line up with the pricing on their website. But the fact that they shared it with me indicated to me that my use case fell under some kind of "enterprise" usage. Regardless, I would not continue to work with them after 4-days notice and the threatening email. It's too small and app for it to be worth it to me.
I did ask in one of my emails about the pricing discrepancy and got no reply.
A dollar a day is $30/month, and at $1.99/sale that means OP would need to sell 15 apps ONLY to break even, which given the comments in the blog might be close to reality. This is pennies for both sides, but the way that Yelp distastefully contacted OP is probably worth just shutting the whole thing, it's not like OP was making bank on the app to begin with.
Throw on taxes, administrative overheads, etc, they are probably looking at 30-45 sales per month. Which is likely not realistic.
On top of that, this is a continuous payment. Even if I was looking at 5-10x rate of return, I would be very hesitant as that's the rate-of-return today while the sales are forever.
I've been wondering how realistic microsubscriptions are... Say $1-2 dollars a month per user to maintain an app, perhaps limited to just power users, would support a lot of infrastructure.