I got the same email. Despite my hobby project (a random food picker) having been broken and not used for years (because of yelp API updates), they also told me my usage was higher than other developers...
The email also arrived in my spam folder, so I was lucky to even see it. Once I got back to them they did increase the cutoff by a few days but it has since been stopped.
They never answered my question about the discrepancy between these prices and the prices on their website. Both would be too high to make Restaurants low sales sustainable.
That said, I'll say again that they were perfectly in their right to start charging for their API. They just should not have done so with 4-days notice and with such a threatening email.
$10 per request at the lowest tier of enhanced?! The only business case I can think of for this is if you save all the data in your own database then serve requests for your own service out of your own cache. I guess if the Yelp data doesn't change very often, then you just choose to stay 6-months behind on the data or something...
Interesting and an excellent read. The death spiral at Travelocity (pre-Expedia acquisition) worked differently and was partially self-induced.
Some background is required first. Travelocity was the first online travel agency (OTA). I believe they started around 1996.
Through most of its life at that point all, I mean 95%+, of its success came from only two factors: best marketing in industry and growth of the internet. This is severely problematic because marketing only gets you so far in business. Conversely Expedia had really shitty marketing in comparison and yet came to dominate the industry because they were extremely aggressive at growing their supplier relations.
You cannot EVER rely on growth of media adoption, like mom and pop coming online, because once that stops you have no fuel left in the tank. Reliance on growth of a media platform is like a gravy train that you did nothing to build and returns amazing wealth if you are in the right place at the right time, but once it stagnates its like your train derails and everybody dies. That is because everybody expects growth to continue, except you did nothing to earn the growth and now have no answers and nothing to show for it. This was around later 2008 when I joined the company and became unavoidably obvious to everybody over the next year.
So, at that point what do you do? You competitors are far out pacing you by ignoring fun stuff, marketing, technology, and all the other bullshit that technology people look to. Instead they are focusing on core business principles and eating your golden goose while laughing at you. So, what do you do?
In the case of Travelocity all the executives leave. New executives come into trying to figure out what to do. Like every great web business they focus more aggressively on marketing and advertising. This was Travelocity's death spiral.
You have to understand that people DO NOT like advertisements. Really, I know its surprising, but when your site becomes littered with advertisements everywhere and all kinds of hidden telemetry people will leave and never come back. Your wonderful palace has become a trailer park.
The business loves advertising. Revenue from advertisements is immediate. That is really significant. In e-commerce there is a massive lag between each stage of profit, revenue, and sales because you have to account for the cost of operations, sales, and inventory. The more expensive the product the longer the lag and that lag really complicates projections. So advertisements are like cocaine, because they immediately return profit that requires no effort while rotting your health slowly until you are a hollow skeleton.
To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
My learning from this is that a business that earns profit from selling something directly should not fuck up conversion or go out of its way to make customers hate them. When I put that way it sounds obvious, but web business get that wrong all the time because they get distracted by shiny things.
>To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
So in the travel industry you have limited lines of business, primarily: air, hotels, and car rentals. Air is super low margin because its set directly by the supplier plus some arbitrary service fee, typically $7 per ticket. Air is high volume though and is the primary draw for OTA customers, so it cannot be ignored. Rental cars are super low priority because they are both low margin and low volume.
The real interest in the travel agency business is on hotel inventory. This is how Expedia came to dominate the industry, because they aggressively hired geographically focused hotel relationship personnel and Travelocity wasn't keeping up. So Expedia had much greater inventory from various different properties and stronger relationships with those properties for their business prioritization. Hotels, typically hate dealing with OTAs, because OTAs are a transparent barrier between the hotel and the guest staying at the hotel, but the OTAs bring people to the properties. The name of the game for hotels is "heads in beds" and while up-selling is beneficial if you get the "heads in beds" part wrong you cannot exist.
The margin on hotel properties is much higher and highly variable. Hotels in many cases will even eat the cost of airfare to get heads in beds if the pricing is right. That provides a huge opportunity for a marriage between smart hotel systems and OTAs because the hotels are doing the smarter inventory management and the OTAs are supplying the airfare volume and customers to populate that smart inventory management. The hotel data systems, Property Management Systems (PMS), were at that time much smarter than the inventory availability OTAs had, because they had to do much more to account for seasonal volume planning and various customer demands.
Where Travelocity was strong was in writing algorithms to account for these considerations and offer volume pricing discounts the competition could not and also air + hotel package pricing at massive discounts the competition could not compete with. In parallel Travelocity was slowly building up a corporate travel business to take advantage of that volume pricing intelligence and at one point had both WalMart and Lockheed-Martin as customers.
Again, both of those were amazing and could have really helped retake lost market, but conquering competition is a fragile slow process while advertising is immediate.
> they also told me my usage was higher than other developers...
I'm curious to what they compare this. The non-existent free plan where everything higher than 0 is unusual. Or a large mass of dead accounts who have really zero to none usage. Or the actually paying account who are now on their enterprise plan. In the later case, this might indicate some real problem if their usage is low enough that smaller projects are already exceptional.
Author here. I actually discovered something in their terms of use today while looking up something else that someone had asked me in this thread:
"2. CHANGES
Yelp reserves the right to modify the API Agreement at any time. If Yelp reasonably determines that a modification may materially and adversely impact You, Yelp will provide email notice to you using the email address you provided during registration no less than ten (10) days prior to the material adverse modification taking affect. IF YOU DO NOT WISH TO BE BOUND TO ANY NEW OR MODIFIED TERMS, YOU MUST TERMINATE THE API AGREEMENT BY CEASING USE OF THE API WITHIN TEN (10) DAYS OF RECEIPT OF SUCH NOTICE."
So, the 4-days would have violated their original notification terms of use anyway. I think they realized this after the fact of sending all of us the threatening emails and this is why they ultimately closed my API key after 10-days as described in my original post.
Anyway, just an interesting aside that the poorly written, inaccurate original email that I shared in my original post was even inaccurate to their own policies.
"YOU MUST TERMINATE" rather than be held to the original terms you initially signed up for ... because Yelp RESERVES the right.
We have all now been reminded that RESERVED RIGHTS can and will be used against you maliciously. And if you want protection from such malice, there needs to be a RIGHTS of the USER clause.
This should be handled and enforced by government statutes.
(It is an amazing world we live in where rights of the user have to be expressed.)
Seen this story play out so many times. I audited a company years ago that claimed to have excellent, personal almost, relationship with Google and all needed paperwork to use their platform as a core of their business. They went bust 6 months after because of "unexpected" change of Google's product strategy.
Real lesson here is to avoid single points of failure, regardless if it's API, people or partners. Ask yourself a question if there's a single entity that can kill your business and remove that reliance.
If you're depending on the continued existence of any Google product beyond GMail, Google Cloud and advertising you should really consider fallback positions. The existence of Youtube is also guaranteed, but the existence of any particular API or service that you might depend on? Not so much. Hell they'd probably consider killing GMail if they didn't need the accounts to tie advertising impressions to individuals.
That's a ton of money, (and I've heard that they paid Apple even more last year) but it's kind of understandable. Apple customers don't usually have android phones snitching on them, are less likely to use chrome, and tend to have their devices and data in the Apple ecosystem. Google has fewer ways to spy on Apple user's lives and push ads at them. Search, gmail, and youtube probably give them the best opportunities. I wouldn't doubt that Apple users are getting as frustrated with Google's search engine as everyone else is though.
Google used to need Search in order to learn about the intimate details of our lives. It's how they knew what we were interested in, what our medical problems were, what we were learning, and what we were thinking about.
Now they have chrome giving google people's entire browsing history, android devices collecting realtime data on what people are doing offline, where they are and who they are with. Google also has products like fitbit, nest, and gmail that gather still more data for google. Google doesn't need Search to spy on us anymore, so they haven't invested in keeping it useful.
In fact, it's better for google if you can't find what you want and have to make multiple searches for information because it gives them more chances to throw ads at you, and the harder it is for people to find websites using Search the more sites might feel like they have no choice but to pay Google to keep them at the top of the search results.
People who find Search increasingly useless though are turning to alternatives and for many people AI could end up replacing google's Search product as the first thing they turn to.
I would not count on Search staying around forever.
That is only half-true. On the one hand, I agree and I would never build a serious business on another single entity. But on the other hand, OP built an app that ran successfully for 10 years, generating some revenue.
Most products have an expected expiration date and you can provide a useful service (that generates revenue) by building on other platforms, even if it won't last forever.
By now I missed more opportunities by having the mindset of "not relying on other people's APIs" than real changes/shutdowns would warrant this kind of caution.
It's so sad that we're at such a devoid barren state.
There was so much hype & excitement in web2 age, about APIs & intertwingularity. Humanity was going to keep improving & iterating & crafting ever neater ideas.
Now it's all bureaucracy & lawyers arrayed to defend the cloud-keeps, moats & walls.
The user & agency of the world has been steady dimished, eroded away by ever advancing legal IP Protectionism.
That's not going to work in practice as money and time are limited. Choose your partners wisely but don't obsess over the scenario of them going bankrupt.
I built a service around helping podcasters automatically convert their audio podcast into a YouTube channel. I went through tons of review with Google in order to get access to the YouTube API and make sure everything I was doing was in compliance with their terms - literally months of back and forth. I had been testing in my development and staging environments against their API for 6+ months. I launched in production, got a few videos uploaded to YouTube, and they disabled my API key. I spent months emailing them and never got anything more than the same boilerplate copy/pasted answer. I could have pivoted or something, but I just shut it down and moved on. Lesson learned.
I worked for a very early billpay company where you could pay your bills online to vendors, even if the vendor didn't support it. We used API's where we could, but where we couldn't...
We had a whole team dedicated to keeping up the changes vendors would make to their websites that we scraped for info. The team was called, of course, "Scrape and P(r)ay".
If you build your scraper to find data on the page based on the shape of the data itself instead of the structure of the page then it will be resilient to most changes that don't materially change what data is displayed on the page.
So, prefer regex over css selectors, and css selectors over xpath, where possible. And don't select based on nesting or position if possible.
That's because Booking was also committing some type of misrepresentation and taking revenue away from Ryanair through their browser automation. Even then, the infraction was sooo bad that they got a $5k fine.
> In November 2022 the U.S. District Court for the Northern District of California ruled that hiQ had breached LinkedIn's User Agreement and a settlement agreement was reached between the two parties.
Reminder that the earlier ruling was overturned, it is no longer clear whether scraping is legal or not.
My first Saas was built around Facebook, Twitter and Reddits API.
Facebook broke like monthly and required random updates, sometimes documented, sometimes not. Zero support.
Twitter worker flawlessly until I hit some limits and there was no way to increase them because I didn't hit some other limits. No way they would talk to me.
Reddit just worked until I gave up.
What I learned is to never again do any business based on someone else.
A good lesson, but a more nuanced one is to do sure diligence. There are companies I would never build my business around (Google, Oracle, etc.) for a variety of reasons. There are ones I know I can trust (at least until they change). There are many where I need a contract.
Change is also a constant, and there are plenty of good companies give bad, and a few in reverse. So due diligence is an ongoing process.
That dramatically raises the price of SaaS, so I use it much less than mainstream industry practice, but much more than zero.
> What I learned is to never again do any business based on someone else.
That's a little dramatic, don't you think?
Supply chains are a core of many (most?) businesses out there. They depend on others to get their business done. No supply -> no business.
What you probably meant is not building a business based SOLELY on irreplaceable business relationships. Unfortunately in a lot of cases involving the tech giants and little startups, there are no alternatives.
More or less, however I really try to not build on anyone else ever again.
I can't find a single dependency in my serious money generating chain right now that isn't replaceable with a similar product within a few minutes or hours if necessary.
My most recent project didn't even last a full year as there was no replacement that worked for me for ChatGPT in that moment. It was built to die, a gamble essentially. I wouldn't waste my time for things like this if I didn't had the end calculated in already.
Same experience, slightly different. It made me realise why some parts of an MBA (e.g. strategic place/partnership/spring boarding from a brand) actually have value I never saw before for tech companies. There are essentially king makers in certain fields - either by their apathy at first-movers, or their choice on who they allow to continue.
Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul. Reddit, Twitter, and company all got big because they had open APIs and people were able to use them extensively for really creative things.
I agree, Google has swallowed up the video streaming market unfortunately.
I keep thinking back to how Vine was basically TikTok, and they threw it away.
Take it from someone who watched with interest, Reddit and Twitter did not get big due to open APIs. They got their first big steps before they even had APIs... In the first year of twitters launch it was in the news in the UK at 300k users in press releases about how it was a "new form of communication" etc... 2008 was actually when the original set of API-laden websites began to fade because with the recession, we could not have nice things be free so much. Many API services fell into disrepair in 2009, certainly the peak of APIs for the UK was around 2008. Check out Tom Scotts video on this for a picture of the open API internet many people thought would occur (and which did exist in some places until FAANG began to productise and dominate and use accounts across products which do not incentivize APIs). Yes it is true open network graphs/facebook etc had some API access, but the APIs before were more single purpose and numerous; facebook lets you explore their network and to some extent data, but that's not the same as the useful APIs that became restricted, paid or were cut in the aftermath of the recession.
Those startups are their competitors. You don’t have to pay millions of dollars to acquire a competitor that was never started because of your inconsistent API policy.
Google makes enough money that losing several currently non-existent revenue streams that are theoretically $10 million apiece isn’t hurting them. It’s hurting their users.
Google is all about ads - why would they give a shit about the users?
>Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul.
Google made $300B last year. "billions over the long haul" is a lot of money, an unimaginable amount even, to you and I. But to Google?
It's a testimate to the health of our free market that the company throwing away millions, and billions over the long haul, is the overwhelmingly dominant market leader.
Attempted to build a similar thing a few years ago when living in Tokyo.
Provide a selection of restaurants within a 1km radius and automagically provide 3 recommendations based on my preferences.
We had multiple API providers available, Tabelog, Gurunavi and Hotpepper, all required a paid developer license. We still needed to use the Google maps API to get the user's current location though.
This was also just when Google Maps suddenly raised their API pricing. After spending a couple of weekends building a working prototype, we stopped as we couldn't justify the cost of paying multiple API providers for basically 3 guys looking to save 10 mins deciding where to get lunch.
Also, this kind of app is a common theme if you frequent meetups in Tokyo. There's always at least 1 person that has built such a thing.
> We still needed to use the Google maps API to get the user's current location though.
I’m sure as somebody who actually built a prototype, you understand the situation better than me, but I’m curious: why would just getting the user’s lat/long require anything you’d pay for? Wouldn’t the OS or browser provide that to you?
Oops. I meant that we were using the maps/places API to get lat/long of the restaurants using the street address from the other API providers, then using that to calculate a 'N mins walk away'.
We were just testing the abilities of the various APIs. Google was the only that provided travelling time estimates.
I was thinking about how to solve this given that one of the primary problems is that of fast, global content distribution. I like the idea of paying people in crypto as part of a ledger transaction to host and serve content, like bittorrent with a crypto payment. Unfortunately I can't also think of a way to prevent such a system from being abused to distribute harmful media such as CP. I guess it's not like this isn't a problem with BitTorrent today though.
Regulation might here - something like minimum mandatory 3 months notice for shutting down the API keys. Considering the average age of our politicians, I doubt they’ll understand what “API” is, much less be willing to take on giant tech corporations even if they did understand the problems
Competitive markets are by far more the exception than the rule. It's just not how capitalism works (because pretty much any competitive market is ripe for “consolidation”, which increases aggregate shareholders value by reducing competitive pressure).
If you want competitive market in a capitalist economy, then you need very active state enforcement.
The markets are absolutely competitive. Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
They are now hemorrhaging search to OpenAI that popped into public existence just recently.
To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
If you wanna see lack of competition, look at government granted monopolies on utilities. Guaranteed but capped rates means you reduce investment right to $0 and cut costs as much as possible since there is no other way to make money. That “state enforcement” you are calling for is how you end up with PG&E and scenarios like all insurances companies pulling out of the state.
> Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
Phew, here I was stressing out about lack of competition, but you helped me relax.
Google has to invest into Youtube lest it loses out to Twitch (Amazon), Netflix ($35bn per year), HBO Max (Warner Bros, $50bn per year), (Amazon) Prime, Plus (Disney, $90bn per year).
My faith in humanity is restored now that there are alternatives.
> To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
Please tell me how many “product advancement” in Google search or YouTube over the past decade… From a consumer's perspective all that happened was enshitification and despite being owned by a search engine company, YouTube never managed to ship a functional search on their platform.
> If you wanna see lack of competition, look at government granted monopolies on utilities.
Utilities are “natural monopolies” though, and as such they should be state owned. Making a natural monopoly owned for profit is a recipe for rent seeking, and that's why it was promoted …
> That “state enforcement” you are calling for
No, the state enforcement I'm calling for is proper enforcement of antitrust laws, forbidding consolidation through M&A and disbanding companies megacorps. That is to say, what existed in US's golden age.
>Competitive markets are by far more the exception than the rule it's just not how capitalism works
Empirically this absolutely isn't the case; the majority of listed companies have fairly low margins, especially non-tech companies, which can be trivially seen from their financial statements. A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
> A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
Not necessarily: if prices are elastic then even a monopoly can aim for low profit margin (in percentage) in order to increase profit. What matters is how much total profit are being made, margin only matters when measuring risk.
Also, corporations are social structures, and low competition also encourage complacency in the corporate structure itself, which drives costs up and reduces profit margin.
Do you know if there is any open source tools in this field that a programmer can work with/automate, I have mainly textual and graphic content, a little bit of video, but thinking to branch out to podcasts and music soon, unfortunately the other products supporting text seem pricey and geared to big teams.
on edit: obviously I can write my own, but I am hoping for a project that has already done a bunch of things that I can extend for my needs, as writing my own would be at the point where it would be more cost benefit to purchase.
I believe I am entirely in compliance with YouTube's terms and policies, but I was flagged for some bullshit as well and got the boilerplate responses. So what I decided to do was hide it for logged out users so it looked like that specific feature was entirely removed to the dudes in India that review this stuff... One year later and my fingers are crossed we don't get another audit.
We went through a three-week Facebook API shutdown due to a clear glitch - Meta support couldn't go outside the script, or escalate in any way beyond "a supervisor will email you". Only resolved when our CEO found some VP at Facebook on LinkedIn and got them to escalate it internally.
(The dev community thread is full of people still impacted, so I think they literally just edited our app's flags directly.)
Even tried to invoke GDPR's rights to be exempt from automatic decision making, but their privacy questions email address responds with "nope fuck off" to those.
Basically any hope of solving a problem with a Meta property is to know employees who can escalate your issue. Sometimes your friend at Meta also needs to argue with the bug owner to not close it frivolously.
Within the customer service industry, they'd claim a great "deflection rate". That's a metric that all of these large companies hold around any kind of help/support channel that may involve humans because people are a cost. It's often covered by some kind of satisfaction metric based off post-issue surveys, but fundamentally, if you just go away that's success.
There are often complaints here about what amounts to bean counting affecting other aspects of business. Customer service at larger scale is costly so attracts a section of very analytical leaders. They don't get, or ever prioritize, the human elements. It's only when satisfaction numbers are bad, or another exec outside of customer service takes action, that things improve.
The scale part kills the customer. You can have great support at one size. Once you grow then leadership, structure, and the culture change. These analytical leaders don't want to carry over the culture and structure because it comes at a cost. It really needs force and support from outside of customer support leadership to maintain it.
Had this issue with Ubisoft recently. Installed Trackmania through Steam for free, then had to create a Ubisoft account, then had to buy the Trackmania annual pass for $20. I guess I followed the wrong flow or something, but I ended up with the purchase on my Ubisoft account, but I guess their system also auto-creates a Ubisoft account for Steam accounts? So the purchase on the Ubisoft account ended up on the account not tied to Steam.
Emailed support about it on 13/07/2024.
Got back a reply on 23/07/2024:
> Our Support Team is currently experiencing high case volumes, so we are reaching out with a message to check-in and make sure you still need help!
> If you've found a solution to your problem already, there's nothing you need to do. Just ignore this message and your case will automatically close in 2 days.
Didn't see the reply until the 3rd day.
> As we have not heard from you in the last 2 days your case has been closed automatically.
I think people, even on HN, grossly underestimate how much customer service costs at any major company that works with the public.
Most calls to customer service are easily self-serviceable. My brother used to work for X-Box Live support, and he said over half his calls were for password resets, something people could easily do on their X-Box.
But then there's bad actors. Another significant chunk of his calls were from people complaining about being banned, and he'd see the reports of their spamming racial slurs.
Not to mention how many people try to abuse customer support in attempts to take over someone else's account.
Yeah...customer service is expensive. Even if you automate as much as you can, but offer an appeal process, that appeal process with be absolutely abused by bad actors, so that will end up automated as well.
I am curious if you would be interested to retry the idea? I might have an in with the YouTube team. I feel like it's a shame to let this go. Would you be open to chat? Please reach out r@pehul.com
No. The default API quota is not large enough to upload videos. You would have to contact Google, explain your use case, and jump through several hoops to get the quota increased... which is a huge process.
Yeah. Uploading legit videos is non-trivial. And if you ever upload the same video twice (which you think you might do during testing, right?), it's a violation of their terms and they disable your access.
I think Yelp is shutting off their API for the same reason as Reddit: to ensure that AI training makes money for them. It sucks that you’re a drive-by casualty of that, and if I’d bought this app, then - same as Apollo iOS - I would not request a refund.
YouTube already uses Widevine for movies. they will never use it for regular videos, because it will kill their website. widevine is a resource killer, for both client and server.
> To wit, there is exactly one business strategy: When you are small, be nice. When you are big, pull that ladder up behind you.
I'd add a "in venture capital and big capital" after "business strategy". When you don't have VCs or the stonk market breathing down your neck all the time, incentives massively change.
IMHO, there is only one solution left... once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust that has a clear mandate to run its companies in a way beneficial to society at large. That way the government doesn't have to spend taxpayer money on countless r&d experiments, VC investors have a perspective to payout, and the world gets kept from utter bullshit like API games.
Well if small capital doesn't pay good, I have no incentive to run a small business at all!
> once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust
Interesting, haven't heard that one. I'll say maybe. I'd rather have Facebook not exist at all than have it be an *official* arm of the NSA
> the government doesn't have to spend taxpayer money on countless r&d experiments
If you expect the government to make a tender offer on the largest market-cap companies, they sure as heck _are_ paying for r&d indirectly. And a lot more!
I think it's quite different. It involves money, but QE is all about monetary supply and interest rates alone. Buying entire companies comes with a lot more implications, like decision-making power over the organization. You want that for API keys, but do you want that across the board? Seems like regulation would be much cheaper.
Why? The problem at the core is the mantra of the WSB et al groupthink: stonk only go up.
Of course when WSB makes jokes about it and catapults a junk stock like GME to ridiculous heights, squeezing shortsellers dry, it's pretty funny to watch. But it gets problematic when just about everyone's pension is tied to "stonks go up" as well. That leads to all sorts of perverse incentives.
Just look at Boeing and how far they've fallen. Twice in recent years, literally and not just proverbially, with hundreds dead as the inevitable consequence of stonk market games.
I think the GP had a visceral reaction to your intentional misspelling of "stock" as "stonk".
I tend to agree, though I don't think I'd go so far as to think someone saying "stonk" has "lost all credibility"... but to me, it's like Republicans deliberately mispronouncing Kamala Harris' first name, because they can, and because their base eats it up. It's a cheap shot, and it feels childish, like you don't have a real argument, so instead you act like an elementary school kid on the playground calling the other kid names.
This has been one [unintended?] consequence of AI promulgation. A direct disincentive toward the kind of open access that so used to be common and provided a lot of low-hanging fruit for independent developers trying to increase interoperability within their favorite niches.
So now not only is AI filling the web with garbage that poisons future model development[1], it provides incentive to further close and wall off access to (user-provided!) data.
Yes. We're entering a new era of web applications, whether we want to or not. Companies used to be able to gate their data behind UIs, because the search results had more value than the raw data.
Things are starting to flip. Increasingly, people would rather have access to the raw dataset to pipe into an LLM. The question, as always, is how to control this access and charge people accordingly.
I think you're right. API strategies are being reviewed because AI training data.
That said... I think API policies are also vestiges of a defunct idea set that prevailed circa 2009.
At that time, "twitter is a protocol." APIs and openness were going to be the new worldwideweb. Social media was going to be an ecosystem of independent apps and services with complex interactions mitigated by protocols.
YC even did an "rfs" recruiting startups to build off the twitter API.
That didn't happen, but APIs intended to enable it were already in the wild.
As API access becomes strategic again (because AI), "why do we even have this?" is likely to come up.
I think the general lack of willingness to help in Big Tech is very problematic. You can almost never get through those thick-skulled reps that email you out of the blue...
The problem is that by having a thick-skulled rep walk in and send a few e-mails, Yelp has already lost more money on API customer acquisition than the app developer was willing to pay. That's why these APIs had free tiers: they covered these kinds of micro-usages that would be far too cheap for a sales rep to cover.
The reason why those free tiers went away is that AI came along. Not so much that the AI scrapers were abusing Yelp free tier[0], but that they could. And once companies realized how much money was floating around in selling data access, non-abusive free tier users went from "a cool goodwill gesture" to "freeloading parasites".
David Kopec and Restauraunts got steamrolled in a case of technological gentrification. If you're selling data access for $TOO_CHEAP_TO_METER/call to a random indie, Apple, Google, and/or Microsoft will use that as a comparable for why Yelp should charge peanuts. Or they'll just acquihire him. They need him and his app to go away because he is inconvenient to the long-term valuation plan of Yelp, an old guard Web 2.0 business that never quite became sovereign.
[0] Though, to be clear, AI scrapers are absolutely abusive in general.
A couple years ago, I was confused about why my Apple TV remote was behaving weirdly, and a few taps in the support app and I had a real person call me and fix my issue within minutes.
I was looking for a completely random Molex adapter from the early 1980's for a hobby project. After reaching out to the company they had someone call me and attempt to help. They didn't find the part (since it hasn't existed for 30 years) but they did give me some good leads.
Amazon’s AWS has good customer service. I get good customer service at the Apple Store, and I also got good customer service when Microsoft had retail stores. I have used Azure’s customer service and it was OK. Amazon’s AWS is know for outstanding customer service. My main point is some big tech companies have terrible service. Some don’t.
I meant, if you have the ability to initiate a chargeback (pay money, and have the ability to take it back out of their control for bad service), then they will probably provide customer service.
I'd like to see how someone can see a directly employed Google support agent / repair technician in person for support like you can with the Genius Bar.
Additionally, as far as I know, the kind of support you get calling AppleCare is unmatched by anyone else.
> You can almost never get through those thick-skulled reps that email you out of the blue...
He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
> He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
> What more would you want the rep to do?
Not send a threatening, inaccurate email with a 4-day (1 business day Friday->Monday) deadline in the first place?
Also for the record, this was not a sales manager. This was someone in "growth" that sent the email. I guess maybe that just means "sales."
> Your API usage is higher than lots of other Yelp Fusion developers
You're right it wasn't just inaccurate it was outright deceptive, trying to frame this as something specific to OP that he is responsible for rather than a change in their business applying to all API users.
Employ a shred of critical thinking and realize this app probably drives more value to Yelp than the cost to run the API, and flag it for an exception.
Customer service or sales people like that are usually at the lowest point on the totem pole. It’s likely that they have zero leeway, either you’re gonna pay exorbitantly or you’re out. It’s an institutional problem.
In that case it's not customer service but a customer firewall. If the CS employees are not empowered to induce solutions even if they benefit both the customers and the company then they are literally useless.
And yes, many companies have "customer service" like that designed to waste your time until you go away. That doesn't mean it's the only possible way.
But Yelp is not big tech, certainly not by market cap. It could be old tech, in the way that a 2010-era data API may have been a growth hack intended to create an app ecosystem. But 10 years later, there’s no app ecosystem and the company can’t afford to support a (stagnant) lone indie developer. Especially in the current era of LLM scrapers.
Author here—seems they noticed that people were unhappy. Just got this email from Yelp (August 1, 2024):
> Earlier this month, we sent you an email about your Yelp Fusion API usage. That email gave developers until July 23rd to contact us if they want to continue using Yelp’s data for use in their app. We realize you might need more time and are extending your free access for an additional 90 days starting today. Your access should be available now.
> We’re sorry for any inconvenience or frustration this abbreviated transition might have caused. Please respond to this email or contact us at api@yelp.com if you have any questions.
> First of all, I was not “trialing” the Yelp API. I had been using it for a decade and had official permission from Yelp to create Restaurants.
This looks like a Face/Off situation to me.
(Spoiler alert for a 1997 action movie: in Face/Off a cop surgically exchanges faces with an imprisoned villain in order to go undercover in their organization, but the villain then murders everyone who knew about the swap and steals the cop's life.)
Somebody at Yelp in 2014 knew that you had been approved to build this app. That person almost certainly no longer worked at Yelp ten years later, so the institutional knowledge of that agreement had likely been lost.
I absolutely wouldn't expect there to be notes against an API key.
Heck, in many companies I have encountered that they don't even know who is using API keys they've issued to third parties, especially over generations of systems. I am impressed that after getting email approval for an app a decade ago, Yelp still had contact details enough to let the O/P know!
None of that negates the issue of 4-days notice and Yelp shooting themselves in the foot by removing access to an app which redirects people to their website. But someone in executive management made a decision on the basis that they wanted revenue from the API overhead, wanted a slice of other people's pie and that's that.
To the O/Ps point about subscription, there was a UK food app which gave you restaurant food safety scores taken from the food safety agency "Scores on The Doors" it's gone now but I paid a few £ a year for it and never had an issue. If the Yelp pricing is sustainable at that level it's not a bad idea to pay for API access, Yelp has to pay for their servers and a portion of the API calls won't be converted to traffic. Who knows if Yelp still has a sustainable business? Maybe that API thing doesn't pay for itself? I don't know.
Maybe not for an API key, but for an account, you would think they would have some method of keeping notes. I've never worked for a company that had a concept of accounts, and teams of people that interfaced with accounts, without having a method of keeping notes about an account. That's just a basic necessity, especially when the accounts exchange money with the company, there's usually accounts that pay more than others that require special handling, which is usually documented in an account notes screen of some kind.
On the Yelp developer site you have an app profile where there is the name and description of the app. There is also a separate field for the URL of the app website, which I did have filled in. So, they have that at a minimum. If they didn't retain the records of the screenshots/communications that led to the original approval and raise of the API daily limit then that is poor recording keeping but understandable in a corporate environment after 10 years.
The API key usually has a contact email. After a year that contact email probably doesn’t work anymore, certainly for free tier.
Which underlines what I concluded long ago: the best and most durable form of identity on the internet is rooted in the ability to pay money. Any identity that is free to create is doomed.
Oh I completely agree - 4 days notice is _never_ OK. I get uncomfortable with 30 days notice because I've had the occasional vacation that long!
Ideally they'd have a notes field against developer apps and a robust process for recording this kind of thing - but I've worked for companies, so it doesn't surprise me at all that there's no good mechanism in place for that.
In real life, sometimes months isn't enough. On August 5, UPS will complete their API transition to OAuth based authorization after many months of process. Many of my customers haven't responded to my attempts to warn them about the change. I've resigned myself at this point to August 5 being a crazy day.
Yeah, you'd think so. But as many of us have learned, many companies can't and won't be considerate. Sorry that you got hit with it, but at least it's a quick and clean break.
Don't sweat the refunds too much (unless someone is being really rude about it). Apple certainly won't.
Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
> Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
Right, the biggest thing is a failure of communication. There should have been emails months ago, not 4-days (1 business day Friday->Monday) before.
There should have been an email months ago saying they were converting their API to be paid. Not a 4-day threat to shutoff your API key or pay-up. As I mentioned in my post, I was not aware that they were converting to a paid program and had received no prior emails.
My first thoughts while reading the story was that their champion at Yelp quit and whoever's left didn't know/care enough about them. 10 years ago, Yelp would still be scrappy start up and there was a motivated employees willing to go outside the painted lines. Now, it's just a faceless person following orders.
or in the departed. Martin Sheen is the only one who knew dicaprio was undercover. With him dead the protection goes with it. Only a real contract would CYA. When a company goes from friendly to unfriendly they're happy they were 'scrappy' not to write everything down.
I’m actually surprised the API access lasted for so long. The person that provisioned it is probably long gone from Yelp. Similarly, the company’s values and priorities have changed as well.
Also, I’m guessing this new API pricing policy is they way of combating data scraping to train AI.
If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions, or the service in question was built poorly from the start.
There is little room to blame behavior like this on bad actors and not bad decision making. 10s of thousands of companies have managed to figure out combatting API abuse without shooting themselves in the foot.
> If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions
You are entirely right, yet I see this train of thought happen so very often in different companies. Some of these people are a bit saner than others and actually do listen to others explaining why it is an idiotic train of thought. Others really are just see numbers in isolation and somehow refuse to look at them in context.
Depending on what kind of approvals they gave him 10 years ago, it MIGHT be possible that doing this violates a contract. It sounds like they had some kind of understanding when they gave him access that he had some sort of informal approval. Even if nothing was signed, that still forms a contract. Even if there was a formal contract or terms of service (TOS) agreement, there are certain restrictions around when and how a company can change their TOS. In particular, there's often requirements about how much advance notice has to be given if the terms of service change.
It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
> It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
Yes, the monetary stakes are too small for it to be worth it for me to pursue. I could probably dig up some old emails from 10+ years ago but it just wouldn't be worth it. Exposing this kind of bad behavior (4-days notice!) is enough.
What's needed to form a contract is an offer, acceptance, and consideration - if an offer was made and accepted and something of value was exchanged, and there wasn't confusion about the terms (after 10 years, there wouldn't be), that's good enough for a contract to have legal force.
Not forever, no. But usually with any sort of contract, when one side wants to break or alter the contract they have to give the other side adequate notice. In certain circumstances a contract can eliminate such a requirement, and the definition of “adequate” would vary a lot based on specifics, but that is very much a possibility in this case.
It sounds like he requested API access in order to make a native Mac application for Yelp. The specifics matter a lot here, but “if you develop a native Mac application for Yelp, we’ll give you free API access” sounds a lot like consideration. That could be completely false based on exactly how that went down, of course.
> Developing an app to use someone's free API is not an exchange of consideration.
I would never pursue anything further over this app that had about $2,000 of revenue over 10 years. That said, for the record there was consideration.
I had to go through an official approval process that required providing evidence of the app's functionality and a few emails back and forth. I think I actually sent them a prototype of it. And based on that process they decided how many daily API calls to give me. A normal free user did not receive 25,000 API calls per day. I believe if you didn't go through approval back then you got something like 1,000 per month. So there was a consideration process on their part and a determination to green light my use case.
You could argue that the indefinite API access was in exchange for writing the app (service for service), but if this were me, I probably wouldn't bother. Maybe I'd write an adversarial-interoperability backend to replace the API, or open-source the app to allow other interested parties to do so. Or maybe I'd just say "It was a nice run", and let it die.
If you care enough to send a polite email, you could say that Yelp has a prior agreement, you'd appreciate them not reneging, and if they do, you'd appreciate compensation for your labour (minus, of course, the money you made from the software). Probably won't go anywhere, but…
I would be upset too, but this sort of sounds like someone did him a favor 10 years ago, management changed, and now management wants him to pay. This sucks, but near infinite free api calls sounds unsustainable
We received the exact same letter from Yelp. Our usage is significantly higher and when we talked to them, the prices they quoted were ridiculously high (thousands of dollars a month).
Exactly, we're not. We're closing our products. They are perfectly welcome to charge too high a price for us. They just should've communicated it better.
We have a analytics SaaS which displays, among many other pieces of data, ratings from different providers, including Yelp.
I assumed that Yelp had been doing this all along and we ran above some predetermined limit, but the email we received was identical to yours. So I'm thinking that Yelp is trying its best to monetize all API users
A 25,000 calls per day limit for an API sounds absurdly low. Any idea if the API calls actually would have used much in the way of backend resources, or do you reckon it was more just mostly a database lookup?
Honestly, making people pay for an app that only uses a public API you’re not paying for, and no form of fallback is asking for trouble. This is not a responsible way to do business and I hope people reading this thread will understand that.
Since ~15 years, people got the idea that all the WhatsApps, Instagrams or Twitters around them aren't companies, but kind of public infrastructure. They all knew that it's a wrong assumption, but it's sooo convenient, and at some point, people around you force you in that direction, even if you don't like all those walled gardens. And once this wrong philosophy was established, people started escalating it to some smaller shops, I guess. Those are also the guys that complain when some 3rd party Reddit or Youtube hobby clients must disappear. They just have no understanding at all about some basic mechanics of human interaction. The biggest problem: There are entire generations of people nowadays with that misunderstanding.
but the author pointed out that it drove traffic to Yelp, and that almost certainly seems to be true.
In fact, little integration plays like this app might have been the only thing keeping Yelp sort-of alive after Google got into the game.
Killing off your bizdev partners seems incredibly short-sighted and foolish. (Also, feeding the reviews into an AI... for what, exactly? To train a model on how to write reviews? Or perhaps to detect fake reviews -- actually, that was an issue on Yelp even before AI, so it seems like it wouldn't be the best training content.)
I think Apple has been developing a yelp clone internally; and have been slowly releasing bits and pieces.
Apple has added rating systems for destinations. Amenities available at location (ie, “accepts Apple Pay”, dog/cat friendly). Maybe in the next decade Yelp integration will be phased out completely
They've already got something in Maps that switches locations from Yelp reviews to the Apple thumbs up/down system once they hit some critical mass of data, and the same for location photos.
Was this a move to try and get some profit from AI companies trying to train on their data? With the API providing limited results, I’d think it would be of limited use. I always found Apple’s use of it in Maps frustrating, because seeing POI details basically required I also have Yelp installed.
If this move was AI driven, like with Reddit and others, I’m starting to dislike AI more and more… at least the rent seeking end of it, which seems to be slowing killing the open internet.
SaaS can provide “open core” or better yet simply sell a hosted version of their fully open source code. If the provider fails to provide, you can fall back to self hosting.
The API equivalent would be open sourcing the data. This is the OpenStreetMap model. If the API provider fails to provide, you can fallback to the underlying data.
That's asking too much. SaaS should give an option for you to export all your own data in a simple, parseable format (like JSON). That's about it. They don't owe anyone their source code, and they don't owe any ethically sourced data (such as employees researching and manually entering).
API access needs better terms. Like guaranteed access for X years at $Y price with Z days notice if there's a change, where Z > 3 months or so.
This was the hope behind the StackExchange data dumps, that the community at large could always take their contributions elsewhere if the service jumped the shark.
Well, before the SE organization tried to kill the data exports off in an attempt to commercialize it towards AI companies, but thats a whole other issue.
That would probably as big or bigger project than the entire rest of the app itself, and since it is such a single developer and not a super profitable app it likely makes no sense to do this.
SerpAPI is a convenient wrapper API for scraping various sites. I assume they've vetted all the legality of things. They have a YouTube API: https://serpapi.com/yelp-search-api
Still they have a boilerplate ToS with some glaring mistakes:
> These Terms of Service and any separate agreements whereby we provide you Services shall be governed by and construed in accordance with the laws of 5540 N Lamar Blvd #12, Austin, TX, 78756, United States. (sic)
In that case, the court found 3Taps was criminally guilty for scraping publicly available craigslist data while logged out because 3Taps knew their use was not authorized.
This person has just received an email from Yelp telling them their free usage is not authorized, so circumventing that may well be illegal, now that they've been given that sort of communication, even if it might be questionably legal for other serpapi users.
The courts are altering their views on scrapping. This [0] is a good paper that explored the last 20 years of rulings (although it hasn't been updated with the most recent cases).
Not sure why this is getting downvoted, because it's a reasonable question - the answer being that such scraping requires a lot more time and technical expertise to engineer than a simple API call. Also devolves into a cat and mouse game between your application's backend and whatever proxy they put in front of yelp like cloudflare, which you'll probably lose or will be prohibitively expensive.
It was never different. Just, with the advent of iPhone, FB, WhatsApp, Youtube, etc, a lot of people basically decided to ignore this fact of life. So good and for so long that they completely forgot it. They forgot it good enough that even their childs got no education in that regard. And every some days, someone gets a lesson. ;)
Why should Yelp not throw you under the bus for money?
The problem was that (impersonal) you stopped understanding basic mechanisms of human interaction. For no real reason.
> It seemed they were in fact encouraging me to finish the app and release it.
That was quite the assumption. They gave you access to something for free, not encouragement. I do feel bad for OP, but they weren't paying for the API, and should not have had any assumption that it would last forever because there was no contract or terms or anything. This is the risk we take by building our house on someone else's foundation.
The gist I got from my communications with them 10 years ago was that they were encouraging me to finish it.
I agree with you there was no expectation of it being free forever. I never said there was. What was unreasonable was the 4-days notice that it was coming to an end. That's just not enough time.
That statement is an outright lie too. Not only are they operating at less than 1% of what the free plan offered, but the email doesn't tell them the honest truth: That the free plan is reaching sunset, and they want all API users to switch to a paid plan.
This isn't unreasonable, but they absolutely should provide more than 1 business day notice.
Of course it's generic and perhaps not a technically correct statement. And now? Why should they spend time on writing you nicer emails? What was your value to them? Zero? Or is that too optimistic actually?
> if you utilize a third-party API for the core of your app, you are at their whim
More than that, if you aren't paying for use of that third-party API, the people who run it will not care about you, and will think nothing of shutting you down.
I think Yelp handled this poorly, and Restaurants was probably a net positive for their business -- a positive that they were getting for near-zero cost. It's a shame that companies are so short-sighted like this.
But ultimately if you build on top of someone else's platform, with no backups and no alternatives, it's not really truly your app.
Enshittification is at an all time high, and API access is being tightened and monetized in response to many industry factors including "freeloading" AI trainers.
Cory Doctorow is right, if you want to disrupt, or make any improvement to an existing large platform, adversarial interoperability(that is, reverse engineering) is the only way forward and has to be explicitly legalized in cases where it's a tool for progress.
My previous statement is arrogant, as it assumes developers are entitled to take any data they want and profit from it. It also puts services in a situation where harmful crawling like what is performed by some new AI actors with no experience is an expected thing. This is of course wrong, but I want to argue that had Yelp and other actors not wanted such a future, they shouldn't have tightened free access to their proper APIs where they have the ability to set ground rules and have the ability to talk to their users.
Big companies are amazingly bad at keeping track of things internally - a promise in an e-mail is easily forgotten 10 years later. But why should the user be punished for Yelps lack of control?
I'm sorry to read that. Unfortunately, it isn't an uncommon story.
> if you utilize a third-party API for the core of your app, you are at their whim.
That's the money quote, there. I avoid using third-party APIs like the plague. I have written backend aggregators and facias, to avoid having to use the API.
I Just. Will. Not. embed an opaque codeball into my app. I'm a cranky old bastard, I know, but I sleep well at night.
Amen. Doing the wrong thing is easy. Doing the right thing is hard. These are thermodynamically-mandated rules that cannot ever be circumvented by cleverness or money or hard work.
yelp had so much potential but they pissed it all away. Google brain raped them [1,2]. Then they never recovered from it.
Then there is ongoing issues between merchants and yelp [3]
Yelp used to be a great place to find some decent place to eat in a new city. But the platform
has gotten stale. Reviews are less reliable. Star rating often not useful.
Probably what happened was 1 support rep helped you out but did it through some undocumented backdoor to unblock you and not a formal contract. Later a completely unrelated set of employees are tasked with figuring out who the biggest API users are and to either cut them loose or get them to start paying.
Yes, this was surely somewhat tied to employee turnover and poor record keeping. But I was almost certainly not one of the "biggest API users."
The thing is, regardless of the turnover or situation, 4-days (actually 1-business day Friday->Monday) is not a reasonable timeframe to threaten to shutoff someone's API key who hasn't violated any terms of service. They have the right to do it, but it doesn't make you want to work with them in the future.
> The other thing this taught me is the danger of an up-front paid model for apps that depend on ongoing access to third-party services. If users were continually paying for the app, paying for Yelp’s APIs would not be as much of an issue. And I wouldn’t feel as guilty about the app being discontinued since if the fees were charged on a monthly basis, they would just end at the same time the app ceased to exist instead of facing an expectation upon purchase of “forever access.” On the other hand, how would you charge a monthly fee for an app that people are only willing to spend less than $5 for upfront?
There are apps that I like and would like to purchase, but paying e.g. 24 euros per year feels like too much. So I stick with the free version...
A hard lesson I learned after several times getting burned in my time long ago doing indie stuff is “never depend on a single company without a backup plan”.
Sometimes this can be pretty difficult to even see. If you were a Mac shareware dev depending on VersionTracker and MacUpdate to drive downloads, with no actual marketing budget, you might not have noticed that you were implicitly relying on Apple not creating an App Store which would turn the entire ecosystem upside down.
This is one of the reasons that it’s so tough to make a business work at a small scale. In some ways, you have more flexibility than a big company, but the lack of capital means that certain events that a larger company would shrug off can totally upend everything you’re doing.
I have seen many notices like this in the past few months. My guess is anbody of signficant size whose reason for existing includes curating user-generated content is trying to ensure that gen-AI ingestors don’t swallow up their data through unpaid API access or scraping. (Maybe a podcast or article popular with CxO and IT directors on the topic??)
Which they seemingly already have done anyway. Another unintended side-effect of the borderline illegal and generally immoral “AI” companies efforts to get as many data sets as possible.
It seems to be mostly a US phenomenon. As I pointed out in the post almost all of Restaurants sales were to the US. The directory is very up-to-date with active reviews everywhere that I've travelled in this country.
I got the same email regarding my hobby website "The Wheel of Lunch" (https://wheelof.com/lunch). The site makes no money and sends traffic to Yelp. The cheapest plan offered was $230 a month! I can't afford that.
I have stopped calling the Yelp API for local listings and put up a notice on the site. It was fun while it lasted!
We've been bitten by going the "free 3rd-party <x>" route so many times, so I sympathise with the problem here.
Off-topic, but kudos to OP for still engaging with the threads. This certainly wasn't a dump and run post. They've probably spent more time answering questions here than they did developing the app!
Yes, for sure. Some people explain it away as "well if you buy Wrangler jeans from Walmart and there's something wrong with them you ask Walmart for a refund." So if you buy X's app from Apple you ask Apple for a refund. That kind of makes sense. But if Wrangler does a recall, they have a way of getting that across to their customers.
I'd say, no you had no shared customers. Yelp had the customers, and you wrote some app that uses Yelp. For no real reason you assumed that you could be part of them (as with Facebook earlier). Sure you had some kind of informal permission at first. And then they revoked it. That's their right to do, right? Maybe you should not write Facebook or Yelp apps, if you are neither Facebook nor Yelp nor asked by them to write clients for their walled gardens. Or if you really want, set up an actual license agreement with them. Those networks are walled gardens and they don't want to cooperate with you that way. They are just not those kinds of shops which you can cuddle with. If you want to create something that has an own value, create something more than just another Faceyelptwitterddit frontend.
The good thing: You took your lesson. Maybe this time it was sufficient to actually learn sth.
Person who paid for my app also later uses the Yelp website via a link for a specific review from my app. Also a customer.
Therefore shared customers.
> Sure you had some kind of informal permission at first.
I don't know why you would define it as informal. They had an official review process for deciding what apps to grant higher daily API limits. My app went through that review process 10 years ago including considering its functionality, screenshots, and I believe I even sent them a prototype back then. We had a few emails back and forth to confirm my intentions to only develop for the Mac and what I was building.
I'm sure that got lost in the corporate shuffle over 10 years but I clearly had their permission to build the app and in fact given the relatively high API limit they gave (25000 per day versus the free at that time 10 years ago I think being 1000 per month), arguably blessing.
have you thought about writing your own API instead of using the de facto API? it may involve screen scraping but it also will be a lot more reliable and there's no limits, especially when using proxies....
I've written thousands of tools that scrape websites and never used the apis for this reason, you can never trust the API, either because of the reliability, cost, limits imposed, etc... Nobody wants you pulling data from their site anyway, so you're back to scraping anyway, its better to start out there then to have to end up there years later for some other reason...
That was a long blog post to tell us that they are no longer entitled to valuable free API access. It was a mistake to think it would last. This story has been repeated so many times here. As soon as the author started making money (they conveniently excluded their sales revs), they should have negotiated a written contract for access for a fee. It could be cut of app revs or pure API charges. Also, I am sure some smart managers looked at all their free API giveaways and decided there we no longer useful to their business model.
The author disclosed his sales revenues here: https://news.ycombinator.com/item?id=41124971 . He sold 467 copies of the software after 10 years, for a total revenue of around $2000. He never exceeded 200 API calls per day. I don't think it would have been worth Yelp's time to negotiate for a percentage of that.
Drama aside, the guy signed up for the free Yelp API 10 years ago, which has since been discontinued. He was offered the option to switch to a paid API, which he chose not to consider.
Yes, Yelp was a bit clumsy in handling this, but discontinuing the free API after 10 years is totally within their rights. The developer didn't even bother getting their pricing proposal, which might have been totally reasonable (or not), considering his app is paid.
He's didn't argue it wasn't within their rights. He called them "quite rude", which seems hard to deny.
No matter the rate Yelp set, the apps economics no longer make sense. The existing customers, already paid, and he has no way to transition them to a subscription.
I guess you didn't read my blog post because I addressed everything you wrote. The issue is not that it went paid, it's the 4-days notice. They are perfectly in their right to start charging for their API, they just can't give us 4-days notice.
Well they can do whatever timeframe they want of course. And I can write about how rude it is. 4-days (really 1-business day in the original email Friday->Monday) is not a reasonable timeframe within which to threaten to cutoff an app with real users.
That's when you take matters into your own hands and extract the API key out of one of their official apps. At least that's what I would've done.
Though I'm not sure how legal that would've been if done in a paid app. It feels like a serious difference between just providing a better UX for someone else's service through adversarial interoperability for free, and profiting off of it.
I sold 467 copies over 10 years for less than $5 per copy. It's about $2000 in revenue. Surely not a good use of my time. But I liked the app and so did its few users. We should build things we want to use ourselves right? I think it was a much nicer and faster frontend to the Yelp restaurant directory than the Yelp website.
Your other question about the cost is answered in the blog post and in several other comments I have left in this thread. In short, when I started the app 10 years ago there was no paid API (perhaps there was some kind of "enterprise" version but I'm not sure). The point of my post is how poorly they handled the transition from free->paid.
The difference, I think, is that the former are platforms to run applications while the latter are communication platforms. There used to be a time when the phone company could press charges if you connected your "app" (a tone-generating machine) to their "API" (the phone line) to "do things the official app didn't support" (place long distance calls for free). It will always be beneficial to someone who owns a communication channel to assume as much control of the pipe as possible. If Twitter could jack directly into your brain stem and refuse to work until you install its client on your wetware, it would.
Not surprising at all. Yelp, Paypal and their ilk deserve all of their earned derision for, amazingly, being simultaneously user, customer, and developer hostile.
Related to the earlier "old & grumpy" 3rd party API post, I've seen far too many, otherwise, outstanding businesses held hostage and summarily executed by either sudden un-explained usage-tier/throttling policies without economic recompense or the outright deactivation of API keys w/o notice or explanation.
When will people realize that using any big tech company API is a recipe for disaster? These companies, and their revolving door of employees, could care less about you, your app, or your users.
Same as the Reddit stunt versus the Apollo app dev, except that his app had a big enough audience you'd think they'd have figured something out.
Nope. The paid app meant money from users to the app dev instead of from advertisers to the site.
What's strange is Reddit didn't seem to do the math on how little they should have charged the app dev for API access if all they wanted was to offset revenue from that user base. Perhaps the fear was much as with TV streaming: they know advertisers want audiences willing to pay, not only the audiences seeking free.
> "People still use yelp? I thought it was widely known that they suppress bad reviews for money, and suppress good reviews if you don't pay."
A judge said that it wasn't extortion for yelp to hide good reviews until a company pays yelp, and that it wasn't extortion for yelp to remove bad reviews for companies that do pay them money though so I guess we're supposed to act like that's acceptable behavior and that their reviews can be considered trustworthy
Question-- with the yelp api now being paywalled-- and obviously you'd have to write code to handle scraping-- but can you legally scrape yelp to do this?
Does the linkedin scraping lawsuit permit scraping yelp?
I wrote a review for a grocery store turned food service shovel basically said "this will be the first store that charges for air." Guess what? I use their bathroom every other day,for free. I think it's been months since I bought anything, and I help myself to utensils.
Every customer will eventually turn anything they can use for free, into a profitable feature for themselves.
There was a book called the 1 minute manager: it was followed by a book called the 59 second employee.
Never rely on an unscupulus third party, or if a third party becomes unscrupulous... Hunt them down.
I disliked Yelp, and held it in deep distain, then while working for a service company, Yelp called. "For a fee we can remove all negative reviews." This is the definition of ensh*tification. So... I told him I would get back to him. I called a few numbers at Yelp and was basically able to socially engineer a vast list of their revinue growth supervisors.
I made a list of every business in that city and the next, and related my experence, and how worthless a Yelp review was. Almost all the business responded to poor reviews with a 'i bet you are a Yelp employee drumming up money for fake reviews.' after a few weeks, I gathered up a lot of these, and sent a three page letter of them in a package with 50 of the Yelp peoples's names in it. I let that stew for a few days, and then called the idiot back. He said "we don't care." They don't. They do not care in the least. Google, OAth, and Facebook do not either. Hurt them in their advertising review and then they will listen.
i use to be that guy telling people not to put their eggs in other peoples basket. Some laughed at me for a decade, almost convincingly and then...
Of course my stuff also breaks because you cant really do anything anymore without trusting anything even against better judgement.
I think we will eventually get expensive quality terms of service for those who think it might be fun if things work forever, like html documents and megalithic structures.
For a site that caters to a startup and entrepreneurial crowd, it's hilarious the number of comments here that amount to "tough cookies, bud" and "Yelp can do whatever they want, and because they can, you should just shut up."
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
> which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
Not everyone. Business that build on top of other company's APIs will arrange contracts with their API providers. Those contracts generally include warning periods for changes or discontinuation and penalties for early termination.
The key here is that it was a free API with no contract or guarantees. Four days is short notice and frustrating, but it wouldn't have really changed the trajectory of his business if they had given him 180 days. If he didn't intend to pay for the API, he couldn't really sell an app that was going to stop working in a few months.
So I know we're supposed to be angry about the 4 days thing. It's not good, obviously. However, I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
I hear you, but this story keeps happening over and over and over. The reality is once these companies have you and your product by the balls, they will start squeezing. You can pay money to reduce the pressure, or leave and not be squeezed. I would argue using an unpaid API takes you into the unknown with considerable risk.
Almost everyone is working on projects that “use APIs” in some general sense, sure. But I don’t think it’s the case that all or even most people are working on a project that entirely depends on a single third party’s API and is useless without it.
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users.
Or, they completely get it but they work for large platforms that leverage API access commercially or strategically, so their response to unruly peasants is to figuratively chop their heads off.
Thanks—yeah I actually think they mostly just didn't read the whole post since I addressed this in detail in the last two bold sections "Development Ends" and "Lessons Learned."
> They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
This has been known for like 20 years now. We all know that if you're relying on someone else's API that's a massive risk to your business. What more is there to say at this point? What sympathy is there to give when the inevitable happens?
I think this is the giant internet trick... do anything to scale, then when you reach critical mass (or someone wants OKRs on their status report), pull out the rug.
To be fair there wasn’t much scale. We’re talking about an app that sold 467 copies over its 10 year span. But it was an app I really liked, the people who paid for it really liked, and just drove traffic to Yelp, so they should’ve liked too. It was also a nice portfolio piece for me.
I think Yelp needs to just die at this point. Reviews are gamed, usability is worse and worse. Not sure anyone still uses or trusts Yelp. They’re just in inertia from their better times.
Is there a more open reviews platform not associated with a company? OpenReviews sort of thing or perhaps an open POI database associated with OSM, with reviews as attributes?
Somewhat silly we’re still relying on big companies to collect this data when they then gate it. The timing is right with LLMs that can make maintaining and rating reviews less burdensome on human labor (“moderation” broadly speaking). It’s just text blobs and possibly images linked to people and places.
I really hate that they force you to install the app to be able to view reviews that people have freely contributed. This is especially a problem on iOS where you cannot change your user agent so that it will serve you the desktop version.
I mean, as at least one other comment noted, anything like Yelp is going to be gamed above and beyond any particularly questionable processes of Yelp itself.
Zagat worked pretty well for "foodies" back in a day when raters had to physically mail in hardcopy questionnaires and people had to buy guidebooks in stores. I'm not sure how you duplicate that today absent physical pay-walls on both sides which I can't really see working.
I do think the various rating sites--Yelp, TripAdvisor, Google--"work" in the sense that they're probably better than picking at random. And you can't really call the local newspaper's food critic on the phone (which is what my dad used to do). So I'm not sure what the alternative really is. Yes, there's some word of mouth but that's pretty random.
Yep I specifically keep Google Maps around to find restaurants even though I far prefer the audible navigation from Apple Maps nowadays once I actually want to drive there.
I've noticed that Apple Maps incorporated their own rating system some time ago (I want to say within the last one or two years); it's simple, just asking you to give a thumbs up or thumbs down on certain businesses, sometimes with a bit of granularity (e.g., rate the atmosphere, food, value, and service separately). There appears to be some threshold for how many ratings they have when they switch from displaying Yelp reviews to their own stuff, although it's not clear what the heuristic is.
Anyway, tl;dr: I think they're working on replacing Yelp.
Hang on, they didn't give his app 4 days - they gave him 4 days to respond with some screenshots, and if he didn't respond, they would shut down his access. They didn't say they'll shut it down in 4 days regardless.
I mean, don't take this as me defending Yelp - they're scumbags, and deserve any hate coming their way - but I don't think that the headline is an accurate description of what happened.
I did respond with the required information and they still shut down my access (10-days post though, not 4-days in the end). But regardless, no, I don't think sending you a request on a Friday for information due on a Monday or shutting down all of your access is reasonable. As I mentioned in my post, what if I had been on vacation? 1-business day or shut you down is just not a reasonable time frame to make anyone want to work with you in the future.
Which seems to indicate a base price of $229 per month. I have no idea why that doesn't line up with the pricing on their website. But the fact that they shared it with me indicated to me that my use case fell under some kind of "enterprise" usage. Regardless, I would not continue to work with them after 4-days notice and the threatening email. It's too small and app for it to be worth it to me.
I did ask in one of my emails about the pricing discrepancy and got no reply.
A dollar a day is $30/month, and at $1.99/sale that means OP would need to sell 15 apps ONLY to break even, which given the comments in the blog might be close to reality. This is pennies for both sides, but the way that Yelp distastefully contacted OP is probably worth just shutting the whole thing, it's not like OP was making bank on the app to begin with.
Throw on taxes, administrative overheads, etc, they are probably looking at 30-45 sales per month. Which is likely not realistic.
On top of that, this is a continuous payment. Even if I was looking at 5-10x rate of return, I would be very hesitant as that's the rate-of-return today while the sales are forever.
I've been wondering how realistic microsubscriptions are... Say $1-2 dollars a month per user to maintain an app, perhaps limited to just power users, would support a lot of infrastructure.
I got the same email. Despite my hobby project (a random food picker) having been broken and not used for years (because of yelp API updates), they also told me my usage was higher than other developers...
The email also arrived in my spam folder, so I was lucky to even see it. Once I got back to them they did increase the cutoff by a few days but it has since been stopped.
Their new prices seemed insane to me.
“they also told me my usage was higher than other developers...”
Next you’ll tell me that they’re NOT actually experiencing higher-than-normal call volumes!
"We've understaffed our call center" just doesn't have the same ring to it.
The prices in the deck they shared with me are on slide 3 if anyone is interested: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
They never answered my question about the discrepancy between these prices and the prices on their website. Both would be too high to make Restaurants low sales sustainable.
That said, I'll say again that they were perfectly in their right to start charging for their API. They just should not have done so with 4-days notice and with such a threatening email.
$10 per request at the lowest tier of enhanced?! The only business case I can think of for this is if you save all the data in your own database then serve requests for your own service out of your own cache. I guess if the Yelp data doesn't change very often, then you just choose to stay 6-months behind on the data or something...
No, it's $10 per thousand on average (you get 1,000 calls per day) with a $299/month minimum commitment by my reading.
> Their new prices seemed insane to me.
I call this the "going out of business sale".
The death spiral.
https://steveshuconsulting.com/2006/03/the_pricing_dea/
Interesting and an excellent read. The death spiral at Travelocity (pre-Expedia acquisition) worked differently and was partially self-induced.
Some background is required first. Travelocity was the first online travel agency (OTA). I believe they started around 1996.
Through most of its life at that point all, I mean 95%+, of its success came from only two factors: best marketing in industry and growth of the internet. This is severely problematic because marketing only gets you so far in business. Conversely Expedia had really shitty marketing in comparison and yet came to dominate the industry because they were extremely aggressive at growing their supplier relations.
You cannot EVER rely on growth of media adoption, like mom and pop coming online, because once that stops you have no fuel left in the tank. Reliance on growth of a media platform is like a gravy train that you did nothing to build and returns amazing wealth if you are in the right place at the right time, but once it stagnates its like your train derails and everybody dies. That is because everybody expects growth to continue, except you did nothing to earn the growth and now have no answers and nothing to show for it. This was around later 2008 when I joined the company and became unavoidably obvious to everybody over the next year.
So, at that point what do you do? You competitors are far out pacing you by ignoring fun stuff, marketing, technology, and all the other bullshit that technology people look to. Instead they are focusing on core business principles and eating your golden goose while laughing at you. So, what do you do?
In the case of Travelocity all the executives leave. New executives come into trying to figure out what to do. Like every great web business they focus more aggressively on marketing and advertising. This was Travelocity's death spiral.
You have to understand that people DO NOT like advertisements. Really, I know its surprising, but when your site becomes littered with advertisements everywhere and all kinds of hidden telemetry people will leave and never come back. Your wonderful palace has become a trailer park.
The business loves advertising. Revenue from advertisements is immediate. That is really significant. In e-commerce there is a massive lag between each stage of profit, revenue, and sales because you have to account for the cost of operations, sales, and inventory. The more expensive the product the longer the lag and that lag really complicates projections. So advertisements are like cocaine, because they immediately return profit that requires no effort while rotting your health slowly until you are a hollow skeleton.
To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
My learning from this is that a business that earns profit from selling something directly should not fuck up conversion or go out of its way to make customers hate them. When I put that way it sounds obvious, but web business get that wrong all the time because they get distracted by shiny things.
>To be fair they were doing amazing things with inventory and pricing that was vastly superior to what the competition was doing after the leadership turn over. This was too little too late though. These innovations could have saved the business provided more time and the same level of discipline, but not when you are already in a death spiral.
I am curious what these amazing things were.
So in the travel industry you have limited lines of business, primarily: air, hotels, and car rentals. Air is super low margin because its set directly by the supplier plus some arbitrary service fee, typically $7 per ticket. Air is high volume though and is the primary draw for OTA customers, so it cannot be ignored. Rental cars are super low priority because they are both low margin and low volume.
The real interest in the travel agency business is on hotel inventory. This is how Expedia came to dominate the industry, because they aggressively hired geographically focused hotel relationship personnel and Travelocity wasn't keeping up. So Expedia had much greater inventory from various different properties and stronger relationships with those properties for their business prioritization. Hotels, typically hate dealing with OTAs, because OTAs are a transparent barrier between the hotel and the guest staying at the hotel, but the OTAs bring people to the properties. The name of the game for hotels is "heads in beds" and while up-selling is beneficial if you get the "heads in beds" part wrong you cannot exist.
The margin on hotel properties is much higher and highly variable. Hotels in many cases will even eat the cost of airfare to get heads in beds if the pricing is right. That provides a huge opportunity for a marriage between smart hotel systems and OTAs because the hotels are doing the smarter inventory management and the OTAs are supplying the airfare volume and customers to populate that smart inventory management. The hotel data systems, Property Management Systems (PMS), were at that time much smarter than the inventory availability OTAs had, because they had to do much more to account for seasonal volume planning and various customer demands.
Where Travelocity was strong was in writing algorithms to account for these considerations and offer volume pricing discounts the competition could not and also air + hotel package pricing at massive discounts the competition could not compete with. In parallel Travelocity was slowly building up a corporate travel business to take advantage of that volume pricing intelligence and at one point had both WalMart and Lockheed-Martin as customers.
Again, both of those were amazing and could have really helped retake lost market, but conquering competition is a fragile slow process while advertising is immediate.
Thank you for this comment. Very interesting! And very applicable to everything we are doing again.
Thank you for the link, nicely explained!
Or they want to get the traffic back directly and overprice on purpose.
Doesn’t Google own yelp?
No, it is a publicly listed company:
https://www.macrotrends.net/stocks/charts/YELP/yelp/net-inco...
Been going sideways for a decade now though:
https://www.macrotrends.net/stocks/charts/YELP/yelp/market-c...
Doesn't seem like they do. They attempted to acquire them back in 2009 but it didn't work out.
> they also told me my usage was higher than other developers...
I'm curious to what they compare this. The non-existent free plan where everything higher than 0 is unusual. Or a large mass of dead accounts who have really zero to none usage. Or the actually paying account who are now on their enterprise plan. In the later case, this might indicate some real problem if their usage is low enough that smaller projects are already exceptional.
Author here. I actually discovered something in their terms of use today while looking up something else that someone had asked me in this thread:
"2. CHANGES Yelp reserves the right to modify the API Agreement at any time. If Yelp reasonably determines that a modification may materially and adversely impact You, Yelp will provide email notice to you using the email address you provided during registration no less than ten (10) days prior to the material adverse modification taking affect. IF YOU DO NOT WISH TO BE BOUND TO ANY NEW OR MODIFIED TERMS, YOU MUST TERMINATE THE API AGREEMENT BY CEASING USE OF THE API WITHIN TEN (10) DAYS OF RECEIPT OF SUCH NOTICE."
So, the 4-days would have violated their original notification terms of use anyway. I think they realized this after the fact of sending all of us the threatening emails and this is why they ultimately closed my API key after 10-days as described in my original post.
Anyway, just an interesting aside that the poorly written, inaccurate original email that I shared in my original post was even inaccurate to their own policies.
"YOU MUST TERMINATE" rather than be held to the original terms you initially signed up for ... because Yelp RESERVES the right.
We have all now been reminded that RESERVED RIGHTS can and will be used against you maliciously. And if you want protection from such malice, there needs to be a RIGHTS of the USER clause.
This should be handled and enforced by government statutes.
(It is an amazing world we live in where rights of the user have to be expressed.)
10 days.
How very generous of them.
Seen this story play out so many times. I audited a company years ago that claimed to have excellent, personal almost, relationship with Google and all needed paperwork to use their platform as a core of their business. They went bust 6 months after because of "unexpected" change of Google's product strategy.
Real lesson here is to avoid single points of failure, regardless if it's API, people or partners. Ask yourself a question if there's a single entity that can kill your business and remove that reliance.
If you're depending on the continued existence of any Google product beyond GMail, Google Cloud and advertising you should really consider fallback positions. The existence of Youtube is also guaranteed, but the existence of any particular API or service that you might depend on? Not so much. Hell they'd probably consider killing GMail if they didn't need the accounts to tie advertising impressions to individuals.
Not even search makes the cut?
Search is already dying. Google doesn't need it.
Well, they pay $25B+ per year to be the default search engine on various platforms, so that seems unlikely in the near term.
(https://www.cnbc.com/amp/2023/10/27/google-paid-26-billion-i...)
That's a ton of money, (and I've heard that they paid Apple even more last year) but it's kind of understandable. Apple customers don't usually have android phones snitching on them, are less likely to use chrome, and tend to have their devices and data in the Apple ecosystem. Google has fewer ways to spy on Apple user's lives and push ads at them. Search, gmail, and youtube probably give them the best opportunities. I wouldn't doubt that Apple users are getting as frustrated with Google's search engine as everyone else is though.
Do you have a source for this? I suspect search makes at least 75% of Google’s profits.
see https://news.ycombinator.com/item?id=30347719
Google used to need Search in order to learn about the intimate details of our lives. It's how they knew what we were interested in, what our medical problems were, what we were learning, and what we were thinking about.
Now they have chrome giving google people's entire browsing history, android devices collecting realtime data on what people are doing offline, where they are and who they are with. Google also has products like fitbit, nest, and gmail that gather still more data for google. Google doesn't need Search to spy on us anymore, so they haven't invested in keeping it useful.
In fact, it's better for google if you can't find what you want and have to make multiple searches for information because it gives them more chances to throw ads at you, and the harder it is for people to find websites using Search the more sites might feel like they have no choice but to pay Google to keep them at the top of the search results.
People who find Search increasingly useless though are turning to alternatives and for many people AI could end up replacing google's Search product as the first thing they turn to.
I would not count on Search staying around forever.
Given how management is behaving lately, I wouldn't rule out any "shooting yourself in the foot" moves by them
That is only half-true. On the one hand, I agree and I would never build a serious business on another single entity. But on the other hand, OP built an app that ran successfully for 10 years, generating some revenue.
Most products have an expected expiration date and you can provide a useful service (that generates revenue) by building on other platforms, even if it won't last forever.
By now I missed more opportunities by having the mindset of "not relying on other people's APIs" than real changes/shutdowns would warrant this kind of caution.
It's so sad that we're at such a devoid barren state.
There was so much hype & excitement in web2 age, about APIs & intertwingularity. Humanity was going to keep improving & iterating & crafting ever neater ideas.
Now it's all bureaucracy & lawyers arrayed to defend the cloud-keeps, moats & walls.
The user & agency of the world has been steady dimished, eroded away by ever advancing legal IP Protectionism.
That's not going to work in practice as money and time are limited. Choose your partners wisely but don't obsess over the scenario of them going bankrupt.
I built a service around helping podcasters automatically convert their audio podcast into a YouTube channel. I went through tons of review with Google in order to get access to the YouTube API and make sure everything I was doing was in compliance with their terms - literally months of back and forth. I had been testing in my development and staging environments against their API for 6+ months. I launched in production, got a few videos uploaded to YouTube, and they disabled my API key. I spent months emailing them and never got anything more than the same boilerplate copy/pasted answer. I could have pivoted or something, but I just shut it down and moved on. Lesson learned.
I used to not bother with APIs and instead use selenium to drive the websites. It neatly avoided all of the politics around API access.
HTTP is an API of sorts…
> HTTP is an API of sorts…
True, but it is an API that they can't easily deprecate on a whim.
But they can quickly change the "structure" of that API.
I worked for a very early billpay company where you could pay your bills online to vendors, even if the vendor didn't support it. We used API's where we could, but where we couldn't...
We had a whole team dedicated to keeping up the changes vendors would make to their websites that we scraped for info. The team was called, of course, "Scrape and P(r)ay".
I believe this is where smarter tools like Kadoa [0] can be of help. It detects data structure changes for existing workflows, and adapts to them.
[0] https://www.kadoa.com/
A small problem compared to not having an API key and being stonewalled as to why.
If you build your scraper to find data on the page based on the shape of the data itself instead of the structure of the page then it will be resilient to most changes that don't materially change what data is displayed on the page.
So, prefer regex over css selectors, and css selectors over xpath, where possible. And don't select based on nesting or position if possible.
Isn't that similar to deleting your API key except you can at least fix the structure of the selenium one.
You're one CAPTCHA added to the flow from this access being effectively deprecated.
In the AI era, is this as true?
That may be a CFAA violation, a felony in the United States. See Ryanair DAC v. Booking Holdings, Inc.
That's because Booking was also committing some type of misrepresentation and taking revenue away from Ryanair through their browser automation. Even then, the infraction was sooo bad that they got a $5k fine.
I think the hiQ Labs vs LinkedIn case is a better representation that scraping is generally allowed: https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
> In November 2022 the U.S. District Court for the Northern District of California ruled that hiQ had breached LinkedIn's User Agreement and a settlement agreement was reached between the two parties.
Reminder that the earlier ruling was overturned, it is no longer clear whether scraping is legal or not.
When it comes to CFAA violations, corporations get a $5K fine, while individuals get hounded to suicide.
Start a company then do my nefarious work under that, got it!
Just say you train an AI on it and it's all fair game.
Good thing I am not an American, don't live in the US and have no intentions of visiting then.
(I gave you an upvote, I don't think your comment deserve to be nuked for bringing up a possible legal risk)
Very Probably No way they could find me.
My first Saas was built around Facebook, Twitter and Reddits API.
Facebook broke like monthly and required random updates, sometimes documented, sometimes not. Zero support.
Twitter worker flawlessly until I hit some limits and there was no way to increase them because I didn't hit some other limits. No way they would talk to me.
Reddit just worked until I gave up.
What I learned is to never again do any business based on someone else.
A good lesson, but a more nuanced one is to do sure diligence. There are companies I would never build my business around (Google, Oracle, etc.) for a variety of reasons. There are ones I know I can trust (at least until they change). There are many where I need a contract.
Change is also a constant, and there are plenty of good companies give bad, and a few in reverse. So due diligence is an ongoing process.
That dramatically raises the price of SaaS, so I use it much less than mainstream industry practice, but much more than zero.
> What I learned is to never again do any business based on someone else.
That's a little dramatic, don't you think?
Supply chains are a core of many (most?) businesses out there. They depend on others to get their business done. No supply -> no business.
What you probably meant is not building a business based SOLELY on irreplaceable business relationships. Unfortunately in a lot of cases involving the tech giants and little startups, there are no alternatives.
More or less, however I really try to not build on anyone else ever again.
I can't find a single dependency in my serious money generating chain right now that isn't replaceable with a similar product within a few minutes or hours if necessary.
My most recent project didn't even last a full year as there was no replacement that worked for me for ChatGPT in that moment. It was built to die, a gamble essentially. I wouldn't waste my time for things like this if I didn't had the end calculated in already.
Same experience, slightly different. It made me realise why some parts of an MBA (e.g. strategic place/partnership/spring boarding from a brand) actually have value I never saw before for tech companies. There are essentially king makers in certain fields - either by their apathy at first-movers, or their choice on who they allow to continue.
>Lesson learned.
And that lesson is don't base your business model around playing in someone else's waled garden.
Precisely that.
Funnily, if I understood correctly, he already had that lesson with Facebook. Let's see what comes next...
Their loss of additional content, traffic and ad revenue.
The impact on google losing $10m of revenue is exactly equal to the impact of John Doe losing $10m of revenue
In a competitive market John Doe would go to one of googles competitors. We don't have a competitive market. We have winner takes all market.
Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul. Reddit, Twitter, and company all got big because they had open APIs and people were able to use them extensively for really creative things.
I agree, Google has swallowed up the video streaming market unfortunately.
I keep thinking back to how Vine was basically TikTok, and they threw it away.
Take it from someone who watched with interest, Reddit and Twitter did not get big due to open APIs. They got their first big steps before they even had APIs... In the first year of twitters launch it was in the news in the UK at 300k users in press releases about how it was a "new form of communication" etc... 2008 was actually when the original set of API-laden websites began to fade because with the recession, we could not have nice things be free so much. Many API services fell into disrepair in 2009, certainly the peak of APIs for the UK was around 2008. Check out Tom Scotts video on this for a picture of the open API internet many people thought would occur (and which did exist in some places until FAANG began to productise and dominate and use accounts across products which do not incentivize APIs). Yes it is true open network graphs/facebook etc had some API access, but the APIs before were more single purpose and numerous; facebook lets you explore their network and to some extent data, but that's not the same as the useful APIs that became restricted, paid or were cut in the aftermath of the recession.
Those startups are their competitors. You don’t have to pay millions of dollars to acquire a competitor that was never started because of your inconsistent API policy.
Google makes enough money that losing several currently non-existent revenue streams that are theoretically $10 million apiece isn’t hurting them. It’s hurting their users.
Google is all about ads - why would they give a shit about the users?
>Sure, its $10m today, but think of all the other potential startups that they're doing this to, and how many millions they're throwing away, and billions over the long haul.
Google made $300B last year. "billions over the long haul" is a lot of money, an unimaginable amount even, to you and I. But to Google?
> Google made $300B last year.
No, Google made $73.795B last year. Revenue is not profit, net income is profit and what you ‘make’ as a company.
Revenue is not profit.
It's a testimate to the health of our free market that the company throwing away millions, and billions over the long haul, is the overwhelmingly dominant market leader.
Attempted to build a similar thing a few years ago when living in Tokyo.
Provide a selection of restaurants within a 1km radius and automagically provide 3 recommendations based on my preferences.
We had multiple API providers available, Tabelog, Gurunavi and Hotpepper, all required a paid developer license. We still needed to use the Google maps API to get the user's current location though.
This was also just when Google Maps suddenly raised their API pricing. After spending a couple of weekends building a working prototype, we stopped as we couldn't justify the cost of paying multiple API providers for basically 3 guys looking to save 10 mins deciding where to get lunch.
Also, this kind of app is a common theme if you frequent meetups in Tokyo. There's always at least 1 person that has built such a thing.
> We still needed to use the Google maps API to get the user's current location though.
I’m sure as somebody who actually built a prototype, you understand the situation better than me, but I’m curious: why would just getting the user’s lat/long require anything you’d pay for? Wouldn’t the OS or browser provide that to you?
Oops. I meant that we were using the maps/places API to get lat/long of the restaurants using the street address from the other API providers, then using that to calculate a 'N mins walk away'.
We were just testing the abilities of the various APIs. Google was the only that provided travelling time estimates.
I was thinking about how to solve this given that one of the primary problems is that of fast, global content distribution. I like the idea of paying people in crypto as part of a ledger transaction to host and serve content, like bittorrent with a crypto payment. Unfortunately I can't also think of a way to prevent such a system from being abused to distribute harmful media such as CP. I guess it's not like this isn't a problem with BitTorrent today though.
Regulation might here - something like minimum mandatory 3 months notice for shutting down the API keys. Considering the average age of our politicians, I doubt they’ll understand what “API” is, much less be willing to take on giant tech corporations even if they did understand the problems
> […] much less be willing to take on giant tech corporations […]
I’d argue that, on the contrary, it seems to be all the rage lately.
Competitive markets are by far more the exception than the rule. It's just not how capitalism works (because pretty much any competitive market is ripe for “consolidation”, which increases aggregate shareholders value by reducing competitive pressure).
If you want competitive market in a capitalist economy, then you need very active state enforcement.
The markets are absolutely competitive. Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
They are now hemorrhaging search to OpenAI that popped into public existence just recently.
To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
If you wanna see lack of competition, look at government granted monopolies on utilities. Guaranteed but capped rates means you reduce investment right to $0 and cut costs as much as possible since there is no other way to make money. That “state enforcement” you are calling for is how you end up with PG&E and scenarios like all insurances companies pulling out of the state.
> Google has to pour money into youtube to keep it actively developed and popular. Yet they still lose live streamers to Twitch, video uploads to all kinds of niche specific platforms, and their paid content isn’t coming close to things like Netflix, max, prime, etc.
Phew, here I was stressing out about lack of competition, but you helped me relax.
Google has to invest into Youtube lest it loses out to Twitch (Amazon), Netflix ($35bn per year), HBO Max (Warner Bros, $50bn per year), (Amazon) Prime, Plus (Disney, $90bn per year).
My faith in humanity is restored now that there are alternatives.
From other MegaCorps.
> To claim there isn’t competition in these markets is completely ignorant. “A big player eats 80%” isn’t anything like a monopoly/duopoly scenario where there literally isn’t any competition or product advancement for decades.
Please tell me how many “product advancement” in Google search or YouTube over the past decade… From a consumer's perspective all that happened was enshitification and despite being owned by a search engine company, YouTube never managed to ship a functional search on their platform.
> If you wanna see lack of competition, look at government granted monopolies on utilities.
Utilities are “natural monopolies” though, and as such they should be state owned. Making a natural monopoly owned for profit is a recipe for rent seeking, and that's why it was promoted …
> That “state enforcement” you are calling for
No, the state enforcement I'm calling for is proper enforcement of antitrust laws, forbidding consolidation through M&A and disbanding companies megacorps. That is to say, what existed in US's golden age.
>Competitive markets are by far more the exception than the rule it's just not how capitalism works
Empirically this absolutely isn't the case; the majority of listed companies have fairly low margins, especially non-tech companies, which can be trivially seen from their financial statements. A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
> A low profit margin means a competitive market (because if it wasn't a competitive market the firm could raise its prices to obtain higher margins).
Not necessarily: if prices are elastic then even a monopoly can aim for low profit margin (in percentage) in order to increase profit. What matters is how much total profit are being made, margin only matters when measuring risk.
Also, corporations are social structures, and low competition also encourage complacency in the corporate structure itself, which drives costs up and reduces profit margin.
Might there be other reasons to report lower profits?
Cool idea, I was almost about to do the same app/service. I didn't know it could end like that. (Naive, I know.)
Have you seen https://repurpose.io? They existed before I started working on my service and they do the same thing.
Do you know if there is any open source tools in this field that a programmer can work with/automate, I have mainly textual and graphic content, a little bit of video, but thinking to branch out to podcasts and music soon, unfortunately the other products supporting text seem pricey and geared to big teams.
on edit: obviously I can write my own, but I am hoping for a project that has already done a bunch of things that I can extend for my needs, as writing my own would be at the point where it would be more cost benefit to purchase.
I run a webapp that uses the YouTube API too.
I believe I am entirely in compliance with YouTube's terms and policies, but I was flagged for some bullshit as well and got the boilerplate responses. So what I decided to do was hide it for logged out users so it looked like that specific feature was entirely removed to the dudes in India that review this stuff... One year later and my fingers are crossed we don't get another audit.
We went through a three-week Facebook API shutdown due to a clear glitch - Meta support couldn't go outside the script, or escalate in any way beyond "a supervisor will email you". Only resolved when our CEO found some VP at Facebook on LinkedIn and got them to escalate it internally.
(The dev community thread is full of people still impacted, so I think they literally just edited our app's flags directly.)
Even tried to invoke GDPR's rights to be exempt from automatic decision making, but their privacy questions email address responds with "nope fuck off" to those.
Basically any hope of solving a problem with a Meta property is to know employees who can escalate your issue. Sometimes your friend at Meta also needs to argue with the bug owner to not close it frivolously.
It's the same with Google.
Within the customer service industry, they'd claim a great "deflection rate". That's a metric that all of these large companies hold around any kind of help/support channel that may involve humans because people are a cost. It's often covered by some kind of satisfaction metric based off post-issue surveys, but fundamentally, if you just go away that's success.
There are often complaints here about what amounts to bean counting affecting other aspects of business. Customer service at larger scale is costly so attracts a section of very analytical leaders. They don't get, or ever prioritize, the human elements. It's only when satisfaction numbers are bad, or another exec outside of customer service takes action, that things improve.
The scale part kills the customer. You can have great support at one size. Once you grow then leadership, structure, and the culture change. These analytical leaders don't want to carry over the culture and structure because it comes at a cost. It really needs force and support from outside of customer support leadership to maintain it.
Had this issue with Ubisoft recently. Installed Trackmania through Steam for free, then had to create a Ubisoft account, then had to buy the Trackmania annual pass for $20. I guess I followed the wrong flow or something, but I ended up with the purchase on my Ubisoft account, but I guess their system also auto-creates a Ubisoft account for Steam accounts? So the purchase on the Ubisoft account ended up on the account not tied to Steam.
Emailed support about it on 13/07/2024.
Got back a reply on 23/07/2024:
> Our Support Team is currently experiencing high case volumes, so we are reaching out with a message to check-in and make sure you still need help!
> If you've found a solution to your problem already, there's nothing you need to do. Just ignore this message and your case will automatically close in 2 days.
Didn't see the reply until the 3rd day.
> As we have not heard from you in the last 2 days your case has been closed automatically.
Excellent customer experience.
> Customer service at larger scale is costly
I think people, even on HN, grossly underestimate how much customer service costs at any major company that works with the public.
Most calls to customer service are easily self-serviceable. My brother used to work for X-Box Live support, and he said over half his calls were for password resets, something people could easily do on their X-Box.
But then there's bad actors. Another significant chunk of his calls were from people complaining about being banned, and he'd see the reports of their spamming racial slurs.
Not to mention how many people try to abuse customer support in attempts to take over someone else's account.
Yeah...customer service is expensive. Even if you automate as much as you can, but offer an appeal process, that appeal process with be absolutely abused by bad actors, so that will end up automated as well.
I've heard that serving them process is another way to talk to a human being.
My sympathy is veeery limited. But once you have an app that does the exact opposite, please tell me!!
And, just in that particular case: I'm veery okay if that would be a cloud service.
YouTube to podcast is way easier than the other way around. I'm sure solutions already exist.
Are you based in EU? If yes, then I would a complaint letter to the competition bureau. And you can CC Google offices in your country.
I am curious if you would be interested to retry the idea? I might have an in with the YouTube team. I feel like it's a shame to let this go. Would you be open to chat? Please reach out r@pehul.com
If only we weren't living in a world run by abusive mafia... o_O
open source your code ?
Couldn't you have bypassed them and used `yt-dlp` or something similar?
Why get Google/YouTube's permission at all about this?
I was creating YouTube channels and uploading videos to them on behalf of my users. That requires using the YouTube API.
Can you let the user bring their own API key?
No. The default API quota is not large enough to upload videos. You would have to contact Google, explain your use case, and jump through several hoops to get the quota increased... which is a huge process.
I once tried using a Python library to upload videos with an API key. All videos uploaded got forced private due to "spam".
Yeah. Uploading legit videos is non-trivial. And if you ever upload the same video twice (which you think you might do during testing, right?), it's a violation of their terms and they disable your access.
reread: The project was going in the other direction, From a pure audio source to a Youtube video.
I think Yelp is shutting off their API for the same reason as Reddit: to ensure that AI training makes money for them. It sucks that you’re a drive-by casualty of that, and if I’d bought this app, then - same as Apollo iOS - I would not request a refund.
I think nobody wants an API. The same reason why YouTube plays cat-and-mouse with yt-dl and longs for the sweet nectar of Widevine DRM
To wit, there is exactly one business strategy: When you are small, be nice. When you are big, pull that ladder up behind you.
YouTube already uses Widevine for movies. they will never use it for regular videos, because it will kill their website. widevine is a resource killer, for both client and server.
It's just more of Cory Doctrow's observation of Enshittification.
Or, perhaps more simply, The Oatmeal's Reaching People on the Internet: https://theoatmeal.com/comics/reaching_people
> To wit, there is exactly one business strategy: When you are small, be nice. When you are big, pull that ladder up behind you.
I'd add a "in venture capital and big capital" after "business strategy". When you don't have VCs or the stonk market breathing down your neck all the time, incentives massively change.
IMHO, there is only one solution left... once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust that has a clear mandate to run its companies in a way beneficial to society at large. That way the government doesn't have to spend taxpayer money on countless r&d experiments, VC investors have a perspective to payout, and the world gets kept from utter bullshit like API games.
Well if small capital doesn't pay good, I have no incentive to run a small business at all!
> once a publicly traded company gains critical market dominance and is reasonably profitable, the government buys all shares at the current market price and places the company in a public-good trust
Interesting, haven't heard that one. I'll say maybe. I'd rather have Facebook not exist at all than have it be an *official* arm of the NSA
> the government doesn't have to spend taxpayer money on countless r&d experiments
If you expect the government to make a tender offer on the largest market-cap companies, they sure as heck _are_ paying for r&d indirectly. And a lot more!
Isn't that kind of what they do with QE?
I think it's quite different. It involves money, but QE is all about monetary supply and interest rates alone. Buying entire companies comes with a lot more implications, like decision-making power over the organization. You want that for API keys, but do you want that across the board? Seems like regulation would be much cheaper.
[flagged]
Why? The problem at the core is the mantra of the WSB et al groupthink: stonk only go up.
Of course when WSB makes jokes about it and catapults a junk stock like GME to ridiculous heights, squeezing shortsellers dry, it's pretty funny to watch. But it gets problematic when just about everyone's pension is tied to "stonks go up" as well. That leads to all sorts of perverse incentives.
Just look at Boeing and how far they've fallen. Twice in recent years, literally and not just proverbially, with hundreds dead as the inevitable consequence of stonk market games.
I think the GP had a visceral reaction to your intentional misspelling of "stock" as "stonk".
I tend to agree, though I don't think I'd go so far as to think someone saying "stonk" has "lost all credibility"... but to me, it's like Republicans deliberately mispronouncing Kamala Harris' first name, because they can, and because their base eats it up. It's a cheap shot, and it feels childish, like you don't have a real argument, so instead you act like an elementary school kid on the playground calling the other kid names.
This has been one [unintended?] consequence of AI promulgation. A direct disincentive toward the kind of open access that so used to be common and provided a lot of low-hanging fruit for independent developers trying to increase interoperability within their favorite niches.
So now not only is AI filling the web with garbage that poisons future model development[1], it provides incentive to further close and wall off access to (user-provided!) data.
1. https://www.nature.com/articles/s41586-024-07566-y
Yes. We're entering a new era of web applications, whether we want to or not. Companies used to be able to gate their data behind UIs, because the search results had more value than the raw data.
Things are starting to flip. Increasingly, people would rather have access to the raw dataset to pipe into an LLM. The question, as always, is how to control this access and charge people accordingly.
Reminds me of this video about the old, open web:
https://www.youtube.com/watch?v=BxV14h0kFs0
I was just wondering why - for multiple screens of text - nobody came and just posted some pointless youtube link.
But this case seems to be funny at least (acc to your text): A YT link about the 'old, open web'. YMMD!
I think you're right. API strategies are being reviewed because AI training data.
That said... I think API policies are also vestiges of a defunct idea set that prevailed circa 2009.
At that time, "twitter is a protocol." APIs and openness were going to be the new worldwideweb. Social media was going to be an ecosystem of independent apps and services with complex interactions mitigated by protocols.
YC even did an "rfs" recruiting startups to build off the twitter API.
That didn't happen, but APIs intended to enable it were already in the wild.
As API access becomes strategic again (because AI), "why do we even have this?" is likely to come up.
Why would anyone ever want to train an AI off Yelp reviews? That sounds horrifying.
Local recommendations is a big category of LLM questions that they're mostly bad at today that they can sell ads for in the future.
I think the general lack of willingness to help in Big Tech is very problematic. You can almost never get through those thick-skulled reps that email you out of the blue...
The problem is that by having a thick-skulled rep walk in and send a few e-mails, Yelp has already lost more money on API customer acquisition than the app developer was willing to pay. That's why these APIs had free tiers: they covered these kinds of micro-usages that would be far too cheap for a sales rep to cover.
The reason why those free tiers went away is that AI came along. Not so much that the AI scrapers were abusing Yelp free tier[0], but that they could. And once companies realized how much money was floating around in selling data access, non-abusive free tier users went from "a cool goodwill gesture" to "freeloading parasites".
David Kopec and Restauraunts got steamrolled in a case of technological gentrification. If you're selling data access for $TOO_CHEAP_TO_METER/call to a random indie, Apple, Google, and/or Microsoft will use that as a comparable for why Yelp should charge peanuts. Or they'll just acquihire him. They need him and his app to go away because he is inconvenient to the long-term valuation plan of Yelp, an old guard Web 2.0 business that never quite became sovereign.
[0] Though, to be clear, AI scrapers are absolutely abusive in general.
One of the things I most admire and also loathe about Big Tech is how much they normalized nonexistent customer service.
A couple years ago, I was confused about why my Apple TV remote was behaving weirdly, and a few taps in the support app and I had a real person call me and fix my issue within minutes.
I was looking for a completely random Molex adapter from the early 1980's for a hobby project. After reaching out to the company they had someone call me and attempt to help. They didn't find the part (since it hasn't existed for 30 years) but they did give me some good leads.
Amazon’s AWS has good customer service. I get good customer service at the Apple Store, and I also got good customer service when Microsoft had retail stores. I have used Azure’s customer service and it was OK. Amazon’s AWS is know for outstanding customer service. My main point is some big tech companies have terrible service. Some don’t.
github support is nil in my experiences
Par for the course for any product or service owned by Microsoft.
on the other hand, gitlab support has impressed me a few times with the speed of the response and the knowledge level of the support techs.
Maybe if you can initiate a chargeback, you get customer service.
I have never had to initiate a chargeback to get customer service.
I meant, if you have the ability to initiate a chargeback (pay money, and have the ability to take it back out of their control for bad service), then they will probably provide customer service.
Hey now, don't lump Apple into this.
I'd like to see how someone can see a directly employed Google support agent / repair technician in person for support like you can with the Genius Bar.
Additionally, as far as I know, the kind of support you get calling AppleCare is unmatched by anyone else.
> You can almost never get through those thick-skulled reps that email you out of the blue...
He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
What more would you want the rep to do?
> He did get through to the sales rep. The responses are directly in the article. The sales rep responded within hours and showed him how to sign up for the free trial option to extend the free usage period longer while he decided.
> What more would you want the rep to do?
Not send a threatening, inaccurate email with a 4-day (1 business day Friday->Monday) deadline in the first place?
Also for the record, this was not a sales manager. This was someone in "growth" that sent the email. I guess maybe that just means "sales."
The email wasn't inaccurate: They really were going to shut down his API key and then they did.
> Your API usage is higher than lots of other Yelp Fusion developers
You're right it wasn't just inaccurate it was outright deceptive, trying to frame this as something specific to OP that he is responsible for rather than a change in their business applying to all API users.
> What more would you want the rep to do?
Employ a shred of critical thinking and realize this app probably drives more value to Yelp than the cost to run the API, and flag it for an exception.
Customer service or sales people like that are usually at the lowest point on the totem pole. It’s likely that they have zero leeway, either you’re gonna pay exorbitantly or you’re out. It’s an institutional problem.
In that case it's not customer service but a customer firewall. If the CS employees are not empowered to induce solutions even if they benefit both the customers and the company then they are literally useless.
And yes, many companies have "customer service" like that designed to waste your time until you go away. That doesn't mean it's the only possible way.
Completely agree with you.
This is a strategy, like quiet firing, to make people go away.
Rather than being up front, this is an underhanded way of making folks leave.
But Yelp is not big tech, certainly not by market cap. It could be old tech, in the way that a 2010-era data API may have been a growth hack intended to create an app ecosystem. But 10 years later, there’s no app ecosystem and the company can’t afford to support a (stagnant) lone indie developer. Especially in the current era of LLM scrapers.
And I'd put Apple App Store Review team (specifically, the appeals/rejection team) at the top of this list
Author here—seems they noticed that people were unhappy. Just got this email from Yelp (August 1, 2024):
> Earlier this month, we sent you an email about your Yelp Fusion API usage. That email gave developers until July 23rd to contact us if they want to continue using Yelp’s data for use in their app. We realize you might need more time and are extending your free access for an additional 90 days starting today. Your access should be available now.
> We’re sorry for any inconvenience or frustration this abbreviated transition might have caused. Please respond to this email or contact us at api@yelp.com if you have any questions.
> First of all, I was not “trialing” the Yelp API. I had been using it for a decade and had official permission from Yelp to create Restaurants.
This looks like a Face/Off situation to me.
(Spoiler alert for a 1997 action movie: in Face/Off a cop surgically exchanges faces with an imprisoned villain in order to go undercover in their organization, but the villain then murders everyone who knew about the swap and steals the cop's life.)
Somebody at Yelp in 2014 knew that you had been approved to build this app. That person almost certainly no longer worked at Yelp ten years later, so the institutional knowledge of that agreement had likely been lost.
That's fair, but you'd think their admin interface would show how long an app had been active for and any notes about the app from the beginning.
Regardless, even if it had been active for 10 months not 10 years, 4-days notice is unacceptable.
I absolutely wouldn't expect there to be notes against an API key.
Heck, in many companies I have encountered that they don't even know who is using API keys they've issued to third parties, especially over generations of systems. I am impressed that after getting email approval for an app a decade ago, Yelp still had contact details enough to let the O/P know!
None of that negates the issue of 4-days notice and Yelp shooting themselves in the foot by removing access to an app which redirects people to their website. But someone in executive management made a decision on the basis that they wanted revenue from the API overhead, wanted a slice of other people's pie and that's that.
To the O/Ps point about subscription, there was a UK food app which gave you restaurant food safety scores taken from the food safety agency "Scores on The Doors" it's gone now but I paid a few £ a year for it and never had an issue. If the Yelp pricing is sustainable at that level it's not a bad idea to pay for API access, Yelp has to pay for their servers and a portion of the API calls won't be converted to traffic. Who knows if Yelp still has a sustainable business? Maybe that API thing doesn't pay for itself? I don't know.
Maybe not for an API key, but for an account, you would think they would have some method of keeping notes. I've never worked for a company that had a concept of accounts, and teams of people that interfaced with accounts, without having a method of keeping notes about an account. That's just a basic necessity, especially when the accounts exchange money with the company, there's usually accounts that pay more than others that require special handling, which is usually documented in an account notes screen of some kind.
On the Yelp developer site you have an app profile where there is the name and description of the app. There is also a separate field for the URL of the app website, which I did have filled in. So, they have that at a minimum. If they didn't retain the records of the screenshots/communications that led to the original approval and raise of the API daily limit then that is poor recording keeping but understandable in a corporate environment after 10 years.
The API key usually has a contact email. After a year that contact email probably doesn’t work anymore, certainly for free tier.
Which underlines what I concluded long ago: the best and most durable form of identity on the internet is rooted in the ability to pay money. Any identity that is free to create is doomed.
Oh I completely agree - 4 days notice is _never_ OK. I get uncomfortable with 30 days notice because I've had the occasional vacation that long!
Ideally they'd have a notes field against developer apps and a robust process for recording this kind of thing - but I've worked for companies, so it doesn't surprise me at all that there's no good mechanism in place for that.
> I get uncomfortable with 30 days
In real life, sometimes months isn't enough. On August 5, UPS will complete their API transition to OAuth based authorization after many months of process. Many of my customers haven't responded to my attempts to warn them about the change. I've resigned myself at this point to August 5 being a crazy day.
> I get uncomfortable with 30 days notice because I've had the occasional vacation that long!
Four weeks is the statutory minimum annual leave entitlement here in New Zealand. Most people I know get more.
Yeah, you'd think so. But as many of us have learned, many companies can't and won't be considerate. Sorry that you got hit with it, but at least it's a quick and clean break.
Don't sweat the refunds too much (unless someone is being really rude about it). Apple certainly won't.
Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
> Btw, I'm pretty sure what happened was that this conversion was planned and carried out, but nobody was assigned the responsibility to tell developers. It was clearly done last minute using the most convenient form.
Right, the biggest thing is a failure of communication. There should have been emails months ago, not 4-days (1 business day Friday->Monday) before.
why? for what?
There should have been an email months ago saying they were converting their API to be paid. Not a 4-day threat to shutoff your API key or pay-up. As I mentioned in my post, I was not aware that they were converting to a paid program and had received no prior emails.
There was no contract and nothing. Even instant revocation would be okay. Why not?
Usually it would've been a 2 day notice. The great magnanimous Yelp noticed it was a weekend and decided to give the author another 2 days.
/s
John Travolta and Nic Cage at their finest.
My first thoughts while reading the story was that their champion at Yelp quit and whoever's left didn't know/care enough about them. 10 years ago, Yelp would still be scrappy start up and there was a motivated employees willing to go outside the painted lines. Now, it's just a faceless person following orders.
> 10 years ago, Yelp would still be scrappy start up
Yelp was founded 20 years ago :-)
or in the departed. Martin Sheen is the only one who knew dicaprio was undercover. With him dead the protection goes with it. Only a real contract would CYA. When a company goes from friendly to unfriendly they're happy they were 'scrappy' not to write everything down.
I’m actually surprised the API access lasted for so long. The person that provisioned it is probably long gone from Yelp. Similarly, the company’s values and priorities have changed as well.
Also, I’m guessing this new API pricing policy is they way of combating data scraping to train AI.
If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions, or the service in question was built poorly from the start.
There is little room to blame behavior like this on bad actors and not bad decision making. 10s of thousands of companies have managed to figure out combatting API abuse without shooting themselves in the foot.
> If you work for a company and decide that people using 100 API calls per day, hurts your company more than helps it.. then most likely you are either unqualified for making these decisions
You are entirely right, yet I see this train of thought happen so very often in different companies. Some of these people are a bit saner than others and actually do listen to others explaining why it is an idiotic train of thought. Others really are just see numbers in isolation and somehow refuse to look at them in context.
How much money should yelp spend to help Restaurants continue to work?
Depending on what kind of approvals they gave him 10 years ago, it MIGHT be possible that doing this violates a contract. It sounds like they had some kind of understanding when they gave him access that he had some sort of informal approval. Even if nothing was signed, that still forms a contract. Even if there was a formal contract or terms of service (TOS) agreement, there are certain restrictions around when and how a company can change their TOS. In particular, there's often requirements about how much advance notice has to be given if the terms of service change.
It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
> It sounds like the monetary stakes are pretty small, but depending on the author's desire, it might be worth doing some research and potentially going to small claims court and claiming damages for those customers that requested refunds.
Yes, the monetary stakes are too small for it to be worth it for me to pursue. I could probably dig up some old emails from 10+ years ago but it just wouldn't be worth it. Exposing this kind of bad behavior (4-days notice!) is enough.
I would be amazed if it’s somehow possible for a company to implicitly commit themselves to giving you API access forever without a formal contract.
What's needed to form a contract is an offer, acceptance, and consideration - if an offer was made and accepted and something of value was exchanged, and there wasn't confusion about the terms (after 10 years, there wouldn't be), that's good enough for a contract to have legal force.
Not forever, no. But usually with any sort of contract, when one side wants to break or alter the contract they have to give the other side adequate notice. In certain circumstances a contract can eliminate such a requirement, and the definition of “adequate” would vary a lot based on specifics, but that is very much a possibility in this case.
> Depending on what kind of approvals they gave him 10 years ago, it MIGHT be possible that doing this violates a contract.
What contract? He never entered into a contract or even exchanged consideration with Yelp for the API as far as I can tell.
Getting a green light via e-mail to use a free service is not a binding contract and does not come with any obligations.
It sounds like he requested API access in order to make a native Mac application for Yelp. The specifics matter a lot here, but “if you develop a native Mac application for Yelp, we’ll give you free API access” sounds a lot like consideration. That could be completely false based on exactly how that went down, of course.
> but “if you develop a native Mac application for Yelp, we’ll give you free API access” sounds a lot like consideration.
He was using a free API that anyone could sign up for.
They did not exchange anything with him. Developing an app to use someone's free API is not an exchange of consideration.
> Developing an app to use someone's free API is not an exchange of consideration.
I would never pursue anything further over this app that had about $2,000 of revenue over 10 years. That said, for the record there was consideration.
I had to go through an official approval process that required providing evidence of the app's functionality and a few emails back and forth. I think I actually sent them a prototype of it. And based on that process they decided how many daily API calls to give me. A normal free user did not receive 25,000 API calls per day. I believe if you didn't go through approval back then you got something like 1,000 per month. So there was a consideration process on their part and a determination to green light my use case.
Ah, that's not what consideration means in contract law. Consideration means something is exchanged for something else. (See https://en.wikipedia.org/wiki/Consideration_in_English_law)
You could argue that the indefinite API access was in exchange for writing the app (service for service), but if this were me, I probably wouldn't bother. Maybe I'd write an adversarial-interoperability backend to replace the API, or open-source the app to allow other interested parties to do so. Or maybe I'd just say "It was a nice run", and let it die.
If you care enough to send a polite email, you could say that Yelp has a prior agreement, you'd appreciate them not reneging, and if they do, you'd appreciate compensation for your labour (minus, of course, the money you made from the software). Probably won't go anywhere, but…
I would be upset too, but this sort of sounds like someone did him a favor 10 years ago, management changed, and now management wants him to pay. This sucks, but near infinite free api calls sounds unsustainable
Well the thing is they gave me 25000 per day, but the actual use was less than 100. The 25000 number is almost a red herring to the rest of the story.
It’s understandable they went to paid. What’s not understandable is the 4 days notice and tone of the email.
We received the exact same letter from Yelp. Our usage is significantly higher and when we talked to them, the prices they quoted were ridiculously high (thousands of dollars a month).
If the price is too high for you, here is my idea for you: don't use their product. ;)
Exactly, we're not. We're closing our products. They are perfectly welcome to charge too high a price for us. They just should've communicated it better.
Sorry to hear that. What was your product?
We have a analytics SaaS which displays, among many other pieces of data, ratings from different providers, including Yelp.
I assumed that Yelp had been doing this all along and we ran above some predetermined limit, but the email we received was identical to yours. So I'm thinking that Yelp is trying its best to monetize all API users
A 25,000 calls per day limit for an API sounds absurdly low. Any idea if the API calls actually would have used much in the way of backend resources, or do you reckon it was more just mostly a database lookup?
API rate-limits have always been a cash grab and a way to discourage efficient automated use in favor of getting a human to "engage" instead.
I would be surprised if it were much more than a database lookup.
Yeah, it's probably that simple. 25k calls may only be a few seconds worth of actual database runtime too, if that.
Honestly, making people pay for an app that only uses a public API you’re not paying for, and no form of fallback is asking for trouble. This is not a responsible way to do business and I hope people reading this thread will understand that.
Since ~15 years, people got the idea that all the WhatsApps, Instagrams or Twitters around them aren't companies, but kind of public infrastructure. They all knew that it's a wrong assumption, but it's sooo convenient, and at some point, people around you force you in that direction, even if you don't like all those walled gardens. And once this wrong philosophy was established, people started escalating it to some smaller shops, I guess. Those are also the guys that complain when some 3rd party Reddit or Youtube hobby clients must disappear. They just have no understanding at all about some basic mechanics of human interaction. The biggest problem: There are entire generations of people nowadays with that misunderstanding.
> near infinite free api calls
but the author pointed out that it drove traffic to Yelp, and that almost certainly seems to be true.
In fact, little integration plays like this app might have been the only thing keeping Yelp sort-of alive after Google got into the game.
Killing off your bizdev partners seems incredibly short-sighted and foolish. (Also, feeding the reviews into an AI... for what, exactly? To train a model on how to write reviews? Or perhaps to detect fake reviews -- actually, that was an issue on Yelp even before AI, so it seems like it wouldn't be the best training content.)
I think Yelp marketing is on an incredible shortsighted and profit-losing path.
I too used the macos App.
Looking forward to Apple Map dumping Yelp, because that combo doesn't work for me and I do not want Yelp cluttering my Apple map.
I think Apple has been developing a yelp clone internally; and have been slowly releasing bits and pieces.
Apple has added rating systems for destinations. Amenities available at location (ie, “accepts Apple Pay”, dog/cat friendly). Maybe in the next decade Yelp integration will be phased out completely
They've already got something in Maps that switches locations from Yelp reviews to the Apple thumbs up/down system once they hit some critical mass of data, and the same for location photos.
It's not even popular outside of the US, I'm not sure how they can grow with that.
If I look at my closest city in the UK at restaurants, nothing has been reviewed recently (2018, 2019) and it's mostly from US visitors coming here.
Was this a move to try and get some profit from AI companies trying to train on their data? With the API providing limited results, I’d think it would be of limited use. I always found Apple’s use of it in Maps frustrating, because seeing POI details basically required I also have Yelp installed.
If this move was AI driven, like with Reddit and others, I’m starting to dislike AI more and more… at least the rent seeking end of it, which seems to be slowing killing the open internet.
My son, it is time for you to embrace the good news of our lord and savior web scraping.
You reminded me of this "meme" about APIs vs scraping: https://i.imgur.com/koPo3M0.png
"his business is profitable" is a little too close to home.
We need to move towards “zero trust” for APIs.
SaaS can provide “open core” or better yet simply sell a hosted version of their fully open source code. If the provider fails to provide, you can fall back to self hosting.
The API equivalent would be open sourcing the data. This is the OpenStreetMap model. If the API provider fails to provide, you can fallback to the underlying data.
That's asking too much. SaaS should give an option for you to export all your own data in a simple, parseable format (like JSON). That's about it. They don't owe anyone their source code, and they don't owe any ethically sourced data (such as employees researching and manually entering).
API access needs better terms. Like guaranteed access for X years at $Y price with Z days notice if there's a change, where Z > 3 months or so.
This was the hope behind the StackExchange data dumps, that the community at large could always take their contributions elsewhere if the service jumped the shark.
Well, before the SE organization tried to kill the data exports off in an attempt to commercialize it towards AI companies, but thats a whole other issue.
If the api was as basic as you say, can you replace it with some screen scraping on yelp’s site?
Not only would that violate their terms of service, but the monetary stakes are also too low for it to be worth it to me even if it didn't.
That would probably as big or bigger project than the entire rest of the app itself, and since it is such a single developer and not a super profitable app it likely makes no sense to do this.
Probably not without violating other terms of service.
It should be ok as long as you do it logged out.
SerpAPI is a convenient wrapper API for scraping various sites. I assume they've vetted all the legality of things. They have a YouTube API: https://serpapi.com/yelp-search-api
> I assume they've vetted all the legality of things.
They claim they did: https://serpapi.com/blog/scraping-public-pages-legality/
Still they have a boilerplate ToS with some glaring mistakes:
> These Terms of Service and any separate agreements whereby we provide you Services shall be governed by and construed in accordance with the laws of 5540 N Lamar Blvd #12, Austin, TX, 78756, United States. (sic)
It's not quite so clear-cut. We even have a historical precedent here:
https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc.
In that case, the court found 3Taps was criminally guilty for scraping publicly available craigslist data while logged out because 3Taps knew their use was not authorized.
This person has just received an email from Yelp telling them their free usage is not authorized, so circumventing that may well be illegal, now that they've been given that sort of communication, even if it might be questionably legal for other serpapi users.
The courts are altering their views on scrapping. This [0] is a good paper that explored the last 20 years of rulings (although it hasn't been updated with the most recent cases).
[0] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3221625
I don't care about Yelp's terms of service, do you?
You will when a process server hands you a notice.
A notice of what? It's not illegal to violate someone's random wishlist. It's not like you've signed a contract with them.
It's so cute watching people play pretend lawyer on the internet.
Corporate policy is not the law. If you didn't sign a contract with them they should have no legal power. The DMCA was a mistake.
Not sure why this is getting downvoted, because it's a reasonable question - the answer being that such scraping requires a lot more time and technical expertise to engineer than a simple API call. Also devolves into a cat and mouse game between your application's backend and whatever proxy they put in front of yelp like cloudflare, which you'll probably lose or will be prohibitively expensive.
Sorry to hear how Yelp treated the OP...
Given how many stories of greed and throwing people under the bus for money I hear nowadays, we might already be living in a dystopia.
It was never different. Just, with the advent of iPhone, FB, WhatsApp, Youtube, etc, a lot of people basically decided to ignore this fact of life. So good and for so long that they completely forgot it. They forgot it good enough that even their childs got no education in that regard. And every some days, someone gets a lesson. ;)
Why should Yelp not throw you under the bus for money?
The problem was that (impersonal) you stopped understanding basic mechanisms of human interaction. For no real reason.
> It seemed they were in fact encouraging me to finish the app and release it.
That was quite the assumption. They gave you access to something for free, not encouragement. I do feel bad for OP, but they weren't paying for the API, and should not have had any assumption that it would last forever because there was no contract or terms or anything. This is the risk we take by building our house on someone else's foundation.
The gist I got from my communications with them 10 years ago was that they were encouraging me to finish it.
I agree with you there was no expectation of it being free forever. I never said there was. What was unreasonable was the 4-days notice that it was coming to an end. That's just not enough time.
I am in the same boat, sadly.
I am particularly disappointed by the generic "Your API usage is higher than lots of other Yelp Fusion developers" statement.
My giveback service has a tiny user-base and find it hard to believe my API usage level can be higher than average.
https://try-something-new.web.app was built a couple years ago.
That statement is an outright lie too. Not only are they operating at less than 1% of what the free plan offered, but the email doesn't tell them the honest truth: That the free plan is reaching sunset, and they want all API users to switch to a paid plan.
This isn't unreasonable, but they absolutely should provide more than 1 business day notice.
Absolutely. You wrote it better than I did myself in the original post.
Perhaps theere are a lot of inactive API users with zero usage so the statement is technically true ;)
Textbook half truths. :-|
Of course it's generic and perhaps not a technically correct statement. And now? Why should they spend time on writing you nicer emails? What was your value to them? Zero? Or is that too optimistic actually?
If I knew about this app before, I would've definitely bought it! The Yelp site and app are incredibly slow and tedious to use
Thank you. Yup that was the main thing people liked about it. It was really fast and had no ads.
> if you utilize a third-party API for the core of your app, you are at their whim
More than that, if you aren't paying for use of that third-party API, the people who run it will not care about you, and will think nothing of shutting you down.
I think Yelp handled this poorly, and Restaurants was probably a net positive for their business -- a positive that they were getting for near-zero cost. It's a shame that companies are so short-sighted like this.
But ultimately if you build on top of someone else's platform, with no backups and no alternatives, it's not really truly your app.
Enshittification is at an all time high, and API access is being tightened and monetized in response to many industry factors including "freeloading" AI trainers.
Cory Doctorow is right, if you want to disrupt, or make any improvement to an existing large platform, adversarial interoperability(that is, reverse engineering) is the only way forward and has to be explicitly legalized in cases where it's a tool for progress.
My previous statement is arrogant, as it assumes developers are entitled to take any data they want and profit from it. It also puts services in a situation where harmful crawling like what is performed by some new AI actors with no experience is an expected thing. This is of course wrong, but I want to argue that had Yelp and other actors not wanted such a future, they shouldn't have tightened free access to their proper APIs where they have the ability to set ground rules and have the ability to talk to their users.
Big companies are amazingly bad at keeping track of things internally - a promise in an e-mail is easily forgotten 10 years later. But why should the user be punished for Yelps lack of control?
I'm sorry to read that. Unfortunately, it isn't an uncommon story.
> if you utilize a third-party API for the core of your app, you are at their whim.
That's the money quote, there. I avoid using third-party APIs like the plague. I have written backend aggregators and facias, to avoid having to use the API.
I Just. Will. Not. embed an opaque codeball into my app. I'm a cranky old bastard, I know, but I sleep well at night.
"Opaque codeball" is a very large category. Are you saying you won't use any 3rd-party libraries at all?
Amen. Doing the wrong thing is easy. Doing the right thing is hard. These are thermodynamically-mandated rules that cannot ever be circumvented by cleverness or money or hard work.
yelp had so much potential but they pissed it all away. Google brain raped them [1,2]. Then they never recovered from it.
Then there is ongoing issues between merchants and yelp [3]
Yelp used to be a great place to find some decent place to eat in a new city. But the platform has gotten stale. Reviews are less reliable. Star rating often not useful.
[1] https://www.theguardian.com/technology/2015/mar/20/google-il...
[2] https://techcrunch.com/2015/02/06/google-takes-on-yelp-elite...
[3] https://en.m.wikipedia.org/wiki/Yelp
Yelp suffers from the classic issue of not having a way to monetize that isn't a conflict of interest with their core business model.
Google search has the same issue.
Probably what happened was 1 support rep helped you out but did it through some undocumented backdoor to unblock you and not a formal contract. Later a completely unrelated set of employees are tasked with figuring out who the biggest API users are and to either cut them loose or get them to start paying.
Yes, this was surely somewhat tied to employee turnover and poor record keeping. But I was almost certainly not one of the "biggest API users."
The thing is, regardless of the turnover or situation, 4-days (actually 1-business day Friday->Monday) is not a reasonable timeframe to threaten to shutoff someone's API key who hasn't violated any terms of service. They have the right to do it, but it doesn't make you want to work with them in the future.
This feels important:
> The other thing this taught me is the danger of an up-front paid model for apps that depend on ongoing access to third-party services. If users were continually paying for the app, paying for Yelp’s APIs would not be as much of an issue. And I wouldn’t feel as guilty about the app being discontinued since if the fees were charged on a monthly basis, they would just end at the same time the app ceased to exist instead of facing an expectation upon purchase of “forever access.” On the other hand, how would you charge a monthly fee for an app that people are only willing to spend less than $5 for upfront?
There are apps that I like and would like to purchase, but paying e.g. 24 euros per year feels like too much. So I stick with the free version...
A hard lesson I learned after several times getting burned in my time long ago doing indie stuff is “never depend on a single company without a backup plan”.
Sometimes this can be pretty difficult to even see. If you were a Mac shareware dev depending on VersionTracker and MacUpdate to drive downloads, with no actual marketing budget, you might not have noticed that you were implicitly relying on Apple not creating an App Store which would turn the entire ecosystem upside down.
This is one of the reasons that it’s so tough to make a business work at a small scale. In some ways, you have more flexibility than a big company, but the lack of capital means that certain events that a larger company would shrug off can totally upend everything you’re doing.
> is that if you utilize a third-party API for the core of your app, you are at their whim.
Well, yes. I think people figured that out more than 10 years ago
And business models can and do change
And by checking the pricing page, Yelp's commercial API is $15 per 1000 API calls per month. Which sounds ok?
Right, they never answered my questions about the discrepancy between that and the pricing deck they specifically sent me for my app. See slide 3 with a $229 per month base price: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
The economics of Restaurants actually mean it wouldn't be profitable either way though. That's how low the sales are.
They're welcome to start charging for their API whatever they want, but they should give more than 4-days (Friday->Monday, 1 business day) notice.
I have seen many notices like this in the past few months. My guess is anbody of signficant size whose reason for existing includes curating user-generated content is trying to ensure that gen-AI ingestors don’t swallow up their data through unpaid API access or scraping. (Maybe a podcast or article popular with CxO and IT directors on the topic??)
Which they seemingly already have done anyway. Another unintended side-effect of the borderline illegal and generally immoral “AI” companies efforts to get as many data sets as possible.
Is Yelp even still relevant these days?
At least where I am in Europe I find it incredibly outdated, showing restaurants that have closed a long long time ago and none of the new ones.
It seems to be mostly a US phenomenon. As I pointed out in the post almost all of Restaurants sales were to the US. The directory is very up-to-date with active reviews everywhere that I've travelled in this country.
I got the same email regarding my hobby website "The Wheel of Lunch" (https://wheelof.com/lunch). The site makes no money and sends traffic to Yelp. The cheapest plan offered was $230 a month! I can't afford that.
I have stopped calling the Yelp API for local listings and put up a notice on the site. It was fun while it lasted!
Sorry to hear that. Remember your site from 15 years ago when I was an intern trying to figure out where to eat.
We've been bitten by going the "free 3rd-party <x>" route so many times, so I sympathise with the problem here.
Off-topic, but kudos to OP for still engaging with the threads. This certainly wasn't a dump and run post. They've probably spent more time answering questions here than they did developing the app!
> But due to the way that the Mac App Store works we don’t have our customers’ email nor any way to directly refund them.
> if you utilize a third-party API for the core of your app, you are at their whim.
I think you might also want to revisit your relationship with Apple Incorporated.
Yes, for sure. Some people explain it away as "well if you buy Wrangler jeans from Walmart and there's something wrong with them you ask Walmart for a refund." So if you buy X's app from Apple you ask Apple for a refund. That kind of makes sense. But if Wrangler does a recall, they have a way of getting that across to their customers.
I'd say, no you had no shared customers. Yelp had the customers, and you wrote some app that uses Yelp. For no real reason you assumed that you could be part of them (as with Facebook earlier). Sure you had some kind of informal permission at first. And then they revoked it. That's their right to do, right? Maybe you should not write Facebook or Yelp apps, if you are neither Facebook nor Yelp nor asked by them to write clients for their walled gardens. Or if you really want, set up an actual license agreement with them. Those networks are walled gardens and they don't want to cooperate with you that way. They are just not those kinds of shops which you can cuddle with. If you want to create something that has an own value, create something more than just another Faceyelptwitterddit frontend.
The good thing: You took your lesson. Maybe this time it was sufficient to actually learn sth.
> I'd say, no you had no shared customers.
Person pays for my app. That was a customer.
Person who paid for my app also later uses the Yelp website via a link for a specific review from my app. Also a customer.
Therefore shared customers.
> Sure you had some kind of informal permission at first.
I don't know why you would define it as informal. They had an official review process for deciding what apps to grant higher daily API limits. My app went through that review process 10 years ago including considering its functionality, screenshots, and I believe I even sent them a prototype back then. We had a few emails back and forth to confirm my intentions to only develop for the Mac and what I was building.
I'm sure that got lost in the corporate shuffle over 10 years but I clearly had their permission to build the app and in fact given the relatively high API limit they gave (25000 per day versus the free at that time 10 years ago I think being 1000 per month), arguably blessing.
have you thought about writing your own API instead of using the de facto API? it may involve screen scraping but it also will be a lot more reliable and there's no limits, especially when using proxies....
I've written thousands of tools that scrape websites and never used the apis for this reason, you can never trust the API, either because of the reliability, cost, limits imposed, etc... Nobody wants you pulling data from their site anyway, so you're back to scraping anyway, its better to start out there then to have to end up there years later for some other reason...
That’s trash. I’ll delete yelp over that, bad customer or dev relations sucks.
What is the precise meaning of 'dev relations' here? What is the actual relation? They haven't ordered him to dev sth.
It seems that you still can spin the Wheel of Lunch, but there are only generic proposals like Pizza, Ramen, Thai Food, and so on.
That was a long blog post to tell us that they are no longer entitled to valuable free API access. It was a mistake to think it would last. This story has been repeated so many times here. As soon as the author started making money (they conveniently excluded their sales revs), they should have negotiated a written contract for access for a fee. It could be cut of app revs or pure API charges. Also, I am sure some smart managers looked at all their free API giveaways and decided there we no longer useful to their business model.
The author disclosed his sales revenues here: https://news.ycombinator.com/item?id=41124971 . He sold 467 copies of the software after 10 years, for a total revenue of around $2000. He never exceeded 200 API calls per day. I don't think it would have been worth Yelp's time to negotiate for a percentage of that.
Building atop centralized platforms, you run this risk. In a decentralized environment, you’d be able to keep a fork running just as it was.
Control your own future, or you’ll have no future.
Reminds me of something Reddit did just recently...
I think when the AI scraping funding models go away, all these APIs will magically open up again.
Drama aside, the guy signed up for the free Yelp API 10 years ago, which has since been discontinued. He was offered the option to switch to a paid API, which he chose not to consider.
Yes, Yelp was a bit clumsy in handling this, but discontinuing the free API after 10 years is totally within their rights. The developer didn't even bother getting their pricing proposal, which might have been totally reasonable (or not), considering his app is paid.
He's didn't argue it wasn't within their rights. He called them "quite rude", which seems hard to deny.
No matter the rate Yelp set, the apps economics no longer make sense. The existing customers, already paid, and he has no way to transition them to a subscription.
I guess you didn't read my blog post because I addressed everything you wrote. The issue is not that it went paid, it's the 4-days notice. They are perfectly in their right to start charging for their API, they just can't give us 4-days notice.
Well they can do whatever timeframe they want of course. And I can write about how rude it is. 4-days (really 1-business day in the original email Friday->Monday) is not a reasonable timeframe within which to threaten to cutoff an app with real users.
That's when you take matters into your own hands and extract the API key out of one of their official apps. At least that's what I would've done.
Though I'm not sure how legal that would've been if done in a paid app. It feels like a serious difference between just providing a better UX for someone else's service through adversarial interoperability for free, and profiting off of it.
Digital sharecropping. I don't know what the answer is, but it's sad to see it unfolding!
Curious how much money the app developer made, and how much they paid for Yelp API access.
I sold 467 copies over 10 years for less than $5 per copy. It's about $2000 in revenue. Surely not a good use of my time. But I liked the app and so did its few users. We should build things we want to use ourselves right? I think it was a much nicer and faster frontend to the Yelp restaurant directory than the Yelp website.
Your other question about the cost is answered in the blog post and in several other comments I have left in this thread. In short, when I started the app 10 years ago there was no paid API (perhaps there was some kind of "enterprise" version but I'm not sure). The point of my post is how poorly they handled the transition from free->paid.
Yahoo used to have a decent restaurants API. I assume that's dead now.
If building an app on a platform does not increase the networks effects of that platform, this will always be the outcome.
Good platforms get more valuable with more apps: iOS, Windows, etc
Bad platforms don't get better with more apps: reddit, netflix, twitter. So they always end up killing the API.
The difference, I think, is that the former are platforms to run applications while the latter are communication platforms. There used to be a time when the phone company could press charges if you connected your "app" (a tone-generating machine) to their "API" (the phone line) to "do things the official app didn't support" (place long distance calls for free). It will always be beneficial to someone who owns a communication channel to assume as much control of the pipe as possible. If Twitter could jack directly into your brain stem and refuse to work until you install its client on your wetware, it would.
Real scumbag move from Yelp, sorry to read this man
Not surprising at all. Yelp, Paypal and their ilk deserve all of their earned derision for, amazingly, being simultaneously user, customer, and developer hostile.
Related to the earlier "old & grumpy" 3rd party API post, I've seen far too many, otherwise, outstanding businesses held hostage and summarily executed by either sudden un-explained usage-tier/throttling policies without economic recompense or the outright deactivation of API keys w/o notice or explanation.
Yelp is a scumbag company, I'm surprised it's still in business as it's reviled by business owners and TripAdvisor crushes it in its segment.
When will people realize that using any big tech company API is a recipe for disaster? These companies, and their revolving door of employees, could care less about you, your app, or your users.
Dollars to donut stores, what's likely going on with the switch to expensive API is this:
https://business.yelp.com/products/yelp-ads/
Same as the Reddit stunt versus the Apollo app dev, except that his app had a big enough audience you'd think they'd have figured something out.
Nope. The paid app meant money from users to the app dev instead of from advertisers to the site.
What's strange is Reddit didn't seem to do the math on how little they should have charged the app dev for API access if all they wanted was to offset revenue from that user base. Perhaps the fear was much as with TV streaming: they know advertisers want audiences willing to pay, not only the audiences seeking free.
Could pivot to customer provided api keys :\
This really sucks for OP, but my first thought on seeing yelp was:
"People still use yelp? I thought it was widely known that they suppress bad reviews for money, and suppress good reviews if you don't pay."
Yelp's path to monetization has always been kind of scummy IMO.
> "People still use yelp? I thought it was widely known that they suppress bad reviews for money, and suppress good reviews if you don't pay."
A judge said that it wasn't extortion for yelp to hide good reviews until a company pays yelp, and that it wasn't extortion for yelp to remove bad reviews for companies that do pay them money though so I guess we're supposed to act like that's acceptable behavior and that their reviews can be considered trustworthy
Have you considered building your own restaurant database? You could add features to the app to allow users to submit and update entries.
This is just pure boneheadedness across the Yelp management chain, all the way down to the IC level.
- IC should have recognized the site was driving traffic to Yelp and flagged to management.
- Management should have realized that some API usage is beneficial to Yelp overall, and crafted a plan around this.
Just pure insanity. If I were a VP I'd fire everyone involved for a lack of basic critical thinking skills.
Question-- with the yelp api now being paywalled-- and obviously you'd have to write code to handle scraping-- but can you legally scrape yelp to do this?
Does the linkedin scraping lawsuit permit scraping yelp?
The big takeaway here is to never rely on a third party, they will inevitibly fuck you over.
Every business will eventually turn anything that they can into a profitable feature for them.
I wrote a review for a grocery store turned food service shovel basically said "this will be the first store that charges for air." Guess what? I use their bathroom every other day,for free. I think it's been months since I bought anything, and I help myself to utensils.
Every customer will eventually turn anything they can use for free, into a profitable feature for themselves.
There was a book called the 1 minute manager: it was followed by a book called the 59 second employee.
Never rely on an unscupulus third party, or if a third party becomes unscrupulous... Hunt them down.
I disliked Yelp, and held it in deep distain, then while working for a service company, Yelp called. "For a fee we can remove all negative reviews." This is the definition of ensh*tification. So... I told him I would get back to him. I called a few numbers at Yelp and was basically able to socially engineer a vast list of their revinue growth supervisors.
I made a list of every business in that city and the next, and related my experence, and how worthless a Yelp review was. Almost all the business responded to poor reviews with a 'i bet you are a Yelp employee drumming up money for fake reviews.' after a few weeks, I gathered up a lot of these, and sent a three page letter of them in a package with 50 of the Yelp peoples's names in it. I let that stew for a few days, and then called the idiot back. He said "we don't care." They don't. They do not care in the least. Google, OAth, and Facebook do not either. Hurt them in their advertising review and then they will listen.
i use to be that guy telling people not to put their eggs in other peoples basket. Some laughed at me for a decade, almost convincingly and then...
Of course my stuff also breaks because you cant really do anything anymore without trusting anything even against better judgement.
I think we will eventually get expensive quality terms of service for those who think it might be fun if things work forever, like html documents and megalithic structures.
For a site that caters to a startup and entrepreneurial crowd, it's hilarious the number of comments here that amount to "tough cookies, bud" and "Yelp can do whatever they want, and because they can, you should just shut up."
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
> which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
Not everyone. Business that build on top of other company's APIs will arrange contracts with their API providers. Those contracts generally include warning periods for changes or discontinuation and penalties for early termination.
The key here is that it was a free API with no contract or guarantees. Four days is short notice and frustrating, but it wouldn't have really changed the trajectory of his business if they had given him 180 days. If he didn't intend to pay for the API, he couldn't really sell an app that was going to stop working in a few months.
So I know we're supposed to be angry about the 4 days thing. It's not good, obviously. However, I don't think it actually changes the situation at all if he wasn't going to sign up anyway.
I hear you, but this story keeps happening over and over and over. The reality is once these companies have you and your product by the balls, they will start squeezing. You can pay money to reduce the pressure, or leave and not be squeezed. I would argue using an unpaid API takes you into the unknown with considerable risk.
> That should be concerning to anyone working on projects that use APIs
Well, free APIs anyway. If you are paying for API access, you hopefully have a contract which gives you power.
Almost everyone is working on projects that “use APIs” in some general sense, sure. But I don’t think it’s the case that all or even most people are working on a project that entirely depends on a single third party’s API and is useless without it.
Please don't sneer, including at the rest of the community.
https://news.ycombinator.com/newsguidelines.html
They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users.
Or, they completely get it but they work for large platforms that leverage API access commercially or strategically, so their response to unruly peasants is to figuratively chop their heads off.
Thanks—yeah I actually think they mostly just didn't read the whole post since I addressed this in detail in the last two bold sections "Development Ends" and "Lessons Learned."
> They miss the spirit of this blog post entirely, which is to point out the overt hostility to and powerlessness of API users. That should be concerning to anyone working on projects that use APIs, which is, um... almost everyone, these days.
This has been known for like 20 years now. We all know that if you're relying on someone else's API that's a massive risk to your business. What more is there to say at this point? What sympathy is there to give when the inevitable happens?
[dead]
I think this is the giant internet trick... do anything to scale, then when you reach critical mass (or someone wants OKRs on their status report), pull out the rug.
To be fair there wasn’t much scale. We’re talking about an app that sold 467 copies over its 10 year span. But it was an app I really liked, the people who paid for it really liked, and just drove traffic to Yelp, so they should’ve liked too. It was also a nice portfolio piece for me.
I think Yelp needs to just die at this point. Reviews are gamed, usability is worse and worse. Not sure anyone still uses or trusts Yelp. They’re just in inertia from their better times.
Is there a more open reviews platform not associated with a company? OpenReviews sort of thing or perhaps an open POI database associated with OSM, with reviews as attributes?
Somewhat silly we’re still relying on big companies to collect this data when they then gate it. The timing is right with LLMs that can make maintaining and rating reviews less burdensome on human labor (“moderation” broadly speaking). It’s just text blobs and possibly images linked to people and places.
I have never in it's lifetime used Yelp. It has been plagued with shit like this from the beginning.
I really hate that they force you to install the app to be able to view reviews that people have freely contributed. This is especially a problem on iOS where you cannot change your user agent so that it will serve you the desktop version.
I mean, as at least one other comment noted, anything like Yelp is going to be gamed above and beyond any particularly questionable processes of Yelp itself.
Zagat worked pretty well for "foodies" back in a day when raters had to physically mail in hardcopy questionnaires and people had to buy guidebooks in stores. I'm not sure how you duplicate that today absent physical pay-walls on both sides which I can't really see working.
I do think the various rating sites--Yelp, TripAdvisor, Google--"work" in the sense that they're probably better than picking at random. And you can't really call the local newspaper's food critic on the phone (which is what my dad used to do). So I'm not sure what the alternative really is. Yes, there's some word of mouth but that's pretty random.
>Not sure anyone still uses or trusts Yelp.
The results clearly indicate it is.
Remember the world is far more vast than just Hacker News.
Yelp sucks. I wish Apple Maps would drop it like a hot potato.
Apple Maps Yelp integration is irritating. Any click on a photo prompts Yelp install. I wish they had a "Don't Show Me Again" option
Yep I specifically keep Google Maps around to find restaurants even though I far prefer the audible navigation from Apple Maps nowadays once I actually want to drive there.
I've noticed that Apple Maps incorporated their own rating system some time ago (I want to say within the last one or two years); it's simple, just asking you to give a thumbs up or thumbs down on certain businesses, sometimes with a bit of granularity (e.g., rate the atmosphere, food, value, and service separately). There appears to be some threshold for how many ratings they have when they switch from displaying Yelp reviews to their own stuff, although it's not clear what the heuristic is.
Anyway, tl;dr: I think they're working on replacing Yelp.
Their website should be a primary case study for normalizing ux dark patterns
[flagged]
Surely nothing like this will happen to the folks that are using LLMs at the core of their app. /s
Who could have predicted that a company with a scammy business model would have such shady behavior when it comes to changing API pricing models?
/s
Hang on, they didn't give his app 4 days - they gave him 4 days to respond with some screenshots, and if he didn't respond, they would shut down his access. They didn't say they'll shut it down in 4 days regardless.
I mean, don't take this as me defending Yelp - they're scumbags, and deserve any hate coming their way - but I don't think that the headline is an accurate description of what happened.
I did respond with the required information and they still shut down my access (10-days post though, not 4-days in the end). But regardless, no, I don't think sending you a request on a Friday for information due on a Monday or shutting down all of your access is reasonable. As I mentioned in my post, what if I had been on vacation? 1-business day or shut you down is just not a reasonable time frame to make anyone want to work with you in the future.
That's a very fair point.
[flagged]
Surprised the guy never even inquired as to how much the API access to support 100 API calls per day would cost.
After 2 seconds of idle research, I have found that it would cost less than a dollar a day.
https://docs.developer.yelp.com/page/start-your-free-trial
It's just not worth working with a partner that will shut you off at 4-days notice.
In addition though, they shared the following pricing deck with me: https://drive.google.com/file/d/1Cb_8laDpxZdfwJPtYBmibZgvLZ8...
Which seems to indicate a base price of $229 per month. I have no idea why that doesn't line up with the pricing on their website. But the fact that they shared it with me indicated to me that my use case fell under some kind of "enterprise" usage. Regardless, I would not continue to work with them after 4-days notice and the threatening email. It's too small and app for it to be worth it to me.
I did ask in one of my emails about the pricing discrepancy and got no reply.
A dollar a day is $30/month, and at $1.99/sale that means OP would need to sell 15 apps ONLY to break even, which given the comments in the blog might be close to reality. This is pennies for both sides, but the way that Yelp distastefully contacted OP is probably worth just shutting the whole thing, it's not like OP was making bank on the app to begin with.
Throw on taxes, administrative overheads, etc, they are probably looking at 30-45 sales per month. Which is likely not realistic.
On top of that, this is a continuous payment. Even if I was looking at 5-10x rate of return, I would be very hesitant as that's the rate-of-return today while the sales are forever.
I've been wondering how realistic microsubscriptions are... Say $1-2 dollars a month per user to maintain an app, perhaps limited to just power users, would support a lot of infrastructure.