Author must not have worked in enterprise software before.
That's a classic trick where the developer will push back on the bug author and say "I can't reproduce this, can you verify it with the latest version?" without actually doing anything. And if it doesn't get confirmed then they can close it as User Error or Not Reproducible.
Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
From experience with Microsoft (paid) support (after doing 5 tickets because it's never the right team and apparently moving tickets internally is for losers), they will ask for proof of the reproduction. And they will take every opportunity to shift the blame ("Oh I can see in the log you're running an antivirus, open a ticket with them. Closed").
I recompiled OpenSSL to make s_server -www return the correct, static XML blob for a .NET application that was buggy to make a reproducer for them that didn't rely on our product at all and which could be self-contained on a very barren windows VM they could play with to their heart's content and which didn't even care about the network because everything was connecting via loopback, so they couldn't blame that, eitehr.
Turns out there was a known bug in Microsoft schannel that had yet to be patched and they'd wasted weeks of our effort by not searching their own bug tracker properly.
I hate that so much. It's everywhere. An example is a bug with discord. They wanted me to restart my phone, reinstall the app, what are my versions, what phone am I on, what settings, etc. After all of that they go "oh that's a known issue." Whyyyyyyyyyyy. I get that multiple things can have the same symptom, but maybe start with that. Not like I signed any NDA so they aren't hiding it's an issue from the public.
puts on paranoid hat It could be to demoralize you so you subconsciously decide to not file a bug next time, knowing all the rigamarole you'd have to go through. takes off paranoid hat
My favourite variant of this merrygoround is when they ask you to demonstrate the issue live in a Teams session, you do so, and there's this moment of silence followed by an "Oh... I see".
Then you assume, naively, that this means that they've recognised that there really is a product problem and will go off and fix it. However, then in turn the support tech needs to reproduce the the issue to the development team.
They invariably fail to do so for any number of reasons, such as: This only happens in my region, not others. Or the support tech's lab environment doesn't actually allow them to spin up the high-spec thing that's broken. Or whatever.
Then the ticket gets rejected with "can't reproduce" after you've reproduced the issue, with a recorded video and everything as evidence.
If you then navigate that gauntlet, the ticket is most typically rejected with "It is broken like that by design, closed."
It'd kind of sad, how the market went. I suppose there are pluses too.
But back in the 80s and 90s, margins were significantly higher. If you look at hardware, I recall selling hardware with 30% margin, if not more... even 80% on some items.
Yet what came with that was support, support, support. And when you sell 5 computers a month, instead of 500, well.. you need that margin to even have a store. Which you need, because no wide-scale internet.
On the software side, it was sort of the same. I remember paying $80 for some pieces of software, which would be like $200 today. You'd pay $1 on an app store for such software, but I'd also call the author if there was a bug. He'd send an update in the mail.
I guess my point is, in those days, it was fun to fix issues. The focus was more specific, there was time to ply the trade, to enjoy it, to have performant, elegant fixes.
Now, it's all "my boss is hassling me and another bug will somehow mean I have to work harder", which is .. well, sad.
It really depends, support is usually the first thing companies adjust when they want to improve their margins.
Even when you're paying millions to AWS you have to get through their first line of support and they will ask silly questions until you can convince them to escalate.
Not really, you get "really dedicated support" at most, but not a "really good" one, otherwise all those decades-old bugs common in many software producs would've been fixed since they affect people at all tiers
Back then, computers didn't had competition from the analog world, so vendors had to provide excellent service such that users would be convinced into switching over to the digital way if doing things. Now comouters have a monopoly on how we work and live, so vendors care as little as possible.
This is simply a bug, it's an implementation mistake, it's even possible to imagine from what we do know about the implementation inside Windows to imagine how you'd likely write that bug, simply you're writing the "lock stealing" code and you realise you need some context -- are we stealing the write lock or the read lock? You realise that context won't fit in your tiny flag budget (flag bits are hidden in the bottom of a pointer) and you forget that you actually know this context at the exact moment you need it - you were asked for either a write lock or a read lock, that's what you're stealing. So, you write code which does what it can without the context, always steal the write lock. Oops. Bug.
And yet several people insist that this wasn't a bug it's actually the proper way for this to function. Not only in this github ticket, and in the Microsoft internal bug, but I saw several third parties defend the bug as obviously the correct way for this to work.
Fortunately it seems STL understood that and the internal ticket was eventually fixed and (presumably) in Windows 11 today this bug is fixed.
That kind of attitude disgusts me. Like it's someone else's job to have a sense of accountability. They would not remain employed in my company.
When I developed software I would jump right on top of any bug reports immediately, and work until they were fixed. I was grateful to my customers for bringing them to my attention.
It is different when you have a billion customers, all with different setups. At that scale, you notice real defects through product telemetry, support ticket volume, or trusted channels. You receive a high volume of bug reports that are due to user confusion, misconfiguration, or misbehavior of other software on the device - where solving an issue for one customer doesn't result in improvements for the other billion. Triage, filtering, and winnowing are necessary here.
I got a lot of those too, it meant I inevitably did a little bit of free tech support for my customers. In the end I felt it was worth it as they raved about the quality of support and it was a real differentiator - not to mention built a lot of brand loyalty (and internal staff loyalty too once I grew enough to build out a team - they derived real satisfaction from actually solving problems instead of playing ping-pong).
I agree regarding the need to triage at scale, unfortunately most large companies I've encountered fail to do this well and seem ill-equipped to accept high quality bug reports of edge case defects generated by expert users (save for the odd exception that arrives by social media from someone who happens to have enough followers to get their attention outside the regular support pipeline).
In my experience this doesn't usually boil down to a systems issue (the ticketing systems etc. exist that should theoretically allow for eventual escalation to the right engineer/developer) but a corporate culture thing (the company just doesn't prioritize customer feedback especially at the level where staff who actually deal with customers interface with the teams that write/maintain the software). Often it's genuinely valued at the C-level (the Bezos story of calling Amazon"s tech support line during an exec meeting is a fun example) but diluted somewhere between them and the rank-and-file.
(Ps. I'm not arguing with you and appreciate you took the time to craft a thoughtful reply)
It should be the other way around - at billion customer scale you should be responsible for how your product interacts with other software whose developers have less resources than you.
My guess it's just the emergent behavior that results when a company doesn't provide developers time to fix bugs.
If their week is already booked full just trying to keep up with the roadmap deadlines, a bug ticket feels like being tossed a 25lb weight when you're drowning.
You could say: "but have pride in your work!"
But if your company only values shipping, not fixing, that attitude doesn't make it through the first performance review.
What I've found to be most effective for program management is to set aside a maintenance team separate from the feature teams. The roadmap is then planned without counting anything for the maintenance team and they deal with bug tickets as they come in. Rotate the assignment periodically so that every developer has to occasionally spend a few months on the maintenance team.
Doesn’t this lead to problems like the feature team pushing buggy code and having no accountability or responsibility to deal with it?
My preference is to treat the defects like feature work, size and plan. Yes you might not get all the feature work done but the team is accountable for everything they make
There's a lot more to effective program quality management than I can explain in a comment here. Forcing all developers to rotate through the maintenance team is one incentive not to ship crap because they might end up having to deal with it anyway. But more importantly you have to shift left the quality assurance and control activities to minimize the risk of defect leakage in the first place. And set up a closed-loop system where any leaked defect triggers a rigorous root-cause analysis that results in further process improvement.
You’ve just described AGILE development, a way for product owners to backlog code rot while empowering developers to feel like they have a say in things.
Yep. On the other side of the curtain this often isn't nefarious. It's a simple cost/benefit analysis of spending time on something that one user is complaining about versus a backlog of higher business priorities. I've seen this in my work and it makes me sad for the user, but it often does take a bit of effort to spear these bug reports through.
I totally understand that from the perspective of individual employees: they have little incentive to do more than the bare minimum to close tickets. But this behavior is typically a symptom of broken corporate culture and failure to align internal metrics. For every customer who takes the trouble to submit a formal bug report there are likely many others who just live with it, and badmouth you to other customers. Doing deep investigations of even minor bug reports also tends to expose other, more serious latent bugs. And root cause analysis allows you to create closed-loop solutions to prevent similar future bugs.
Large monopolistic tech companies like Apple and Microsoft can afford to ignore this stuff for years because there are few realistic alternatives. But longer term eventually a disruptive competitor comes along who takes product quality and customer service more seriously.
There's also going to be mountains of bugs resulting from cosmic rays hitting the computer, defective ram chips, weird modifications of the system the reporter hasn't mentioned.
You could sink an infinite amount of time investigating and find nothing. At some point you have to cut off the time investment when only one person has reported it and no devs have been able to reproduce it.
Quite often reproduction information will only reproduce the bug in the customers environment, hence there is a lot of incomplete state on what is actually causing the problem.
It's pretty terrible in enterprise because there is so much 3rd party crap touching things it shouldn't that love to cause fun problems.
There's obviously some nuance here, but the fact is that much modern software is riddled with bugs, and this is sub-optimal for everyone (both software users and software builders). Most of the bugs which frustrate and irritate software users are not due to uncontrollable events such as cosmic rays flipping a bit. Most of them are plain old code defects.
But, you do have a valid point. Allow me to rephrase it this way: The answer is not for software companies to spend unbounded amounts of engineer time chasing every reported bug.
But there are ways that we, as an industry, can do better, and it's not by pouring all our time into chasing hard-to-diagnose bugs. Here are a few ways that I personally see:
1. Some very powerful technologies for finding many bugs with little engineering effort already exist, but are not widely used. As an example, coverage-guided fuzzing is amazingly good at finding all kinds of obscure bugs. The idea of coverage-guided fuzzing was known from the 1990's, but it took AFL (in ~2013) to take it mainstream. Even now, much of the industry is not benefiting from the awesome power of coverage-guided fuzzing. And there are other, equally powerful techniques which have been known for a long time, but are even less accessible to most software developers.
So: spread the word about such techniques, and for programming language/platform developers, work on making them more easily applicable. This could help many software companies to catch a great number of bugs before they ever go to production.
2. Similarly, there are extant historical computing systems which had very powerful debugging facilities, much better than what is currently available to most developers. The ideas on how to make our platforms more debuggable are already out there; it's now a matter of popularizing those ideas and making them readily accessible and applicable.
3. Since it's widely known that many bugs (real bugs, not "cosmic rays") are extremely hard to reproduce, an admirable target for us to aim for as developers is to implement debug logging in a way which allows us to root-cause most obscure bugs just by examining the logs (i.e. no need to search for a reproducer). Some real-world systems have achieved that goal, with very good results.
4. While there is currently much buzz about using LLM-based coding agents to write code, I think an almost better use case for coding agents is in triaging bug reports, diagnosing the bugs, finding reproducers, etc.
I've recently had a couple of shocking experiences where, just given a written description of an intermittent, hard-to-diagnose bug, a coding agent was able to search an entire codebase, identify the exact cause, and write a reproducer test case. (And this after multiple experienced human programmers had looked at the issue but failed to identify the cause.)
In summary, I think there are ways to "cut the Gordian knot" of bug reports.
What if no devs even tried to reproduce it, and they have no reason to believe they've fixed the bug with any other changes?
That seems to be the case described in the article. In such a situation, I think it's dishonest to ask the reporter to expend even more effort when you've spent zero. Just close it if you don't want to do it, you don't have to be a jerk to your customers, too, by sending them off on a wild goose chase.
Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
> Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
Right. The problem isn’t closing the ticket, it’s pretending more work is happening than actually is.
“Needs verification” is fine if someone has actually tried to reproduce it. Otherwise it’s just a nicer way of saying “we’re not going to look at this.”
Most of the time, there isn't any reason to believe it could be fixed, i.e. there were not any non-trivial changes around that area. What you're describing happens less frequently, and in such cases, the devs should discuss that with the reporter.
In this specific example, it looks like Apple gave no indications that such changes had happened, and no indications they had even spent a nonzero amount of effort following the reproduction instructions with either the old code or the new code.
"Please consider cosmic rays hitting the computer, defective ram chips, weird modifications of the system before submitting the bug. Unlesss you explicitly acknowledge that, your bug will be closed automatically in 30 days. Thank you very much"
> For every customer who takes the trouble to submit a formal bug report there are likely many others who just live with it
This reminds me of a fairly old but famous story about ignoring bugs from Linux users. I couldn't find the HN post but here's slashdot
| Though only 5.8% of his game's buyers were playing on Linux, they generated over 38% of the bug reports. Not because the Linux platform was buggier, either. Only 3 of the roughly 400 bug reports submitted by Linux users were platform specific
The short is that they initially ignored it, triaging, but it was a mistake. Especially since the culture of Linux users is to submit more detailed bug reports. That their submissions help general users.
Don't just a bug report by its cover, judge it by its merits. We're all biased to dismiss them and find an excuse to ignore them. But that just leads to bad software.
Funny you mention Microsoft because I used to see bug reports for Windows. I can tell you there was a ton of low quality "SOMEONE HACKED MY COMPUTER" or similar feedback (and sometimes just unintelligible ranting) that was completely inactionable or unreproducible. I otherwise do agree with your premise that large monopolistic businesses can sit on large swaths of feedback without worrying about competition - and that this is a problem.
However, for most software projects and businesses, the lack of repeated feedback is a signal that the issue isn't important.
As a user I would hope that the software author/publisher is prioritizing important problems. Closing one ticket is not indicative of organizational rot, as you say.
Yeah competition works. I don't like nexus that much but they accept every ticket I've opened and fix it the next release. Two things I think affect that. One, my ticket has the name of a fortune 100 next to it. Two, artifactory will eat them alive if they don't keep customers happy.
> this often isn't nefarious. It's a simple cost/benefit analysis of spending time on something that one user is complaining about versus a backlog of higher business priorities.
You can triage without closing tickets. So it is nefarious. It is metric hacking
If you're having trouble reproducing, tag "needs verification" or something else. But closing a ticket isn't triaging, it is sweeping problems under the rug
It's a false dichotomy - something being "a simple cost/benefit analysis" doesn't remove the ethical dimension, and can absolutely be nefarious. A movie villain saying "it was just business" doesn't make their actions less villainous.
I’d argue that there should be no higher business priority than shipping a product you already sold. If you sold a product and your customer spends their time documenting exactly why and how you sold them something that’s broken, you should make that a high priority. As a natural progression, you’ll start shipping less buggy / better tested products and that’s how you unlock yourself from the obligation you made to your existing customers to do other work.
Not directed at you of course, just the proverbial “you” from the frustration of a purchaser of software.
Careful saying that too loudly, the “ship new features at all costs” gang will come for your head. They don’t approve of things like “quality software” and “making stuff that works past the demo and cursory inspection” or “actual user utility”.
I can sort of back that for desktop apps but telemetry is so trivial for webapps needing a reproducer is almost an embarrassing admission the operator has no clue what they're doing.
Error tracking and tracing make it fairly straight forward to retroactively troubleshoot unreproducible issues.
The telemetry is almost always not voluntary and neither is the relation ship with the companies in the first place unless you mean that technically you could become a hermit living in the woods.
Or even non-software tickets at large corporations. I reported a water dispenser filling too slowly at my office because it took me a few tries just to fill my 1L water bottle. They said it was fixed and closed it.
It was not fixed. So I took a video of myself refilling my water bottle, attached it to the ticket, and re-opened it. They actually fixed it after that. The video was 2m12s long (and I spent god knows how long making the video file small enough to attach to the ticket lol)
this is actually a good example of how a more detailed issue will have a higher chance to be addressed. I don't know what information that's your previous report is lacking, but the video certainly give more information that the maintainer can pinpoint the cause and act on it. The ability to pinpoint the cause from the report is a godsent for maintainers, it drastically reduce the time to investigate the cause, thus able to act immediately.
Some of the information in this can may be:
* how "slow" exactly the process is related with normal behavior. If it's just said "slow" on previous report, it's easy to be dismissed
* the dispenser's behavior, such as if the water flow is consistently low volume or clogged intermittently, or if the dispenser is struggling to fetch from water source, etc
I'd say it was both. I gave a pretty detailed explanation before, far more detailed than my post here, including a timeline of when it filled in one shot, then two shots, and then three or four (can't remember). I doubt they actually checked before the video. But I was very motivated to fix the issue so I gave them proof lol
More importantly it shows how the reporter actually used the system to trigger the undesired behavior. Just because something is obvious to you doesn't mean it will be obvious to whoever is looking at the bug report.
As an open source maintainer, I feel that statement is really unfair. Yes, we do sometimes close bug reports without evidence they are fixed. But:
- We owe you nothing! And the fact that people still expect maintainers to work for them is really sad, IMHO.
- Unlike corporate workers, nobody is measuring our productivity therefore we have no incentive to close issues if we believe they are unfixed. That means that when we close the issue, we believe it has a high chance of being fixed, and also we weigh the cost of having many maybe-fixed open issues against maybe closing a standing issue, and (try to) choose what's best for the project.
It's not about expectation of work (well, there's some entitled people sure.)
It's about throwing away the effort the reporter put into filing the issue. Stale bots disincentivise good quality issues, makes them less discoverable, and creates the burden of having to collate discussion across N previously raised issues about the same thing.
Bug reports and FRs are also a form of work. They might have a selfish motive, but they're still raised with the intention of enriching the software in some way.
IMO closing issues via stale bot is fine, the problem is locking issues so that no further conversation is allowed on the issue. Multiple times, I've encountered multi-year old issues (which is usually not fixed due to the fix not being simple or compatible with the current architecture). There's usually a good amount of conversation between users offering workarounds (and those workarounds updated for newer versions) - till stale bot locks the issue.
This 1000%. Whoever came up with the idea of closing and locking issues because no one has posted on them for awhile is at best not all that bright and at worst downright sinister.
Closing an issue due to staleness is one thing, locking it is another.
> That means that when we close the issue, we believe it has a high chance of being fixed
I agree with this iff it's being done manually after reading the issue. stalebot is indiscriminate and as far as "owing" the user, that's fair, but I'd assume that the person reporting the bug is also doing you a favor by helping you make things more stable and contributing to your repo/tool's community.
I partially agree, but even with stalebots nobody is measuring the maintainers' productivity. So when they made the choice to use stalebots, they did that because they believe that's best for the project. It's different from corporate.
Nobody is measuring their productivity, but people definitely look at how many open issues they have and potentially how long those issues have existed. They’re likely incentivized to close issues for appearances.
With a popular open source project, you'll quickly get to a number of bug reports that you have no chance of ever solving. You will have to focus on the worst ones and ones affecting most users.
At the same time, you want to communicate to users that this is the case so they don't have wrong expectation. But also, psychologically it is demotivating to have a 1000+ open bugs queue with no capacity to re-triage and only two maintainers able to out a few fours in every month or every week.
In open source, "won't fix" means either "not in scope — feel free to fork" or "no capacity ever expected — feel free to provide a fix".
The optimization problem is how do you get the most out of very limited time from very few people, and having 1000+ open bugs that nobody can keep in their head or look for duplicates in is mentally draining and stops the devs from fixing even the top 3 bugs users do face.
The problem is that your users also have limited time and if it's clear you're not even looking at issues where someone has put in lots of effort to help you then you're only going to get lazy issues and it will actually take more effort from you to do all that work yourself if you want to reach the same software quality.
I think you are missing the point: a user putting in a lot of effort into a bug report is usually trying to help themselves get the bug fixed.
As a maintainer, you will obviously look at that bug with more appreciation: but if you estimate it will take you 3 months of active development to fix it that you will have to spread over a full year of your weekends (which you can't afford), what would you do?
And what would a reasonable user rather see? Yes, this is an issue, but very hard to fix, and I don't have the time, or just letting the bug linger?
> We owe you nothing! And the fact that people still expect maintainers to work for them is really sad, IMHO.
Users also don't owe you anything either. Auto-closing reports without even looking at them is like asking for donations only to throw 90% of what you get straight into the trash. Not cool. If you don't want bug reports, state that up front or at least leave bugs open for other users to see and talk about. Otherwise, users are free to warn others to stay away from you and your projects.
And that's before getting into more complex issues like what responsibility you have if you take on maintenance of existing software and end up breaking something what was working perfectly for some users.
> Unlike corporate workers, nobody is measuring our productivity therefore we have no incentive to close issues if we believe they are unfixed.
There are plenty incentives, e.g. pride.
> That means that when we close the issue, we believe it has a high chance of being fixed, and also we weigh the cost of having many maybe-fixed open issues against maybe closing a standing issue, and (try to) choose what's best for the project.
That's fine, but bots that auto-close issues unless the reporter dances for them is the opposite of that.
I got reeeeally good at producing repro gifs that I could plug straight inline into email replies to "can't repro"; it's forever clear that most developers either don't know how to test the product they are building, or simply can't be bothered to try.
> Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
I'm not going to lie. That's not who I am. If Apple really wants to close a bug report when the bug isn't fixed, that's on their conscience, if they have one.
I've been on both sides of this. Absolutely sucks as a user falling prey to stalebot or some poor sap pretending to be stalebot, but when I was working in enterprise tech support it was a huge relief to close a case and get it off my plate, for any reason. We had to take 2 new cases per day minimum, update each case (I often had 20+) every few days minimum. Only a small minority were a quick and easy solution (like security vulns with a fix ready we could send the customer were the easiest). We were stuck with our cases also, you couldn't give them to someone else unless you were out sick pretty much, and you'd get them back when you returned unless by some miracle the other guy fixed their problem. An inactivity close on a stalled case was comforting, I was finally free. I think they started cracking down after a bit and said you had to check in with them 3 times x days apart first instead of just immediately closing after no word for 2 weeks, and then they wanted you to call them first. Absolute nightmare.
I think as long as the issue isn't stuck with any one person then it's easier to leave open until it's actually fixed, like the 20+ year old Mozilla bug reports. Big corpo bureaucratic nonsense just ruins everything.
If you are a veteran of software in a big company, we all know there will be weekly or bi-weekly meetings that some PM will set up. All the PM will do is go over the JIRA tickets and be like "is this still happening". Default answer is "no", as in "I didn't even try to reproduce it, do you think I have time to even do it?". Default answer by spineless QA person is also "didn't try it again yet". Then, the PM closes the ticket. It is much easier for QA person to say "Yes I verified it" if you are remote and developer cannot see the lies on your bad poker face.
Ooh this gives me an interesting passive-aggressive idea to counter pointless "is this still relevant" questions. "No, I haven't hit this in the last 2 days." "No, I haven't hit this since I gave up trying to do it with your tool." And so forth.
The less passive-aggressive version is to use this obviously-unhelpful answer of the obviously-unhelpful question, to actually have a conversation to get the PM to recognize that the default state of a ticket is in fact "no change." Ultimately that may turn into a stale bot if the PM realizes the policy they actually want is some sort of timeout, but at least it's not a time consuming meeting!
(Note, a cathartic thought experiment, but not really good manners to actually do!)
I've worked with enterprise software. The result its that people will eventually just wait a few hours/days and lie if they even care enough to do that. The perverse incentives destroy what utility a bug tracker could bring. Int theory transparency could help by changing the incentives if third parties analyze the metrics and call out bullshit to an audicence that matters.
> the developer will push back on the bug author and say "I can't reproduce this, can you verify it with the latest version?" without actually doing anything.
We do this. Because frankly, very often the bug has been reported by others and has been fixed, we just can't connect the dots in our ticketing system.
That's of course less than ideal, but given that a lot of tickets we get are often very poorly described it's hard. It's one aspect I have genuine hope AI can help us with.
Is your argument "it's bad everywhere, so it's ok"? As a software developer I do understand how enterprises operate, as a customer and a user I'd put Apple under higher scrutiny and would expect better.
I wish someone had told me how common this was back when I worked myself to death fixing every UI abnormality that no one except some misincentivized testers used to report at my first job. At the time I thought it was dishonest to say something was irreproducible and it'd be beneath me to patch an issue knowing it'll sprout ten others.
I'm proud of fixing everything properly but I won't repeat it ever unless the company actually has that high a bar across the board.
Hi, bigcorp employee getting showered with tickets here.
I don't have enough time in the day to deal with the tickets where the reporter actually tries, let alone the tickets where they don't.
If I tell you to update your shit, it's because it's wildly out of date, to the point that your configuration is impossible for me to reproduce without fucking up my setup to the point that I can't repro 8 other tickets.
Back when I worked at Apple I would just try it in whatever I had installed. If it didn't reproduce I'd write "Cannot reproduce in 10.x.x" and close it. Maybe a third were like that, duplicates of some other issue that was resolved long ago.
Anyone that attached a repro file to their issue got attention because it was easy enough to test. Sometimes crash traces got attention, I'd open the code and check out what it was. If it was like a top 15 crash trace then I'd spend a lot longer on it.
If the ticket was long and involved like "make an iMovie and tween it in just such and such a way" then probably I'd fiddle around for 10-15 minutes before downgrading its priority and hope a repro file would come about.
There were a bunch of bug reports for a deprecated codec that I closed and one guy angrily replied that I couldn't just close issues I didn't want to fix!
Guess what buddy, nobody's ever going to fix it.
The oldest bug like that I ever fixed was a QuickDraw bug that was originally written when I was 8 years old but it was just an easy bounds check one liner.
But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
> But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
Except this same shit keeps happening with multiple teams.
Judging from your mention of QuickDraw, which was removed entirely from macOS in 2012, perhaps your Apple experience is now out of date.
> That the ~50000 engineers at Apple are conspiring to close your tickets in the exact same way. It's ridiculou
It's pretty clear from experience that the organization policy is to not provide feedback on bug submissions. Getting a 'check it if still reproduces or we'll close it in two weeks' message after 3 years is actually a fast turnaround.
Best I've gotten was on an issue I routed to a friend who worked at Apple who promised it would get looked at, but that I wouldn't hear back.
Microsoft wouldn't fix my issues either, but at least they got back to me in a timely fashion. Usually telling me it was a known issue that they weren't going to fix.
You don’t hear back because almost always your bug is a duplicate of some other one. They can’t share the original with you because it contains data from another customer or from inside the company.
Almost nobody is the first reporter in an OS with billions of users. The only useful thing about those long dupe lists was being able to scan them for one with easier repro steps.
But sometimes that duplicate marking is wrong or some subtly different issue so they ask you if it still reproduces in whatever version contains the fix before closing it.
That makes sense. But when you take 3-5 years to respond to my bug report, I'm going to take at least 3 months to respond to your response. And I'm probably not filing more bugs, because chances are I won't be at my current employer by the time you reply.
When you consitently burn bug reporters, sooner or later there's nobody to file bugs.
Because that's probably how long it took for someone to prioritize it.
Even if it's not fixed by the dupe ticket, the volume of bug reports makes it almost certain another ticket for the same issue will come up. And if it doesn't then it probably wasn't that relevant to anyone.
Not my tickets specifically. I don't think they're out to get me individually. On the contrary, this is a common practice, which affects many developers. I just happen to be relatively loud, as far as blogging is concerned.
Yes I understand that. ~50000 engineers aren't conspiring to close all tickets that way. It's a stupid line of thinking.
More than likely your steps to reproduce are too laborious to receive attention relative to the value fixing the bug would provide. That's why they're asking you to verify it still happens. Seems pretty simple right?
There's also a strong chance your ticket was linked as a duplicate of some other issue that was fixed in the beta and they want you to verify that's the case but they won't expose their internal issue to you for a variety of reasons.
> ~50000 engineers aren't conspiring to close all tickets that way.
I didn't say that either. It's happened to me only sporadically, but multiple times.
I agree with you that teams within Apple manage their own tickets. Perhaps some individual teams are declaring bug bankruptcy at some point, so only their bugs would go out for verification. I don't really know. I wish I did. What I do know is that multiple teams have done this at different points.
There's indisputably a company-wide DevBugs canned response for this. It's the same exact language every time. You can even Google it.
I think that's entirely dependent on the workload the company is placing on their support staff. If Apple decides the techs should be handling 10 tickets at once, then the techs have a choice:
1. Tell everyone to update their shit, and close tickets if they don't.
2. Waste several hours per day uninstalling and reinstalling 10 versions of the same program.
One of these will allow you to close lots of tickets immediately, and handle the remaining ones as efficiently as possible. Yay! Good job, peon! You get a raise!
The other approach will result in a deep backlog, slow turnaround times, and lower apparent output from management's perspective. Boo! Bad job, peon! You're fired!
Please tell us where you work so we can avoid all of your company’s software. Unless it’s Microsoft, because we’ve already seen the results of that attitude there.
I don't see how it's an unreasonable request. If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically. You will be prioritized last, because my effectiveness is measured by how many tickets I close.
They're not. If there's nothing wrong with it, one could ask whether the person here would be okay sitting in a room with their supervisor, the head of the company, and 10 customers, say the same things they're saying here, and get a consensus that this is how this should all work out.
> If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically.
You completely missed the point of the blog post. Apple was in the process of developing macOS 26.4 beta 4, and they wanted me to install the beta just to "verify" the bug.
Apple could test my bug with 26.4 beta 4 a heck of a lot easier than I could. Nobody was asking Apple to install some ancient version.
> my effectiveness is measured by how many tickets I close.
That was one of the points of the blog post: this is a perverse incentive from management.
Note what you did not say: "my effectiveness is measured by how many bugs I fix." So engineers are incentivized to close tickets even if the bugs they report are unfixed. This is how a company ends up with crappy, buggy software.
The parent comment is talking about the broader practice of people telling you to update and then repro again. That's a completely legitimate thing to ask, given both the perverse corporate incentives and the basic reality that version toggling makes a tech far less efficient at solving all tickets, not just yours.
I also hate this pressure of it being on the user to come up with a minimal reproducing example. That means that any bug of any moderate complexity will never get fixed because you can't always reduce them to a few steps and they may be statistical.
A bug is a bug, no matter the developers' opinion or the complexity of the bug.
However there are "bugs" that actually do turn out to be just cosmic rays flipping bits or plain user error. If you as the reporter don't provide enough information for the developer to be sure they are not going on a wild goose chase then it's fair for the developer to not invest too much time.
Sure. But I'm biased because I was a "customer" of a former (large) company's products while also working at that company. So the bugs I would file were the type that a customer would file, but since I was inside, I saw how they were handled. The tactics that my fellow R&D developers would do to claim something wasn't a bug or reproducible were nearly endless.
I mean if the customer stops complaining, either the bug was fixed, or the bug was not too important to begin with, or they are not a customer anymore and nobody else cares about that niche bug. In all of the above closing the ticket sounds reasonable.
What is not reasonable is that they close issues with thousands of “I have this issue too” with active complains and full repros
> perhaps praying that the bug had magically disappeared on its own, with no effort from Apple.
I suspect that this is a common approach. It maybe even works, often enough, to make it standard practice.
For myself, I've stopped submitting bug reports.
It's not the being ignored, that bothers me; it's when they pay attention, they basically insist that I become an unpaid systems engineering QC person, and go through enormous effort to prove the bug exists.
> they basically insist that I become an unpaid systems engineering QC person
Microsoft support is guilty of this, especially for Azure & 365 issues.
Like sorry, but you aren't paying me to debug your software. Here's a report, and here's proof of me reproducing the problem & some logs. That's all I'm going to provide. It's your software, you debug it.
And based on my own personal experience, even if you persevere and force them to acknowledge the problematic behaviour, they can turn around and say it's not a problem and working as intended.
For example, when using Azure Front Door, it's apparently absolutely not a problem that as yet un-cached file in their CDN downloads from their own Azure Blob storage have a maximum download speed of around 2MB (16Mb) per second:
Them:
> Hello Jonathan,
> I hope you are doing well!
> I sincerely apologize for the significant delay in our response, which was necessary to conduct further internal testing.
> After a comprehensive review, we have determined that the behavior you are experiencing is typical for this type of operation.
> This is primarily due to the connection not being entirely directly, as it must pass through Azure Front Door. This process also involves distributing the cache among point-of-presence (POP) servers, which inevitably impacts the > operation's speed. Let me provide you with documentation covering that matter:
Me:
> So to be clear, Azure Front Door maxes out at less than 2MB/s (16Mbit/s) for uncached items even when everything is on Microsoft’s own servers?
Them:
> Hello Jonathan,
> Thank you for getting back to me.
> These values may vary by region, but those particular ones apply for South Africa North.
I also tested this behaviour in US and EU regions (from an Azure VM requesting a file from Azure Blob storage in the same region as the VM but via Azure Front Door) and in EU it was also similarly limited while in the US it was only a tiny bit better.
We use Cloudflare now, cheaper, faster, configuration UI which isn't painfully slow. Not without their own recent incidents, but better than Azure Front Door 99.99% of the time.
While I'd love to take your tack, unfortunately, I find that if I actually want the fix, I have to become their unpaid engineer.
Which is ridiculous, because at the same time my company is paying a separate support fee, large enough to literally employ a dedicated engineer for my company!
I will do the work for them (typically paid for by my employer) iff I can expect them to fix it.
Blackbox debugging is a PITA, which is part of why I prefer open source, but it is what it is... If something is broken, and I can get it fixed by putting in the time to get a good report, and etc and they fix the thing, then I'll do it.
But if they don't fix the stuff, I have no shortage of things to fix myself.
Damn. I've put quite a lot of effort into open source tools w.r.t. debugging and bugfixing, but yeah putting that for a corporate product that doesn't even respect you must be draining.
I filed an issue, they told me to get in GitHub, fix it and file a PR myself O_o. In fairness they bugged me and cajoled me and offered to help until I actually did it, which I actually appreciated in the end because the experience was useful. (Give a man a fish, etc...)
All kinds of open source projects do this too. It's really annoying. It's one thing if the authors actually try and fail to verify the bug, but these days it seems like most projects just close "stale" bugs as a matter of course. This is equivalent to assuming that any given bug is automatically fixed after X amount of time, which is pretty absurd.
It's rather unreasonable to be annoyed. The maintainers may have entirely different priorities, which is fine. They're also likely being spammed with low-effort bug reports (not yours necessarily but from others).
The great thing about open source projects you can just fix the bug yourself and submit a PR, or fork the whole project if the maintainers won't merge your changes. If you don't have the time or skills yourself then you can even pay a contractor to do it for you.
I disagree. If you discover that a bug that makes an open source library unusable to you, after spending time on learning and using that library, and the authors close the bug as a wontfix, I think being annoyed is quite reasonable, even expected.
If that type of thing annoys you then you should restrict your use of open source projects to those backed by corporations with a paid support business model.
It's open source software. If you discover the bug, have written a failing test that demonstrates it, and a proposed solution to it, then maybe you can be annoyed when the authors close it as wontfix.
Otherwise OSS is pretty much as-is, where-is, with the exception of very widely used and corporately supported projects.
If the maintainer merely doesn't fix the bug, then yes. If they close the bug report so it gets lost and other contributors are discouraged from working on it, then no.
Closed reports are not lost, they are still searchable/linkable, they are just not in the list of work to do.
This is entirely up to the maintainer, who puts in the work and gives up their time/money to do so. If you want to be in charge on a given repo, put in the work and become a real contributor, if not accept the rules the maintainers choose.
Dupe reports are a signal all by themselves, that's really not harmful, nor does something being closed implied solved.
You shouldn't presume to know what is best for an open source maintainer of any given project - projects vary, reports vary in quality, and the job of maintenance is not an easy one.
It is not about "priorities". Putting work into writing bug is utterly useless, because if the maintainer does not fix it in 2 months, the bug will be closed as stale. I am actually fine with open source project maintainer to prioritize stuff and hey, maybe that bug will be fixed in 6 months when they have time. But I am not fine being told "we did not had time for 2 moths, therefore we are closing your bug" as is standard on github now.
Do not throw the ball back with "this happens because bugs are not well written". Stalebot closes bug regardless of whether they are well written and ensures no one will put effort into writing another well written bug again.
Is that a serious question? It works like any contract programming gig. You give the contractor money and in exchange they give you code (including copyright assignment). You can go through a freelancer site like Upwork if you don't know an appropriate contractor yourself.
> Adding filters so that developers only look at actionable tickets would be much more sane.
That's a reasonable approach, but I don't understand how it's any more or less sane than autoclosing them with a stale label.
Whether these sorts of bugs are "open but stale" or "closed because stale" seems like it depends on whether the project defines "closed" as "no work planned" or "fixed", which both seem valid.
Either way these bugs will be hidden from developer dashboards but still available in the database so there's no practical difference, you just need to make sure everyone is on the same page about the meaning of "closed".
There is some bot that will match your issue to some other 3 vaguely related issue, then auto close in 3 days. The other vaguely related issues are auto closed for inactivity. Nothing is ever fixed, which is why they can't keep the thing from messing with your scroll position for years now.
I recognize that this is annoying from a user perspective, but I do understand it. Not all bugs are easily reproducible (and even if they are 100% reproducible for the user, it's not always so easy for the developers). Also sometimes you make a change to the code that you think might be in a related area, and so sometimes the most "efficient" thing is just to ask the user to re-test.
When I close an old bug that is not actionable, I do feel bad about it. But keeping the bug open when realistically I can't really do anything with it might be worse.
Back in another part of my career I worked a lot with putting Macs on ActiveDirectory. And there was a common refrain from Apple about bugs in that implementation: "works on 17!".
The joke is that Apple owns the 17.x.x.x class-A range on the Internet (they got in early, the also have a second class-B and used to have a second class-B that they gave back), and what engineers were really saying is that they could not reproduce on the AD systems that Apple had setup (lots of times it was because AD had been setup with a .local domain, a real no-no, but it was in Microsoft's training materials as an example at the time...).
I used to think that there is no harm in keeping the bug open. I think if you honestly feel that you have the time and resources to go back to the bug and fix it, then by all means keep it open.
But I find that sometimes I can tell from experience that the IR is not actionable and that it will never be fixed. Some examples:
* There's not enough info to reproduce the issue and the user either can't or won't be able to reproduce it themselves. Intermittent bugs generally fall into this category.
* The bug was filed against some version of the software that's no longer in production (think of the cloud context where the backend service has been upgraded to a newer version).
Sometimes the cost to investigate a bug is so high relative to the pain caused that it just closed as a WONTFIX. These sometimes suck the most because they are often legitimate bugs with possible fixes, but they will never be prioritized high enough to get fixed.
Or sometimes the bug is only reproducible using some proprietary data that I don't have access to and so you sometimes have no choice but to ask the bug filer "can you still reproduce this?".
Computer systems are complicated. And real-world systems consisting of multiple computer systems are even more complicated.
I think asking someone if they can still reproduce an issue is valid. Especially if it was trivially reproducible for them, and now it isn't, that seems like a fine resolution, and the bug should be closed.
But in the other cases, closing the bug seems to me to be a way to perturb metrics. It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?
For your specific scenarios:
- lack of information on how to reproduce or resolve a bug doesn't mean it doesn't exist, just that it's not well understood.
- For the "new version" claim, I've seen literal complete rewrites contain the same defects as the previous version. IMHO the author of the new version needs to confirm that the bug is fixed (and how/why it was fixed)
- I agree there are high cost bugs that nobody has resources to fix, but again, that doesn't mean they don't exist (important for errata)
- Similarly with proprietary data, if you aren't allowed to access it, but it still triggers the bug, then the defect exists
In general my philosophy is to treat the existence of open bugs as the authoritative record of known issues. Yes, some of them will never be solved. But having them in the record is important in and of itself.
> It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?
Yes, fully agreed. But closing a bug doesn't preclude that. A closed bug isn't refutation or denial of a defect. It's just an indication that there is no plan to fix the bug. Not every bug system works like this though. My bug tracker works like this, and I should have more clearly described what a "closed bug" is in my earlier posts.
Bug reports are not known defects, at any kind of scale half of them will be already fixed, misunderstandings, bad data in, or related to an unusual setup.
Closing the bug is a way of saying: sorry this doesn’t look too important and we don’t have time to look at this given the other more important things (bugs/features) we plan to work on.
If it’s closed as stale after 6-12 months (multiple humans will have seen it) OR triaged by a human and marked as won’t fix I think that’s reasonable.
> at any kind of scale half of them will be already fixed, misunderstandings, bad data in,
Here you're referring to a class of bug reports that's uninteresting for this discussion, because they're invalid (i.e. they don't represent an actual bug). We're talking about valid bugs that have not been fixed.
> or related to an unusual setup
Unusual, but ostensibly supported? Then there exists a bug.
What is the use in keeping it open when no one will ever look at it again after it goes stale? It still exists in the system if you ever wanted to find it again or if someone reports the same issue again. But after a certain time without reconfirming the bug exists, there is no point investigating because you will never know if you just haven't found it yet or if it was fixed already.
See my reply to eminence32 - bug tracking serves as a list of known defects, not as a list of work the engineers are going to do this [day/month/year].
The primary purpose is not usually a list of known defects and many ‘bugs’ are not actually bugs but feature requests or misunderstandings from users (e.g. RFC disallows the data you want my html parser to allow).
The people who filed them would disagree and many would vehemently argue that their bug is in fact a bug, and is the most important bug and how dare you close it.
Conversely, what's the harm in closing the bug? (As long as you don't lock or delete it, I agree that's bad.)
People focused on the work often interpret "open" to mean "requires work" and "closed" to mean "no planned work" in which case keeping an unreproducible bug open is dishonest because it falsely implies that someone might continue to work on it.
Whereas people focused on the problem often interpret "open" to mean "not fixed" and "closed" to mean "fixed" in which case closing an unreproducible bug is dishonest because it falsely implies that it's no longer a problem.
Neither seems right or wrong as long as everyone on the project agrees which interpretation you're using.
How is that worse? Leaving it open signals to anyone searching about it that's it's still an issue of concern. It will show up in filters for active bugs, etc. Closing it without fixing it just obfuscates the situation. It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.
To some people "open" means "not fixed" whereas to others it means "more work planned". I've worked on projects with both interpretations and it's fine as long as everyone is on the same page.
> It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.
In our case we omit bugs we couldn't reproduce from the issues list due to practicality, not pride -- our software has tens of thousands of unreproducible bugs and having them show up in reports would drown out planned work.
And it's not like anyone deleted or locked the unreproducible bugs, they are either tracked as "open but unreproducible" or "closed because unreproducible". Either way they're still in the database in case more information comes along, but still filtered out of the vast majority of dashboards.
Apple did not say they couldn't reproduce it. Neither did they say that they thought they fixed it. They refused to say anything except "Verify with macOS 26.4 beta 4".
> and even if they are 100% reproducible for the user, it's not always so easy for the developers
It's not easy for the user! Like I said in the blog post, I don't usually run the betas, so it would have been an ordeal to install macOS 26.4 beta 4 just to test this one bug. If anything, it's easier for Apple to test when they're developing the beta.
> the most "efficient" thing is just to ask the user to re-test.
Efficient from Apple's perspective, but grossly inefficient from the bug reporter's perspective.
> realistically I can't really do anything with it
In this case, I provided Apple with a sample Xcode project and explicit steps to reproduce. So realistically, they could have tried that.
I suspect that your underlying assumption is incorrect: I don't think Apple did anything with my bug report. This is not the first time Apple has asked me to "verify" an unfixed bug in a beta version. This seems to be a perfunctory thing they do before certain significant OS releases, clear out some older bug reports. Maybe they want to focus now on macOS 27 for WWDC and pretend that there are no outstanding issues remaining. I don't know exactly what's going through their corporate minds, but what spurred me to blog about it is that they keep doing this same shit.
I don't work at Apple, so I can't comment on that. But that doesn't always help. There's been plenty of times where I have a full HAR file from the user and I can clearly see that something went wrong, but that doesn't always mean I can reproduce the issue. (I recognize a HAR file doesn't represent the complete state of the world, but it's often one of the best things a backend developer can get)
It always helps. Even if you can't determine the root cause you can at least add an extra assertion check or logging statement at that point so that next time the bug gets triggered you'll at least get more useful diagnostic data and can get a step close. Iterate until you find the root cause.
That’s easy enough. The hard part is doing so without capturing a bunch of email, messages, and other private data that happens to be in memory at the time.
Ignorant question, if privacy didn’t matter and they had an atomically identical machine, would there still be plenty of edge cases where it was the printer or the Wi-Fi causing the issue?
In any case I would have said it sounds difficult on every front
I should be more precise. Capturing the system state isn’t too hard. Turning that into a reproducer may be quite hard, because of things like you say. There are certainly a lot of bugs that such a capture would make easier to figure out, but it wouldn’t be a panacea.
Story time. I used to work for Facebook (and Google) and lots of games were played around bugs.
At some point the leadership introduced an SLA for high then medium priority bugs. Why? because bugs would sit in queues for years. The result? Bugs would often get downgraded in priority at or close to the SLA. People even wrote automated rules to see if their bugs filed got downgraded to alert them.
Another trick was to throw it back to the user, usually after months, ostensibly to request information, to ask "is this still a problem?" or just adding "could not reproduce". Often you'd get no response. sometimes the person was no longer on the team or with the company. Or they just lost interest or didn't notice. Great, it's off your plate.
If you waited long enough, you could say it was "no longer relevant" because that version of the app or API had been deprecated. It's also a good reason to bounce it back with "is still this relevant?"
Probably the most Machiavellian trick I saw was to merge your bug with another one vaguely similar that you didn't own. Why? Because this was hard to unwind and not always obvious.
Anyone who runs a call center or customer line knows this: you want to throw it back at the customer because a certain percentage will give up. It's a bit like health insurance companies automatically sending a denial for a prior authorization: to make people give up.
I once submitted some clear bugs to a supermarket's app and I got a response asking me to call some 800 number and make a report. My bug report was a complete way to reproduce the issue. I knew what was going on. Somebody simply wanted to mark the issue as "resolved". I'm never going to do that.
I don't think you can trust engineering teams (or, worse, individuals) to "own" bugs. They're not going to want to do them. They need to be owned by a QA team or a program team that will collate similar bugs and verify something is actually fixed.
Google had their own versions of things. IIRC bugs had both a priority and s everity for some reason (they were the same 99% of the time) between 0 and 4. So a standard bug was p2/s2. p0/s0 was the most severe and meant a serious user-facing outage. People would often change a p2/s2 to p3/s3, which basically meant "I'm never going to do this and I will never look at it again".
I've basically given up on filing bug reports because I'm aware of all these games and getting someone to actually pay attention is incredibly difficult. So much of this comes down to stupid organizational-level metrics about bug resolution SLAs and policies.
> Google had their own versions of things. IIRC bugs had both a priority and s everity for some reason (they were the same 99% of the time) between 0 and 4. So a standard bug was p2/s2. p0/s0 was the most severe and meant a serious user-facing outage. People would often change a p2/s2 to p3/s3, which basically meant "I'm never going to do this and I will never look at it again".
Yeah, I've done that. I find it much more honest than automatically closing it as stale or asking the reporter to repeatedly verify it even if I'm not going to work on it. The record still exists that the bug is there. Maybe some day the world will change and I'll have time to work on it.
I'm sure the leadership who set SLAs on medium-priority bugs anticipated a lot of bugs would become low-priority. They forced triage; that's the point.
> People even wrote automated rules to see if their bugs filed got downgraded to alert them.
This part though is a sign people are using the "don't notify" box inappropriately, denying reporters/watchers the opportunity to speak up if they disagree about the downgrade.
> Google had their own versions of things. IIRC bugs had both a priority and severity for some reason (they were the same 99% of the time) between 0 and 4.
At the company I worked with (not Google, but a major one) this was the same. We used Salesforce, the "Lightning Experience" or whatever it was called [0]. Our version was likely customized for our company, but I think the idea was the same - one, I think the "priority", was for our eyes only, one was for the customer (the "severity"). If the customer was insistent on raising the severity, we'd put it as sev1, but the priority was what we actually thought it was. I was actually surprised that for the ~4 years I was there no one made the mistake of telling the customer the priority as a mistake, especially when a lot of people were sloppily copy-pasting text from Slack or other internal tools that sometimes referred to a case as either the severity or the priority.
Those were heavy customers with SLAs, though, not supermarket apps or anything like that.
What was sad was that our internal tools, no matter how badly written, with 90's UI and awful security practices, our tools were 50 times as fast as whatever Salesforce garbage we had to deal with. Of course, there was a lot of unneeded redundancy between the tools so the complexity didn't stay in the Salesforce tool. But somehow the internal tools written by someone 10 years ago, barely maintained, who had to still deal with complex databases of who-what-when-how, felt like you had the DB locally on a supercomputer while SF felt like you were actually asking a very overworked person to manually give you your query right on each click. I'm exaggerating, but just by a bit.
[0] That name was funny because it was slow as shit. Each click took 5 to 20 seconds to update the view. I wonder what the non-Lightning version was.
> IIRC bugs had both a priority and s everity for some reason (they were the same 99% of the time) between 0 and 4. So a standard bug was p2/s2. p0/s0 was the most severe and meant a serious user-facing outage
I've seen this at a couple places... I think it's supposed to help model things like if something is totally down, that's an S0... But if it's the site for the Olympics and it's a year with no Olympics, it's not a P0.
Personally, that kind of detail doesn't seem to matter to me, and it's hard to get people to agree to standards about it, so the data quality isn't likely to be good, so it can't be used for reporting. A single priority value is probably more useful. Priority helps responsible parties decide what issue to fix first, and helps reporters guess when their issue might be addressed.
> People would often change a p2/s2 to p3/s3, which basically meant "I'm never going to do this and I will never look at it again".
I learned this behavior because closing with wontfix would upset people who filed issues for things that I understand, but am not going to change. I'm done with it, but you're going to reopen it if I close it, so whatever, I'll leave it open and ignore it. Stalebot is terrible, but it will accept responsibility for closing these kinds of things.
> I don't think you can trust engineering teams (or, worse, individuals) to "own" bugs. They're not going to want to do them.
I will disagree there. The engineers often want to fix the bugs. Management is telling them they need it's all hands on deck for (Insert company goal here. Probably AI right now).
Followed by management also telling them they have too many bugs, of course. In a condescending tone.
I’ve been dealing with ElevenLabs pulling this same garbage.
I’ll fill out a bug report, wait a few days to a week to get a response, which are often AI generated, and then 48 hours afterward their bot marks it as stale. Telling me to check if it’s still broken or they assume it’s fixed lol
Sad to hear this. How are you submitting the bug report? If you submit an issue via any of our open source repos on GitHub you'll get a timely human response.
Observation: Long, long ago I submitted a bug to Microsoft. I was new at the time and didn't distill it down to the minimum, just gave a scenario that would 100% reproduce. I was contacted months later because someone looked at it and couldn't reproduce.
Yeah, I had found one manifestation of something else that they fixed by the time someone looked at it. The fix in the notes didn't look anything like my bug, only by observing that it now worked I was able to figure out that I had been the blind man trying to describe an elephant.
Former Apple employee here. This is a deeper quirk of Apple culture than one would guess.
Each and every Radar (Apple's internal issue tracker is called Radar, and each issue is called a Radar) follows a state machine, going from the untriaged state to the done state. One hard-coded state in this is Verify. Each and every bug, once Fixed, cannot move to Closed without passing through the Verify state. It seems like a cool idea on the surface. It means that Apple assumes and demands that everything must be verified as fixed (or feature complete) by someone. Quite the corporate value to hold the line on, and it goes back decades.
I seriously hated the Verify state. It caused many pathologies. Imagine trying to run a burndown of your sprint when zero of the Radars are closed, because they have to be verified in production before being closed, meaning you cannot verify until after the release. Another pathology is that lots (thousands and thousands) of Radars end up stranded in Verify. Many, many engineers finish their fix, check it in, it gets released and then they move on. This led to a pathology that the writer of this post got caught up in: There is lots of "org health" reporting that goes out showing how many Radars are unverified and how long your Radars stay in the unverified state on average. A lot of teams simply close Radars that remain unverified for some amount of time because they are being "graded" on this.
> Imagine trying to run a burndown of your sprint when zero of the Radars are closed, because they have to be verified in production before being closed, meaning you cannot verify until after the release.
I think most teams use verify as a "closed" state to hide all that messiness. But sure, zero bugs is a project management fiction and produces perverse outcomes.
I was literally just coming in here to comment "in before someone says this is fine and there's no issue." and the first(!) comment is effectively "this is fine and there's no issue."
The sentiment feels like software folks are optimizing for the local optimum.
It's the programmer equivalent of "if it's important they'll call back." while completely ignoring the real world first and second-order effects of such a policy.
Feeling overwhelmed by insurmountable mountain of bugs and issues is not the way either. We can argue that closing the tickets is not the best way, but if realistically nobody will ever look at them, why not make the developers feel better.
Either you truly need to fix the bugs, in which case the feeling is good and maybe more effort should go that way (more resources assigned to it or whatever), or you're at a scale where tackling everything is impossible and you shouldn't feel overwhelmed by seeing the noise then.
But I think modern industry pretends all it's fine to convince themselves that it's ok to chase the next feature instead.
Move them to a deficated status. “Never triaged”, “lost”, “won’t do”, what have you.
That way, you’re at least not deluding yourself about your own capacity to triage and fix problems, and can hopefully search for and reopen issues that are resurfaced.
It’s really a question of whether a team believes bugs are defects that deserve to be fixed, or annoyances that get in the way of shipping features. And all too often, KPIs and promotions are tied to the features, not the bugs.
Plus, I’ve been in jobs where fixing bugs ends up being implicitly discouraged; if you fix a bug then it invites questions from above for why the bug existed, whether the fix could cause another bug, how another regression will be prevented and so on. But simply ignoring bug reports never triggered attention.
Considering Apple is one of the largest companies in the world, raking in money, what consequential effects are you talking about? It certainly doesn't seem to hurt their bottom line, which is the only thing they care about.
As a software developer, I don't have any problem with this. If a bug doesn't bother somebody enough for them to follow up, then spend time fixing bugs for people who will. Apple isn't obligated to fix anybody's bug.
It's not like they were nagging him about it - it's been years, and they had major releases in the mean time. Quite possible it was fixed as a side effect of something else.
> It certainly doesn't seem to hurt their bottom line, which is the only thing they care about.
I want to draw out this comment because it's so antithetical to what Apple marketed that it stood for (if you remember, the wonderful 1984 commercial Apple created; which was very much against the big behemoths of the day and the way they operated).
We're at the point where we've normalized crappy behavior and crappy software so long as the bottom line keeps moving up and to the right on the graph.
Not, "Let's build great software that people love.", but "How much profit can we squeeze out? Let's try to squeeze some more."
We've optimized for profit instead of happiness and customer satisfaction. That's why it feels like quality in general is getting worse, profit became the end goal, not the by-product of a customer-centric focus. We've numbed ourselves to the pain and discomfort we endure and cause every single day in the name of profit.
> We've optimized for profit instead of happiness and customer satisfaction.
It's easy to blame the companies, but if the consumers keep buying the shitty products then there's no reason for the companies to spend money fixing stuff. I stopped using Apple long ago because I thought the software quality was going to crap. A lot of people won't do that, and so they get what Apple gives them.
Still, as a software engineer, I don't see a problem closing out old, abandoned bugs. Even for a company of Apple's size, there's limited time and sometimes it can be literally impossible to fix all of the bugs in a way that satisfies the people opening them. Given that, their approach for deciding which bugs to close seems reasonable and more fair then other ways of doing it.
If you are looking at it from a business perspective, there is little value to fixing a bug that is not impacting your revenue.
Of course, the developers should be determining if the bug may have a greater impact that will or does cause a problem that impacts revenue before closing it - not doing that is negligent.
I have been on the other side where I can't replicate/verfiy and the think the user would tell me if it was fixed. After exhausting myself and contacting the user only to find out it was resolved.
I mean it can be useful to do that every year on old (say 2y+ tickets) but the way it is usually done is completely aisine
Sensible way would be probably something like this
* run cleaning yearly, on bugs say not touched (which is different than age!) for last 2 years
* mark bug waiting for answer, add automated comment with "is it still happening/can you reproduce it on newest version?"
* if that gets unanswered for say 3 months THEN close it.
that way at least it's "real" issue and even if solution is not being worked on maybe someone will see workaround that sometimes someone posts in the comment. Not create new one that gets closed for being duplicate...
Meanwhile I've seen shit as aisine as making bug stale 30 days after reporting.
Basically every single old bug report I've ever seen is essentially a red-herring that is usually not able to be reproduced anymore after N years and takes away time from focusing on newer and more solvable issues. I don't see the issue with removing that noise if it's no longer being reported, but to each their own I suppose.
Sure. So try to reproduce on a current build, and close with a "No longer reproduceable on ___". That'd be good practice. Closing silently because no one can be bothered to evaluate at all is horrendous, and creates the user expectation that "no one looks at these, so I'm not going to keep reporting it" which "justifies" developers closing old bugs.
I agree with you about that, but why would an ill-defined report be kept open in the first place? It shouldn't be. Give the user an opportunity to provide more detail - for my own use I have some auto-text "scripts" set up, to make prompt questions easy - and then auto-close after a few days.
[Edit, answering my own question: they're left open because they were ignored to begin with.]
I write excellent bug reports, the vast majority of which (I'm thinking of one service-provider in particular, may they live in shame) get ignored. Or escalated and ignored; somehow that feels worse, though I don't know if it should. I guess it's the hope. It's the hope that kills you.
Every other month I get an email from a legacy pre-GH bug tracker that's either a "me too" or "bug fixed in latest release" a decade after I filed these one-offs you would be so quick to throw away. Bugs with no activity for years on end.
the right thing to do is to actually ping the original reporter if possible, or a developer that you might assign the bug to and try to drive it to a conclusion.
if the answer is 'everything in that part of the code has been rewritten' or 'yeah, that was a dup, we fixed that' or 'there isn't enough information here to try to reproduce it even if we wanted to' or 'this a feature request that we would never even consider' or some other similar thing, then sure delete it.
otherwise you're just throwing away useful information.
edit: I think this difference of option is due to a cultural difference between (a) the software should be as correct as reasonably possible and (b) if no one is complaining then there isn't a problem
Closing bugs because of a rewrite is probably the most harmful practice in the whole industry. The accumulated unresolved issues of your existing code base are a rich resource of test cases. Writing the new code base without checking to see if it fixes the old bugs is a mistake.
sure. But you can say put "please verify whether it is still present" via bot before doing so. Which apple did and I'm not sure why blogpost author is complaining about that
> you can say put "please verify whether it is still present" via bot before doing so. Which apple did
No, that's not what Apple did. They said, "Please verify this issue with macOS 26.4 beta 4", a very specific version, implying that they actually fixed the bug in that specific beta version (spoiler: they didn't). And I would have to go out of my way to install that specific beta just to "verify" the bug. Moreover, they gave me only 2 weeks to verify before closing the bug that they hadn't responded to at all in 3 years.
They suddenly created artificial urgency for no apparent reason.
That only works if the "please verify" bot isn't just prodding for noise and then autoclosing anyway, which is exactly what the author keep running into, over and over, even when the report already has enough detail to reproduce it.
Worse, they ask again after a video.
If you want signal, add a flag or make verification mean something in triage, not just another loop.
As set up now, you're rewarded more for patience than for actually fixing teh issue at all.
At work I literally just spent a half hour meeting with colleagues doing backlog management to clear out old bugs that were random one-offs and never came up again.
My only positive experience reporting bugs post early startup was with the chromium team, i get usually assigned to a dedicated reproducer that verifies and is reachable for helping them recreate in a matter of a few days. I had two experiences where bugs were taking less than a week from report to fix in canary.
I’ve had the worst experiences with Chromium. I’ve found multiple regressions in Chromium, tracked down regression points to a single commit, provided clear reproduction steps, shown that they do not reproduce with either Safari and Firefox or even older Chromium... And then the bugs were just ignored for years at this point. Sometimes some people played ping pong with the assignee field, before leaving it unassigned, and that’s it.
I don’t report any new Chromium regressions I find anymore.
That's impressive. I've only reported one bug to Chromium, years ago. It was a bug in their CSS engine and I included an HTML file with a full repro. It took them a few years to actually fix it since the person who was initially assigned it never bothered, eventually left Google, and nobody picked it back up for a while. But they did eventually fix it, so that's something, I suppose.
Edit: this comment elsewhere in the thread is closer to my experience: https://news.ycombinator.com/item?id=47523107 Certainly in my own stint at Google I saw the same thing--bugs below a certain priority level would just never get looked at.
This class of problem will soon be fixed with LLM agent that can just say "Yes, I can verify this is still not fixed". You just post the issue and then hook it up to your agent and forget about it. It can keep posting that there is an issue. It could even "bump! this is affecting crucial internal workflows" or similar every now and then.
In Scotland, they close an issue by taking a vote of "OK", "Broken", or "Not Proven".
I believe they also have attorneys. Perhaps that's how Apple could make bug-tracking more effective -- hire a prosecuting attorney and a defending attorney for each bug.
I was an development tools engineering manager who was in enumerable "bug scrubs" to triage the flow.
Sometimes I would advocate based on business reasons to fix the bug. Or to de-prioritize it or close it. I took every side possible, depending. As did the more pragmatic of the engineers.
I miss the give and take, if not the feeling of perpetual technical debt.
It’s quite possible (likely, even) for there to be more bugs reported than Apple has capacity to investigate. I assume this is just a filter they use to get the queue down to a more reasonable size and remove bug reports that are especially old (trusting that if they’re still issued they’ll be re-reported). This kind of culling happens all the time with low pri stuff and even sometimes medium pri if there’s a clear workaround.
I 100% agree when it comes to security issues. They would have to host it themselves on their intranet through AppleConnect.
I'm a bit removed from what software Apple uses nowadays. Back when I worked there ~2021 it was still a mix of native apps (Radar, Phantom) and self-hosted enterprise versions of popular development tools. (eg. GitHub)
The experience is similar if you call for end-user support. I did this once with an Apple Home issue. I called 133-MAC (Australia), got through to someone immediately, had a nice chat, was very impressed, felt supported, and got myself a case number. Of course, it wasn't resolved.
And then, no matter what I did, I could never, ever get a single word out of anyone about that case again. I often wonder if it's still open.
I had an even more time consuming experience like this. I worked with Apple Support over the phone for a few months. They had me install a profile on the iPhone to collect more diagnostic logs, had me perform various steps to reproduce the issue, followed up for more information, etc. After a few months, the person assigned to the case went on vacation or something and another person was assigned. Coincidentally, it was getting closer to a new iOS release date. My whole case went completely dead and there was no way to revive it.
Replit customer service did the same thing to me as a paying customer.
Their customer service threw me around because fixing my locked git processes that their system locks you out of for security reasons was too much work for them. My project service was unusable and they just auto-closed the ticket after never following up on their commitments. That was despite my consistently putting in work for them and doing software engineering debugging and delivering to them why it needed to be manually reset on their end.
After I complained on a twitter post tagging their CEO, someone reached out again finally and expected me to open a brand new fresh ticket because "their system needs this". Ok yeah no thank you, the team avoiding responsibility by auto-closing unresolved tickets expects me to put in more work and open a new ticket because you can't figure out how to re-open one or create one on my behalf. Lazy.
> FB22057274 “Pinned tabs: slow-loading target="_blank" links appear in the wrong tab
If you're not testing your code under extreme latency it will almost certainly fail in all kinds of hilarious ways.
I spend a lot of time with 4G as my only internet connection. It makes me feel that most software is quickly produced, poorly tested, and thrown out the door on a whim.
That would be an accurate summary of almost all software.
Either it's quickly produced and thrown out the door as it's a startup trying to iterate and find market fit asap or because it's a bigcorp who's metrics are all not related to software.
> Why do I file bug reports with Apple Feedback Assistant? I plead insanity.
As do I.
> In the three years since I filed the bug report, I received no response whatsoever from Apple… until a couple of weeks ago, when Apple asked me to “verify” the issue with macOS 26.4 beta 4 and update my bug report.
The author is extremely lucky to even get a response. I’ve filed several issue reports (as an end user, not as a developer) on Feedback Assistant over the years. Not only do the issues not get fixed, but there’s nary a response or any indication that anyone has looked or is planning to look at it. Apple does not even bother to close my issue reports. They just stay open.
Sometimes, some issues may get fixed. But no notice of the fix being done. I’d never know at all.
One being that the most recent version is on their cdn but not their [npm package](https://www.npmjs.com/package/livephotoskit?activeTab=readme) which was never updated for 7 years.
You know what they did with this issue? They've marked it as "Unable to diagnose".
Also I've mentioned something about their documentation not being up to date for a function definition. This issue has remained open for 4 years now.
The replies here suggest that many of us have been on both sides and that Apple's behavior it's a great way to trade bug triaging time on the org side for a few frustrated reporters on the customer side.
The problem is it frustrates the most diligent of bug reporters who put time into filing high quality issues resulting in overall lower bug submission quality.
A good compromise might be select high quality bugs or users with good rep and disable auto-closing for them. In the age of AI it shouldn't be too hard to correlate all those low quality duplicates and figure out what's worth keeping alive, no?
I love that when I search for an odd behaviour or bug in macos or iOS, most of the time I will find a years old bug report with some irrelevant or useless "work around".
This is not too unusual. I've completely given up on bug reports, it's almost always a complete waste of my time.
I'm currently going around in circles with a serious performance issue with two different vendors. They want logs, process lists and now real time data. It's an issue multiple people have complained about in their forums and on reddit. The fact that this exact same thing is going on with TWO different companies ...
> I received no response whatsoever from Apple… until a couple of weeks ago, when Apple asked me to “verify” the issue with macOS 26.4 beta 4 and update my bug report
Since this is your typical automated bot garbage process, couldn't you just respond with your own bot voice and say it's "verified" and is still an issue?
I looked at the bug report. You don’t use the packet filter, but expect packet filtering? Seems to be a misunderstanding. The flow filter needs a flow to filter, which requires a TCP handshake to establish.
Seriously, auto-closing issues that haven't seen activity in 3–6 months is one of the best things you can do for your project.
If nobody's touched it in that long, it's time to accept it's never getting prioritized -- it's just collecting dust and making your backlog feel way heavier than it actually is.
So let it go. Let it go! (It feels good to channel your inner Elsa!)
A clean backlog is a healthy backlog. You'll actually be able to find the stuff people care about instead of wading through years of noise. And if something truly matters? Don't worry... those issues come back, they always do.
It's the only reasonable approach. Any software that used by general public (even general developers) will eventually be flooded by bug reports that are not reproducible. Keeping them open helps no one. If a bug hasn't been touched for 2000 days the chance someone will suddenly care about it on the 2001st day is negligible.
Oh you sweet summer child. Everyone else does this.
Yes, I hate it too.
Put yourself in the position of the employee on the other side. They currently have 647 bugs in their backlog. And they also have actual work to do that's not even related to these bugs.
You come to work. Over night there's 369 emails (after many filters have been applied), 27 new bugs (14 of which are against a previous version). You triage. If you think 8h is enough to deal with 369 emails (67 of which are actionable. But which 67?) and actually close 27 bugs, then… well then you'd be assigned another 82 bugs and get put on email lists for advisory committees.
Before you jump to "why don't they just…", you should stop yourself and acknowledge that this in an unsolved problem. Ignore them, let them pile up? That's not a solution? Close them? No! It's still a problem! Ask you to verify it (and implicitly confirm that you still care)? That's… a bit better actually.
"Just hire more experts"… experts who are skilled enough, yet happy to work all day trying to reproduce these bugs? Sure, you can try. But it's extremely not a "why don't they just…".
Author must not have worked in enterprise software before.
That's a classic trick where the developer will push back on the bug author and say "I can't reproduce this, can you verify it with the latest version?" without actually doing anything. And if it doesn't get confirmed then they can close it as User Error or Not Reproducible.
Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
From experience with Microsoft (paid) support (after doing 5 tickets because it's never the right team and apparently moving tickets internally is for losers), they will ask for proof of the reproduction. And they will take every opportunity to shift the blame ("Oh I can see in the log you're running an antivirus, open a ticket with them. Closed").
I recompiled OpenSSL to make s_server -www return the correct, static XML blob for a .NET application that was buggy to make a reproducer for them that didn't rely on our product at all and which could be self-contained on a very barren windows VM they could play with to their heart's content and which didn't even care about the network because everything was connecting via loopback, so they couldn't blame that, eitehr.
Turns out there was a known bug in Microsoft schannel that had yet to be patched and they'd wasted weeks of our effort by not searching their own bug tracker properly.
I hate that so much. It's everywhere. An example is a bug with discord. They wanted me to restart my phone, reinstall the app, what are my versions, what phone am I on, what settings, etc. After all of that they go "oh that's a known issue." Whyyyyyyyyyyy. I get that multiple things can have the same symptom, but maybe start with that. Not like I signed any NDA so they aren't hiding it's an issue from the public.
You know what was the saddest thing? After all that effort, I got some blow-back for the fix taking so long.
The actual soln is to use oss stuff and aggressively send claude code into the weeds whenever you find a big, then upstream the fixes.
I switched from IntelliJ to emacs for this reason and it’s been great
puts on paranoid hat It could be to demoralize you so you subconsciously decide to not file a bug next time, knowing all the rigamarole you'd have to go through. takes off paranoid hat
Why would they do that when they could waste your time instead?
[flagged]
My favourite variant of this merrygoround is when they ask you to demonstrate the issue live in a Teams session, you do so, and there's this moment of silence followed by an "Oh... I see".
Then you assume, naively, that this means that they've recognised that there really is a product problem and will go off and fix it. However, then in turn the support tech needs to reproduce the the issue to the development team.
They invariably fail to do so for any number of reasons, such as: This only happens in my region, not others. Or the support tech's lab environment doesn't actually allow them to spin up the high-spec thing that's broken. Or whatever.
Then the ticket gets rejected with "can't reproduce" after you've reproduced the issue, with a recorded video and everything as evidence.
If you then navigate that gauntlet, the ticket is most typically rejected with "It is broken like that by design, closed."
It'd kind of sad, how the market went. I suppose there are pluses too.
But back in the 80s and 90s, margins were significantly higher. If you look at hardware, I recall selling hardware with 30% margin, if not more... even 80% on some items.
Yet what came with that was support, support, support. And when you sell 5 computers a month, instead of 500, well.. you need that margin to even have a store. Which you need, because no wide-scale internet.
On the software side, it was sort of the same. I remember paying $80 for some pieces of software, which would be like $200 today. You'd pay $1 on an app store for such software, but I'd also call the author if there was a bug. He'd send an update in the mail.
I guess my point is, in those days, it was fun to fix issues. The focus was more specific, there was time to ply the trade, to enjoy it, to have performant, elegant fixes.
Now, it's all "my boss is hassling me and another bug will somehow mean I have to work harder", which is .. well, sad.
High end enterprise products still come with support. That's literally what customers are paying for: a single throat to choke.
Exactly! The "pay a lot of money but get really good support" tier still exists just about everywhere. You just didn't do the first part.
It really depends, support is usually the first thing companies adjust when they want to improve their margins.
Even when you're paying millions to AWS you have to get through their first line of support and they will ask silly questions until you can convince them to escalate.
So build barely usable products that force people to pay for support as an upsell.
Aka the Red Hat business model. It's all you have when access to the product itself is free. Gotta keep yourself in the loop somehow.
‘Oh you want to access help documents indexed by google? Please show us your enterprise licence to continue.’
Not really, you get "really dedicated support" at most, but not a "really good" one, otherwise all those decades-old bugs common in many software producs would've been fixed since they affect people at all tiers
Back then, computers didn't had competition from the analog world, so vendors had to provide excellent service such that users would be convinced into switching over to the digital way if doing things. Now comouters have a monopoly on how we work and live, so vendors care as little as possible.
Outsiders can't see the internal ticket, but e.g. https://github.com/microsoft/STL/issues/4448
This is simply a bug, it's an implementation mistake, it's even possible to imagine from what we do know about the implementation inside Windows to imagine how you'd likely write that bug, simply you're writing the "lock stealing" code and you realise you need some context -- are we stealing the write lock or the read lock? You realise that context won't fit in your tiny flag budget (flag bits are hidden in the bottom of a pointer) and you forget that you actually know this context at the exact moment you need it - you were asked for either a write lock or a read lock, that's what you're stealing. So, you write code which does what it can without the context, always steal the write lock. Oops. Bug.
And yet several people insist that this wasn't a bug it's actually the proper way for this to function. Not only in this github ticket, and in the Microsoft internal bug, but I saw several third parties defend the bug as obviously the correct way for this to work.
Fortunately it seems STL understood that and the internal ticket was eventually fixed and (presumably) in Windows 11 today this bug is fixed.
That kind of attitude disgusts me. Like it's someone else's job to have a sense of accountability. They would not remain employed in my company.
When I developed software I would jump right on top of any bug reports immediately, and work until they were fixed. I was grateful to my customers for bringing them to my attention.
It is different when you have a billion customers, all with different setups. At that scale, you notice real defects through product telemetry, support ticket volume, or trusted channels. You receive a high volume of bug reports that are due to user confusion, misconfiguration, or misbehavior of other software on the device - where solving an issue for one customer doesn't result in improvements for the other billion. Triage, filtering, and winnowing are necessary here.
I got a lot of those too, it meant I inevitably did a little bit of free tech support for my customers. In the end I felt it was worth it as they raved about the quality of support and it was a real differentiator - not to mention built a lot of brand loyalty (and internal staff loyalty too once I grew enough to build out a team - they derived real satisfaction from actually solving problems instead of playing ping-pong).
I agree regarding the need to triage at scale, unfortunately most large companies I've encountered fail to do this well and seem ill-equipped to accept high quality bug reports of edge case defects generated by expert users (save for the odd exception that arrives by social media from someone who happens to have enough followers to get their attention outside the regular support pipeline).
In my experience this doesn't usually boil down to a systems issue (the ticketing systems etc. exist that should theoretically allow for eventual escalation to the right engineer/developer) but a corporate culture thing (the company just doesn't prioritize customer feedback especially at the level where staff who actually deal with customers interface with the teams that write/maintain the software). Often it's genuinely valued at the C-level (the Bezos story of calling Amazon"s tech support line during an exec meeting is a fun example) but diluted somewhere between them and the rank-and-file.
(Ps. I'm not arguing with you and appreciate you took the time to craft a thoughtful reply)
It should be the other way around - at billion customer scale you should be responsible for how your product interacts with other software whose developers have less resources than you.
My guess it's just the emergent behavior that results when a company doesn't provide developers time to fix bugs.
If their week is already booked full just trying to keep up with the roadmap deadlines, a bug ticket feels like being tossed a 25lb weight when you're drowning.
You could say: "but have pride in your work!"
But if your company only values shipping, not fixing, that attitude doesn't make it through the first performance review.
What I've found to be most effective for program management is to set aside a maintenance team separate from the feature teams. The roadmap is then planned without counting anything for the maintenance team and they deal with bug tickets as they come in. Rotate the assignment periodically so that every developer has to occasionally spend a few months on the maintenance team.
Doesn’t this lead to problems like the feature team pushing buggy code and having no accountability or responsibility to deal with it?
My preference is to treat the defects like feature work, size and plan. Yes you might not get all the feature work done but the team is accountable for everything they make
There's a lot more to effective program quality management than I can explain in a comment here. Forcing all developers to rotate through the maintenance team is one incentive not to ship crap because they might end up having to deal with it anyway. But more importantly you have to shift left the quality assurance and control activities to minimize the risk of defect leakage in the first place. And set up a closed-loop system where any leaked defect triggers a rigorous root-cause analysis that results in further process improvement.
You’ve just described AGILE development, a way for product owners to backlog code rot while empowering developers to feel like they have a say in things.
[flagged]
None of those were full rewrites. Not sure what point you are trying to make.
https://www.jwz.org/doc/cadt.html
That didn't prove anything. Modules get rewritten, but does that make it a full rewrite? That's Ship of Theseus material there...
Yep. On the other side of the curtain this often isn't nefarious. It's a simple cost/benefit analysis of spending time on something that one user is complaining about versus a backlog of higher business priorities. I've seen this in my work and it makes me sad for the user, but it often does take a bit of effort to spear these bug reports through.
I totally understand that from the perspective of individual employees: they have little incentive to do more than the bare minimum to close tickets. But this behavior is typically a symptom of broken corporate culture and failure to align internal metrics. For every customer who takes the trouble to submit a formal bug report there are likely many others who just live with it, and badmouth you to other customers. Doing deep investigations of even minor bug reports also tends to expose other, more serious latent bugs. And root cause analysis allows you to create closed-loop solutions to prevent similar future bugs.
Large monopolistic tech companies like Apple and Microsoft can afford to ignore this stuff for years because there are few realistic alternatives. But longer term eventually a disruptive competitor comes along who takes product quality and customer service more seriously.
There's also going to be mountains of bugs resulting from cosmic rays hitting the computer, defective ram chips, weird modifications of the system the reporter hasn't mentioned.
You could sink an infinite amount of time investigating and find nothing. At some point you have to cut off the time investment when only one person has reported it and no devs have been able to reproduce it.
If bug contains instructions for reproduction most of that will be eliminated.
99.99% of bug reports do not
You're lucky if there is an accurate description of what produced the bug on the customers' specific setup at the time of reporting.
well then closing it with inactivity is fine.
Quite often reproduction information will only reproduce the bug in the customers environment, hence there is a lot of incomplete state on what is actually causing the problem.
It's pretty terrible in enterprise because there is so much 3rd party crap touching things it shouldn't that love to cause fun problems.
There's obviously some nuance here, but the fact is that much modern software is riddled with bugs, and this is sub-optimal for everyone (both software users and software builders). Most of the bugs which frustrate and irritate software users are not due to uncontrollable events such as cosmic rays flipping a bit. Most of them are plain old code defects.
But, you do have a valid point. Allow me to rephrase it this way: The answer is not for software companies to spend unbounded amounts of engineer time chasing every reported bug.
But there are ways that we, as an industry, can do better, and it's not by pouring all our time into chasing hard-to-diagnose bugs. Here are a few ways that I personally see:
1. Some very powerful technologies for finding many bugs with little engineering effort already exist, but are not widely used. As an example, coverage-guided fuzzing is amazingly good at finding all kinds of obscure bugs. The idea of coverage-guided fuzzing was known from the 1990's, but it took AFL (in ~2013) to take it mainstream. Even now, much of the industry is not benefiting from the awesome power of coverage-guided fuzzing. And there are other, equally powerful techniques which have been known for a long time, but are even less accessible to most software developers.
So: spread the word about such techniques, and for programming language/platform developers, work on making them more easily applicable. This could help many software companies to catch a great number of bugs before they ever go to production.
2. Similarly, there are extant historical computing systems which had very powerful debugging facilities, much better than what is currently available to most developers. The ideas on how to make our platforms more debuggable are already out there; it's now a matter of popularizing those ideas and making them readily accessible and applicable.
3. Since it's widely known that many bugs (real bugs, not "cosmic rays") are extremely hard to reproduce, an admirable target for us to aim for as developers is to implement debug logging in a way which allows us to root-cause most obscure bugs just by examining the logs (i.e. no need to search for a reproducer). Some real-world systems have achieved that goal, with very good results.
4. While there is currently much buzz about using LLM-based coding agents to write code, I think an almost better use case for coding agents is in triaging bug reports, diagnosing the bugs, finding reproducers, etc.
I've recently had a couple of shocking experiences where, just given a written description of an intermittent, hard-to-diagnose bug, a coding agent was able to search an entire codebase, identify the exact cause, and write a reproducer test case. (And this after multiple experienced human programmers had looked at the issue but failed to identify the cause.)
In summary, I think there are ways to "cut the Gordian knot" of bug reports.
What if no devs even tried to reproduce it, and they have no reason to believe they've fixed the bug with any other changes?
That seems to be the case described in the article. In such a situation, I think it's dishonest to ask the reporter to expend even more effort when you've spent zero. Just close it if you don't want to do it, you don't have to be a jerk to your customers, too, by sending them off on a wild goose chase.
Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
> Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
Truenas literally takes this approach to bugs.
Right. The problem isn’t closing the ticket, it’s pretending more work is happening than actually is.
“Needs verification” is fine if someone has actually tried to reproduce it. Otherwise it’s just a nicer way of saying “we’re not going to look at this.”
Most of the time there is some reason to believe that the bug could be fixed though, i.e. there were non-trivial code changes around that area.
Most of the time, there isn't any reason to believe it could be fixed, i.e. there were not any non-trivial changes around that area. What you're describing happens less frequently, and in such cases, the devs should discuss that with the reporter.
In this specific example, it looks like Apple gave no indications that such changes had happened, and no indications they had even spent a nonzero amount of effort following the reproduction instructions with either the old code or the new code.
I'll steal this to my projects bug template! /s
"Please consider cosmic rays hitting the computer, defective ram chips, weird modifications of the system before submitting the bug. Unlesss you explicitly acknowledge that, your bug will be closed automatically in 30 days. Thank you very much"
Don't just a bug report by its cover, judge it by its merits. We're all biased to dismiss them and find an excuse to ignore them. But that just leads to bad software.
https://m.slashdot.org/story/391921
Yes and no.
Funny you mention Microsoft because I used to see bug reports for Windows. I can tell you there was a ton of low quality "SOMEONE HACKED MY COMPUTER" or similar feedback (and sometimes just unintelligible ranting) that was completely inactionable or unreproducible. I otherwise do agree with your premise that large monopolistic businesses can sit on large swaths of feedback without worrying about competition - and that this is a problem.
However, for most software projects and businesses, the lack of repeated feedback is a signal that the issue isn't important.
As a user I would hope that the software author/publisher is prioritizing important problems. Closing one ticket is not indicative of organizational rot, as you say.
Yeah competition works. I don't like nexus that much but they accept every ticket I've opened and fix it the next release. Two things I think affect that. One, my ticket has the name of a fortune 100 next to it. Two, artifactory will eat them alive if they don't keep customers happy.
If you're having trouble reproducing, tag "needs verification" or something else. But closing a ticket isn't triaging, it is sweeping problems under the rug
Exactly. Thank you for saying this.
It's a false dichotomy - something being "a simple cost/benefit analysis" doesn't remove the ethical dimension, and can absolutely be nefarious. A movie villain saying "it was just business" doesn't make their actions less villainous.
I’d argue that there should be no higher business priority than shipping a product you already sold. If you sold a product and your customer spends their time documenting exactly why and how you sold them something that’s broken, you should make that a high priority. As a natural progression, you’ll start shipping less buggy / better tested products and that’s how you unlock yourself from the obligation you made to your existing customers to do other work.
Not directed at you of course, just the proverbial “you” from the frustration of a purchaser of software.
Careful saying that too loudly, the “ship new features at all costs” gang will come for your head. They don’t approve of things like “quality software” and “making stuff that works past the demo and cursory inspection” or “actual user utility”.
I can sort of back that for desktop apps but telemetry is so trivial for webapps needing a reproducer is almost an embarrassing admission the operator has no clue what they're doing.
Error tracking and tracing make it fairly straight forward to retroactively troubleshoot unreproducible issues.
And total 24/7 surveillance also makes police work easier, but it's still wrong.
...you're claiming error tracking and monitoring are wrong?
We're talking about companies you voluntarily have a relationship with, not the government
The telemetry is almost always not voluntary and neither is the relation ship with the companies in the first place unless you mean that technically you could become a hermit living in the woods.
[flagged]
> Author must not have worked in enterprise software before.
Or with open source projects. Fucking stalebot.
Or even non-software tickets at large corporations. I reported a water dispenser filling too slowly at my office because it took me a few tries just to fill my 1L water bottle. They said it was fixed and closed it.
It was not fixed. So I took a video of myself refilling my water bottle, attached it to the ticket, and re-opened it. They actually fixed it after that. The video was 2m12s long (and I spent god knows how long making the video file small enough to attach to the ticket lol)
this is actually a good example of how a more detailed issue will have a higher chance to be addressed. I don't know what information that's your previous report is lacking, but the video certainly give more information that the maintainer can pinpoint the cause and act on it. The ability to pinpoint the cause from the report is a godsent for maintainers, it drastically reduce the time to investigate the cause, thus able to act immediately.
Some of the information in this can may be:
* how "slow" exactly the process is related with normal behavior. If it's just said "slow" on previous report, it's easy to be dismissed
* the dispenser's behavior, such as if the water flow is consistently low volume or clogged intermittently, or if the dispenser is struggling to fetch from water source, etc
I'd say it was both. I gave a pretty detailed explanation before, far more detailed than my post here, including a timeline of when it filled in one shot, then two shots, and then three or four (can't remember). I doubt they actually checked before the video. But I was very motivated to fix the issue so I gave them proof lol
More importantly it shows how the reporter actually used the system to trigger the undesired behavior. Just because something is obvious to you doesn't mean it will be obvious to whoever is looking at the bug report.
Take a look at Anthropic's repo. They auto-close issues after just a few weeks.
I don't think I've seen an issue of theirs that wasn't auto-closed.
I just randomly plugged in numbers to look up random Claude issues on github, and out of the 20 I checked, only one was closed as fixed. :-(
Wait, isn’t software engineering a solved problem?
Yes, that’s why they have such great up time. They don’t go down multiple times per day.
Yes
Fuck stalebot.
All my homies hate stalebot
What about OSS weekend ?
As an open source maintainer, I feel that statement is really unfair. Yes, we do sometimes close bug reports without evidence they are fixed. But:
- We owe you nothing! And the fact that people still expect maintainers to work for them is really sad, IMHO.
- Unlike corporate workers, nobody is measuring our productivity therefore we have no incentive to close issues if we believe they are unfixed. That means that when we close the issue, we believe it has a high chance of being fixed, and also we weigh the cost of having many maybe-fixed open issues against maybe closing a standing issue, and (try to) choose what's best for the project.
It's not about expectation of work (well, there's some entitled people sure.)
It's about throwing away the effort the reporter put into filing the issue. Stale bots disincentivise good quality issues, makes them less discoverable, and creates the burden of having to collate discussion across N previously raised issues about the same thing.
Bug reports and FRs are also a form of work. They might have a selfish motive, but they're still raised with the intention of enriching the software in some way.
IMO closing issues via stale bot is fine, the problem is locking issues so that no further conversation is allowed on the issue. Multiple times, I've encountered multi-year old issues (which is usually not fixed due to the fix not being simple or compatible with the current architecture). There's usually a good amount of conversation between users offering workarounds (and those workarounds updated for newer versions) - till stale bot locks the issue.
This 1000%. Whoever came up with the idea of closing and locking issues because no one has posted on them for awhile is at best not all that bright and at worst downright sinister.
Closing an issue due to staleness is one thing, locking it is another.
> That means that when we close the issue, we believe it has a high chance of being fixed
I agree with this iff it's being done manually after reading the issue. stalebot is indiscriminate and as far as "owing" the user, that's fair, but I'd assume that the person reporting the bug is also doing you a favor by helping you make things more stable and contributing to your repo/tool's community.
I partially agree, but even with stalebots nobody is measuring the maintainers' productivity. So when they made the choice to use stalebots, they did that because they believe that's best for the project. It's different from corporate.
Nobody is measuring their productivity, but people definitely look at how many open issues they have and potentially how long those issues have existed. They’re likely incentivized to close issues for appearances.
With a popular open source project, you'll quickly get to a number of bug reports that you have no chance of ever solving. You will have to focus on the worst ones and ones affecting most users.
At the same time, you want to communicate to users that this is the case so they don't have wrong expectation. But also, psychologically it is demotivating to have a 1000+ open bugs queue with no capacity to re-triage and only two maintainers able to out a few fours in every month or every week.
In open source, "won't fix" means either "not in scope — feel free to fork" or "no capacity ever expected — feel free to provide a fix".
The optimization problem is how do you get the most out of very limited time from very few people, and having 1000+ open bugs that nobody can keep in their head or look for duplicates in is mentally draining and stops the devs from fixing even the top 3 bugs users do face.
The problem is that your users also have limited time and if it's clear you're not even looking at issues where someone has put in lots of effort to help you then you're only going to get lazy issues and it will actually take more effort from you to do all that work yourself if you want to reach the same software quality.
I think you are missing the point: a user putting in a lot of effort into a bug report is usually trying to help themselves get the bug fixed.
As a maintainer, you will obviously look at that bug with more appreciation: but if you estimate it will take you 3 months of active development to fix it that you will have to spread over a full year of your weekends (which you can't afford), what would you do?
And what would a reasonable user rather see? Yes, this is an issue, but very hard to fix, and I don't have the time, or just letting the bug linger?
> We owe you nothing! And the fact that people still expect maintainers to work for them is really sad, IMHO.
Users also don't owe you anything either. Auto-closing reports without even looking at them is like asking for donations only to throw 90% of what you get straight into the trash. Not cool. If you don't want bug reports, state that up front or at least leave bugs open for other users to see and talk about. Otherwise, users are free to warn others to stay away from you and your projects.
And that's before getting into more complex issues like what responsibility you have if you take on maintenance of existing software and end up breaking something what was working perfectly for some users.
> Unlike corporate workers, nobody is measuring our productivity therefore we have no incentive to close issues if we believe they are unfixed.
There are plenty incentives, e.g. pride.
> That means that when we close the issue, we believe it has a high chance of being fixed, and also we weigh the cost of having many maybe-fixed open issues against maybe closing a standing issue, and (try to) choose what's best for the project.
That's fine, but bots that auto-close issues unless the reporter dances for them is the opposite of that.
Why do you close the issue then?
Because I have a reason to believe it's fixed, I have many more like it and it's difficult to reproduce. Simple :)
You have no evidence that it's fixed, but you have reason to believe it's fixed?
>Because I have a reason to believe it's fixed
What reason?
Because open source is corporate now
I got reeeeally good at producing repro gifs that I could plug straight inline into email replies to "can't repro"; it's forever clear that most developers either don't know how to test the product they are building, or simply can't be bothered to try.
> Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
I'm not going to lie. That's not who I am. If Apple really wants to close a bug report when the bug isn't fixed, that's on their conscience, if they have one.
Companies have no consciences.
Companies are made of humans who do.
I think you and I have had vastly different experiences of the normal level of conscience in companies.
Possibly, though generally speaking you don't have to look hard to find people with conscience and ethics in companies, they're abundant.
I guess that explains why social media companies try to maximize engagement even though it is almost bad for your mental health in every way.
Or does it?
So was the Nazi regime, and yet…
That's what we call a "non sequitur" in critical thinking class.
I've been on both sides of this. Absolutely sucks as a user falling prey to stalebot or some poor sap pretending to be stalebot, but when I was working in enterprise tech support it was a huge relief to close a case and get it off my plate, for any reason. We had to take 2 new cases per day minimum, update each case (I often had 20+) every few days minimum. Only a small minority were a quick and easy solution (like security vulns with a fix ready we could send the customer were the easiest). We were stuck with our cases also, you couldn't give them to someone else unless you were out sick pretty much, and you'd get them back when you returned unless by some miracle the other guy fixed their problem. An inactivity close on a stalled case was comforting, I was finally free. I think they started cracking down after a bit and said you had to check in with them 3 times x days apart first instead of just immediately closing after no word for 2 weeks, and then they wanted you to call them first. Absolute nightmare.
I think as long as the issue isn't stuck with any one person then it's easier to leave open until it's actually fixed, like the 20+ year old Mozilla bug reports. Big corpo bureaucratic nonsense just ruins everything.
If you are a veteran of software in a big company, we all know there will be weekly or bi-weekly meetings that some PM will set up. All the PM will do is go over the JIRA tickets and be like "is this still happening". Default answer is "no", as in "I didn't even try to reproduce it, do you think I have time to even do it?". Default answer by spineless QA person is also "didn't try it again yet". Then, the PM closes the ticket. It is much easier for QA person to say "Yes I verified it" if you are remote and developer cannot see the lies on your bad poker face.
Ooh this gives me an interesting passive-aggressive idea to counter pointless "is this still relevant" questions. "No, I haven't hit this in the last 2 days." "No, I haven't hit this since I gave up trying to do it with your tool." And so forth.
The less passive-aggressive version is to use this obviously-unhelpful answer of the obviously-unhelpful question, to actually have a conversation to get the PM to recognize that the default state of a ticket is in fact "no change." Ultimately that may turn into a stale bot if the PM realizes the policy they actually want is some sort of timeout, but at least it's not a time consuming meeting!
(Note, a cathartic thought experiment, but not really good manners to actually do!)
Absolutely spot on LOL
I've worked with enterprise software. The result its that people will eventually just wait a few hours/days and lie if they even care enough to do that. The perverse incentives destroy what utility a bug tracker could bring. Int theory transparency could help by changing the incentives if third parties analyze the metrics and call out bullshit to an audicence that matters.
> the developer will push back on the bug author and say "I can't reproduce this, can you verify it with the latest version?" without actually doing anything.
We do this. Because frankly, very often the bug has been reported by others and has been fixed, we just can't connect the dots in our ticketing system.
That's of course less than ideal, but given that a lot of tickets we get are often very poorly described it's hard. It's one aspect I have genuine hope AI can help us with.
Is your argument "it's bad everywhere, so it's ok"? As a software developer I do understand how enterprises operate, as a customer and a user I'd put Apple under higher scrutiny and would expect better.
I wish someone had told me how common this was back when I worked myself to death fixing every UI abnormality that no one except some misincentivized testers used to report at my first job. At the time I thought it was dishonest to say something was irreproducible and it'd be beneath me to patch an issue knowing it'll sprout ten others.
I'm proud of fixing everything properly but I won't repeat it ever unless the company actually has that high a bar across the board.
Hi, bigcorp employee getting showered with tickets here.
I don't have enough time in the day to deal with the tickets where the reporter actually tries, let alone the tickets where they don't.
If I tell you to update your shit, it's because it's wildly out of date, to the point that your configuration is impossible for me to reproduce without fucking up my setup to the point that I can't repro 8 other tickets.
Back when I worked at Apple I would just try it in whatever I had installed. If it didn't reproduce I'd write "Cannot reproduce in 10.x.x" and close it. Maybe a third were like that, duplicates of some other issue that was resolved long ago.
Anyone that attached a repro file to their issue got attention because it was easy enough to test. Sometimes crash traces got attention, I'd open the code and check out what it was. If it was like a top 15 crash trace then I'd spend a lot longer on it.
If the ticket was long and involved like "make an iMovie and tween it in just such and such a way" then probably I'd fiddle around for 10-15 minutes before downgrading its priority and hope a repro file would come about.
There were a bunch of bug reports for a deprecated codec that I closed and one guy angrily replied that I couldn't just close issues I didn't want to fix!
Guess what buddy, nobody's ever going to fix it.
The oldest bug like that I ever fixed was a QuickDraw bug that was originally written when I was 8 years old but it was just an easy bounds check one liner.
But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
> But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
Except this same shit keeps happening with multiple teams.
Judging from your mention of QuickDraw, which was removed entirely from macOS in 2012, perhaps your Apple experience is now out of date.
[flagged]
What specifically do you claim I'm making up?
That the ~50000 engineers at Apple are conspiring to close your tickets in the exact same way. It's ridiculous
> That the ~50000 engineers at Apple are conspiring to close your tickets in the exact same way. It's ridiculou
It's pretty clear from experience that the organization policy is to not provide feedback on bug submissions. Getting a 'check it if still reproduces or we'll close it in two weeks' message after 3 years is actually a fast turnaround.
Best I've gotten was on an issue I routed to a friend who worked at Apple who promised it would get looked at, but that I wouldn't hear back.
Microsoft wouldn't fix my issues either, but at least they got back to me in a timely fashion. Usually telling me it was a known issue that they weren't going to fix.
You don’t hear back because almost always your bug is a duplicate of some other one. They can’t share the original with you because it contains data from another customer or from inside the company.
Almost nobody is the first reporter in an OS with billions of users. The only useful thing about those long dupe lists was being able to scan them for one with easier repro steps.
But sometimes that duplicate marking is wrong or some subtly different issue so they ask you if it still reproduces in whatever version contains the fix before closing it.
That makes sense. But when you take 3-5 years to respond to my bug report, I'm going to take at least 3 months to respond to your response. And I'm probably not filing more bugs, because chances are I won't be at my current employer by the time you reply.
When you consitently burn bug reporters, sooner or later there's nobody to file bugs.
Because that's probably how long it took for someone to prioritize it.
Even if it's not fixed by the dupe ticket, the volume of bug reports makes it almost certain another ticket for the same issue will come up. And if it doesn't then it probably wasn't that relevant to anyone.
Not my tickets specifically. I don't think they're out to get me individually. On the contrary, this is a common practice, which affects many developers. I just happen to be relatively loud, as far as blogging is concerned.
Yes I understand that. ~50000 engineers aren't conspiring to close all tickets that way. It's a stupid line of thinking.
More than likely your steps to reproduce are too laborious to receive attention relative to the value fixing the bug would provide. That's why they're asking you to verify it still happens. Seems pretty simple right?
There's also a strong chance your ticket was linked as a duplicate of some other issue that was fixed in the beta and they want you to verify that's the case but they won't expose their internal issue to you for a variety of reasons.
> ~50000 engineers aren't conspiring to close all tickets that way.
I didn't say that either. It's happened to me only sporadically, but multiple times.
I agree with you that teams within Apple manage their own tickets. Perhaps some individual teams are declaring bug bankruptcy at some point, so only their bugs would go out for verification. I don't really know. I wish I did. What I do know is that multiple teams have done this at different points.
There's indisputably a company-wide DevBugs canned response for this. It's the same exact language every time. You can even Google it.
> It's a stupid line of thinking.
Please respect the HN guidelines: https://news.ycombinator.com/newsguidelines.html
> More than likely your steps to reproduce are too laborious to receive attention. That's why they're asking you to do it.
It was much more laborious for me, because I do not install the macOS betas.
> Seems pretty simple right?
No, it doesn't explain why specifically, after 3 years, they were suddenly asking me to verify with macOS 26.4 beta 4.
Yes that's a thing, but never with external customers in public betas
I think that's entirely dependent on the workload the company is placing on their support staff. If Apple decides the techs should be handling 10 tickets at once, then the techs have a choice:
1. Tell everyone to update their shit, and close tickets if they don't.
2. Waste several hours per day uninstalling and reinstalling 10 versions of the same program.
One of these will allow you to close lots of tickets immediately, and handle the remaining ones as efficiently as possible. Yay! Good job, peon! You get a raise!
The other approach will result in a deep backlog, slow turnaround times, and lower apparent output from management's perspective. Boo! Bad job, peon! You're fired!
Please tell us where you work so we can avoid all of your company’s software. Unless it’s Microsoft, because we’ve already seen the results of that attitude there.
I don't see how it's an unreasonable request. If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically. You will be prioritized last, because my effectiveness is measured by how many tickets I close.
just bc everyone is calling you insane: you are being extremely reasonable.
Flagging isn't supposed to be used as a super downvote.
There's a term for the bizarre behavior and thought processes (read: justification) by the person you're responding to. <https://en.wikipedia.org/wiki/Occupational_psychosis>
> you are being extremely reasonable
They're not. If there's nothing wrong with it, one could ask whether the person here would be okay sitting in a room with their supervisor, the head of the company, and 10 customers, say the same things they're saying here, and get a consensus that this is how this should all work out.
Did I say it's how it should be, or did I say this is how it is?
It's a reasonable request given the unreasonable nature of my working conditions, a thing I have no power to change.
You are an exasperatingly selfish and un-self-aware person. You should not be mentoring children. This is enshittification personified:
> One of these will allow you to[…] get a raise
> given the unreasonable nature of my working conditions, a thing I have no power to change
You know that there are people who experience actual hardship, right?
[dead]
> If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically.
You completely missed the point of the blog post. Apple was in the process of developing macOS 26.4 beta 4, and they wanted me to install the beta just to "verify" the bug.
Apple could test my bug with 26.4 beta 4 a heck of a lot easier than I could. Nobody was asking Apple to install some ancient version.
> my effectiveness is measured by how many tickets I close.
That was one of the points of the blog post: this is a perverse incentive from management.
Note what you did not say: "my effectiveness is measured by how many bugs I fix." So engineers are incentivized to close tickets even if the bugs they report are unfixed. This is how a company ends up with crappy, buggy software.
I'm with you on the Apple thing, that's asinine.
The parent comment is talking about the broader practice of people telling you to update and then repro again. That's a completely legitimate thing to ask, given both the perverse corporate incentives and the basic reality that version toggling makes a tech far less efficient at solving all tickets, not just yours.
[flagged]
You missed the point entirely.
I also ask to say what company you're working for so I can avoid your bullshit attitude.
And hey, you will have less bug reports that way so please tell us
I also hate this pressure of it being on the user to come up with a minimal reproducing example. That means that any bug of any moderate complexity will never get fixed because you can't always reduce them to a few steps and they may be statistical.
A bug is a bug, no matter the developers' opinion or the complexity of the bug.
However there are "bugs" that actually do turn out to be just cosmic rays flipping bits or plain user error. If you as the reporter don't provide enough information for the developer to be sure they are not going on a wild goose chase then it's fair for the developer to not invest too much time.
Sure. But I'm biased because I was a "customer" of a former (large) company's products while also working at that company. So the bugs I would file were the type that a customer would file, but since I was inside, I saw how they were handled. The tactics that my fellow R&D developers would do to claim something wasn't a bug or reproducible were nearly endless.
I mean if the customer stops complaining, either the bug was fixed, or the bug was not too important to begin with, or they are not a customer anymore and nobody else cares about that niche bug. In all of the above closing the ticket sounds reasonable.
What is not reasonable is that they close issues with thousands of “I have this issue too” with active complains and full repros
[dead]
[dead]
> perhaps praying that the bug had magically disappeared on its own, with no effort from Apple.
I suspect that this is a common approach. It maybe even works, often enough, to make it standard practice.
For myself, I've stopped submitting bug reports.
It's not the being ignored, that bothers me; it's when they pay attention, they basically insist that I become an unpaid systems engineering QC person, and go through enormous effort to prove the bug exists.
> they basically insist that I become an unpaid systems engineering QC person
Microsoft support is guilty of this, especially for Azure & 365 issues.
Like sorry, but you aren't paying me to debug your software. Here's a report, and here's proof of me reproducing the problem & some logs. That's all I'm going to provide. It's your software, you debug it.
And based on my own personal experience, even if you persevere and force them to acknowledge the problematic behaviour, they can turn around and say it's not a problem and working as intended.
For example, when using Azure Front Door, it's apparently absolutely not a problem that as yet un-cached file in their CDN downloads from their own Azure Blob storage have a maximum download speed of around 2MB (16Mb) per second:
Them:
> Hello Jonathan,
> I hope you are doing well!
> I sincerely apologize for the significant delay in our response, which was necessary to conduct further internal testing.
> After a comprehensive review, we have determined that the behavior you are experiencing is typical for this type of operation.
> This is primarily due to the connection not being entirely directly, as it must pass through Azure Front Door. This process also involves distributing the cache among point-of-presence (POP) servers, which inevitably impacts the > operation's speed. Let me provide you with documentation covering that matter:
Me:
> So to be clear, Azure Front Door maxes out at less than 2MB/s (16Mbit/s) for uncached items even when everything is on Microsoft’s own servers?
Them:
> Hello Jonathan,
> Thank you for getting back to me.
> These values may vary by region, but those particular ones apply for South Africa North.
I also tested this behaviour in US and EU regions (from an Azure VM requesting a file from Azure Blob storage in the same region as the VM but via Azure Front Door) and in EU it was also similarly limited while in the US it was only a tiny bit better.
We use Cloudflare now, cheaper, faster, configuration UI which isn't painfully slow. Not without their own recent incidents, but better than Azure Front Door 99.99% of the time.
While I'd love to take your tack, unfortunately, I find that if I actually want the fix, I have to become their unpaid engineer.
Which is ridiculous, because at the same time my company is paying a separate support fee, large enough to literally employ a dedicated engineer for my company!
It makes a lot of sense. I would also try to get my customers to do work for me if I were confident they would never churn.
I will do the work for them (typically paid for by my employer) iff I can expect them to fix it.
Blackbox debugging is a PITA, which is part of why I prefer open source, but it is what it is... If something is broken, and I can get it fixed by putting in the time to get a good report, and etc and they fix the thing, then I'll do it.
But if they don't fix the stuff, I have no shortage of things to fix myself.
Damn. I've put quite a lot of effort into open source tools w.r.t. debugging and bugfixing, but yeah putting that for a corporate product that doesn't even respect you must be draining.
I filed an issue, they told me to get in GitHub, fix it and file a PR myself O_o. In fairness they bugged me and cajoled me and offered to help until I actually did it, which I actually appreciated in the end because the experience was useful. (Give a man a fish, etc...)
[dead]
All kinds of open source projects do this too. It's really annoying. It's one thing if the authors actually try and fail to verify the bug, but these days it seems like most projects just close "stale" bugs as a matter of course. This is equivalent to assuming that any given bug is automatically fixed after X amount of time, which is pretty absurd.
It's rather unreasonable to be annoyed. The maintainers may have entirely different priorities, which is fine. They're also likely being spammed with low-effort bug reports (not yours necessarily but from others).
The great thing about open source projects you can just fix the bug yourself and submit a PR, or fork the whole project if the maintainers won't merge your changes. If you don't have the time or skills yourself then you can even pay a contractor to do it for you.
> It's rather unreasonable to be annoyed.
I disagree. If you discover that a bug that makes an open source library unusable to you, after spending time on learning and using that library, and the authors close the bug as a wontfix, I think being annoyed is quite reasonable, even expected.
If that type of thing annoys you then you should restrict your use of open source projects to those backed by corporations with a paid support business model.
For feature requests, sure, but not for bug reports
It's open source software. If you discover the bug, have written a failing test that demonstrates it, and a proposed solution to it, then maybe you can be annoyed when the authors close it as wontfix.
Otherwise OSS is pretty much as-is, where-is, with the exception of very widely used and corporately supported projects.
Not really no, you got the support you were willing to pay for.
If the maintainer merely doesn't fix the bug, then yes. If they close the bug report so it gets lost and other contributors are discouraged from working on it, then no.
Closed reports are not lost, they are still searchable/linkable, they are just not in the list of work to do.
This is entirely up to the maintainer, who puts in the work and gives up their time/money to do so. If you want to be in charge on a given repo, put in the work and become a real contributor, if not accept the rules the maintainers choose.
You know what I mean. If the issue is closed, it looks like it's been solved. A new issue may be created that duplicates it, etc.
Obviously it's up to the maintainer. I'm saying what the maintainer should do, not what they can do.
Dupe reports are a signal all by themselves, that's really not harmful, nor does something being closed implied solved.
You shouldn't presume to know what is best for an open source maintainer of any given project - projects vary, reports vary in quality, and the job of maintenance is not an easy one.
You're right in that it's unreasonable to expect someone else work for you for free
But I would also say a quality bug report is a contribution in and of itself
Closing it without reason is also, literally, unreasonable
It is not about "priorities". Putting work into writing bug is utterly useless, because if the maintainer does not fix it in 2 months, the bug will be closed as stale. I am actually fine with open source project maintainer to prioritize stuff and hey, maybe that bug will be fixed in 6 months when they have time. But I am not fine being told "we did not had time for 2 moths, therefore we are closing your bug" as is standard on github now.
Do not throw the ball back with "this happens because bugs are not well written". Stalebot closes bug regardless of whether they are well written and ensures no one will put effort into writing another well written bug again.
You pay contractors to fix open source bugs? Tell me how that works
Is that a serious question? It works like any contract programming gig. You give the contractor money and in exchange they give you code (including copyright assignment). You can go through a freelancer site like Upwork if you don't know an appropriate contractor yourself.
Stalebot closing is a problem. There's no problem with stalebot adding a stale label (but really, a filter with last update x time ago does the same).
Adding filters so that developers only look at actionable tickets would be much more sane.
> Adding filters so that developers only look at actionable tickets would be much more sane.
That's a reasonable approach, but I don't understand how it's any more or less sane than autoclosing them with a stale label.
Whether these sorts of bugs are "open but stale" or "closed because stale" seems like it depends on whether the project defines "closed" as "no work planned" or "fixed", which both seem valid.
Either way these bugs will be hidden from developer dashboards but still available in the database so there's no practical difference, you just need to make sure everyone is on the same page about the meaning of "closed".
My favorite is the Claude Code bugtracker, on GitHub of course: https://github.com/anthropics/claude-code/issues
There is some bot that will match your issue to some other 3 vaguely related issue, then auto close in 3 days. The other vaguely related issues are auto closed for inactivity. Nothing is ever fixed, which is why they can't keep the thing from messing with your scroll position for years now.
They are also closing issues automatically that has no "activity" in 30 days, so you have to spam those issues.
Sounds like a job for an agentic tool that can produce human like sentences on interval …
> that will match your issue to some other 3 vaguely related issue, then auto close
Well, it was trained on StackOverflow.
Oh so it jumping to the top happens to others too?
Yep I've never successfully installed claude code, and this is exactly what happened to the issue lol
this scroll position thing is mentally damaging
I recognize that this is annoying from a user perspective, but I do understand it. Not all bugs are easily reproducible (and even if they are 100% reproducible for the user, it's not always so easy for the developers). Also sometimes you make a change to the code that you think might be in a related area, and so sometimes the most "efficient" thing is just to ask the user to re-test.
When I close an old bug that is not actionable, I do feel bad about it. But keeping the bug open when realistically I can't really do anything with it might be worse.
Back in another part of my career I worked a lot with putting Macs on ActiveDirectory. And there was a common refrain from Apple about bugs in that implementation: "works on 17!".
The joke is that Apple owns the 17.x.x.x class-A range on the Internet (they got in early, the also have a second class-B and used to have a second class-B that they gave back), and what engineers were really saying is that they could not reproduce on the AD systems that Apple had setup (lots of times it was because AD had been setup with a .local domain, a real no-no, but it was in Microsoft's training materials as an example at the time...).
> keeping the bug open when realistically I can't really do anything with it might be worse
I've heard this from others before but I really don't understand the mindset.
What's the harm in keeping the bug open?
I used to think that there is no harm in keeping the bug open. I think if you honestly feel that you have the time and resources to go back to the bug and fix it, then by all means keep it open.
But I find that sometimes I can tell from experience that the IR is not actionable and that it will never be fixed. Some examples:
* There's not enough info to reproduce the issue and the user either can't or won't be able to reproduce it themselves. Intermittent bugs generally fall into this category. * The bug was filed against some version of the software that's no longer in production (think of the cloud context where the backend service has been upgraded to a newer version).
Sometimes the cost to investigate a bug is so high relative to the pain caused that it just closed as a WONTFIX. These sometimes suck the most because they are often legitimate bugs with possible fixes, but they will never be prioritized high enough to get fixed.
Or sometimes the bug is only reproducible using some proprietary data that I don't have access to and so you sometimes have no choice but to ask the bug filer "can you still reproduce this?".
Computer systems are complicated. And real-world systems consisting of multiple computer systems are even more complicated.
I think asking someone if they can still reproduce an issue is valid. Especially if it was trivially reproducible for them, and now it isn't, that seems like a fine resolution, and the bug should be closed.
But in the other cases, closing the bug seems to me to be a way to perturb metrics. It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?
For your specific scenarios:
- lack of information on how to reproduce or resolve a bug doesn't mean it doesn't exist, just that it's not well understood.
- For the "new version" claim, I've seen literal complete rewrites contain the same defects as the previous version. IMHO the author of the new version needs to confirm that the bug is fixed (and how/why it was fixed)
- I agree there are high cost bugs that nobody has resources to fix, but again, that doesn't mean they don't exist (important for errata)
- Similarly with proprietary data, if you aren't allowed to access it, but it still triggers the bug, then the defect exists
In general my philosophy is to treat the existence of open bugs as the authoritative record of known issues. Yes, some of them will never be solved. But having them in the record is important in and of itself.
> It might be true that you'll never fix a given bug, but shouldn't there be a record of the "known defects", or "errata" as some call them?
Yes, fully agreed. But closing a bug doesn't preclude that. A closed bug isn't refutation or denial of a defect. It's just an indication that there is no plan to fix the bug. Not every bug system works like this though. My bug tracker works like this, and I should have more clearly described what a "closed bug" is in my earlier posts.
Bug reports are not known defects, at any kind of scale half of them will be already fixed, misunderstandings, bad data in, or related to an unusual setup.
Closing the bug is a way of saying: sorry this doesn’t look too important and we don’t have time to look at this given the other more important things (bugs/features) we plan to work on.
If it’s closed as stale after 6-12 months (multiple humans will have seen it) OR triaged by a human and marked as won’t fix I think that’s reasonable.
> at any kind of scale half of them will be already fixed, misunderstandings, bad data in,
Here you're referring to a class of bug reports that's uninteresting for this discussion, because they're invalid (i.e. they don't represent an actual bug). We're talking about valid bugs that have not been fixed.
> or related to an unusual setup
Unusual, but ostensibly supported? Then there exists a bug.
These rules are in place because that first class of 'bugs' is the majority of bug reports when you are at scale.
[dead]
What does it cost you to keep the bug open?
Their team's bug close metrics
Attention very time you look at the bugs list.
That's what labels are for.
What is the use in keeping it open when no one will ever look at it again after it goes stale? It still exists in the system if you ever wanted to find it again or if someone reports the same issue again. But after a certain time without reconfirming the bug exists, there is no point investigating because you will never know if you just haven't found it yet or if it was fixed already.
See my reply to eminence32 - bug tracking serves as a list of known defects, not as a list of work the engineers are going to do this [day/month/year].
The primary purpose is not usually a list of known defects and many ‘bugs’ are not actually bugs but feature requests or misunderstandings from users (e.g. RFC disallows the data you want my html parser to allow).
> The primary purpose is not usually a list of known defects and many ‘bugs’ are not actually bugs but feature requests
IME there are separate mechanisms to track feature work, bug trackers are for... bugs.
> or misunderstandings from users (e.g. RFC disallows the data you want my html parser to allow).
Again, this is a class of bug report that nobody is arguing should stay open.
The people who filed them would disagree and many would vehemently argue that their bug is in fact a bug, and is the most important bug and how dare you close it.
Surely a few years worth of open but unverified bugs would cause some issues with reporting and such.
> What's the harm in keeping the bug open?
Conversely, what's the harm in closing the bug? (As long as you don't lock or delete it, I agree that's bad.)
People focused on the work often interpret "open" to mean "requires work" and "closed" to mean "no planned work" in which case keeping an unreproducible bug open is dishonest because it falsely implies that someone might continue to work on it.
Whereas people focused on the problem often interpret "open" to mean "not fixed" and "closed" to mean "fixed" in which case closing an unreproducible bug is dishonest because it falsely implies that it's no longer a problem.
Neither seems right or wrong as long as everyone on the project agrees which interpretation you're using.
makes it very hard to lie with your metrics
Craig shows up under your bed.
How is that worse? Leaving it open signals to anyone searching about it that's it's still an issue of concern. It will show up in filters for active bugs, etc. Closing it without fixing it just obfuscates the situation. It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.
To some people "open" means "not fixed" whereas to others it means "more work planned". I've worked on projects with both interpretations and it's fine as long as everyone is on the same page.
> It costs nothing (except pride?) to leave "Issues (1)" if there is indeed an Issue.
In our case we omit bugs we couldn't reproduce from the issues list due to practicality, not pride -- our software has tens of thousands of unreproducible bugs and having them show up in reports would drown out planned work.
And it's not like anyone deleted or locked the unreproducible bugs, they are either tracked as "open but unreproducible" or "closed because unreproducible". Either way they're still in the database in case more information comes along, but still filtered out of the vast majority of dashboards.
> Not all bugs are easily reproducible
Apple did not say they couldn't reproduce it. Neither did they say that they thought they fixed it. They refused to say anything except "Verify with macOS 26.4 beta 4".
> and even if they are 100% reproducible for the user, it's not always so easy for the developers
It's not easy for the user! Like I said in the blog post, I don't usually run the betas, so it would have been an ordeal to install macOS 26.4 beta 4 just to test this one bug. If anything, it's easier for Apple to test when they're developing the beta.
> the most "efficient" thing is just to ask the user to re-test.
Efficient from Apple's perspective, but grossly inefficient from the bug reporter's perspective.
> realistically I can't really do anything with it
In this case, I provided Apple with a sample Xcode project and explicit steps to reproduce. So realistically, they could have tried that.
I suspect that your underlying assumption is incorrect: I don't think Apple did anything with my bug report. This is not the first time Apple has asked me to "verify" an unfixed bug in a beta version. This seems to be a perfunctory thing they do before certain significant OS releases, clear out some older bug reports. Maybe they want to focus now on macOS 27 for WWDC and pretend that there are no outstanding issues remaining. I don't know exactly what's going through their corporate minds, but what spurred me to blog about it is that they keep doing this same shit.
A company like Apple should have complex enough tools to perfectly capture system state at the time of the bug so that they can reproduce it
I don't work at Apple, so I can't comment on that. But that doesn't always help. There's been plenty of times where I have a full HAR file from the user and I can clearly see that something went wrong, but that doesn't always mean I can reproduce the issue. (I recognize a HAR file doesn't represent the complete state of the world, but it's often one of the best things a backend developer can get)
Reminds me of this Raymond Chen Microsoft blog post:
https://devblogs.microsoft.com/oldnewthing/20241108-00/?p=11...
It always helps. Even if you can't determine the root cause you can at least add an extra assertion check or logging statement at that point so that next time the bug gets triggered you'll at least get more useful diagnostic data and can get a step close. Iterate until you find the root cause.
That’s easy enough. The hard part is doing so without capturing a bunch of email, messages, and other private data that happens to be in memory at the time.
Ignorant question, if privacy didn’t matter and they had an atomically identical machine, would there still be plenty of edge cases where it was the printer or the Wi-Fi causing the issue?
In any case I would have said it sounds difficult on every front
I should be more precise. Capturing the system state isn’t too hard. Turning that into a reproducer may be quite hard, because of things like you say. There are certainly a lot of bugs that such a capture would make easier to figure out, but it wouldn’t be a panacea.
Story time. I used to work for Facebook (and Google) and lots of games were played around bugs.
At some point the leadership introduced an SLA for high then medium priority bugs. Why? because bugs would sit in queues for years. The result? Bugs would often get downgraded in priority at or close to the SLA. People even wrote automated rules to see if their bugs filed got downgraded to alert them.
Another trick was to throw it back to the user, usually after months, ostensibly to request information, to ask "is this still a problem?" or just adding "could not reproduce". Often you'd get no response. sometimes the person was no longer on the team or with the company. Or they just lost interest or didn't notice. Great, it's off your plate.
If you waited long enough, you could say it was "no longer relevant" because that version of the app or API had been deprecated. It's also a good reason to bounce it back with "is still this relevant?"
Probably the most Machiavellian trick I saw was to merge your bug with another one vaguely similar that you didn't own. Why? Because this was hard to unwind and not always obvious.
Anyone who runs a call center or customer line knows this: you want to throw it back at the customer because a certain percentage will give up. It's a bit like health insurance companies automatically sending a denial for a prior authorization: to make people give up.
I once submitted some clear bugs to a supermarket's app and I got a response asking me to call some 800 number and make a report. My bug report was a complete way to reproduce the issue. I knew what was going on. Somebody simply wanted to mark the issue as "resolved". I'm never going to do that.
I don't think you can trust engineering teams (or, worse, individuals) to "own" bugs. They're not going to want to do them. They need to be owned by a QA team or a program team that will collate similar bugs and verify something is actually fixed.
Google had their own versions of things. IIRC bugs had both a priority and s everity for some reason (they were the same 99% of the time) between 0 and 4. So a standard bug was p2/s2. p0/s0 was the most severe and meant a serious user-facing outage. People would often change a p2/s2 to p3/s3, which basically meant "I'm never going to do this and I will never look at it again".
I've basically given up on filing bug reports because I'm aware of all these games and getting someone to actually pay attention is incredibly difficult. So much of this comes down to stupid organizational-level metrics about bug resolution SLAs and policies.
> Google had their own versions of things. IIRC bugs had both a priority and s everity for some reason (they were the same 99% of the time) between 0 and 4. So a standard bug was p2/s2. p0/s0 was the most severe and meant a serious user-facing outage. People would often change a p2/s2 to p3/s3, which basically meant "I'm never going to do this and I will never look at it again".
Yeah, I've done that. I find it much more honest than automatically closing it as stale or asking the reporter to repeatedly verify it even if I'm not going to work on it. The record still exists that the bug is there. Maybe some day the world will change and I'll have time to work on it.
I'm sure the leadership who set SLAs on medium-priority bugs anticipated a lot of bugs would become low-priority. They forced triage; that's the point.
> People even wrote automated rules to see if their bugs filed got downgraded to alert them.
This part though is a sign people are using the "don't notify" box inappropriately, denying reporters/watchers the opportunity to speak up if they disagree about the downgrade.
> Google had their own versions of things. IIRC bugs had both a priority and severity for some reason (they were the same 99% of the time) between 0 and 4.
At the company I worked with (not Google, but a major one) this was the same. We used Salesforce, the "Lightning Experience" or whatever it was called [0]. Our version was likely customized for our company, but I think the idea was the same - one, I think the "priority", was for our eyes only, one was for the customer (the "severity"). If the customer was insistent on raising the severity, we'd put it as sev1, but the priority was what we actually thought it was. I was actually surprised that for the ~4 years I was there no one made the mistake of telling the customer the priority as a mistake, especially when a lot of people were sloppily copy-pasting text from Slack or other internal tools that sometimes referred to a case as either the severity or the priority.
Those were heavy customers with SLAs, though, not supermarket apps or anything like that.
What was sad was that our internal tools, no matter how badly written, with 90's UI and awful security practices, our tools were 50 times as fast as whatever Salesforce garbage we had to deal with. Of course, there was a lot of unneeded redundancy between the tools so the complexity didn't stay in the Salesforce tool. But somehow the internal tools written by someone 10 years ago, barely maintained, who had to still deal with complex databases of who-what-when-how, felt like you had the DB locally on a supercomputer while SF felt like you were actually asking a very overworked person to manually give you your query right on each click. I'm exaggerating, but just by a bit.
[0] That name was funny because it was slow as shit. Each click took 5 to 20 seconds to update the view. I wonder what the non-Lightning version was.
> IIRC bugs had both a priority and s everity for some reason (they were the same 99% of the time) between 0 and 4. So a standard bug was p2/s2. p0/s0 was the most severe and meant a serious user-facing outage
I've seen this at a couple places... I think it's supposed to help model things like if something is totally down, that's an S0... But if it's the site for the Olympics and it's a year with no Olympics, it's not a P0.
Personally, that kind of detail doesn't seem to matter to me, and it's hard to get people to agree to standards about it, so the data quality isn't likely to be good, so it can't be used for reporting. A single priority value is probably more useful. Priority helps responsible parties decide what issue to fix first, and helps reporters guess when their issue might be addressed.
> People would often change a p2/s2 to p3/s3, which basically meant "I'm never going to do this and I will never look at it again".
I learned this behavior because closing with wontfix would upset people who filed issues for things that I understand, but am not going to change. I'm done with it, but you're going to reopen it if I close it, so whatever, I'll leave it open and ignore it. Stalebot is terrible, but it will accept responsibility for closing these kinds of things.
> I don't think you can trust engineering teams (or, worse, individuals) to "own" bugs. They're not going to want to do them.
I will disagree there. The engineers often want to fix the bugs. Management is telling them they need it's all hands on deck for (Insert company goal here. Probably AI right now).
Followed by management also telling them they have too many bugs, of course. In a condescending tone.
What an interesting display of a principal-agent problem.
I’ve been dealing with ElevenLabs pulling this same garbage.
I’ll fill out a bug report, wait a few days to a week to get a response, which are often AI generated, and then 48 hours afterward their bot marks it as stale. Telling me to check if it’s still broken or they assume it’s fixed lol
(I work at ElevenLabs)
Sad to hear this. How are you submitting the bug report? If you submit an issue via any of our open source repos on GitHub you'll get a timely human response.
Observation: Long, long ago I submitted a bug to Microsoft. I was new at the time and didn't distill it down to the minimum, just gave a scenario that would 100% reproduce. I was contacted months later because someone looked at it and couldn't reproduce.
Yeah, I had found one manifestation of something else that they fixed by the time someone looked at it. The fix in the notes didn't look anything like my bug, only by observing that it now worked I was able to figure out that I had been the blind man trying to describe an elephant.
Former Apple employee here. This is a deeper quirk of Apple culture than one would guess.
Each and every Radar (Apple's internal issue tracker is called Radar, and each issue is called a Radar) follows a state machine, going from the untriaged state to the done state. One hard-coded state in this is Verify. Each and every bug, once Fixed, cannot move to Closed without passing through the Verify state. It seems like a cool idea on the surface. It means that Apple assumes and demands that everything must be verified as fixed (or feature complete) by someone. Quite the corporate value to hold the line on, and it goes back decades.
I seriously hated the Verify state. It caused many pathologies. Imagine trying to run a burndown of your sprint when zero of the Radars are closed, because they have to be verified in production before being closed, meaning you cannot verify until after the release. Another pathology is that lots (thousands and thousands) of Radars end up stranded in Verify. Many, many engineers finish their fix, check it in, it gets released and then they move on. This led to a pathology that the writer of this post got caught up in: There is lots of "org health" reporting that goes out showing how many Radars are unverified and how long your Radars stay in the unverified state on average. A lot of teams simply close Radars that remain unverified for some amount of time because they are being "graded" on this.
Interesting insight to what should be a good internal process if users followed up.
In this case the bug wasn't fixed.
> A lot of teams simply close Radars that remain unverified for some amount of time because they are being "graded" on this.
The simple solution here: you should also be graded on closing bugs that get re-opened.
> Imagine trying to run a burndown of your sprint when zero of the Radars are closed, because they have to be verified in production before being closed, meaning you cannot verify until after the release.
I think most teams use verify as a "closed" state to hide all that messiness. But sure, zero bugs is a project management fiction and produces perverse outcomes.
I think you're incorrectly assuming two things:
1. Apple engineers actually attempted to fix the bug.
2. Feedback Assistant "Please verify with the latest beta" matches the Radar "Verify" state.
I don't believe either of those are true.
to be fair this is pretty common spring cleaning in any bugzilla...
I was literally just coming in here to comment "in before someone says this is fine and there's no issue." and the first(!) comment is effectively "this is fine and there's no issue."
The sentiment feels like software folks are optimizing for the local optimum.
It's the programmer equivalent of "if it's important they'll call back." while completely ignoring the real world first and second-order effects of such a policy.
I've seen this in many teams and it always drives me nuts: "hey this ticket is old and we didn't bother, let's delete it to keep the board clean".
You feeling accomplished by seeing an empty list is not the goal!
Feeling overwhelmed by insurmountable mountain of bugs and issues is not the way either. We can argue that closing the tickets is not the best way, but if realistically nobody will ever look at them, why not make the developers feel better.
Either you truly need to fix the bugs, in which case the feeling is good and maybe more effort should go that way (more resources assigned to it or whatever), or you're at a scale where tackling everything is impossible and you shouldn't feel overwhelmed by seeing the noise then.
But I think modern industry pretends all it's fine to convince themselves that it's ok to chase the next feature instead.
Because it isn’t about the developers feelings, it’s about the users. Why not set aside a day or half day a week to fix those bugs instead?
Move them to a deficated status. “Never triaged”, “lost”, “won’t do”, what have you.
That way, you’re at least not deluding yourself about your own capacity to triage and fix problems, and can hopefully search for and reopen issues that are resurfaced.
Deficated?
I like this. It should be a status.
"I deficated this issue. Closed."
This was supposed to be "dedicated", but I might also be coming around :)
number-1 type bugs and number-2 type bugs...
It’s really a question of whether a team believes bugs are defects that deserve to be fixed, or annoyances that get in the way of shipping features. And all too often, KPIs and promotions are tied to the features, not the bugs.
Plus, I’ve been in jobs where fixing bugs ends up being implicitly discouraged; if you fix a bug then it invites questions from above for why the bug existed, whether the fix could cause another bug, how another regression will be prevented and so on. But simply ignoring bug reports never triggered attention.
Is it really programmers doing this, though?
These auto-closing policies usually originate from somewhere else.
Considering Apple is one of the largest companies in the world, raking in money, what consequential effects are you talking about? It certainly doesn't seem to hurt their bottom line, which is the only thing they care about.
As a software developer, I don't have any problem with this. If a bug doesn't bother somebody enough for them to follow up, then spend time fixing bugs for people who will. Apple isn't obligated to fix anybody's bug.
It's not like they were nagging him about it - it's been years, and they had major releases in the mean time. Quite possible it was fixed as a side effect of something else.
> It certainly doesn't seem to hurt their bottom line, which is the only thing they care about.
I want to draw out this comment because it's so antithetical to what Apple marketed that it stood for (if you remember, the wonderful 1984 commercial Apple created; which was very much against the big behemoths of the day and the way they operated).
We're at the point where we've normalized crappy behavior and crappy software so long as the bottom line keeps moving up and to the right on the graph.
Not, "Let's build great software that people love.", but "How much profit can we squeeze out? Let's try to squeeze some more."
We've optimized for profit instead of happiness and customer satisfaction. That's why it feels like quality in general is getting worse, profit became the end goal, not the by-product of a customer-centric focus. We've numbed ourselves to the pain and discomfort we endure and cause every single day in the name of profit.
> We've optimized for profit instead of happiness and customer satisfaction.
It's easy to blame the companies, but if the consumers keep buying the shitty products then there's no reason for the companies to spend money fixing stuff. I stopped using Apple long ago because I thought the software quality was going to crap. A lot of people won't do that, and so they get what Apple gives them.
Still, as a software engineer, I don't see a problem closing out old, abandoned bugs. Even for a company of Apple's size, there's limited time and sometimes it can be literally impossible to fix all of the bugs in a way that satisfies the people opening them. Given that, their approach for deciding which bugs to close seems reasonable and more fair then other ways of doing it.
This is exactly the mindset to which GP and I object.
> It certainly doesn't seem to hurt their bottom line
…yet
>anybody's bug.
:)
Funny at first but I’m coming around to that perspective
If you are looking at it from a business perspective, there is little value to fixing a bug that is not impacting your revenue.
Of course, the developers should be determining if the bug may have a greater impact that will or does cause a problem that impacts revenue before closing it - not doing that is negligent.
I have been on the other side where I can't replicate/verfiy and the think the user would tell me if it was fixed. After exhausting myself and contacting the user only to find out it was resolved.
I mean it can be useful to do that every year on old (say 2y+ tickets) but the way it is usually done is completely aisine
Sensible way would be probably something like this
* run cleaning yearly, on bugs say not touched (which is different than age!) for last 2 years * mark bug waiting for answer, add automated comment with "is it still happening/can you reproduce it on newest version?" * if that gets unanswered for say 3 months THEN close it.
that way at least it's "real" issue and even if solution is not being worked on maybe someone will see workaround that sometimes someone posts in the comment. Not create new one that gets closed for being duplicate...
Meanwhile I've seen shit as aisine as making bug stale 30 days after reporting.
It's very common but it's still a poor practice.
Basically every single old bug report I've ever seen is essentially a red-herring that is usually not able to be reproduced anymore after N years and takes away time from focusing on newer and more solvable issues. I don't see the issue with removing that noise if it's no longer being reported, but to each their own I suppose.
Sure. So try to reproduce on a current build, and close with a "No longer reproduceable on ___". That'd be good practice. Closing silently because no one can be bothered to evaluate at all is horrendous, and creates the user expectation that "no one looks at these, so I'm not going to keep reporting it" which "justifies" developers closing old bugs.
>creates the user expectation that "no one looks at these
Apple has done the best job of creating this expectation.
Apple Feedback = compliments (and ideas)
Public Web = complaints & bug reports
Apple Support = important bug reports (can create feedback first then call immmediately)
—
Prev comment w/link (2mo ago): https://news.ycombinator.com/item?id=46591541
> try to reproduce on a current build
Good luck doing that when the bug report (like virtually all bug reports in nature) doesn't provide sufficient reproduction steps.
I agree with you about that, but why would an ill-defined report be kept open in the first place? It shouldn't be. Give the user an opportunity to provide more detail - for my own use I have some auto-text "scripts" set up, to make prompt questions easy - and then auto-close after a few days.
[Edit, answering my own question: they're left open because they were ignored to begin with.]
I write excellent bug reports, the vast majority of which (I'm thinking of one service-provider in particular, may they live in shame) get ignored. Or escalated and ignored; somehow that feels worse, though I don't know if it should. I guess it's the hope. It's the hope that kills you.
Every other month I get an email from a legacy pre-GH bug tracker that's either a "me too" or "bug fixed in latest release" a decade after I filed these one-offs you would be so quick to throw away. Bugs with no activity for years on end.
Closing bugs because they can no longer be reproduced: obviously fine.
Closing bugs automatically after a cron job demanded that the user verify reproducibility for the 11th time: obviously bad.
the right thing to do is to actually ping the original reporter if possible, or a developer that you might assign the bug to and try to drive it to a conclusion.
if the answer is 'everything in that part of the code has been rewritten' or 'yeah, that was a dup, we fixed that' or 'there isn't enough information here to try to reproduce it even if we wanted to' or 'this a feature request that we would never even consider' or some other similar thing, then sure delete it.
otherwise you're just throwing away useful information.
edit: I think this difference of option is due to a cultural difference between (a) the software should be as correct as reasonably possible and (b) if no one is complaining then there isn't a problem
Closing bugs because of a rewrite is probably the most harmful practice in the whole industry. The accumulated unresolved issues of your existing code base are a rich resource of test cases. Writing the new code base without checking to see if it fixes the old bugs is a mistake.
sure. But you can say put "please verify whether it is still present" via bot before doing so. Which apple did and I'm not sure why blogpost author is complaining about that
> you can say put "please verify whether it is still present" via bot before doing so. Which apple did
No, that's not what Apple did. They said, "Please verify this issue with macOS 26.4 beta 4", a very specific version, implying that they actually fixed the bug in that specific beta version (spoiler: they didn't). And I would have to go out of my way to install that specific beta just to "verify" the bug. Moreover, they gave me only 2 weeks to verify before closing the bug that they hadn't responded to at all in 3 years.
They suddenly created artificial urgency for no apparent reason.
That only works if the "please verify" bot isn't just prodding for noise and then autoclosing anyway, which is exactly what the author keep running into, over and over, even when the report already has enough detail to reproduce it. Worse, they ask again after a video.
If you want signal, add a flag or make verification mean something in triage, not just another loop. As set up now, you're rewarded more for patience than for actually fixing teh issue at all.
[dead]
At work I literally just spent a half hour meeting with colleagues doing backlog management to clear out old bugs that were random one-offs and never came up again.
Pretty standard process.
My only positive experience reporting bugs post early startup was with the chromium team, i get usually assigned to a dedicated reproducer that verifies and is reachable for helping them recreate in a matter of a few days. I had two experiences where bugs were taking less than a week from report to fix in canary.
I’ve had the worst experiences with Chromium. I’ve found multiple regressions in Chromium, tracked down regression points to a single commit, provided clear reproduction steps, shown that they do not reproduce with either Safari and Firefox or even older Chromium... And then the bugs were just ignored for years at this point. Sometimes some people played ping pong with the assignee field, before leaving it unassigned, and that’s it.
I don’t report any new Chromium regressions I find anymore.
That's impressive. I've only reported one bug to Chromium, years ago. It was a bug in their CSS engine and I included an HTML file with a full repro. It took them a few years to actually fix it since the person who was initially assigned it never bothered, eventually left Google, and nobody picked it back up for a while. But they did eventually fix it, so that's something, I suppose.
Edit: this comment elsewhere in the thread is closer to my experience: https://news.ycombinator.com/item?id=47523107 Certainly in my own stint at Google I saw the same thing--bugs below a certain priority level would just never get looked at.
This class of problem will soon be fixed with LLM agent that can just say "Yes, I can verify this is still not fixed". You just post the issue and then hook it up to your agent and forget about it. It can keep posting that there is an issue. It could even "bump! this is affecting crucial internal workflows" or similar every now and then.
And if the maintainer does close the issue, you can have the agent make a disparaging blog post automatically.
It's Apple. They won't even notice. If all of 4chan and HN decided to abandon Apple they wouldn't even register the change.
In Scotland, they close an issue by taking a vote of "OK", "Broken", or "Not Proven".
I believe they also have attorneys. Perhaps that's how Apple could make bug-tracking more effective -- hire a prosecuting attorney and a defending attorney for each bug.
I was an development tools engineering manager who was in enumerable "bug scrubs" to triage the flow.
Sometimes I would advocate based on business reasons to fix the bug. Or to de-prioritize it or close it. I took every side possible, depending. As did the more pragmatic of the engineers.
I miss the give and take, if not the feeling of perpetual technical debt.
Not any more, 'Not proven' was abolished at the start of this year.
That happens constantly everywhere, see github bots sometimes outright closing "stale" issues with nobody even trying to look at them
It’s quite possible (likely, even) for there to be more bugs reported than Apple has capacity to investigate. I assume this is just a filter they use to get the queue down to a more reasonable size and remove bug reports that are especially old (trusting that if they’re still issued they’ll be re-reported). This kind of culling happens all the time with low pri stuff and even sometimes medium pri if there’s a clear workaround.
This is where a company that categorizes customer feedback like unwrap.ai or enterpret could help with volume and priority
No.
Sheesh, you see suggestions here all the time. Just trying to be helpful
For this Apple would have to trust a third party with their bugs and also not have tried to do this themselves
I 100% agree when it comes to security issues. They would have to host it themselves on their intranet through AppleConnect.
I'm a bit removed from what software Apple uses nowadays. Back when I worked there ~2021 it was still a mix of native apps (Radar, Phantom) and self-hosted enterprise versions of popular development tools. (eg. GitHub)
The experience is similar if you call for end-user support. I did this once with an Apple Home issue. I called 133-MAC (Australia), got through to someone immediately, had a nice chat, was very impressed, felt supported, and got myself a case number. Of course, it wasn't resolved.
And then, no matter what I did, I could never, ever get a single word out of anyone about that case again. I often wonder if it's still open.
I had an even more time consuming experience like this. I worked with Apple Support over the phone for a few months. They had me install a profile on the iPhone to collect more diagnostic logs, had me perform various steps to reproduce the issue, followed up for more information, etc. After a few months, the person assigned to the case went on vacation or something and another person was assigned. Coincidentally, it was getting closer to a new iOS release date. My whole case went completely dead and there was no way to revive it.
Replit customer service did the same thing to me as a paying customer.
Their customer service threw me around because fixing my locked git processes that their system locks you out of for security reasons was too much work for them. My project service was unusable and they just auto-closed the ticket after never following up on their commitments. That was despite my consistently putting in work for them and doing software engineering debugging and delivering to them why it needed to be manually reset on their end.
After I complained on a twitter post tagging their CEO, someone reached out again finally and expected me to open a brand new fresh ticket because "their system needs this". Ok yeah no thank you, the team avoiding responsibility by auto-closing unresolved tickets expects me to put in more work and open a new ticket because you can't figure out how to re-open one or create one on my behalf. Lazy.
> FB22057274 “Pinned tabs: slow-loading target="_blank" links appear in the wrong tab
If you're not testing your code under extreme latency it will almost certainly fail in all kinds of hilarious ways.
I spend a lot of time with 4G as my only internet connection. It makes me feel that most software is quickly produced, poorly tested, and thrown out the door on a whim.
That would be an accurate summary of almost all software.
Either it's quickly produced and thrown out the door as it's a startup trying to iterate and find market fit asap or because it's a bigcorp who's metrics are all not related to software.
> Why do I file bug reports with Apple Feedback Assistant? I plead insanity.
As do I.
> In the three years since I filed the bug report, I received no response whatsoever from Apple… until a couple of weeks ago, when Apple asked me to “verify” the issue with macOS 26.4 beta 4 and update my bug report.
The author is extremely lucky to even get a response. I’ve filed several issue reports (as an end user, not as a developer) on Feedback Assistant over the years. Not only do the issues not get fixed, but there’s nary a response or any indication that anyone has looked or is planning to look at it. Apple does not even bother to close my issue reports. They just stay open.
Sometimes, some issues may get fixed. But no notice of the fix being done. I’d never know at all.
So yes, I certainly do plead insanity.
I've submitted a couple of issues for their [javascript library for Live Photos](https://developer.apple.com/documentation/LivePhotosKitJS).
One being that the most recent version is on their cdn but not their [npm package](https://www.npmjs.com/package/livephotoskit?activeTab=readme) which was never updated for 7 years. You know what they did with this issue? They've marked it as "Unable to diagnose".
Also I've mentioned something about their documentation not being up to date for a function definition. This issue has remained open for 4 years now.
The replies here suggest that many of us have been on both sides and that Apple's behavior it's a great way to trade bug triaging time on the org side for a few frustrated reporters on the customer side. The problem is it frustrates the most diligent of bug reporters who put time into filing high quality issues resulting in overall lower bug submission quality.
A good compromise might be select high quality bugs or users with good rep and disable auto-closing for them. In the age of AI it shouldn't be too hard to correlate all those low quality duplicates and figure out what's worth keeping alive, no?
I love that when I search for an odd behaviour or bug in macos or iOS, most of the time I will find a years old bug report with some irrelevant or useless "work around".
This is not too unusual. I've completely given up on bug reports, it's almost always a complete waste of my time.
I'm currently going around in circles with a serious performance issue with two different vendors. They want logs, process lists and now real time data. It's an issue multiple people have complained about in their forums and on reddit. The fact that this exact same thing is going on with TWO different companies ...
How can a user “verify” that the bug remains unfixed? So when a user reports a report, the user should check if the bug was solved after each update?
The question is not whether they will close it if you don't regularly bump the report, but whether they will fix it if you do.
> I received no response whatsoever from Apple… until a couple of weeks ago, when Apple asked me to “verify” the issue with macOS 26.4 beta 4 and update my bug report
Since this is your typical automated bot garbage process, couldn't you just respond with your own bot voice and say it's "verified" and is still an issue?
No. https://news.ycombinator.com/item?id=47523983
There's a breed of Dilbert Manager that loves to do this everywhere. Optimizing for "fewer open bugs" I imagine.
I looked at the bug report. You don’t use the packet filter, but expect packet filtering? Seems to be a misunderstanding. The flow filter needs a flow to filter, which requires a TCP handshake to establish.
Good news -- you can do this too! https://github.com/actions/stale
Seriously, auto-closing issues that haven't seen activity in 3–6 months is one of the best things you can do for your project.
If nobody's touched it in that long, it's time to accept it's never getting prioritized -- it's just collecting dust and making your backlog feel way heavier than it actually is.
So let it go. Let it go! (It feels good to channel your inner Elsa!)
A clean backlog is a healthy backlog. You'll actually be able to find the stuff people care about instead of wading through years of noise. And if something truly matters? Don't worry... those issues come back, they always do.
The sheer volume of bug reports negates the perceived importance
There are no bugs if we don't check that they are still there. It's like anti-vaxer logic.
Some kind of acknowledgement would be nice, but nost of our feedback reports fall on deaf ears.
It's the only reasonable approach. Any software that used by general public (even general developers) will eventually be flooded by bug reports that are not reproducible. Keeping them open helps no one. If a bug hasn't been touched for 2000 days the chance someone will suddenly care about it on the 2001st day is negligible.
Stop wasting your life chomping at the bit to do unpaid labor for the sole benefit of megacorps.
Can confirm. Java causes a bug in an Apple IO library that hasn't been fixed for years.
The radar count is probably nearing a billion at this point
Expected behaviour, closing article.
> Why do I file bug reports with Apple Feedback Assistant?
It is known for decades that Apple largely ignores bugreports.
Anthropic does this too
Oh you sweet summer child. Everyone else does this.
Yes, I hate it too.
Put yourself in the position of the employee on the other side. They currently have 647 bugs in their backlog. And they also have actual work to do that's not even related to these bugs.
You come to work. Over night there's 369 emails (after many filters have been applied), 27 new bugs (14 of which are against a previous version). You triage. If you think 8h is enough to deal with 369 emails (67 of which are actionable. But which 67?) and actually close 27 bugs, then… well then you'd be assigned another 82 bugs and get put on email lists for advisory committees.
Before you jump to "why don't they just…", you should stop yourself and acknowledge that this in an unsolved problem. Ignore them, let them pile up? That's not a solution? Close them? No! It's still a problem! Ask you to verify it (and implicitly confirm that you still care)? That's… a bit better actually.
"Just hire more experts"… experts who are skilled enough, yet happy to work all day trying to reproduce these bugs? Sure, you can try. But it's extremely not a "why don't they just…".
so what, jetbrains just doesn't fix them
Bug Bankruptcy.
Devious.
[flagged]
[dead]
[dead]
[dead]
[dead]
Fuck those guys.
What else should they do? Stop releasing any updates until they reproduced any obscure bug report?
They could just, not close the bug?
Mozilla is famous for having 20 year old bug reports that gets fixed after all that time.
How about they keep the bug report open until they attempt and confirm the bug is no longer reproducible?