I find most of my references while browsing, and capture from the browser is Zotero's winning feature for me: automatically save the pdf and extract the bibliographic data. And while reading a paper, I get a search engine to look up its references and repeat.
How does Zotero compare to Paperpile these days? When I first found Paperpile, I was astonished how seamless the browser extension based citation and PDF auto-import worked and never looked back. But I imagine Zotero has come a long way as well?
Unlike grepping your BibTeX file, Zotero provides a nice tabbed PDF viewer, useful automatic import options with file renaming, and saved dynamic searches. Those are the main reasons why I still use it -- it's a good tool to manage my BibTeX file via a nice GUI, and faster than manually grepping BibTeX from the CLI.
Also, as my unenlightened colleagues insist on using ridiculous tools like MS Word to author papers in, it's useful to have Zotero installed for the ability to link a BibTeX file to a WYSIWYG word processor when collaborating. Zotero helps you play well with others.
I don't think Zotero is perfect by any stretch, but I'm glad that at least it allowed academia to reject both Endnote and Mendeley. Anything that punches Elsevier in the face gets a tick in my book.
Zotero is not just a front-end to BibTeX. Here are some things I like about Zotero off the top of my head:
Zotero integrates well with various online services. This I think is Zotero's most valuable feature. The simple fact is that not everything exports BibTeX. Can you get a library catalog to export BibTeX? Perhaps in CS, most publications provide good BibTeX, but a lot of journals I (a mechanical engineer) deal with don't provide BibTeX at all to my knowledge. But I can simply press a button in Firefox and import the bibliographic data embedded in the web page into Zotero. (No, Google Scholar is not a good solution here because it's frequently inaccurate and incomplete.)
I'm a technical guy who can handle BibTeX fine, but I still prefer the Zotero UI over using BibTeX files in a text editor and shell, even if I'm only generating BibTeX files.
The author discusses using non-standard fields like keywords to store extra data. I would recommend that to store additional context about the document. Zotero can store even more than that, including web pages, files, and additional notes. I like that Zotero saves a snapshot of the journal article web page in most instances. Journals sometimes do go offline, so it can be nice to have the web page. I often have detailed notes about particular documents in notes in Zotero. Yes, you can do that with BibTeX, but I could see the extra fields cluttering the BibTeX file. (Contrary to what the author states, the note field probably should not be used as they describe because it's printed in some/most? bibliography styles.)
The search in Zotero is more powerful than grep. Try returning bibliographic entries where one field contains X and another field contains Y. I think you can probably come up with a grep solution for that, but it's so convoluted that it's probably never used in practice. You could use a BibTeX searching program like biblook to get around this problem.
Don't get me wrong. Zotero is far from perfect. It can be slow, and at this point I would prefer a TUI reference manager. But overall, it's the best option I've tried.
tbh I never cared about bibtex features from Zotero... Having a interface where I can place and organize pdfs, filter them by author and have everything syncronized between my tablet and my computer is what really makes me use it.
If this kind of feature could be replaced by txt files I probably would be using it, but, no...
I used zotero through both a bachelor's and masters. It is amazing software. And it integrates with nearly everything. It completely takes the friction out of literature reviews and research whilst providing much more that a single text file. The snapshotting and auto pdf download are the best parts.
I like me some cli, but if I was to do something similar it would be with csl-json and jq/nushell. I cannot see the advantage of grepping a bibtex file, let alone having to manually manage the related articles.
I just finished importing and sorting thousands of PDF research papers, about 50 GB I've accumulated over the decades. Originally, I was sorting them in file system directly, but the whole thing got out of hand at some point, and thus the Documents folder became a dumping ground for PDFs. So I decided to take control of the situation and use Zotero to manage all this stuff. First I used Claude Code to generate all the BibTex files for the PDFs, then sort them into categories (still on the filesystem). Then created a master bibtex file that lists all the docs in each category. I used that to bulk import in Zotero, which then normalised all the file names once imported. What's really cool is that you can use Zotero built in web APIs to get an external tool to manipulate your collection directly. I've also setup WebDAV to sync the entire collection across multiple machines. Can you do all these without Zotero? Absolutely. And I did, but can't say I'll be looking back any time soon.
At least with Zotero Version 6, you could have just selected all your PDFs and drag-n-dropped them into the interface.
Like another poster here, trusting an LLM with my reference database -- the ultimate source of truth -- is not a step I'd be willing to take. All it takes is a single hallucinated reference, and your career would be forever tainted. It's not worth the risk.
Sadly, Zotero seems to have removed this killer import feature in later versions, which is the reason I keep using version 6. It feels like later versions have been a route to dumbing-down the interface, prioritising simplicity (and an ultra-low-contrast interface) at the expense of functionality. (If you can still drag-n-drop PDFs straight in with the new versions, someone please let me know?)
Drag-and-drop of the PDF file worked for me in Zotero 9 (latest version). I never used this feature before. This would be greatly preferred to my earlier suggestion to get a LLM to generate a list of DOIs.
I still drag and drop PDFs in Zotero 7. It's less finicky then v6, even. The trick is to drop them to root of the tree on the left. Not to the middle of the interface.
> First I used Claude Code to generate all the BibTex files for the PDFs
I think this has an unnecessary risk of hallucinated bibliographic data. For anyone doing something similar in the future, it would be more reliable to make a LLM generate a list of DOIs and have Zotero import the DOIs.
Using a non-deterministic machine on something which needs to be pretty accurate and precise looks like playing Russian roulette with one's academic integrity.
And moreover, something already mostly solved algorithmically. Eg in zotero you can drag and drop files and it generates a reference based on the metadata as long as that is enough (somebody mentions it is not possible in newer versions, I use the latest version and it works fine).
Each one can choose their own tools of course, but we often see llms shown as solutions to something presented as unsolvable before, while it already was pretty amenable to normal tools.
> but we often see llms shown as solutions to something presented as unsolvable before, while it already was pretty amenable to normal tools.
I see that as a symptom of being intellectually lazy, or not knowing the value in mastering a tool via deliberate use over time.
Once one invests the required time to use a tool properly, they can reap way more benefits from said tool when compared to jumping tools at minute's notice.
I have always stayed away from additional software for bibliography management for similar reasons that the authors cites. However, while I do not use Zotero the software, their ZoteroBib (https://zbib.org/) has been a huge time saver. No login or account needed; just copy the URL or DOI and it generates the BibTeX entry. I find it far more accurate than Google Scholar.
I think Zotero is an excellent tool and I used it for a long time. However, like the author, I also felt that Zotero was not the ideal tool for me. But a straight BibTeX file is also rather unwieldy. Essentially, bibliographic data has provenance (say, a DOI identifier), and it is very useful to keep track of this in a structured way.
A friend and I have been working (slowly, over the past few years, since we are both academics) on an abstraction layer over BibTeX called Autobib: https://github.com/autobib/autobib
Broadly speaking, Autobib is a CLI around SQLite database of BibTeX records. But in addition to plain BibTeX, Autobib is aware of 'external data providers' (like DOI, MathSciNet, OpenLibrary, arXiv, zbMath, etc.) and will automatically retrieve data bibliographic data from these data providers. The provenance of the data is stored alongside the record itself, and this can be used to retrieve updates, prevent duplication of data, etc.
The killer feature is: if you have a file (say `file.tex`) and the keys are in a format which Autobib can automatically recognize (say, you use citation keys like `doi:my/weird/doi`, and there is support for custom formats and aliases) you can run `autobib source file.tex` and it will write to standard output a sorted BibTeX bibliography for your file. This lets you trivially maintain a per-project bibliography which you can check into source control locally and which exactly corresponds to the paper itself.
But otherwise, Autobib is "just a wrapper over a BibTeX bibliography"! When you edit an existing record, you are just editing a BibTeX record. There is integrated search, directly on the BibTeX fields themselves.
There are some extra features, like support for attachments, fuzzy search, undo-tree support, headless edit, auto-normalization, soft deletion, replacement, merging, etc. The database format is relatively simple and open (currently not particularly well documented, but this will change when it stabilizes) to allow introspection by other tools.
The tool also strives to play nicely with other tools (structured output, composable, etc.)
These frontends are necessary however because researchers in non-computer related fields are not trained/proficient at command line tools. Many of them need help installing software. Many of them don’t use raw text files much either. They use MS word files instead etc.
I mean even if you are comfortable with grep, it is still not enough, plus then you have to reinvent all the rest of the functionality, ie an extension to automatically add references from a webpage, download pdfs and keep them in a directory with some local link to be able to find and open them, and then another tool to export a subset of the huge bibtex file for each specific project and sync them, as it would not make sense to share a huge bibtex file on every latex project where you need only a small subset.
I'm not sure if I would ditch Zotero. Using grep over a bib file is nice but I guess you are opening the DOIs in browser every time?
The reason I like Zotero is because I can use it offline, have my all documents synced (the backup feature is one of the best things) when needed and I can organize my papers according to project/paper I'm working on. Actually one thing I do wish Zotero would add is tab groups which will allow me have papers open but grouped by project. It's based on Firefox which does support this so I hope it's a matter of time.
I tried to use Zotero so many times... and then I just tried a few other options, and in the end I realised that Zotero is the standard for academic writing... and I just could never got into it... in the end I used a text file, a literal text file and I am quite proud of my academic writing, I don't wish to do it again, but in the end, what worked for me was a literal piece of crap txt file.
JabRef before version 3 was pretty compact and snappy. It's essentially editing a text file for taking notes, except you can search through, download arxiv and journal articles (with LocalCopy extension), launch the associated PDFs.
This sells Zotero short in so many ways, but you do you, I guess. Have fun hand formatting between the dozen citation/reference formats every other journal chooses, seemingly at random
Why would you need that? 99% of the time, you'll just grab the BibTeX from where you're getting the paper (Google Scholar, arXiv, ResearchGate, etc.) and paste it on the .bib file. LaTeX will take care of formatting it when you \cite.
I find most of my references while browsing, and capture from the browser is Zotero's winning feature for me: automatically save the pdf and extract the bibliographic data. And while reading a paper, I get a search engine to look up its references and repeat.
The rest of the interface is ok.
How does Zotero compare to Paperpile these days? When I first found Paperpile, I was astonished how seamless the browser extension based citation and PDF auto-import worked and never looked back. But I imagine Zotero has come a long way as well?
Unlike grepping your BibTeX file, Zotero provides a nice tabbed PDF viewer, useful automatic import options with file renaming, and saved dynamic searches. Those are the main reasons why I still use it -- it's a good tool to manage my BibTeX file via a nice GUI, and faster than manually grepping BibTeX from the CLI.
Also, as my unenlightened colleagues insist on using ridiculous tools like MS Word to author papers in, it's useful to have Zotero installed for the ability to link a BibTeX file to a WYSIWYG word processor when collaborating. Zotero helps you play well with others.
I don't think Zotero is perfect by any stretch, but I'm glad that at least it allowed academia to reject both Endnote and Mendeley. Anything that punches Elsevier in the face gets a tick in my book.
Zotero is not just a front-end to BibTeX. Here are some things I like about Zotero off the top of my head:
Zotero integrates well with various online services. This I think is Zotero's most valuable feature. The simple fact is that not everything exports BibTeX. Can you get a library catalog to export BibTeX? Perhaps in CS, most publications provide good BibTeX, but a lot of journals I (a mechanical engineer) deal with don't provide BibTeX at all to my knowledge. But I can simply press a button in Firefox and import the bibliographic data embedded in the web page into Zotero. (No, Google Scholar is not a good solution here because it's frequently inaccurate and incomplete.)
I'm a technical guy who can handle BibTeX fine, but I still prefer the Zotero UI over using BibTeX files in a text editor and shell, even if I'm only generating BibTeX files.
The author discusses using non-standard fields like keywords to store extra data. I would recommend that to store additional context about the document. Zotero can store even more than that, including web pages, files, and additional notes. I like that Zotero saves a snapshot of the journal article web page in most instances. Journals sometimes do go offline, so it can be nice to have the web page. I often have detailed notes about particular documents in notes in Zotero. Yes, you can do that with BibTeX, but I could see the extra fields cluttering the BibTeX file. (Contrary to what the author states, the note field probably should not be used as they describe because it's printed in some/most? bibliography styles.)
The search in Zotero is more powerful than grep. Try returning bibliographic entries where one field contains X and another field contains Y. I think you can probably come up with a grep solution for that, but it's so convoluted that it's probably never used in practice. You could use a BibTeX searching program like biblook to get around this problem.
Don't get me wrong. Zotero is far from perfect. It can be slow, and at this point I would prefer a TUI reference manager. But overall, it's the best option I've tried.
tbh I never cared about bibtex features from Zotero... Having a interface where I can place and organize pdfs, filter them by author and have everything syncronized between my tablet and my computer is what really makes me use it.
If this kind of feature could be replaced by txt files I probably would be using it, but, no...
I used zotero through both a bachelor's and masters. It is amazing software. And it integrates with nearly everything. It completely takes the friction out of literature reviews and research whilst providing much more that a single text file. The snapshotting and auto pdf download are the best parts.
I like me some cli, but if I was to do something similar it would be with csl-json and jq/nushell. I cannot see the advantage of grepping a bibtex file, let alone having to manually manage the related articles.
I just finished importing and sorting thousands of PDF research papers, about 50 GB I've accumulated over the decades. Originally, I was sorting them in file system directly, but the whole thing got out of hand at some point, and thus the Documents folder became a dumping ground for PDFs. So I decided to take control of the situation and use Zotero to manage all this stuff. First I used Claude Code to generate all the BibTex files for the PDFs, then sort them into categories (still on the filesystem). Then created a master bibtex file that lists all the docs in each category. I used that to bulk import in Zotero, which then normalised all the file names once imported. What's really cool is that you can use Zotero built in web APIs to get an external tool to manipulate your collection directly. I've also setup WebDAV to sync the entire collection across multiple machines. Can you do all these without Zotero? Absolutely. And I did, but can't say I'll be looking back any time soon.
At least with Zotero Version 6, you could have just selected all your PDFs and drag-n-dropped them into the interface.
Like another poster here, trusting an LLM with my reference database -- the ultimate source of truth -- is not a step I'd be willing to take. All it takes is a single hallucinated reference, and your career would be forever tainted. It's not worth the risk.
Sadly, Zotero seems to have removed this killer import feature in later versions, which is the reason I keep using version 6. It feels like later versions have been a route to dumbing-down the interface, prioritising simplicity (and an ultra-low-contrast interface) at the expense of functionality. (If you can still drag-n-drop PDFs straight in with the new versions, someone please let me know?)
Drag-and-drop of the PDF file worked for me in Zotero 9 (latest version). I never used this feature before. This would be greatly preferred to my earlier suggestion to get a LLM to generate a list of DOIs.
I still drag and drop PDFs in Zotero 7. It's less finicky then v6, even. The trick is to drop them to root of the tree on the left. Not to the middle of the interface.
I drag-drop PDFs and ePubs, read PDFs and ePubs, and sync mobile with laptop, etc.
> First I used Claude Code to generate all the BibTex files for the PDFs
I think this has an unnecessary risk of hallucinated bibliographic data. For anyone doing something similar in the future, it would be more reliable to make a LLM generate a list of DOIs and have Zotero import the DOIs.
Using a non-deterministic machine on something which needs to be pretty accurate and precise looks like playing Russian roulette with one's academic integrity.
It's not my cup of vodka, to be sure.
And moreover, something already mostly solved algorithmically. Eg in zotero you can drag and drop files and it generates a reference based on the metadata as long as that is enough (somebody mentions it is not possible in newer versions, I use the latest version and it works fine).
Each one can choose their own tools of course, but we often see llms shown as solutions to something presented as unsolvable before, while it already was pretty amenable to normal tools.
> but we often see llms shown as solutions to something presented as unsolvable before, while it already was pretty amenable to normal tools.
I see that as a symptom of being intellectually lazy, or not knowing the value in mastering a tool via deliberate use over time.
Once one invests the required time to use a tool properly, they can reap way more benefits from said tool when compared to jumping tools at minute's notice.
I have always stayed away from additional software for bibliography management for similar reasons that the authors cites. However, while I do not use Zotero the software, their ZoteroBib (https://zbib.org/) has been a huge time saver. No login or account needed; just copy the URL or DOI and it generates the BibTeX entry. I find it far more accurate than Google Scholar.
I think Zotero is an excellent tool and I used it for a long time. However, like the author, I also felt that Zotero was not the ideal tool for me. But a straight BibTeX file is also rather unwieldy. Essentially, bibliographic data has provenance (say, a DOI identifier), and it is very useful to keep track of this in a structured way.
A friend and I have been working (slowly, over the past few years, since we are both academics) on an abstraction layer over BibTeX called Autobib: https://github.com/autobib/autobib
Broadly speaking, Autobib is a CLI around SQLite database of BibTeX records. But in addition to plain BibTeX, Autobib is aware of 'external data providers' (like DOI, MathSciNet, OpenLibrary, arXiv, zbMath, etc.) and will automatically retrieve data bibliographic data from these data providers. The provenance of the data is stored alongside the record itself, and this can be used to retrieve updates, prevent duplication of data, etc.
The killer feature is: if you have a file (say `file.tex`) and the keys are in a format which Autobib can automatically recognize (say, you use citation keys like `doi:my/weird/doi`, and there is support for custom formats and aliases) you can run `autobib source file.tex` and it will write to standard output a sorted BibTeX bibliography for your file. This lets you trivially maintain a per-project bibliography which you can check into source control locally and which exactly corresponds to the paper itself.
But otherwise, Autobib is "just a wrapper over a BibTeX bibliography"! When you edit an existing record, you are just editing a BibTeX record. There is integrated search, directly on the BibTeX fields themselves.
There are some extra features, like support for attachments, fuzzy search, undo-tree support, headless edit, auto-normalization, soft deletion, replacement, merging, etc. The database format is relatively simple and open (currently not particularly well documented, but this will change when it stabilizes) to allow introspection by other tools.
The tool also strives to play nicely with other tools (structured output, composable, etc.)
> just glorified front-ends for BibTeX
These frontends are necessary however because researchers in non-computer related fields are not trained/proficient at command line tools. Many of them need help installing software. Many of them don’t use raw text files much either. They use MS word files instead etc.
I mean even if you are comfortable with grep, it is still not enough, plus then you have to reinvent all the rest of the functionality, ie an extension to automatically add references from a webpage, download pdfs and keep them in a directory with some local link to be able to find and open them, and then another tool to export a subset of the huge bibtex file for each specific project and sync them, as it would not make sense to share a huge bibtex file on every latex project where you need only a small subset.
Zotero obviously stores data in an SQL db. It's not quite a text file, but it's been around forever, and it's easy to parse directly, too.
I'm not sure if I would ditch Zotero. Using grep over a bib file is nice but I guess you are opening the DOIs in browser every time?
The reason I like Zotero is because I can use it offline, have my all documents synced (the backup feature is one of the best things) when needed and I can organize my papers according to project/paper I'm working on. Actually one thing I do wish Zotero would add is tab groups which will allow me have papers open but grouped by project. It's based on Firefox which does support this so I hope it's a matter of time.
I tried to use Zotero so many times... and then I just tried a few other options, and in the end I realised that Zotero is the standard for academic writing... and I just could never got into it... in the end I used a text file, a literal text file and I am quite proud of my academic writing, I don't wish to do it again, but in the end, what worked for me was a literal piece of crap txt file.
JabRef before version 3 was pretty compact and snappy. It's essentially editing a text file for taking notes, except you can search through, download arxiv and journal articles (with LocalCopy extension), launch the associated PDFs.
I think that Citavi is extremely useful.
This sells Zotero short in so many ways, but you do you, I guess. Have fun hand formatting between the dozen citation/reference formats every other journal chooses, seemingly at random
Why would you need that? 99% of the time, you'll just grab the BibTeX from where you're getting the paper (Google Scholar, arXiv, ResearchGate, etc.) and paste it on the .bib file. LaTeX will take care of formatting it when you \cite.
I was wrong. I was not aware that biblatex had these features so easily available. Learned something.
Isn't citation format handled entirely in the latex document itself?
Mostly yes, you still need to check capitalisation of special words... But this should be independent of the actual citation style.