It is an interesting tool. I've been struggling with Office Excel's inability to open large files. I always work with csv in python, and if a client must review the data in Excel, I take a random sample to generate a smaller file, then explain to the client that we can't open the whole in Excel. This really doesn't seem like a modern work.
"a slow, old pc with 8GB RAM"
By the way, this struck me like a humour of era. Oh god it has 8GB RAM. Cheers! To the good old days.
Fully agree, I have found that above 300k rows Excel struggles even on a good laptop. Not even mentioning the Python integration into MS Excel that is so unbearably slow that it is much better performing the calculations outside of Excel first.
These days I use DuckDB to read massive excel files. DuckDB now ships with a nice local UI and it also works beautifully with Datagrip, my preferred database IDE. With SQL, it just becomes a matter of applying old grease to do whatever analysis I want.
So I'll take a layman's view here since I've only cursory experience of the big data tasks that this software seems to made for. Or maybe the pitch is still different and it went over my head.
It loads quick, and works with large data. Crucially, you can view and edit visually, not only programmatically.
Assuming those already working with such data have Excel and Python tools etc., the pitch here is that the $39 license fee saves time or effort. So, is it that the user can spot and correct errors that you couldn't otherwise do with either Excel or with other big data tools? And/or otherwise do the necessary data manipulations?
I came across the phrase 'eyes like a shithouse rat' recently, to describe the people doing final checks at a printing press. I think there's probably plenty of people out there who would pay $39 for eyes like a shithouse rat.
I like old-school UIs but I wonder if that look doesn't do the product a disservice. I think most people would find it much more appealing if it looked at least as good as tad and rowzero mentioned in other comments. My first impression was that it is some old, slow software from 30 years ago, and the plots are not what I would show to anyone (especially the 3d bar plots). But that's just the looks. Otherwise, the product is solid.
(Yes, I know that there is plenty of old software that is super fast.)
Yep, the website is being kept simple, in fact too simple and will have to be eventually redesigned.
It seems the classic (and actually up to date) WinAPI GUI shouldn't imply the software is being "slow". One could have it both ways and say it's usually on the contrary where commonly cloned packages weighing hundreds of MB and performing relatively simple tasks get a free ride on users' ten(s) of times faster hardware. Then there is the Wirth's law, May's law etc.
It all sounds very compelling , great work! But I have to ask, what’s the catch? This almost seems like it’s ready to fully replace Excel but I’ve seen many things die in that pursuit. What will Excel users miss by switching?
I’m don’t do a ton to big data stuff, but sometimes despite Excels stated row and column support- I find it effectively melts down if even 100K/100 of data and forget adding formulas.
Thanks. This is just a set of features that offer better (sometimes significantly better) performance and make it possible to work with large data sets in some predictable way, that is, knowing what the performance is with 100K formulas, you should be able to estimate how it'll work with 10 millions no matter how much formatting is applied, added hyperlinks, links to external closed workbooks, lists etc. That is probably correct, in the upcoming years there might be no way to compete directly with (the standalone) Excel. Hopefully it just helps to perform or verify various things way faster than in Excel with add-ins (or often at all).
MS Office Macros are too easy for untrained users (that think of themselves as power users) to create, and once they grow big enough, it's very hard to port them to other applications. They're the monkey paw of spreadsheets.
VBA and the ability to use Office files with embedded VBA are disabled in many corporations… Some malware in the past used VBA for their attacks and Microsoft never added a proper sandbox.
As co-founder of Resolver Systems -- we tried but ultimately failed to take on Excel with a Python-enabled equivalent back in 2007 -- and current employee at Anaconda (providing Python in Excel) I really do hope you get this one to work. Excel is a mess, Python is better, and someone surely will eventually be able to fix the former with the latter. Let's hope it's you :-)
This is a potentially interesting feature - it probably could be relatively easily extended to the shared environment in GS-Calc as there are already separate passwords to protect files, the sheet/view structures and cell ranges and optional sha256 checksums for all the data categories, but there no plans as such. At the moment you can just use two script commands to release the opened file, periodically check whether it's modified, report it and automatically reload the file.
https://rowzero.io/ can handle 1 billion+ rows and offers native Python support. Also compatible with Excel and Google Sheets. However it’s a cloud based solution, and the private hosting option is only available to Enterprise users.
It is an interesting tool. I've been struggling with Office Excel's inability to open large files. I always work with csv in python, and if a client must review the data in Excel, I take a random sample to generate a smaller file, then explain to the client that we can't open the whole in Excel. This really doesn't seem like a modern work.
"a slow, old pc with 8GB RAM"
By the way, this struck me like a humour of era. Oh god it has 8GB RAM. Cheers! To the good old days.
I've been using Tad [0] for this purpose, as it streams in the data.
[0] https://www.tadviewer.com
Fully agree, I have found that above 300k rows Excel struggles even on a good laptop. Not even mentioning the Python integration into MS Excel that is so unbearably slow that it is much better performing the calculations outside of Excel first.
I am sold on the website looks and license model!
These days I use DuckDB to read massive excel files. DuckDB now ships with a nice local UI and it also works beautifully with Datagrip, my preferred database IDE. With SQL, it just becomes a matter of applying old grease to do whatever analysis I want.
So I'll take a layman's view here since I've only cursory experience of the big data tasks that this software seems to made for. Or maybe the pitch is still different and it went over my head.
It loads quick, and works with large data. Crucially, you can view and edit visually, not only programmatically.
Assuming those already working with such data have Excel and Python tools etc., the pitch here is that the $39 license fee saves time or effort. So, is it that the user can spot and correct errors that you couldn't otherwise do with either Excel or with other big data tools? And/or otherwise do the necessary data manipulations?
I came across the phrase 'eyes like a shithouse rat' recently, to describe the people doing final checks at a printing press. I think there's probably plenty of people out there who would pay $39 for eyes like a shithouse rat.
Also the website makes me nostalgic :)
I like old-school UIs but I wonder if that look doesn't do the product a disservice. I think most people would find it much more appealing if it looked at least as good as tad and rowzero mentioned in other comments. My first impression was that it is some old, slow software from 30 years ago, and the plots are not what I would show to anyone (especially the 3d bar plots). But that's just the looks. Otherwise, the product is solid.
(Yes, I know that there is plenty of old software that is super fast.)
Yep, the website is being kept simple, in fact too simple and will have to be eventually redesigned. It seems the classic (and actually up to date) WinAPI GUI shouldn't imply the software is being "slow". One could have it both ways and say it's usually on the contrary where commonly cloned packages weighing hundreds of MB and performing relatively simple tasks get a free ride on users' ten(s) of times faster hardware. Then there is the Wirth's law, May's law etc.
It all sounds very compelling , great work! But I have to ask, what’s the catch? This almost seems like it’s ready to fully replace Excel but I’ve seen many things die in that pursuit. What will Excel users miss by switching?
I’m don’t do a ton to big data stuff, but sometimes despite Excels stated row and column support- I find it effectively melts down if even 100K/100 of data and forget adding formulas.
Thanks. This is just a set of features that offer better (sometimes significantly better) performance and make it possible to work with large data sets in some predictable way, that is, knowing what the performance is with 100K formulas, you should be able to estimate how it'll work with 10 millions no matter how much formatting is applied, added hyperlinks, links to external closed workbooks, lists etc. That is probably correct, in the upcoming years there might be no way to compete directly with (the standalone) Excel. Hopefully it just helps to perform or verify various things way faster than in Excel with add-ins (or often at all).
> What will Excel users miss by switching?
The ability to use legacy spreadsheets and macros.
Let's be real, Excel self perpetuates by at once being awful but also the thing everyone used to use thus must still use.
Lots of spreadsheet apps better than Excel have come and gone over the years...
MS Office Macros are too easy for untrained users (that think of themselves as power users) to create, and once they grow big enough, it's very hard to port them to other applications. They're the monkey paw of spreadsheets.
VBA and the ability to use Office files with embedded VBA are disabled in many corporations… Some malware in the past used VBA for their attacks and Microsoft never added a proper sandbox.
I wonder if anyones gonna eventually just create a wrapper API to just a vm that runs some stock excel open to a spreadsheet...
As co-founder of Resolver Systems -- we tried but ultimately failed to take on Excel with a Python-enabled equivalent back in 2007 -- and current employee at Anaconda (providing Python in Excel) I really do hope you get this one to work. Excel is a mess, Python is better, and someone surely will eventually be able to fix the former with the latter. Let's hope it's you :-)
Thank you. The current integration is described in the online HTML GS-Calc help. Any suggestions would be highly appreciated.
Looks very cool, but Windows only?
Thanks. Yes, that was put off a bit too long, but there are plans to come up with the multiplatform version this year.
LibreOffice has Python integration.
https://help.libreoffice.org/latest/en-US/text/sbasic/python...
are there any plans to add real-time collaboration features, like Google Sheets or Excel Online
This is a potentially interesting feature - it probably could be relatively easily extended to the shared environment in GS-Calc as there are already separate passwords to protect files, the sheet/view structures and cell ranges and optional sha256 checksums for all the data categories, but there no plans as such. At the moment you can just use two script commands to release the opened file, periodically check whether it's modified, report it and automatically reload the file.
https://rowzero.io/ can handle 1 billion+ rows and offers native Python support. Also compatible with Excel and Google Sheets. However it’s a cloud based solution, and the private hosting option is only available to Enterprise users.
[flagged]