It helps to gain a 30,000 ft view before zooming into all the details:
Family Spatial symmetry Example
------ ---------------- -------
Bessel Cylindrical (r,θ,z) circular drum, heatpipes, waveguides
Legendre (Ordinary) Axisymmetric Spherical (θ) earth's bulge spinning on its axis
Associated Legendre Full 3D Angular Spherical (θ,ϕ) spherical harmonics
Laguerre Radial Spherical (r) radial pdf of electron in hydrogen atom
There’s a kind of grief that I feel when I think about how, over the last 70 years, engineers have tended to increasingly rely on tech that makes the math knowledge unnecessary.
The beauty of math has played a huge role in the career choices that I’ve made so far. I changed fields at one point because I realized that the math that practitioners used was almost exclusively all baked into CAD tools. I figured that if that was where the applied math lived, then I wanted to write software for a living.
With a lot of the AI tools now (which I use and appreciate daily FWIW), I’m beginning to see that even in math-heavy sub disciplines of software engineering (e.g. robotics), we’re not exempt from this trend.
This comment reminds me: I was in a used bookstore a few weeks ago with my dad (a retired engineer) and he bought up a bunch of the old math and engineering books because, he said, they approach their subjects differently and explain how things work, they walk you through the math and work to build up a detailed mental model of what's actually happening. His feeling was that modern textbooks make everything so abstract that the subject begins to feel divorced from what you're trying to do or understand, which makes getting an intuitive grasp on a topic really difficult.
He showed me some of the books from the 40's and 50's geared towards teens and honestly they did a better job of explaining things than the college textbooks I had to use -- they didn't go into fine detail, but they gave you a good framework to build on and a way of thinking about something which I feel is lacking from contemporary texts.
Tangentially, Feynman said something in a similar vein about the transition from analogue to digital. Kids couldn't tinker anymore and had no real impetus to learn some of the laws which underpin their workings, and most importantly, had no way to build an intuition around these laws through casual experimentation (i.e. fixing a broken radio with a handful of parts from Radio Shack).
Abstraction can be a very powerful tool, but growing up and learning in an environment which has these abstractions already built-in is a real handicap to building a deep understanding of a subject. You become dependent on the tools themselves instead of developing the knowledge used to build the tools -- in effect, the tools become limitations. Look at kids who grew up with iPhones instead of Amigas, or people already becoming dependent on LLM's.
There's a book called "Quick Calculus" (originally published in 1965, though I have the 1991 second edition) that my high school physics teacher gave me as a way to prep for physics class over the summer. It's a self teaching guide to basic calculus and is written to be used standalone without a lecture or grader. It has choose your own adventure stuff where if you get a question right you can skip over an extra explanation/example.
I've kept it around and used it for self teaching when I went down the rabbit hole of prepping for actuarial exams a year back, and again for prepping for going back to college to change careers to medical physics (yes, I have been looking at a bunch of different options to leave software).
I find it so much more elegant and straightforward than modern textbooks, and it's so much easier to focus on pen and paper vs screens. I've had people suggest Khan Academy and 3Blue1Brown, but I find I learn more deeply and retain better from a good old school textbook.
I completely agree that so much interesting math is hidden behind hyper-proprietary tools that engineers are expected to use.
As you note, all the Finite Element tricks are hidden, all the meshing tricks are hidden, and while you can find some in old research papers, in general the main companies hoard them.
The landscape has been 3 / 4 forces:
1) individual academics that do interesting work but more often than not abandon the codes they produce after a paper or three.
2) Proprietary companies that hire enough expertise to take academic research, generalize it, and sell a hyper-proprietary product with very little competition because the expertise is hard to come by
3) The national labs which do amazing work, that release a body of reliable open source work, though some of it is tailored to their systems - usually hard to use / very few people know about it outside academia.
4) Open projects like OpenFOAM / OpenCascade that struggle along at the edges to create a cohesive whole that can compete with the big companies.
I think that LLMs offer an opportunity free us from this status quo:
1) The open source work of the past is much easier to use / decipher with an LLM
2) The research is there, it just hasn't been productized (through an open source vector) / made accessible yet
3) I do believe that LLMs make learning the past codes / getting ideas for new codes / extending codes easier
I think engineering tools are up for a shakeup, as both LLMs + new paradigms are emerging. There is really interesting research into new meshing methods / CAD representation (Brep vs SDF) / topology optimization.
I believe this is an opportunity for an open source ecosystem to start making inroads into the bottom layers of the engineering stack (CAD kernels / representation), and from there make inroads into meshing + make open source FEA more accessible.
I am excited by what the future might bring, and remain cautiously optimistic that just like no one buys a programming language anymore, the lowest levels of the engineering stack might become open source so that research is accelerated, engineers are freed from total lockin, and in general the ecosystem is much healthier and innovative.
To a degree, yes. That's not _that_ long after Einstein had to seek out math experts because he didn't know the theory behind 5x5 matrices. Math professors I know who started much later than that didn't really get how linear algebra was useful when teaching ODEs. It's not that it wasn't useful, but it wasn't commonly used.
I once had a mentor that was trying to help me write a CPU characterization and being a phd in math every suggestion he proposed involved derivatives. I was like, why would I describe I486 cache misses in terms of differential equations? To this day I still don’t know what he meant.
I think one could down the hardware to gates, then convert that to a partial difference equation (time discrete). In F_2 sum is "xor" and product is "and". "and" is bilinear. But it is not a universal gate. Don't shoot the pianist, but gpt suggestion is passing from linear to affine algebra. There traslating by 1 (traslation being summing, xor) is "not" and is an affine map, just as "and" is affine in each argument because being linear. Nice thing is that affine maps can be encoded as linear ones one dimension higher so "nand" can be seen as bilinear form acting on two 2D args (homogeneous bit coordinates in F^2*F^2), that is a twice covariant tensor. That tensor can be expressed less efficientely as a twice covariant, once contravariant one. The advantage of that being that you can map a circuit of "nands" to a tensor network of same topology so a big linear map L, so the difference operator becomes L-Id and that leads to the difference equation. mathse speaks confusingly both of difference and differential equations. Take this as a napkin reconstruction idea, not an account of what Feynman actually did (note also the above assumes RAM as part of the state of the CPU as "cache"/"mega-register-bank" and no I/O so model passes from Mealy machine to a fixed iterative transition function acting on initialized internal memory).
TBH, this might be a good time to figure out what he was talking about! A good continuous approximation of a discrete problem can radically simplify the situation. And current LLMs are extremely patient tutors.
It helps to gain a 30,000 ft view before zooming into all the details:
There’s a kind of grief that I feel when I think about how, over the last 70 years, engineers have tended to increasingly rely on tech that makes the math knowledge unnecessary.
The beauty of math has played a huge role in the career choices that I’ve made so far. I changed fields at one point because I realized that the math that practitioners used was almost exclusively all baked into CAD tools. I figured that if that was where the applied math lived, then I wanted to write software for a living.
With a lot of the AI tools now (which I use and appreciate daily FWIW), I’m beginning to see that even in math-heavy sub disciplines of software engineering (e.g. robotics), we’re not exempt from this trend.
This comment reminds me: I was in a used bookstore a few weeks ago with my dad (a retired engineer) and he bought up a bunch of the old math and engineering books because, he said, they approach their subjects differently and explain how things work, they walk you through the math and work to build up a detailed mental model of what's actually happening. His feeling was that modern textbooks make everything so abstract that the subject begins to feel divorced from what you're trying to do or understand, which makes getting an intuitive grasp on a topic really difficult.
He showed me some of the books from the 40's and 50's geared towards teens and honestly they did a better job of explaining things than the college textbooks I had to use -- they didn't go into fine detail, but they gave you a good framework to build on and a way of thinking about something which I feel is lacking from contemporary texts.
Tangentially, Feynman said something in a similar vein about the transition from analogue to digital. Kids couldn't tinker anymore and had no real impetus to learn some of the laws which underpin their workings, and most importantly, had no way to build an intuition around these laws through casual experimentation (i.e. fixing a broken radio with a handful of parts from Radio Shack).
Abstraction can be a very powerful tool, but growing up and learning in an environment which has these abstractions already built-in is a real handicap to building a deep understanding of a subject. You become dependent on the tools themselves instead of developing the knowledge used to build the tools -- in effect, the tools become limitations. Look at kids who grew up with iPhones instead of Amigas, or people already becoming dependent on LLM's.
There's a book called "Quick Calculus" (originally published in 1965, though I have the 1991 second edition) that my high school physics teacher gave me as a way to prep for physics class over the summer. It's a self teaching guide to basic calculus and is written to be used standalone without a lecture or grader. It has choose your own adventure stuff where if you get a question right you can skip over an extra explanation/example.
I've kept it around and used it for self teaching when I went down the rabbit hole of prepping for actuarial exams a year back, and again for prepping for going back to college to change careers to medical physics (yes, I have been looking at a bunch of different options to leave software).
I find it so much more elegant and straightforward than modern textbooks, and it's so much easier to focus on pen and paper vs screens. I've had people suggest Khan Academy and 3Blue1Brown, but I find I learn more deeply and retain better from a good old school textbook.
I completely agree that so much interesting math is hidden behind hyper-proprietary tools that engineers are expected to use.
As you note, all the Finite Element tricks are hidden, all the meshing tricks are hidden, and while you can find some in old research papers, in general the main companies hoard them.
The landscape has been 3 / 4 forces:
1) individual academics that do interesting work but more often than not abandon the codes they produce after a paper or three.
2) Proprietary companies that hire enough expertise to take academic research, generalize it, and sell a hyper-proprietary product with very little competition because the expertise is hard to come by
3) The national labs which do amazing work, that release a body of reliable open source work, though some of it is tailored to their systems - usually hard to use / very few people know about it outside academia.
4) Open projects like OpenFOAM / OpenCascade that struggle along at the edges to create a cohesive whole that can compete with the big companies.
I think that LLMs offer an opportunity free us from this status quo:
1) The open source work of the past is much easier to use / decipher with an LLM
2) The research is there, it just hasn't been productized (through an open source vector) / made accessible yet
3) I do believe that LLMs make learning the past codes / getting ideas for new codes / extending codes easier
I think engineering tools are up for a shakeup, as both LLMs + new paradigms are emerging. There is really interesting research into new meshing methods / CAD representation (Brep vs SDF) / topology optimization.
I believe this is an opportunity for an open source ecosystem to start making inroads into the bottom layers of the engineering stack (CAD kernels / representation), and from there make inroads into meshing + make open source FEA more accessible.
I am excited by what the future might bring, and remain cautiously optimistic that just like no one buys a programming language anymore, the lowest levels of the engineering stack might become open source so that research is accelerated, engineers are freed from total lockin, and in general the ecosystem is much healthier and innovative.
I have never used more math since llms. My projects have just become more ambitious.
Would be good if it included linear algebra, probability and stats in details
1959.
That time Linear algebra was of no use? Possible may be, Electrical, civil and Mechanical engineerings were mainstream
To a degree, yes. That's not _that_ long after Einstein had to seek out math experts because he didn't know the theory behind 5x5 matrices. Math professors I know who started much later than that didn't really get how linear algebra was useful when teaching ODEs. It's not that it wasn't useful, but it wasn't commonly used.
I once had a mentor that was trying to help me write a CPU characterization and being a phd in math every suggestion he proposed involved derivatives. I was like, why would I describe I486 cache misses in terms of differential equations? To this day I still don’t know what he meant.
I remember a story about Feynman doing the same thing (analyzing a computer program with differential equations). Seems to have worked well for him.
See https://math.stackexchange.com/questions/4752749/how-could-f...
I think one could down the hardware to gates, then convert that to a partial difference equation (time discrete). In F_2 sum is "xor" and product is "and". "and" is bilinear. But it is not a universal gate. Don't shoot the pianist, but gpt suggestion is passing from linear to affine algebra. There traslating by 1 (traslation being summing, xor) is "not" and is an affine map, just as "and" is affine in each argument because being linear. Nice thing is that affine maps can be encoded as linear ones one dimension higher so "nand" can be seen as bilinear form acting on two 2D args (homogeneous bit coordinates in F^2*F^2), that is a twice covariant tensor. That tensor can be expressed less efficientely as a twice covariant, once contravariant one. The advantage of that being that you can map a circuit of "nands" to a tensor network of same topology so a big linear map L, so the difference operator becomes L-Id and that leads to the difference equation. mathse speaks confusingly both of difference and differential equations. Take this as a napkin reconstruction idea, not an account of what Feynman actually did (note also the above assumes RAM as part of the state of the CPU as "cache"/"mega-register-bank" and no I/O so model passes from Mealy machine to a fixed iterative transition function acting on initialized internal memory).
TBH, this might be a good time to figure out what he was talking about! A good continuous approximation of a discrete problem can radically simplify the situation. And current LLMs are extremely patient tutors.
Well it was 36 years ago and not sure where he is but I’m sending him positive vibes (he was old when I met him)!
Maybe he trying looking at cache like some flow problem... Perhaps useful for the cache size, not the misses, it seems
What about all the mathematical discoveries Marc Andreessen claims were kept hidden from academia under the guise of national security?
> What about all the mathematical discoveries Marc Andreessen claims were kept hidden from academia under the guise of national security?
Reference:
https://althouse.blogspot.com/2024/12/during-cold-war-we-cla...