I love how Django just keeps slowly improving at every release. 6.0 is especially cool, including lots of really useful new features. Who said dependable tech was dull - this is the way is should be done. Well done all who contribute.
Same! I've been using it since pre-1.0 and love it. I am currently a few blocks away from it's birthplace.
Probably the wrong time or place but I am also on the market literally as of yesterday so if anyone is looking for an experienced Django guy, I'm your man! oldspiceap@gmail.com
Any code or blog written by Adam is worth spending some time on.
It will be interesting to see how the tasks framework develops and expands. I am sad to see the great Django-Q2 lumped in with the awful Celery though.
Celery is the worst background task framework, except for all the others.
There are bugs and issues, but because so many people are using it, you’re rarely the first to stumble upon a problem. We processed double-digit millions of messages daily with Celery + RabbitMQ without major obstacles. Regardless of what people say, it should be your first go-to.
Celery has way too much magic crammed into it, it is very annoying to debug, and produces interesting bugs. Celery is/was also a "pickle-first" API and this almost always turns out to be the wrong choice. As a rule of thumb, persisting pickles is a really bad idea. Trying to hide IPC / make-believe that it's not there tends to be a bad idea. Trying to hide interfaces between components tends to be a bad idea. Celery combines all of these bad ideas into one blob. The last time I looked the code was also a huge mess, even for old-guard-pythonic-code standards.
I think Celery has a lot of magic happening under it. When the abstractions are so high, it's important they never leak and you don't see anything below the turtles you are supposed to see.
I often prefer designing around explicit queues and building workers/dispatchers. One queuing system I miss is the old Google App Engine one - you set up the queue, the URL it calls with the payload (in your own app), the rate it should use, and that's it.
I tried django-q and I thought it was pretty terrible. The worst was that I couldn't get it to stop retrying stuff that was broken. Sometimes you ship code that does something unexpected, and being able to stop something fast is critical imo.
Fundamentally I think the entire idea behind celery and django-q is mostly misguided. People normally actually need a good scheduler and a bring-your-own queue in tables that you poll. I wrote Urd to cover my use cases and it's been rock solid.
Temporal is an AMAZING piece of software, however I don't believe it's a replacement for something more simple like Celery. Even if you write helpers, the overhead to setting up workflows, invoking them, etc. is just too much for simple jobs like sending an email (imo). I would love to work in a codebase that had access to both, depending on the complexity of what you're trying to background.
It's okay till it's not. Everyone I know who had Celery in production was looking for a substitution (custom or third-party) on a regular basis. Too many moving pieces and nuances (config × logic × backend), too many unresolved problems deep in its core (we've seen some ghosts you can't debug), too much of a codebase to understand or hack. At some point we were able to stabilize it (a bunch of magic tricks and patches) and froze every related piece; it worked well under pressure (thanks, RabbitMQ).
Because it’s a seducer. It does what you need to do and you two are happy together. So you shower more tasks on Celery and it becomes cold and non-responsive at random times.
And debugging is a pain in the ass. Most places I’ve been that have it, I’ve tried to sell them on adding Flower to give better insight and everyone thinks that’s a very good idea but there isn’t time because we need to debug these inscrutable Celery issues.
Although we could say the same thing about Kafka, couldn't we? It's made for much higher throughput and has usually other use cases, but it's also great until it's not great.
At least the last time I used Kafka (which was several years ago so things might have changed) it wasn't at all easy to get started. It was a downright asshole in fact. If you pursue a relationship with an asshole, you shouldn't be surprised when they become cold to you
Celery is great and awful at the same time. In particular, because it is many Python folks' first introduction to distributed task processing and all the things that can go wrong with it. Not to mention, debugging can be a nightmare. Some examples:
- your function arguments aren't serializable
- your side effects (e.g. database writes) aren't idempotent
- discovering what backpressure is and that you need it
- losing queued tasks during deployment / non-compatible code changes
There's also some stuff particular to celery's runtime model that makes it incredibly prone to memory leaks and other fun stuff.
> your side effects (e.g. database writes) aren't idempotent
What does idempotent mean in this context, or did you mean atomic/rollback on error?
I'm confused because how could a database write be idempotent in Django? Maybe if it introduced a version on each entity and used that for crdt on writes? But that'd be a significant performance impact, as it couldn't just be a single write anymore, instead they'd have to do it via multiple round trips
In the context of background jobs idempotent means that if your job gets run for a second time (and it will get run for a second time at some point, they all do at-least-once delivery) there aren't any unfortunate side effects to that. Often that's just a case of checking if the relevant database updates have already been done, maybe not firing a push notification in cases of a repeated job.
If you need idempotent db writes, then use something like Temporal. You can't really blame Celery for not having that because that is not what Celery aims to be.
With Temporal, your activity logic still needs to ensure idempotency e.g. by checking if an event id / idempotency key exists in a table. It's still at-least-once delivery. Temporal does make it easy to mint an idempotency key by concatenating workflow run id and activity id, if you don't have a one provided client-side.
Temporal requires a lot more setup than setting up a Redis instance though. That's the only problem with it. And I find the Python API a bit more difficult to grasp. But otherwise a solid piece of technology.
In my experience async job idempotency is implemented as upserts. Insert all job outputs on the first run. Do (mostly) nothing on subsequent runs. Maybe increment a counter or timestamp.
I'm of the opinion that django task apps should only support a single backend. For example, django-rq for redis only. There's too many differences in backends to make a good app that can handle multiple. That said, I've only used celery in production before, and I'm willing to change my mind.
I've been using Django on and off at work for the past few years. I really like it. That being said, I still find its ORM difficult. I understand it now that since it's an opinionated framework, I need to follow Django way of thinking. The main issue is that at work, I have multiple databases from different business units. So I constantly have to figure out a way to deal with multiple databases and their idiosyncrasies. I ended up doing a lot of hand holding by turning off managed, inspectdb and then manually delete tables I don't want to show via website or other reasons. For green webapps we have, django is as good as it gets.
Agreed, and their DB migration workflow leaves much to be desired. By not storing a schema/DB state alongside code, Django depends on the current DB state to try and figure it out from scratch every time you run a DB command. Not to mention defining DB state from model code is inherently flawed, since models are abstractions on top of database tables. I much prefer the Rails way of composing migrations as specific DB instructions, and then getting models 'for free' on top of those tables.
Yes, we have to in order to use a lot of the features. The core issue for us is really the way Django assumes code represents database state. In normal webapp where the application has full control of the database, that's a good idea. But our databases are overloaded for simple transactions, analytics, users managements, jobs and AI. Business uses the databases in various ways such as Power BI, Aquastudio, etc.. Django app is actually a tiny part of the database. As you can imagine, we duck tape the heck out of the databases, and Django goes bonkers when things aren't matching.
I've used Aldjemy (https://github.com/aldjemy/aldjemy) on a small project and it worked pretty well for allowing me to compose the fairly complex queries needed that the Django ORM couldn't do.
Also don't underestimate setting up e.g. views or materialized views even that you can use through the ORM to query. It helps a lot and allows you to combine fine tuning SQL with ease of use through Django, and get a lot of performance out of it. Just remember to create them in the migration scripts.
I've been using Django for the last 10+ years, its ORM is good-ish. At some point there was a trend to use sqlalchemy instead but it was not worth the effort. The Manager interface is also quite confusing at first. What I find really great is the migration tool.
Since Django has gained mature native migrations there is a lot less point to using SQLAlchemy in a Django project, though SQLAlchemy is undeniably the superior and far more capable ORM. That should be unsurprising though - sqlalchemy is more code than the entire django package, and sqlalchemy + alembic is roughly five times as many LOC as django.db, and both are similar "density" code.
Template partials look good, which is one of the key reasons frameworks like React are as good and popular as they are, because you can reuse small segments of code.
I like this approach. I am especially drawn to the idea of making custom components this way but every time I have experimented with this I get burned by the context which has to be passed down through all functions.
A jinja/django template has an implicit context but for nested functions you really have to pass that context down through every function call.
It inevitably ends up just a big dict blob.
You get some typing support in an IDE but nothing really for function parameters.
iommi is wroth mentioning here. It is different from an HTML generator, but one of the things it does is greatly reduce the amount of HTML you need write.
Yeah, I agree, I find them hard to read. JSX is the best thing I've used. Elsewhere in the thread someone mentioned Cotton which seems to strike a different balance.
To be honest my main problem with templates is they have to be one per file. In principle there's no difference between naming a new file and naming a function, but in practice it just sucks. It's a higher barrier so people are less likely to write smaller components, and refactoring support completely sucks. Even renaming a template is a massive pain whereas renaming a function with decent LSP support is easy.
JSX hits that perfect balance between readability while still being regular functions. Maybe something is possible with the new 3.13 template strings?
As an FYI, I'm with the template strings folks (new feature in 3.14) and we're building an HTML system around the idea of JSX-style components as callables, but with actual strings of HTML. Check our <a href="https://t-strings.github.io/tdom/usage/components.html">components page</a> for more detail.
With template strings, the symbols in the HTML string a just like Python code. Static analysis tooling can step in and do things.
We have ambitions to start an interoperability movement in Python HTML, so htpy and tdom could agree on a common Node structure.
>For nearly all cases, Django’s built-in template language is perfectly adequate. However, if the bottlenecks in your Django project seem to lie in the template system and you have exhausted other opportunities to remedy this, a third-party alternative may be the answer.
>Jinja2 can offer performance improvements, particularly when it comes to speed.
I do a check for `request.htmx` in my views and conditionally return a template partial as needed. This reduced my need for one-off view functions that were only returning partials for htmx. Works pretty well from my experience.
Partialdef inline is the real win. Lets you define parts of a page without needing to place them in another file. Reduces the mental overhead of imagining how the inclusion will look because it’s already there.
The use case is mainly driven by htmx where you will have lots of these partials and the view code renders them as individual responses.
It's just syntactic sugar, making life a bit easier for HTMX users (cf. "htmx was the main motivation for this feature").
I'm using Unpoly and I just render the whole page and let Unpoly swap the content according to the target selectors, so no need for this. Not much difference in perf if you dont generate gigantic pages with heavy header/footer.
Yeah, but I was doing the same thing 10 years ago with include mixed with extends and blocks. I can just include a file inside a template or render it directly.
you're kinda right, {% partial ... %} vs {% include ... %} is not a big difference, but my mind was vaguely thinking that "includes" have often been seen as large templates, whereas partial have been after the component era with the idea of making small blocks. (my 2 cents)
I’m very much an Odoo guy who has dabbled with Django (and notably, celery in the past) but as someone who makes huge use of the Odoo OCA queue module[0], I never understood why Django never made use of the Postgres LISTEN/NOTIFY to offload tasks. Maybe this is my misunderstanding as I’m pretty green with the Django ecosystem as a whole.
Correct. Django 6.0 comes with a standardised API, with 2 testing backends (ImmediateBackend and DummyBackend). You need a third-party backend to store and execute tasks.
django-tasks is the reference implementation for django.tasks, by the same author. It's up for discussion if and when the DatabaseBackend gets merged, and it seems likely to me that a Redis-based backend would be suggested at some point. It takes time and energy though!
I find templates atrocious to use for component fragments like this, that's why I wrote a Python component library when I started using Django with HTMX. Order of magnitude more pleasant to use, works with _every_ Python web framework not just Django: https://compone.kissgyorgy.me/
Dude, I used Django at 1.x - before they even had an ORM. The fact that it is adding a way to run tasks, almost a quarter of a century later, is wild to me.
I am not roasting it or anything, go Django, but just an observation.
I think it is refreshing. They don't half-ass things into the framework. They take the time to do it right. They let every feature fight for its life, and put their effort into LTS and minimizing number of issues and API changes related to the features they do deliver. As a developer I really appreciate this. I don't have to totally rewrite my entire application every new version because the implementation wasn't properly thought through.
I used to have this opinion about ASP.NET then ASP.NET Core and the great churn happened. It's finally settled down again, but boy the in-between years were chaotic.
Not just the Framework -> Core migration itself, but the power to make breaking changes went to their heads, and they started quickly tearing up everything only to change their minds again, such as a short-lived "project.json" syntax.
Django is exactly the technology I'd pick if I wasn't already super familiar with the .NET stack. It's got the "batteries included" feel without the chaotic confusion of a million ways to do things. It doesn't have the breaking changes churn that happens elsewhere too.
Django had ORM from the very beginning. I've been using Django since 0.95 at it had ORM even back then. It was primitive but I hadn't to resort to raw SQL until much later.
Given that Python tends to produce fewer hallucinations when generated by LLMs I wonder if former Django developers using AI tools are secretly having a blast right now.
I think another ace up Django's sleeve is that it has had a remarkable stable API for a long time with very few breaking changes, so almost all blogposts about Django that the LLM has gobbled up will still be mostly correct whether they are a year or a decade old.
I get remarkably good and correct LLM output for Django projects compared to what I get in project with more fast moving and frequently API breaking frameworks.
Whenever I saw people complain about LLMs writing code, I never really understood why they were so adamant that it just didn’t work at all for them.
The moment I did try to use LLMs outside of Django, it became clear that some frameworks are just much easier to work with LLMs than others. I immediately understood their frustrations.
It makes me sad when a secondary meaning, which does not even overcome the main meaning in usage, becomes an obstacle for the normal use of a word. It's like seeing a rainbow as a sexualized symbol not fit for children, because it also happens to be used by LGBTQ+ community. (BTW, since you're a Brit: did people stop using the word "fag" to refer to a cigarette?)
I mean, it is sad. But unfortunately that is what happened with "master", "slave", "whitelist", and "blacklist". No reasonable person construed these as offensive or having any implications about the wider world. But there are people in our profession who are determined to take offense where none is given, and unfortunately they got their way.
Well, "slave" has a pretty direct main meaning of an oppressed person doing forced labor. The word "master" is much milder in this regard (compare "master's degree" and "slave's degree"). The word "nonce" in normal usage seems even more removed from any pejorative secondary meanings.
That didn't stop people from throwing a fit over master-slave terminology in software (having nothing to do with slavery), going so far as to rename long-standing development branch names, as well as put significant effort into removing such terms from the code itself and any documentation.
I love how Django just keeps slowly improving at every release. 6.0 is especially cool, including lots of really useful new features. Who said dependable tech was dull - this is the way is should be done. Well done all who contribute.
Same! I've been using it since pre-1.0 and love it. I am currently a few blocks away from it's birthplace.
Probably the wrong time or place but I am also on the market literally as of yesterday so if anyone is looking for an experienced Django guy, I'm your man! oldspiceap@gmail.com
Any code or blog written by Adam is worth spending some time on.
It will be interesting to see how the tasks framework develops and expands. I am sad to see the great Django-Q2 lumped in with the awful Celery though.
Celery is the worst background task framework, except for all the others.
There are bugs and issues, but because so many people are using it, you’re rarely the first to stumble upon a problem. We processed double-digit millions of messages daily with Celery + RabbitMQ without major obstacles. Regardless of what people say, it should be your first go-to.
Celery has way too much magic crammed into it, it is very annoying to debug, and produces interesting bugs. Celery is/was also a "pickle-first" API and this almost always turns out to be the wrong choice. As a rule of thumb, persisting pickles is a really bad idea. Trying to hide IPC / make-believe that it's not there tends to be a bad idea. Trying to hide interfaces between components tends to be a bad idea. Celery combines all of these bad ideas into one blob. The last time I looked the code was also a huge mess, even for old-guard-pythonic-code standards.
I think Celery has a lot of magic happening under it. When the abstractions are so high, it's important they never leak and you don't see anything below the turtles you are supposed to see.
I often prefer designing around explicit queues and building workers/dispatchers. One queuing system I miss is the old Google App Engine one - you set up the queue, the URL it calls with the payload (in your own app), the rate it should use, and that's it.
OP here, thanks for the praise!
Yeah, I mentioned Celery due to its popularity, no other reason ;)
You are a great writer - thanks for putting this together!
I tried django-q and I thought it was pretty terrible. The worst was that I couldn't get it to stop retrying stuff that was broken. Sometimes you ship code that does something unexpected, and being able to stop something fast is critical imo.
Fundamentally I think the entire idea behind celery and django-q is mostly misguided. People normally actually need a good scheduler and a bring-your-own queue in tables that you poll. I wrote Urd to cover my use cases and it's been rock solid.
I've been using Celery for years. What is the major issues you have with it and how does Django Q2 help?
I also use Kafka on other tech stacks but that's another level completely and use case.
Why is celery awful?
> The Many Problems with Celery:
— https://steve.dignam.xyz/2023/05/20/many-problems-with-celer...
> The problems with (Python’s) Celery:
— https://docs.hatchet.run/blog/problems-with-celery
> Dramatiq motivation:
— https://dramatiq.io/motivation.html
Here are some alternatives:
Dramatiq: https://github.com/Bogdanp/dramatiq
RQ: https://github.com/rq/rq
Huey: https://github.com/coleifer/huey
Hatchet: https://github.com/hatchet-dev/hatchet
Would you consider tools like Temporal, DBOS, Absurd Workflows, PGQueuer as alternatives?
https://temporal.io/
https://docs.dbos.dev/
https://news.ycombinator.com/item?id=45797228
https://python-absurd-client.readthedocs.io/en/latest/quicks...
https://pgqueuer.readthedocs.io/en/latest/
Temporal is an AMAZING piece of software, however I don't believe it's a replacement for something more simple like Celery. Even if you write helpers, the overhead to setting up workflows, invoking them, etc. is just too much for simple jobs like sending an email (imo). I would love to work in a codebase that had access to both, depending on the complexity of what you're trying to background.
django-q2: https://github.com/django-q2/django-q2
It's okay till it's not. Everyone I know who had Celery in production was looking for a substitution (custom or third-party) on a regular basis. Too many moving pieces and nuances (config × logic × backend), too many unresolved problems deep in its core (we've seen some ghosts you can't debug), too much of a codebase to understand or hack. At some point we were able to stabilize it (a bunch of magic tricks and patches) and froze every related piece; it worked well under pressure (thanks, RabbitMQ).
Because it’s a seducer. It does what you need to do and you two are happy together. So you shower more tasks on Celery and it becomes cold and non-responsive at random times.
And debugging is a pain in the ass. Most places I’ve been that have it, I’ve tried to sell them on adding Flower to give better insight and everyone thinks that’s a very good idea but there isn’t time because we need to debug these inscrutable Celery issues.
https://flower.readthedocs.io/en/latest/
Although we could say the same thing about Kafka, couldn't we? It's made for much higher throughput and has usually other use cases, but it's also great until it's not great.
At least the last time I used Kafka (which was several years ago so things might have changed) it wasn't at all easy to get started. It was a downright asshole in fact. If you pursue a relationship with an asshole, you shouldn't be surprised when they become cold to you
Yes, absolutely. It's still pretty much that way. Especially if you want to make changes to a running installation, add nodes etc.
Celery is great and awful at the same time. In particular, because it is many Python folks' first introduction to distributed task processing and all the things that can go wrong with it. Not to mention, debugging can be a nightmare. Some examples:
- your function arguments aren't serializable - your side effects (e.g. database writes) aren't idempotent - discovering what backpressure is and that you need it - losing queued tasks during deployment / non-compatible code changes
There's also some stuff particular to celery's runtime model that makes it incredibly prone to memory leaks and other fun stuff.
Honestly, it's a great education.
> your side effects (e.g. database writes) aren't idempotent
What does idempotent mean in this context, or did you mean atomic/rollback on error?
I'm confused because how could a database write be idempotent in Django? Maybe if it introduced a version on each entity and used that for crdt on writes? But that'd be a significant performance impact, as it couldn't just be a single write anymore, instead they'd have to do it via multiple round trips
In the context of background jobs idempotent means that if your job gets run for a second time (and it will get run for a second time at some point, they all do at-least-once delivery) there aren't any unfortunate side effects to that. Often that's just a case of checking if the relevant database updates have already been done, maybe not firing a push notification in cases of a repeated job.
If you need idempotent db writes, then use something like Temporal. You can't really blame Celery for not having that because that is not what Celery aims to be.
With Temporal, your activity logic still needs to ensure idempotency e.g. by checking if an event id / idempotency key exists in a table. It's still at-least-once delivery. Temporal does make it easy to mint an idempotency key by concatenating workflow run id and activity id, if you don't have a one provided client-side.
Temporal requires a lot more setup than setting up a Redis instance though. That's the only problem with it. And I find the Python API a bit more difficult to grasp. But otherwise a solid piece of technology.
Here is a nice guide from AWS https://docs.aws.amazon.com/wellarchitected/latest/framework...
In my experience async job idempotency is implemented as upserts. Insert all job outputs on the first run. Do (mostly) nothing on subsequent runs. Maybe increment a counter or timestamp.
From your experience, what is a better alternative guys?
Not the comment that you replied to but I use my own Urd. It's a fancier Cron that you can stop fast. Which is imo what you normally want.
Task queues are like email. It's what everyone is used to so people ask for more of it, but it's not actually good/the right tool.
There’s no alternative (while prototyping), and anything else is better (when you properly defined your case).
DjangoQ2 is a fine alternative during early development
I’m currently stuck with the tech debt of Celery myself. I understand that! Does Django Tasks support async functions?
Computer, load up Celery Man please.
I'm of the opinion that django task apps should only support a single backend. For example, django-rq for redis only. There's too many differences in backends to make a good app that can handle multiple. That said, I've only used celery in production before, and I'm willing to change my mind.
With that logic, the Django orm should only support one database.
Why have a backend then?
I've been using Django on and off at work for the past few years. I really like it. That being said, I still find its ORM difficult. I understand it now that since it's an opinionated framework, I need to follow Django way of thinking. The main issue is that at work, I have multiple databases from different business units. So I constantly have to figure out a way to deal with multiple databases and their idiosyncrasies. I ended up doing a lot of hand holding by turning off managed, inspectdb and then manually delete tables I don't want to show via website or other reasons. For green webapps we have, django is as good as it gets.
Agreed, and their DB migration workflow leaves much to be desired. By not storing a schema/DB state alongside code, Django depends on the current DB state to try and figure it out from scratch every time you run a DB command. Not to mention defining DB state from model code is inherently flawed, since models are abstractions on top of database tables. I much prefer the Rails way of composing migrations as specific DB instructions, and then getting models 'for free' on top of those tables.
Do you use Django's multiple databases support ? (https://docs.djangoproject.com/en/6.0/topics/db/multi-db/)
Yes, we have to in order to use a lot of the features. The core issue for us is really the way Django assumes code represents database state. In normal webapp where the application has full control of the database, that's a good idea. But our databases are overloaded for simple transactions, analytics, users managements, jobs and AI. Business uses the databases in various ways such as Power BI, Aquastudio, etc.. Django app is actually a tiny part of the database. As you can imagine, we duck tape the heck out of the databases, and Django goes bonkers when things aren't matching.
I've used Aldjemy (https://github.com/aldjemy/aldjemy) on a small project and it worked pretty well for allowing me to compose the fairly complex queries needed that the Django ORM couldn't do.
Also don't underestimate setting up e.g. views or materialized views even that you can use through the ORM to query. It helps a lot and allows you to combine fine tuning SQL with ease of use through Django, and get a lot of performance out of it. Just remember to create them in the migration scripts.
Any docs? Django migration is a HUUGE pain point for us.
An extremely common thing to do. Also great with materialized views. I bet it's documented somewhere in Django's docs.
I've been using Django for the last 10+ years, its ORM is good-ish. At some point there was a trend to use sqlalchemy instead but it was not worth the effort. The Manager interface is also quite confusing at first. What I find really great is the migration tool.
Since Django has gained mature native migrations there is a lot less point to using SQLAlchemy in a Django project, though SQLAlchemy is undeniably the superior and far more capable ORM. That should be unsurprising though - sqlalchemy is more code than the entire django package, and sqlalchemy + alembic is roughly five times as many LOC as django.db, and both are similar "density" code.
Makes sense as sqlalchemy’s docs are also 5x as confusing.
Maybe this shows my data analyst tendencies, but why not use SQL?
That’s what we do now. But it gets repetitive and not leveraging Django core features.
Template partials look good, which is one of the key reasons frameworks like React are as good and popular as they are, because you can reuse small segments of code.
Key benefit for reusability and composability in React is IMHO that they don't use templates at all, but everything is a function.
Exactly. There are a few libraries to achieve a similar thing in Python:
* https://htpy.dev/
* https://pypi.org/project/fast_html/
* https://fastht.ml/ (different to above, I think)
* https://github.com/volfpeter/fasthx
Probably others. I strongly prefer this to templating, but I find it makes dyed in the wool Django people squirm.
I like this approach. I am especially drawn to the idea of making custom components this way but every time I have experimented with this I get burned by the context which has to be passed down through all functions.
A jinja/django template has an implicit context but for nested functions you really have to pass that context down through every function call.
It inevitably ends up just a big dict blob.
You get some typing support in an IDE but nothing really for function parameters.
Maybe I am doing wrong?
htpy supports passing data between multiple levels components with its context (very similar to React):
https://htpy.dev/usage/#passing-data-with-context
iommi is wroth mentioning here. It is different from an HTML generator, but one of the things it does is greatly reduce the amount of HTML you need write.
There are a lot of cool things about these, one that they are less typo prone and also they are often much faster.
The downside is I find them hard to read.
I think the template approach isn't quite right and yet neither is the functional approach.
At the end of the day these are a type of tree structure; I think we could conjure a new mechanism that gets the best of most/both worlds.
Yeah, I agree, I find them hard to read. JSX is the best thing I've used. Elsewhere in the thread someone mentioned Cotton which seems to strike a different balance.
To be honest my main problem with templates is they have to be one per file. In principle there's no difference between naming a new file and naming a function, but in practice it just sucks. It's a higher barrier so people are less likely to write smaller components, and refactoring support completely sucks. Even renaming a template is a massive pain whereas renaming a function with decent LSP support is easy.
JSX hits that perfect balance between readability while still being regular functions. Maybe something is possible with the new 3.13 template strings?
As an FYI, I'm with the template strings folks (new feature in 3.14) and we're building an HTML system around the idea of JSX-style components as callables, but with actual strings of HTML. Check our <a href="https://t-strings.github.io/tdom/usage/components.html">components page</a> for more detail.
With template strings, the symbols in the HTML string a just like Python code. Static analysis tooling can step in and do things.
We have ambitions to start an interoperability movement in Python HTML, so htpy and tdom could agree on a common Node structure.
The most obvious value here is for HTMX, which requires a lot of partial templates.
React allows for encapsulation of state in a reusable component, its more than just templating.
React also requires you to know the long list of do's and dont's and is littered with minefields that most average developers are not even aware of.
Everyone just busts out "React" for every small thing, but few commit to actually learning this pretty complicated technology.
The last two recent Cloudflare outages were because of React.
Really?
They're a neat design. I started using them on my blog the other day as part of trying out Django 6: https://github.com/simonw/simonwillisonblog/blob/faec3532183...
There've been a variety of open source attempts at this idea. Is this official one now the best to use, or are the others still compelling?
https://django-cotton.com/ is component-based. I used it a bit, it's nice if you're used to the ways of front-end frameworks, I guess.
https://github.com/django-components/django-components also looked interesting
While using Cotton my thoughts were "ok, it's kinda cool... but do I really need it ? No. Is it worth the extra dependency ? No."
There is something very appeasing in just pulling Django and have all the basics covered. It's nice to have options when needed though.
Amazing that Django didn't have this until 2025
It's had includes and custom template tags for over a decade. Partials are a slightly nicer design for a subset of that pattern.
Wouldn’t Jinja2 macros count?
I stayed away from Jinja2 ... was under the impression it has lower performance. But I could have been wrong all these years.
>For nearly all cases, Django’s built-in template language is perfectly adequate. However, if the bottlenecks in your Django project seem to lie in the template system and you have exhausted other opportunities to remedy this, a third-party alternative may be the answer.
>Jinja2 can offer performance improvements, particularly when it comes to speed.
https://docs.djangoproject.com/en/6.0/topics/performance/#al...
But you could already reuse templates in Django by including them. What am I missing?
Check out the HTMX example in the blog, this helped me better understand how it could be used
https://adamj.eu/tech/2025/12/03/django-whats-new-6.0/#rende...
I'm an avid HTMX user but never did I ever think "I'm using so many includes, I wish I didn't have to use include so much."
What I would like is a way to cut down the sprawl of urls and views.
I do a check for `request.htmx` in my views and conditionally return a template partial as needed. This reduced my need for one-off view functions that were only returning partials for htmx. Works pretty well from my experience.
Partialdef inline is the real win. Lets you define parts of a page without needing to place them in another file. Reduces the mental overhead of imagining how the inclusion will look because it’s already there.
The use case is mainly driven by htmx where you will have lots of these partials and the view code renders them as individual responses.
It's just syntactic sugar, making life a bit easier for HTMX users (cf. "htmx was the main motivation for this feature").
I'm using Unpoly and I just render the whole page and let Unpoly swap the content according to the target selectors, so no need for this. Not much difference in perf if you dont generate gigantic pages with heavy header/footer.
indeed the vintage templating was a logical bottleneck
How is it different from include? Just less files from my perspective
The "inline partials" feature is neat, means you can use and define a partial at the same time.
The way you can render just a named partial from both the render() shortcut and the include tag is nice too:
https://docs.djangoproject.com/en/6.0/ref/templates/language...
Yeah, but I was doing the same thing 10 years ago with include mixed with extends and blocks. I can just include a file inside a template or render it directly.
you're kinda right, {% partial ... %} vs {% include ... %} is not a big difference, but my mind was vaguely thinking that "includes" have often been seen as large templates, whereas partial have been after the component era with the idea of making small blocks. (my 2 cents)
I asked the same question
I’m very much an Odoo guy who has dabbled with Django (and notably, celery in the past) but as someone who makes huge use of the Odoo OCA queue module[0], I never understood why Django never made use of the Postgres LISTEN/NOTIFY to offload tasks. Maybe this is my misunderstanding as I’m pretty green with the Django ecosystem as a whole.
[0] https://github.com/oca/queue
Template Partials and HTMX seems like the Django equivalent of View Components and Stimulus for Rails, which is nice.
Also, good to see first class support for Tasks, among a lot of other niceties!
If I understood correctly, to use Tasks in production right now you need to use this as well:
https://github.com/RealOrangeOne/django-tasks
Is that correct?
Correct. Django 6.0 comes with a standardised API, with 2 testing backends (ImmediateBackend and DummyBackend). You need a third-party backend to store and execute tasks.
Good to know. So no need for Django Q2 or Celery anymore either. I guess unless one has a specific reason.
Has there been discussion about adopting/embedding django-tasks into Django 6.x?
django-tasks is the reference implementation for django.tasks, by the same author. It's up for discussion if and when the DatabaseBackend gets merged, and it seems likely to me that a Redis-based backend would be suggested at some point. It takes time and energy though!
Rails renders partials
https://guides.rubyonrails.org/layouts_and_rendering.html#pa...
More discussion: https://news.ycombinator.com/item?id=46153116
I have been using Django on and off for the last 5-6 years, and it does have the advantage of having batteries loaded but it just feel heavy overall
I find templates atrocious to use for component fragments like this, that's why I wrote a Python component library when I started using Django with HTMX. Order of magnitude more pleasant to use, works with _every_ Python web framework not just Django: https://compone.kissgyorgy.me/
Dude, I used Django at 1.x - before they even had an ORM. The fact that it is adding a way to run tasks, almost a quarter of a century later, is wild to me.
I am not roasting it or anything, go Django, but just an observation.
I think it is refreshing. They don't half-ass things into the framework. They take the time to do it right. They let every feature fight for its life, and put their effort into LTS and minimizing number of issues and API changes related to the features they do deliver. As a developer I really appreciate this. I don't have to totally rewrite my entire application every new version because the implementation wasn't properly thought through.
I used to have this opinion about ASP.NET then ASP.NET Core and the great churn happened. It's finally settled down again, but boy the in-between years were chaotic.
Not just the Framework -> Core migration itself, but the power to make breaking changes went to their heads, and they started quickly tearing up everything only to change their minds again, such as a short-lived "project.json" syntax.
Django is exactly the technology I'd pick if I wasn't already super familiar with the .NET stack. It's got the "batteries included" feel without the chaotic confusion of a million ways to do things. It doesn't have the breaking changes churn that happens elsewhere too.
That is indeed - rare, and one of the reasons I am wary of heavy frameworks. You buy into its code AND its legacy.
Django had ORM from the very beginning. I've been using Django since 0.95 at it had ORM even back then. It was primitive but I hadn't to resort to raw SQL until much later.
Correction - it was an "ORM". I remember now, but it was so rudimentary and useless that I never really thought of it that way.
Nothing wrong with that. One had to start somewhere.
Given that Python tends to produce fewer hallucinations when generated by LLMs I wonder if former Django developers using AI tools are secretly having a blast right now.
If Python produces less hallucinations it's not because of the syntax, it's because there's so much training data.
I think another ace up Django's sleeve is that it has had a remarkable stable API for a long time with very few breaking changes, so almost all blogposts about Django that the LLM has gobbled up will still be mostly correct whether they are a year or a decade old.
I get remarkably good and correct LLM output for Django projects compared to what I get in project with more fast moving and frequently API breaking frameworks.
The "one way" / "batteries included" aspect of Django may also make it easier for LLMs
Whenever I saw people complain about LLMs writing code, I never really understood why they were so adamant that it just didn’t work at all for them. The moment I did try to use LLMs outside of Django, it became clear that some frameworks are just much easier to work with LLMs than others. I immediately understood their frustrations.
What a lot of people don’t know is that SWE-bench is over 50% Django code, so all of the top labs hyper optimize to perform well on it.
I know python is more prevalent in SWE-Bench than any other language, but more than 50% django sounds like a big stretch. Citation?
Edit, it's about 37%, and python-only. https://arxiv.org/pdf/2310.06770v3
[flagged]
We don't need to bring this kind of thing up. We're not school children and most of us are technology professionals, so the meaning is clear.
These guidelines are relevant here:
Eschew flamebait. Avoid generic tangents. Omit internet tropes.
Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead.
Please don't complain about tangential annoyances—e.g. ... name collisions ... . They're too common to be interesting.
https://news.ycombinator.com/newsguidelines.html
Well, https://en.wikipedia.org/wiki/Nonce_word
It makes me sad when a secondary meaning, which does not even overcome the main meaning in usage, becomes an obstacle for the normal use of a word. It's like seeing a rainbow as a sexualized symbol not fit for children, because it also happens to be used by LGBTQ+ community. (BTW, since you're a Brit: did people stop using the word "fag" to refer to a cigarette?)
Yes, words have multiple meanings - but only some words are apparently worthy of censorship - which is my point.
> did people stop using the word "fag" to refer to a cigarette?
Yes, seems so. I've not heard that in at least a decade
I mean, it is sad. But unfortunately that is what happened with "master", "slave", "whitelist", and "blacklist". No reasonable person construed these as offensive or having any implications about the wider world. But there are people in our profession who are determined to take offense where none is given, and unfortunately they got their way.
Well, "slave" has a pretty direct main meaning of an oppressed person doing forced labor. The word "master" is much milder in this regard (compare "master's degree" and "slave's degree"). The word "nonce" in normal usage seems even more removed from any pejorative secondary meanings.
More to your point, yes, taking offense can be turned into a weapon: https://nassimtaleb.org/2016/08/intolerant-wins-dictatorship...
Amusing to have my throwaway comment replied to with links to earnest points from prominent essayists. Never change, hacker news!
American hegemony, and all that.
In the US they spell it as nonze.
No we don't.
Pretty positive that was a joke/bait…
It absolutely was a joke
Slightly absurdist non-sensical humour I’ll admit, but none the less, a joke :-)
The best kind :)
https://en.wikipedia.org/wiki/Context
That didn't stop people from throwing a fit over master-slave terminology in software (having nothing to do with slavery), going so far as to rename long-standing development branch names, as well as put significant effort into removing such terms from the code itself and any documentation.