Finding real, useful info is hard
This isn’t about LLM-generated articles or rampant astroturfing. Those definitely contribute to the problem, but even without them, it can be hard to find good info on things. Especially if those things are software development.
TL;DR
Navigating the vast sea of information available to software developers can be daunting, especially when seeking guidance that bridges the gap between beginner tutorials and advanced expertise. While the abundance of resources is undeniable, the challenge lies in discerning the truly valuable insights from the noise. By sharing experiences, engaging with the community, and documenting our learning journeys, we can contribute to a more comprehensive understanding of the tools and frameworks we use. This article is a step in that direction, aiming to spark conversations and encourage others to think aloud, explore publicly, and ultimately, enrich the collective knowledge of the developer community.
Digital Natives
A friend and colleague and I used to give a talk together about how much information is available for people learning computer science and software engineering and yet people still need people to teach, guide and encourage them to become good software engineers and computer scientists. We talked about “digital natives” and how we used to believe that, any day now, a generation of learners would turn up to our seminars with all the basics covered, just from exposure to tech; and, how they never arrived. It was a fun talk that used the Wizard of Oz as a theme, which we stretched to fit in so many ways. Becoming all powerful, like Oz; pulling back the curtain; having what you needed all along; etc.
Abundance of Information
It’s now many years later and I’ve spent much of that time both teaching software development (among other things) and doing commercial development work. In that time, the amount of freely available information for developers has grown, but mostly in one end of the spectrum or the other, but not the middle.
If you’re a beginner or advanced hobbyist, you’re covered. First steps in any language or with whatever library, infrastructure or framework is out there and you can follow any number of tutorials.
If you’re a developer working with a particular stack and you need API docs, blogs on trends or expert opinion on current architectures, you will find plenty.
The Missing Middle Ground
But what if you sit in the middle? For example, you’re a competent programmer but need to now build a web application. Or you’re a web developer that wants to switch to a new stack. The writing on how to go from a tutorial or quick test to something that is production-ready, makes use of modern best practices and utilises the best-in-class components is missing.
Worse, you often find writing that is incorrect and usually in a way that’s hard to spot from the wrong side of the learning curve.
An Example: Django’s new Task Framework
As an example, and the instance that triggered this post, try looking for information on Django’s new task framework. Before Django 6.0, if you wanted to have activity that was not synchronous with requests, you had to either build something separate and find a way to make it work with Django (put it in the DB, have the worker use it as a queue, set fields when done, have the web front end check status, etc.) or use something like Celery.
It makes sense for Django to fold some kind of asynchronous task framework into its core, just like it has done with other functionality. I don’t think Celery will stop being important (it’s a fully-fledged system with an amazing array of features) but it is overkill for lots of solutions.
The Django developers have created the framework but so far stayed away from implementing the actual back-end functionality. Some people argue that this is the wrong move, but I think it makes sense. It leaves room for others to create pluggable implementations of the back-end and doesn’t make them wait until there is a back-end ready and that comes with core Django. Plus whatever might fill that gap in core Django would become the de-facto back-end for many projects, even if the fit isn’t right.
Now, try looking for information on the new Django task framework. There’s the Django documentation, which is good and correct but not really a guide to having an asynchronous task system up and running. There are many posts that follow the usual pattern of explaining things by way of a tutorial application that doesn’t explain why things are as they are, how to make it production-ready, etc. And you also find articles talking about how it’s revolutionary. How now you just need core Django to have all your task needs met, ignoring the fact that core Django doesn’t actually have an asynchronous task completion back-end.
There’s also Django Tasks which is an “implementation and backport of background workers and tasks in Django“ but doesn’t come with “production-grade guarantees”. It’s great, and the docs are good if you know what you want.
The Role of Community and Personal Contribution
To be clear, I don’t think the Django developers or the Django Tasks package developers are failing to communicate. I like the idea of separation of concerns. Django tells you about Django and it’s not up to them to say how it fits your needs, same with Django Tasks. The problem is the proliferation of repetettive tutorial and opinion writing on the subject that’s masking information on how to understand the capability of the Django task framework.
And this is where someone should say “then write it”. And I might. But not until I feel confident I can not give people poor information myself. And maybe that’s the problem. By the time I know the task framework inside out, have found out which back-ends are useful, well written and production-ready, probably started writing a custom back end that I won’t ever complete but benefit from the journey of writing, I’ll have lost the perspective I currently have. I’ll only see the correct paths, not the indecision. I’ll instinctively design things to avoid the obvious (from my new perspective) pitfalls without realising it, making me unable to vocalise them for others.
Moving Forward: Thinking and Exploring Publicly
This is where the past few days of thinking, reading and tinkering got me. This post isn’t the answer, but it is my first step toward providing something that is, I hope, helpful to someone, somewhere. I realised I used to find a lot of useful information on Stack Exchange, but that’s no longer the case. I find lots of good stuff on Hacker News and realised people visibly thinking about things was the real bonus. If there was more of that, I might find the record of someone who has already pondered similar problems and moved toward solutions, or solved the problem entirely. But more important than finding the lines of magic code that solves the issue is seeing how people think.
So, I now have a blog. It’s not my first. The others were all abandoned over the past few decades. This one might be abandoned in the future but for now I’m going to try to write down what could be useful for other people, while I’m in the middle of it.
