A Little Less AI

Learning How to Learn

I've been immersed in my year of intensive self-study for four months now, and it continues to be invigorating, challenging and fun. As well as the different topics I'm studying, I'm also learning about myself and meta-strategies for how I learn optimally.

For example:

• If I just passively watch tutorial videos, a month later I'll remember no more than perhaps 2% of it. Practical exercises are crucial for learning.
• Breaks are essential, not just for balance, but because that's where the learning is internalised. After a burst of intense learning, I often find I need a ten minute nap or to do something completely different like listening to music. These breaks are an important part of the learning process, I've discovered, and not the same thing as procrastination.
• A well structured course makes all the difference. I've done so many courses now, both 'official' ones on Coursera and Udemy and others on YouTube. Finding one with the right pace can be challenging. Too in depth can be painfully slow if you already know some of the material; too shallow you're not learning it well enough.

An Evolving Direction

I'm also learning about which topics I enjoy and which I don't. My main TL/DR conclusion after studying machine learning and deep learning in depth for all this time is I don't want to do it full-time. What I enjoy much more is broader software development, like I used to do professionally many years ago before I become a photographer.

Here's why I don't want to work in AI:

- Specific AI skills go out of date in a matter of months. AI is evolving at a breakneck pace. Even studying it full-time, it's impossible to keep up with all developments. This means that everything I learn quickly becomes obsolete. Fine-tuning a neural network? That's so last year. RAG? That's so two months ago. Agentic workflows might be all the rage now and I could spend a lot of time learning about them but in a year's time none of it would matter anymore.

- Machine learning engineers just don't have the same degree of control as software developers, and I find that a little bit terrifying. Neural networks are often described as a black box. If it doesn't work, or it isn't as good as it should be, you can tweak various hyperparameters like the learning rate or batch size, and you can change the architecture layers but you can't look inside and step through and debug it. I would find it hard to feel fully in control as a ML engineer, even if I got really good at it.

- I predict that much ML work in the near future will be less about extending the capabilities of Large Language Models, and more about optimisation. LLMs already do amazing things but they need lots of VRAM and compute; a lot of their future value will come from being able to run scaled-down versions on local devices. Your AI-powered phone operating system in the near future will need to work natively, not in constant communication through an OpenAI API. The trouble with optimisation, from a personal point of view, is that it's a lot less fun than working on the latest and greatest new innovation.

- Coming up with new architectural deep learning innovations (and writing papers about them, etc) is beyond the reach of most of us mortals. There are a handful of Yann LeCuns and Andrej Karpathys in the world but the odds of me creating a new vision transformer architecture is like an amateur actor hoping to be a Hollywood A-list celebrity. In fact, there are many more A-list actors than A-list ML engineers, plus you need to know maths at a level far beyond anything I can get my brain around.

Image generation continues to fascinate me, but to work in that field would, I suspect, be no fun at all. Imagine working at Midjourney, or Ideogram, or Leonardo, or any of the other image generation companies. Every few weeks one of them releases a new model that outperforms all the others. They are constantly leap-frogging each other. As a consumer of the technology, that's absolutely wonderful. But to work there must be so stressful, with a boss breathing down your neck to keep finding the next innovation that will keep the company relevant and maintain their competitive edge. The same is true with video generation, music generation and LLMs themselves.

If Not AI Then What?

Of the things I've learned over the past few months, the ones that have resonated with me the most are the general software skills, like learning Python, rather than the AI-specific stuff, interesting though that is. So I've come to the conclusion that what I want to do most is general software development, and in a relatively small company where I can 'wear different hats' and do more than one thing.

My core skill as a developer is C#, and I have 8 years professional experience with that (in the distant past) plus another year of C# in Unity as I wrote a computer game as a personal project last year. I used to work in .NET and ASP.NET, and some C++. The industries I worked in were computer vision, then business intelligence data mining, then digital billboards.

While not evolving at the same pace as AI, the world of software development has of course moved on from where it was when I switched to becoming a photographer. So my goal now and for the rest of my studies is to refresh and expand my software skills.

Most of what I've done in the past is back-end (server-side) work. I'll be thoroughly rejuvenating that, but first I'm taking a deliberate step in the 'full-stack' direction by throwing myself into the world of front-end web development. This is something I really knew nothing about. I'm not doing this because I want to be a web developer, but because I want to have a solid foundation in it, to compliment the back-end skills.

To this end, over the last two weeks I've done short courses in HTML and CSS, and an in-depth one in JavaScript. I'm not posting any projects here because what I've done feels embarrassingly simple after some of the AI stuff, but it's given me a solid foundation to build on with the next course, which will be learning React. Specifically: "Hands-on React: 25+ Projects Featuring Next.js, TypeScript, Prisma, Zod, Shadcn, Axios, Router 6, Query 5, Redux Toolkit."

So that will keep me busy for a while!

It means there won't be many flashy machine learning posts on this blog for a while but I will keep linking projects to AI wherever possible as I make progress.

Thanks for reading!

 

Image created by Midjourney. This is exactly what it looks like when I'm hard at work coding; pink holograms and everything.