AI Has Made Model Building Easier. It Hasn't Made Data Science Easier

When I moved into data science fifteen years ago, building a machine learning model was difficult. Today, thanks to generative AI, it’s almost trivial. But, ironically, I think data science itself hasn’t become any easier.

Back then the lure of data science’s capability of being able to predict the future was strong, especially for someone who – up until that point – was primarily working on what historic data meant. It probably also helped that Harvard Business Review had just labelled data scientist “The Sexiest Job of the 21st Century” 😉.

I can’t say the transition from more traditional data roles (database administration and business intelligence) to data science was easy. Instead of having easy-to-use, graphical tools at your disposal that supported you in your day-to-day work, data science required me to go back to the fundamentals: math and statistics.

One massive advantage I did have, however, was knowing how to retrieve, combine and analyse large amounts of data using SQL. Rather than having to learn those skills from scratch, I could build on experience I already had. Looking back, I realise those data skills make up the majority of the work in many data science projects.

When I started public speaking a little bit later, I noticed that I was not the only one in the world of data that ran into the same issues with data science. At that point data science was everywhere, machine learning was rapidly becoming more accessible and many BI professionals either wanted to get more involved or were worried that data science was going to make their roles obsolete if they did not keep up.

So, I thought, “why not help other people facing the same challenges I had when I moved to data science?” and a full-day workshop was born that I was able to present at various data events across Europe.

My message at the time was that you did not need to become a hardcore mathematician or statistician before being able to start exploring, and working in, the field of data science.

You already worked with data on a daily basis and know how to extract, transform and analyse it. And, perhaps most importantly, you probably understood something about the business behind that data. And that is an awesome starting point!

In the last few years, the barrier to entering the field of data science has dropped even further with the availability of generative AI.

Today it is easy and fast to simply ask an LLM to write a Python script to train, test and implement a machine learning model that best fits the results you are after. Simply feed it your data, provide some descriptions and away it goes! As a matter of fact, research on the topic of AI based machine learning agents is booming, ultimately aimed at further automating the role of the data scientist (sounds familiar?).

Tasks that used to require hours of analysing data, debugging Python scripts, measuring results, tuning hyperparameters and, of course, scrolling through Stack Overflow can now sometimes be completed in a matter of minutes while your own attention can be directed on other things.

But there is one major problem: building the machine learning model was never the hardest part of data science.

The 10% problem

One of the slides in my old workshop was always one to disappoint quite a number of attendees:

While the exact percentage of 10% of time is spend on the modelling part of data science might be up for debate, the point still holds. Most of the time spent on data science projects is spent on understanding, retrieving and analysing the data.

However, when people become interested in data science, they are mostly attracted by the algorithm part. That’s where most of the intelligence is happening right?

But as I explained above, a data science project doesn’t start with an algorithm, it starts with a (business) problem. And turning a problem into something you can “solve” using machine learning is far more difficult than you might think.

Imagine your manager coming up to you with the following task:

“We would like to use machine learning AI to predict which customers will leave.”

While that has all the right words (AI, prediction) that would classify it as a typical problem data science can solve, it becomes far more complicated when you start asking some smart questions about the task. Like “what exactly does leave mean in this context” or “what actions is the business taking when the model predicts a customer is leaving”?

We can probably develop a machine learning model that would be very capable at predicting customer churn, but if we don’t spend time on answering the smart questions about the task, the machine learning model will probably provide zero business value.

This problem does not go away when using generative AI. Sure, it can help you fit a machine learning model to the data faster than ever, but it cannot magically decide what problems your organisation should solve.

AI can write the code, but it cannot understand your data for you

People working in BI or with databases have probably experienced this at least a hundred times already. Database A marks inactive customers using a NULL, while database B marks them with a 0. Or even better, the sales department manually updates the value at the end of every quarter, so they don’t mess up the reporting process nobody ever documented.

While you (perhaps unfortunately) know these small details about the data, the AI has no idea about them. Unless, of course, you provide it with the knowledge that Bob from sales manually updates the data at the end of the quarter, and if Bob happens to be sick everything must go on hold until Bob is back.

This is where your experience with the data and the business comes in. You know Bob’s task at the end of each quarter; you know where the data comes from and you know all the little nuances that belong to it. When you feed your data into a generative AI model it has no idea of the story behind that data.

Statistical intuition still matters…a lot

While generative AI is undoubtedly going to play a large role in data science and speed up some parts of the data science process, understanding what choices the generative AI made during model development will remain extremely important. Not only because of the reasons previously mentioned, but also because you need to understand how a model came to a certain prediction. For this at least some statistical intuition will remain very relevant.

Statistical intuition not only helps you understand your data better but also allows you to verify the model development work the generative AI performed. Did it correctly identify the features that have the strongest impact on your predictions? How did it handle missing data? Was it fitted with the correct goal in mind? You should always be able to answer these questions, so your models remain explainable and transparent in how it came to a prediction.

And then there is the problem of bias. Almost every real-world dataset contains some form of bias and it is something you have to take into account. Especially since generative AI models also reflect biases present in both their own training data and the choices made during their development (1).

This is actually good news for BI and data professionals in general

When I started presenting my workshop to help BI and data professionals understand more of the world of data science, I told the audience that their existing skills gave them an advantage. I think that argument has become even stronger than it was all those years ago.

With generative AI making model development easier the coding skills required to build the model will decrease, but at the same time the reliance on your knowledge of the data and the organisation becomes more important. And that is an area you excel in.

Don’t just depend on generative AI to build models

If you are working in BI, or any other data related role, and looking at generative AI to build predictive models that is totally fine. Feel free to experiment and see where it takes you and how it can enhance your existing work. But don’t confuse building a machine learning model with data science.

Remember that 10% problem? While generative AI will make the 10% problem easier, there is still 90% waiting for your human knowledge and skills!

(1) Generative AI models are trained through various feedback cycles where the AI model’s developer will decide what the “wanted” outcome should be.