The Data Solutions Maturity Model
The Data Solutions maturity model
Often a company realizes that they should do ‘something’ with their big data. But where do you start, and what are the next steps?
You can classify the Data Solutions maturity of a company in five different stages. As an example, let’s use the company WatchTime. WatchTime produces and sells clocks and watches. They have heard that big data is something you can use to increase profits, and they want to know how to go about this. Let’s follow their journey.
1. Foundation
At the first stage, you lay the foundation for using big data by collecting the data itself. You might start by collecting data by hand. For example, an administrator at WatchTime writes down in an Excel spreadsheet how many of which clocks they have sold each day. Something to pay attention to here is to collect the data in precise and consistent ways. This is important so that a computer can understand your data as well.
Unfortunately, manual data collection is intensive and inefficient in the long run. It is therefore important to collect data automatically. This practice should be maintained by a data engineer. Thus, a data engineer is a critical first member of any big data team.
WatchTime hires a data engineer to automate data collection. Now, the order details are saved automatically every time they sell a clock. Using this process, they can save a lot more order details than before. For example, they can now keep track of which products they sell at which exact time and date. And of course, now that they have all this data, they can utilize it to their advantage in the next stages!
2. Novice
At the second stage, your data needs to be stored in structured databases. Without this, data cannot be analyzed properly. For example, the data engineer at WatchTime will create databases and transform and structure the data in an optimal way. Additionally, they will actively maintain and improve the databases throughout the next stages.
To automate recurring actions, a data engineer builds pipelines. Pipelines are scripts that can automate things such as data extraction and data transformation. Thus, WatchTime makes sure to hire another data engineer to help with this process.
3. Intermediate
At the third stage, your data is structured and ready to be analyzed. A data analyst is a necessary second addition to a big data team to describe and help a company understand what is going on in the data.
Descriptive analytics can be used to gain insight into how the company or the product is performing, and what could be improved. Additionally, they can use A/B testing to see which version of the product performs better.
For example, WatchTime hires a data analyst to create dashboards to summarize and visualize the performance of each type of clock. By doing this, the data analyst notices that the expensive and traditional style clocks are not selling much but cost a lot to keep in the product range. WatchTime decides to discontinue these products and focus more on their bestselling modern wristwatches. They see an increase in profits the next month.
4. Advanced
At the fourth stage, you are ready for more advanced analytics. By now, you know exactly what happened in the past. But what about predicting what likely will happen in the future? This is where a data scientist comes in.
Data scientists use machine learning for predictive analytics, which can learn from past data to predict future data. This helps companies to manage their products proactively, instead of only being reactive to market trends.
For example, WatchTime hires a data scientist to predict when and how much the sales will increase. Besides the expected large increase in all types of products in December, the data scientist notices all sorts of recurring fluctuations in sales throughout the years. Combining these new expectations with the knowledge of how long it takes to produce each type of product, WatchTime knows exactly when and how much to increase or decrease production. This way, WatchTime stays ahead of the game.
5. Expert
At the fifth stage, you are a big data expert. Now, you can tackle the holy grail of data science: artificial intelligence (AI). This means that you are in the realm of automating traditionally ‘human’ tasks. AI can be used for things such as: processing language, perception, moving and manipulating objects, reasoning, knowledge representation, planning, and learning. The potential for AI is endless (with enough investment).
For example, WatchTime wants to produce smartwatches, so they hire two data scientists to design smart features. The data scientists build an AI algorithm that can analyze wrist movement to detect the sleep quality of the user. This is a greatly desired smart feature that increases sales and helps lift WatchTime to the next level.
So those are the five stages towards big data maturity. After a data engineer helps you to collect and transform your data in stages one and two, a data analyst helps you to understand your data in stage three. Finally, a data scientist will predict future data in stage four, and automate intelligent tasks in stage five.
However, these stages are simply a guideline and not every company goes through a strictly linear process. It is also not always necessary to progress all the way to stage five; this depends on a company’s needs. Let us know what your needs are and we can help you decide upon the next steps.
For more types of maturity models, check out our Continuous Delivery 3.0 Maturity Model.
Eefje Poppelaars started working as a data scientist at NISI in February 2020. She has a background in scientific research, completing a PhD in neuroscience and psychology at Salzburg University (Austria), as well as a Bachelor and Research Master with honours at Leiden University.
As a curious and investigative Data Scientist, she is passionate about finding out what is going on in the data and thinking of creative ways to tackle problems. She is also quick to learn new skills and is continuously developing herself. Eefje is driven to develop data science solutions to help companies get the most out of their data.
A week before the Corona virus situation urged everyone to work from home, she started working for PostNL at the Data Solutions department. Here, she helped develop their 'Adres in Beeld' API, which helps webshops gain insights into addresses in order to optimize their marketing and sales. This challenging project involved aspects of data engineering, data analytics, and data science, and enabled her to use a diverse skillset.
Get In Touch