Alteryx, Inc., revolutionizing business through data science and analytics, today revealed that approximately 54 million data workers around the world face common challenges associated with the complexity, diversity and scale of their organizations' data.
In an increasingly data-driven world, the term "data worker" spans beyond the 54 million identified in this study, but the findings are indicative of the challenges specific to those engaging in significant data work in their day-to-day jobs. The Alteryx-commissioned IDC Infobrief, The State of Data Science and Analytics, uncovered inefficiencies, ineffectiveness and wasted time as many organizations turn to data as the lifeblood of their digital transformation.
Eighty percent of organizations now leverage data across multiple organizational processes, but despite increases in innovation, data workers still waste 44 percent of their time each week because they are unsuccessful in their activities. Data workers spend more than 40 percent of their time searching for and preparing data instead of gleaning insights and, on average, use four to seven different tools to perform data activities, adding to the complexity of the data and analytics process. Among other key findings:
- On average, data workers leverage more than six data sources, 40 million rows of data and seven different outputs along their analytic journey.
- The top frustrations cited by data workers in the survey are indicative of root causes that are responsible for inefficiencies and ineffectiveness. For example, more than 30 percent of data workers say they spend too much time in data preparation, a task that can often be automated.
- Eighty-eight percent of data workers, approximately 47 million people worldwide, use spreadsheets in their data activities. Spreadsheet functions are often used as a proxy for data preparation, analytics and data application development tools but are error-prone and expose the organization to compliance and trust issues.
"Data is at the core of digital transformation, but until organization leaders address these inefficiencies to improve effectiveness, their digital transformation initiatives can only get so far," said Stewart Bond, director of data integration and integrity software research at IDC. "Consolidating platforms and looking for tools that address the needs of any data worker, whether a trained data scientist or an analyst in the line of business, can help reduce the friction that many organizations experience on their path to becoming data-driven."
The survey found that data workers are unsuccessful for a variety of reasons, including lack of collaboration, knowledge gaps and resistance to change. Participants reported the lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the highest-ranked skills gaps responsible for productivity issues, indicative of the pervasive talent gap that exists between data scientists and data workers in the line of business. To overcome these issues and more, many organizations are hiring Chief Data and/or Analytics Officers to streamline analytic processes, build a culture of analytics and encourage data literacy across the enterprise as part of their broader digital transformation strategy.
"Collecting data alone won't digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies. The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyze data effectively to drive real, business-changing results," said Alan Jacobson, chief data and analytics officer (CDAO) of Alteryx. "As the data landscape becomes more complex, this survey exposes the tip of the iceberg when it comes to the sheer volume of workers needing to conduct analysis on a daily basis and the untapped potential for them to drive meaningful business impact."
The State of Data Science and Analytics is based on a comprehensive survey of more than 800 individuals performing data functions across geographies, industries, company sizes and departments. The full IDC Infobrief can be viewed and downloaded here.
