Business 2 CommunityDoes Torturing Your Data Into Submission Yield The Right Insights?
It’s not hard to see that data and analytics have immense power over the dynamics of decision making. Every day, executives heavily rely on data-related opportunities to improve revenues, increase productivity, and create entirely new business models. For many, insight is the oil for running a business. However, how do you know if you are digging in the right areas to make sure you are getting the right insights?
When creating a standard for analytics, most people look towards the modern science community credited with innovating and stabilizing how data is collected, analyzed, reported, and applied. So when a respected scientific journal, “Basic and Applied Social Psychology”, bans the use of a common statistical value (p-value) used to determine the significance of research results, practicing and aspiring data scientists take notice.
Why is this news so critical to the data analytics community? It’s questioning how data is analyzed.
When science changes the art of data analytics
For decades, millions of scientific papers have relied on the p-value to support their results. According to the Scientific American article “Scientists Perturbed by Loss of Stat Tools to Sift Research Fudge from Fact,” “significance testing’s ‘scientific spam filter’ does a poor job of helping researchers separate the true and important effects from the lookalike ones. The implication is that scientific journals might be littered with claims and conclusions that are not likely to be true.”
According to Timo Elliott, Global Innovation Evangelist at SAP, statistical significance has been a problem in science for a long time. “If you don’t account for p-values properly, you can even prove things like ESP. Being good at science doesn’t mean that you’re good at statistics. On the other end of the equation, there are strong incentives – such as getting publishing and tenure – for ignoring the ‘right things.’”
As a result, researchers may find themselves going back in the classroom – this time, as students.
Biostatistician Jeff Leek of Johns Hopkins University believes, “p-values are complicated and require training to understand. Science education has yet to fully adapt to a world in which data are both plentiful and unavoidable, without enough statistical consultants to go around.”
Finding the right answer to uncover the right insights
Recently, the Oxford Economics study “Workforce 2020” revealed that the demand for talent skilled in data science and analytics will rise from 22 to 47 percent in the next five years. And this same sentiment is echoed by McKinsey Global Institute, predicting that the U.S. alone will soon face a shortage between 140,000 and 190,000 people with deep analytical skills, as well as 1.5 million managers and analysts who can apply those insights to sound decision making.
However, when a community that is traditionally hailed as the gold standard in analytics questions how data is analyzed, how can businesses ever expect to have the right talent to scrutinize and mine their data? This is where consistent learning is paramount.
To attract and retain the best workers, companies must rethink how they train and develop their workforce – especially data scientists. Learning is an essential tool in the competitive arsenal that needs to be supported in a world where challenging theories and changing approaches is the norm.
Maybe it’s time to stop torturing your data to find the best story. Rather, data scientists need continuous training to better understand how to use data to find the right story – with the best learning opportunities and technology available.
To find out how your business can train your employees more effectively, download the Center for Business Insight inquiry “5 Ways to Make Corporate Training Deliver More ROI.”
This article was syndicated from Business 2 Community: Does Torturing Your Data Into Submission Yield The Right Insights?
More Technology & Innovation articles from Business 2 Community:
