The Bad Data Problem

History is filled with examples of bad data. Christopher Columbus set sail for Asia but discovered America instead because he was using Roman miles while his geographer Alfraganus was using Arabic miles. As the article “Infographic: The History of Bad Data” explains, “Had Columbus stuck with Alfraganus’ units of measurements and checked with him to make sure they were both correct, the discovery could never have happened.” Another example, described by Kevin Bartley in “8 Jaw-Dropping Ways Bad Data Changed World History,” occurred when the Germans used data from British double agents during World War II. As a result of bad data, countless Londoners’ lives were saved when the Germans overshot their long-range ballistic missile (the V-2) and missed their target. The 2016 US Presidential Election is a more recent example: pollsters and predictors’ bad data led to the faulty prediction that Hillary Clinton would win by a landslide, and may have been the reason why some voters, confident of a Democratic win, stayed home.

Data may be bad for several reasons. In the examples above, the data were bad at the source. Data that are bad at the source will continue to be bad if the root causes go undetected and unfixed. Data also can be bad because they are inherently biased, leading to poor outcomes. This scenario has emerged as a prominent issue in artificial intelligence (AI) and machine learning. Data can also be bad due to manipulation (for example, hacked data) or when it is used for bad intent (for example, to monitor or control).

As the world becomes increasingly digital and data-driven, the problem of bad data will likely grow. “As many firms, including startups, become more data-driven with business models that depend on data (artificial intelligence firms, for example), poor quality data will increasingly become a systemic problem,” writes Mike Davie in “Why bad data could cost entrepreneurs millions.” Startups and other companies that must purchase data in the data economy may be especially vulnerable. According to Davie, “Such high volumes of data are being bought and sold every day, going through multiple levels and exchanging hands so many times that it can become hard to ensure which data is original and which has been tampered with along the way.” The transparency problem makes it more likely that bad data can end up in a company’s decision-making process––and cost them millions.

Companies strategize about how to tap into their data for competitive advantage and new business opportunities. But as Avinash Gowda explains, “The hidden problem many companies face is that while data of good quality is a true business enabler, bad data can set back research, reduce or destroy competitiveness and hinder innovation.” In “Bad Data Can Hurt Your Organisation: Here’s What You Can Do,” Gowda writes, “Practically speaking, poor data wastes sales time, distracts data scientists and consumes IT time spent on syncing systems which cannot communicate to each other.” So how can companies turn bad data into good data? Gowda offers six steps involving data strategy, consolidation, and monitoring that companies can take to clean up their data.

By recognizing that bad data is a business issue and not simply an IT issue, companies can more effectively address their bad data problem. Often there is a gap in data governance, writes Winston Thomas in “Solving the Bad Data Enigma.” Because of stalled demand and increasing competition due to COVID-19, companies are currently taking data governance more seriously. “Bad data can now directly impact business survival,” he writes. In a separate article, “The Looming Data Governance Gap: A CDO Perspective,” Thomas suggests that in order to develop a proper data governance framework, companies need to answer key questions around data definition, access, input, and management. With a proper framework, Thomas writes, companies will “have a single source of truth. It also frees up time for data scientists and analysts to do the work they were employed to do, and not waste time cleaning up bad data or wrestling with erroneous entries.”

Thomas Redman echoes the idea of addressing quality at the source in his article “To Improve Data Quality, Start at the Source.” He uses an example of a health clinic using a flexible and simple process to check its data. The process is straightforward: sort out the data you need, measure the quality of needed data, identify areas where quality could be improved, and identify and eliminate root cause(s). Since the process includes the data customer and the data creator, people need to assume both roles. Redman suggests everyone can “adopt the mindset that ‘data quality means creating data correctly the first time’ within their sphere of influence.” As Redman explains in another article, “Bad Data Costs the U.S. $3 Trillion Per Year,” data users “don’t think to reach out to the data creator, explain their requirements, and help eliminate root causes.” Improving the quality of data, he argues, “enables you to take out costs permanently and to more easily pursue other data strategies.”

In her article, “7 Steps To Ensure and Sustain Data Quality,” Stephanie Shen describes how companies can build a data pipeline they can use to create and sustain quality data from the beginning. She writes that disciplined data governance, rigorous management of incoming data, careful design of data pipelines, and data quality control programs are key for data delivered both externally and internally. Shen highlights five key criteria to measure data quality—accuracy, relevancy, completeness, timeliness, and consistency—and presents seven steps to guarantee it.

Thus far, humans remain indispensable to data creation and data quality. To reduce the likelihood of bad data, companies need effective ways to check for and prevent typos, misspellings, and other data entry errors. They also need to work with their staff to formalize how to collect better data from the outset, as well as how to monitor data collection and quality, and make tweaks where needed. In “Data literacy skills key to cost savings, revenue growth,” Eric Avidon suggests employees need to be educated about data through training programs to give them “the data literacy skills—the ability to evaluate, work with, communicate and apply data—to do their jobs.”

Having good data is ever more important as companies look to harness AI and machine learning capabilities. “The quality demands of machine learning are steep, and bad data can rear its ugly head twice—first in the historical data used to train the predictive model and second in the new data used by that model to make future decisions,” writes Thomas Redman. The data must be right, and you must have the right data, he says in his article “If Your Data Is Bad, Your Machine Learning Tools Are Useless.” As organizations increasingly use machine learning, one predictive model’s outputs will feed into the next predictive model, and so on. One minor error can cause many more. Redman outlines steps companies can take to make data useful for AI.

Another problem with bad data in AI, identified by Katharine Schwab, is that “AI tends to compound the problems of a sexist data set, codifying it into a formula that can appear rational and even just despite its bias. A key solution, then, is better data.” Schwab profiles The Feminist Data Set project in her 2018 article, “This Designer Is Fighting Back Against Bad Data—With Feminism.” Caroline Sinders is an artist, designer, and machine learning researcher behind the Feminist Data Set project, which aspires to ensure collaborative, ethical data collection. Sinders conducts workshops to gather information from the data set’s future users, and participants learn “that data is something that is created by a group of people, and that it reflects the values and positions of those people.” The Feminist Data Set is a multi-year project that analyzes every step of the AI process; it is open source and available for download.

Rather than focusing on critiquing bad data practices, the editors of Good Data aim to provide an optimistic vision of “how digital technologies and data can be used productively and justly to further social, economic, cultural and political goals.” The edited collection includes perspectives from experts from various sectors and disciplines on promoting good and ethical data practices and initiatives. It also provides concrete steps on how to realize good data in practice. “Don’t Let Good Data Go To Waste” offers suggestions to make better use of data. Chris Musser and Scott Dekoster suggest that companies looking to “anchor their strategies in customer insights” need to include those who oversee data collection and analysis in strategic decision-making. Companies need to take customer insights seriously, even though “less than half of business decisions are informed by customer insights” according to a Boston Consulting Group study the authors cite. Leaders must also understand “what data can—and cannot—do” so they can use their expert teams optimally. Companies also need to collect qualitative data; quantitative data, Musser and Dekoster argue, “lack the texture and actionable insights that supporting qualitative data could provide.”

As our world becomes increasingly digital, there will be more and more data for companies to mine and capitalize on. Enhanced computing capabilities will continue to pave the way for increased use of machine learning and AI, which require mounds of data. To benefit from these new opportunities, companies will need to ensure that they consistently have good data. Bad data have costly consequences, which many companies can ill afford.

Review

Leading with Gratitude: Eight Leadership Practices for Extraordinary Business

Adrian Gostick & Chester Elton (New York: HarperCollins, 2019)

How and why does gratitude help individuals and teams achieve their best results? How should leaders communicate gratitude and make it an integral part of their organization’s culture? Adrian Gostick and Chester Elton address these simple yet profound questions in Leading with Gratitude, which is based on a wealth of research and their own experience coaching companies around the world. Most of this 233-page book comprises mini-case studies, brief interviews, and memorable quotes that put concepts into understandable contexts and actionable tactics.

Leading with Gratitude begins with a story from the WD-40 Company during the 2008 global financial crisis. The story introduces readers to the fundamental need for gratitude, its positive power, and the notion of leading with gratitude—that is, “regularly expressing sincere appreciation… to acknowledge and appreciate employees for living the core values.” The authors advise, “while practicing gratitude is easy, it is one of the most misunderstood and misapplied tools of management. That’s a shame, because it is also one of the single-most critical skills for managers to master if they want to enhance their team’s performance and develop their leadership credibility.” Chapter 1 explores this gratitude gap and its impact. Given the convincing evidence the authors provide about gratitude’s benefits, including a 200,000-person study indicating a strong correlation with business metrics for profitability, customer satisfaction, and employee engagement, readers may wonder why a gratitude gap exists.

In Chapters 2–8, the authors attempt to systematically debunk several myths about ingratitude—that is, reasons why leaders don’t show more gratitude. Fear and anxiety can motivate people for short periods of time; however, the authors offer a word of caution by quoting W. Edwards Demming’s observation that “Fear is anathema in a culture of Total Quality Management.” A survey by Glassdoor, which the authors highlight in their book, found that 81 percent of people work harder when their boss shows appreciation, but only 37 percent work harder if they fear losing their jobs. The second ingratitude myth is that some managers feel people want too much praise nowadays; however, the authors note that gratitude not only builds self-esteem but effectively guides an employee’s activities because it “provides clarity about whether the work they are doing is correct, valued by the boss or others, and making a significant contribution to the business.” Gostick and Elton point out that Millennials and Gen Z employees value consistent feedback. They also indicate that a shortage of time is the most frequently given reason for not expressing gratitude. Some managers prioritize building relationships with customers, suppliers, superiors, and other stakeholders before their own team; however, the cost of turnover (especially of high performers) can be significant, the authors suggest.

Gostick and Elton continue with several additional points about ingratitude. Some people say gratitude isn’t their natural style. Neuroscience has indeed found a connection between feelings of gratitude and activity in brain regions for emotional processing and interpersonal bonding, but with practice, the brain can learn new habits of empathy and gratitude. A strategy of praising only top performers is problematic because it neglects most people in an organization whose conscientious work enables the team to deliver on commitments and satisfy customers. Also, a strategy of monetary bonuses as the primary expression of gratitude is suboptimal. The authors highlight studies suggesting that when there’s an equitable compensation structure, knowing your boss appreciates your efforts is a longer-lasting motivator than money alone. Finally, some managers may be reluctant to give praise because it might not be received as authentic.

In Chapters 9–16, the authors present the eight most powerful gratitude practices and include a helpful practice summary for each. The eight gratitude practices are:

• Soliciting and acting on inputs

• Positive intent

• Walk in their shoes

• Small wins

• Give it now, give it often, don’t be afraid

• Tailor to the individual

• Reinforce core values

• Make it peer-to-peer

The first four enable leaders to see opportunities for communicating gratitude. Soliciting and acting on the organization’s inputs is particularly powerful. The authors quote Thoreau: “The greatest compliment anyone gave me was when they asked for my opinion and then attended to my answer.” By observing work being done and speaking with team members, managers can tap into a great source of insights. Assuming positive intent, even when things appear to have gone wrong, will help leaders maintain a sense of gratitude. It is important to speak directly person-to-person, gather the facts, and focus on future improvements (rather than past failures). The authors highlight “walking in their shoes,” which enables leaders to see the challenges with which their people wrestle in their daily work. They share examples of leaders spending full days working together with employees who interact with customers. The authors then suggest that looking for small wins makes it easier to find opportunities for gratitude, explaining that “Every small step in the service of a team’s goals and values is worthy of acknowledgment.”

The second set of four practices guide the expression of gratitude. Gostick and Elston urge leaders not be afraid to give gratitude now, and to give it often. They write, “Like ripe bananas, gratitude does not keep.” Gratitude’s effects are much more powerful when offered soon after the act that merited acknowledgment. The authors assert that expressions of gratitude should be tailored to the individual and the level of business impact. Often employees themselves can help their leaders find the best focus for gratitude and approaches to convey it. When showing gratitude, it should reinforce the core values of the organization. The authors offer the metaphor that, “showing gratitude offers an opportunity to put the flesh of specificity on the bones of core values.” Finally, the authors suggest organizations should foster both manager-employee and peer-to-peer gratitude because they fulfill separate human needs. They highlight studies which indicate that peers have the biggest influence on employee engagement levels. While digital tools provide a platform for acknowledging co-workers’ contributions, social recognition may take many forms—a handwritten note, a round of spontaneous applause, or a traveling trophy.

In the last section, “A Grateful Life,” the authors reflect on how the book’s collection of workplace leadership principles apply to our daily interactions with family and friends. One notable example is Neil Armstrong thanking the people who made his spacesuit, while playing down the importance of his own accomplishments as the first man to walk on the moon. Armstrong’s example is a reminder that true gratitude and humility go hand-in-hand.

The book’s title promises extraordinary business results by focusing on a fundamental element of human relationships—namely, gratitude. Leading with Gratitude delivers insightful and useful principles readers can apply at work and in their daily lives. The book is useful for leaders and team members striving to achieve their organization’s goals, as well as for the broader audience of people seeking better relationships and greater fulfillment in their life. We can be grateful to Gostick and Elton for this compelling, practical guide.

Thomas M. Tirpak, PhD, is a Master Black Belt Quality Coach and member of RTM’s Editorial Board.