The Unintended Consequences of People Analytics

A story about feudalism and horse stirrups seems a strange place to begin an article about people analytics, but bear with me. The metal stirrup was invented in the 5th century to provide lateral support to the rider. It did not take long however for the military potential of the metal stirrup to be recognised. Prior to the stirrup, the foot soldier with his battle-axe served as the mainstay of the kingdom’s military might.  The horse-mounted warrior had little impact on the battlefield as they tended to be lifted from the horse on impact. The metal stirrup was an innovative technology which revolutionised warfare. Now secured to the horse through the stirrup, the rider could deliver more accurate and powerful blows to opposing forces. The foot soldier was no match and the cavalry now reigned supreme.  As argued by Lynn White in ‘Medieval Technology and Social Change’, a new military upper class emerged, one which could afford horses and armour. To keep the heavily armed cavalry troops on side, the monarchy of the day granted plots of land in return for allegiance.  In turn, these knights sublet their land to vassals.  From this arrangement emerged the feudal system and the central reason Europe is divided into nation-states today.   

There is a point to the history lesson. A new technology designed for a specific purpose – a metal horse stirrup to help secure the rider – can have profound unintended consequences – the rise of feudalism. We see a similar pattern today with the introduction of digital technology. For example, email was supposed to make communication more efficient leaving us with an abundance of focus and leisure time. Instead, we are now busier than ever, overloaded in a whirl of incoming and outgoing emails.

This brings us to “people analytics,” a data-driven approach for managing the workforce. Think of the techniques used in the movie Moneyball and apply these to recruiting and developing everyday employees instead of baseball players. Xerox is one company which has embraced people analytics. If you apply for a job in their call centre division, the good news is you won’t have to navigate your way through a stressful interview. Instead, Xerox have built a data model based on an analysis of who their high performing call centre operators already are. As soon as you submit your CV for consideration, it will be instantaneously compared to the data model, and if a reasonable match, you get offered the job. Fully automated, the computer either says yes or no. The advantage of such a data driven approach is the uncovering of patterns which would have traditionally remained invisible. For example, previous call centre experience has no bearing on whether you will be a successful employee in Xerox, but having reliable transportation, moderate participation in social networks, and a creative rather than an inquisitive personality does.  

The use of people analytics is becoming widespread. A 2020 McKinsey survey reports that 70% of executives now see people analytics as a top priority in their organisations. While analytics can be quite effective in some parts of the company, optimising a supply chain for example, quantifying people is far more complex. The intended purpose of people analytics is laudable – using statistical insights from employee data to make talent management decisions – but such quantification approaches may unintentionally alter important attributes such as creativity, behaviour, and accountability. 

Steve Jobs, the legendary Apple co-founder, firmly believed creativity depends on exposure to diverse ideas and people. Jobs saw to it that Apple’s headquarters were designed in a way which encouraged people with diverse backgrounds to mingle together. Dissimilarity forces your mind to make connections it might otherwise miss. While people analytics may provide objectivity, it can potentially stifle creativeness. Take Amazon for instance. A company famed for its success in using algorithms to predict what you want to buy, turned its hand to predicting top talent to recruit. Similar to Xerox, a data model was developed based on existing employee data from the previous 10 years. What the data model gave them was exactly what they already had an abundance of – white males. The analysis of historical employee data to make hiring decisions was not only biased against non-white males, it would have resulted in groupthink and less innovative ideas.  Amazon ultimately abandoned the project.  

Nowhere more than the world of sports has the data and analytics revolution been more heavily worshipped. Heart rate, power output, sleep pattern, field position, acceleration, lateral motion, and speed, to name but a few measures, are all captured and quantified by professionals and amateurs alike desperate to eke out that extra 1% competitive advantage. Such hyper-quantification may well be helping sports teams and athletes improve their performance, but it is also changing behaviours in unintended ways that could transfer to the data driven workplace. In a recent study of professional rugby, players at one particular club revealed how 70 different variables describing a player’s positive and negative actions and physical condition were constantly gathered through wearable devices. As this data was visible to management and all squad members through an app, and also used in contract negotiations, players started to obsess over these metrics and admitted to focusing on improving their own individual stats, even if the resulting behaviour compromised team performance. Similar behaviours have been noted in football and basketball where players opt for the safe pass which improves their tracked pass completion rate but offers no advantage to their team, rather than taking on the risky defence splitting pass which could result in a score. 

Initial indications are that people analytics is starting to lead to similar behaviours in the workplace. One 2020 study found that when a German bank implemented an AI system to assess loan applicant data, loan mangers no longer worked with the customer to enhance their repayment ability. The managers opted for the safe pass by outright rejecting moderately risky loan applications, and the bank’s customer focused image became tarnished as a result.  

In terms of how people analytics might impact accountability, we can draw insights from Stanley Milgram’s controversial experiments at Yale University. Conducted in the 1960s, these experiments fundamentally changed our understanding of accountability and obedience to authority. With his rigged electronic shock machine and confederate in tow, Milgram found that people abandoned their accountability to another human when an authority figure validated their injurious actions. But what if the authority figure is not another human but a complex algorithm? We tend to perceive algorithms as almighty and more accurate than humans. This rose tinted view should be critiqued. Just think back to the debacle with last years leaving cert results when errors in the algorithm led to 14,000 students receiving incorrect grades. The algorithms underpinning people analytics are certainly not flawless either. The promise of people analytics is that employees will benefit from fairer and more transparent decisions. Subservient to the algorithm, pivotal HR decisions such as promotions or staff layoffs, could be made without the responsible manager questioning a flawed recommendation, or even knowing how and why the algorithm came to its decision. When the manager is unaccountable, they cannot explain the rationale behind such life altering decisions. Instead of fairness and transparency, employees could view data driven decisions as opaque, biased, and absurd, which can only lead to frustration and lack of engagement.  

Emerging evidence suggests these unintended repercussions on human accountability are materialising. Numerous Flex delivery drivers in the US have had their contracts terminated after an algorithm decided they were not doing their job properly. Due to factors beyond the driver’s control, such as a locked apartment or snow covered roads, deliveries could not be made on some occasions. A reasonable human manger would be sensitive to such life situations, but the algorithm only saw black or white – a successful or failed delivery. When drivers looked for specific details on how the termination decision was reached, managers outsourced accountability to the algorithm, much the same way as participants in Milgram’s experiments surrendered personal conscience to the lab-coated authority figure. 

The reality is that algorithms can be a blunt instrument when applied to people. NUI Galway has a central role to play in rectifying this situation. As a centre of excellence in data and business analytics, we have committed to realising the potential of emerging technologies. Algorithms tend to be designed by freshly minted technology graduates, like our own, who often have limited experiences of life’s travails, such as the hazards involved in making a delivery in treacherous driving conditions. We need to ensure our students understand that people, and the complexities of the human condition, are central to the design of technological solutions. Moreover, a core theme of our research in this space can be dedicated to identifying the unintended consequences of analytics and emerging technology, and developing solutions when these unexpected outcomes negatively impact society. Realising the potential of new technology requires the ability to recognise bad surprises early enough to do something about them. 

His publications have appeared in Information Systems Journal, Journal of Information Technology, European Journal of Information Systems, MIT Sloan Management Review, and R&D Management. The findings of his research have also been featured in mainstream international outlets such as Forbes, Financial Times, Fortune, Reuters, Irish Independent, and the Irish Times. Eoin serves on the editorial boards of the European Journal of Information Systems and Information & Organization, and has previously led special issues in top basket information systems journals. He also regularly consults corporations on matters relating to his research.