Computer Vision is upping the ante by putting sites under 24/7 safety surveillance

CV spots personnel who are not sufficiently socially distanced

A source of perennial concern for most EHS heads and personnel are Unsafe Acts (USAs) and Unsafe Conditions (USCs) that carry the threat of becoming safety incidents. Many of these remain undetected by the human eye no matter how diligent and thorough the inspection is and when dealing with project sites that either stretch several hundreds of kilometres or are spread over many acres, that involve working at heights or in a marine or similar challenging environment, better safety vigilance is an imperative. “I was seeking to challenge and improve the existing safety requirements,” remarks Michael William Sanderson, Head – EHS, Heavy Civil IC, wrestling with this concern. “Safety and Environment are based on individuals’ legal requirements that are fundamentally aligned to historical common law development. However, predominantly, compliance with any safety or environmental requirements require human interaction and justification and like most things human, they are prone to errors, intentional or otherwise.”

Reducing dependence on the human factor

Just as in the game of cricket the introduction of technologies like the DRS, Ultra Edge and ball tracking are reducing on field human errors, it was felt that the answer to the concern of EHS managers to early detect USAs and USCs lay in a technological intervention. The problem was presented to the Digital team led by S Anantha Sayana, Chief Digital Officer, L&T to study and find a suitable digital solution.

“We already have cameras installed at our project sites that are ‘seeing’ the site 24/7,” points out Anantha. “The question was how to put intelligence into the system to detect unsafe acts and unsafe conditions that the camera could ‘see’. Computer Vision (CV) is an aspect of AI that gives the system cognitive capabilities. By feeding hundreds of different images into the system, it gets trained to recognize and classify images that have elements of unsafe acts or unsafe conditions and there are a variety of Machine Learning techniques and algorithms used to achieve this.” Once the system is trained with the first set of images, it is configured to recognize and alert the appropriate staff about them and it keeps learning from every new image that it processes. “We thus have a system that monitors 24/7 without any human interaction using AI, ML and Computer Vision,” explains Anantha. “We have currently trained the systems about USAs and USCs about whether a workman is wearing his proper PPE like helmet, jacket and shoes or not. In fact, in the present post pandemic phase, it even detects if a person is not wearing a mask or is not sufficiently socially distanced. We are working with the Safety teams to train the system on more conditions.”

Select project sites are now under the constant eye of CV to detect safety anomalies, alert the EHS personnel concerned to act and rectify. “It empowers them (EHS personnel) to conduct root cause analyses to prevent USAs and USCs from reoccurring,” chips in Krithika Venkatesh, Senior Manager – Digital, “and even helps in behaviour-based training.”

AI detects faster, more accurately and ‘sees’ much more

AI picks up what humans do not or cannot. It is not distracted. It does not sleep, rest, or take time off. It does not suffer from the lack of risk knowledge. For a machine, it is always black or white, right or wrong. For Michael and his ilk, it is a 24/7 ‘third eye’ that is constantly working for them. “We have developed models that can recognize if an excavated pit is barricaded properly or not,” explains Krithika. “CV can detect oil spillages and the platform triggers alerts for immediate action. In a controlled environment, we have used CV to even identify trespassers on designated walkways.” A recurrence or similar mistakes could reveal a serious lacuna, nudging EHS personnel to relook and realign their systems and processes. 

Early detection is heaven sent for EHS Heads like Michael for it can significantly reduce incidents. “If a notification is not closed, AI will keep notifying, until corrected. Unlike human beings, it does not give up!” remarks Krithika, with a smile.

The proof of the pudding is at the sites

With Michael’s urgings and the Digital team’s efforts, CV was ready for implementation and rolled out at select Heavy Civil and B&F sites starting with a pilot at the Cochin Dry Dock project site. The results have been more than noteworthy. For EHS In-charge, Deiveegam Murugesan, at the Cochin Dry Dock site, the revelation has been both exciting as well as disturbing. “For the month of December 2020, Computer Vision made 1,642 Safety Observations pertaining to PPE – helmets, jackets, shoes and even masks – as against a mere 221 human observations,” he shares. “While it has certainly made our lives easier as Safety Managers, it is also putting a far bigger responsibility on us to act quicker and more decisively once we have been alerted. This is sure to vastly improve our EHS performance,” he affirms. CV helps to significantly reduce the gap between detection to resolution that can enormously improve safety performance.

“CV opens up a whole world of predictive analysis too,” sums up Anantha Sayana. “EHS managers can now detect trends, identify those safety lapses that occur more frequently than others and take pre-emptive measures because in the final analysis when it comes to safety, prevention is always and infinitely better than cure!”