This year’s judges for ENR’s photo contest were shadowed by a machine-learning, artificial-intelligence system being developed to flag safety hazards in construction photos and video.
A year ago, Smartvid.io Inc., the developer, began marketing a product designed to tag and index frames in videos uploaded to its system. The tags are based on image content and prompted by cues from audio narration during the shooting.
例如，检查员在拍摄视频时描述了位置和功能，而机器学习软件将关联变成适用于类似功能的标签。可以迅速搜索数小时的视频，以显示剪辑具有这些功能。
But the deeper ambition of Josh Kanner, the company’s founder and CEO, is to harness machine learning to create a system that can be taught to find patterns in the pixels of images, matching them to a library of objects and automatically tagging specific features without prompts.
Kanner的团队一直在开发软件it calls VINNIE (“Very Intelligent Neural Network for Insight and Evaluation”), by first “teaching” it to recognize whenever people are present in jobsite images, then, for a first application, to determine whether they are wearing safety colors and hardhats.
坎纳说：“要说'缺少hardhat'，首先，您必须确定'如果人'，然后做一个更难的逻辑问题，以搜索'向我展示所有您看到一个缺少硬汉的人的地方。”
“VINNIE is a long way away from being a safety expert, but what we want it to do is flag an image that has potential risk and raise it to our attention, so humans can do a better job and get broader coverage. It’s the second pair of eyes that never sleeps and always looks to call things to your attention,” Kanner says.
Training and testing VINNIE requires exposing it to thousands of construction images. A number of industry firms, as well as ENR, are contributing construction images for training VINNIE, with the goal of helping to develop a potentially valuable new tool, but also with the understanding that a basic level of VINNIE’s hazard-spotting intelligence will become a free, public utility.
Confirms Kanner, “Indeed, sometime in 2017, we will have a ‘freemium’ version that will offer unlimited-duration, limited-features of the product. In the limited-features bucket can be safety classifiers, so that the functionality is available more broadly. It would, in essence, be like a public utility.”
For the training, ENR contributed thousands of unidentified images from its photo-contest database. Image collections also are being shared by Suffolk Construction, Mortenson Construction, Skanska USA Building, Rogers-O’Brien Construction and other firms that decline to be named.
“We are extremely excited about what Josh is working on,” says Todd Wynne, construction technology manager at Rogers-O’Brien. “It’s like having a smart assistant—that intern that never sleeps and can continually index all the images for you so that, when you come into the office in the morning, you know of specific, actionable items that need to be addressed.”
As Wynne points out, safety managers can’t be everywhere on a project, but with people taking more jobsite images and many more being captured by drones and jobcams, the time is right to leverage the assets to find issues and insights. “What we want to get to is to turn images into not just actionable tasks but automated actions. So, if a drone sees two people aren’t wearing hardhats, it can auto-create a task or issue a report and send it to the safety director,” says Wynne.
VINNIE’s technology works on the same principles that enable autonomous vehicles to detect people and vehicles in real time, allowing the vehicle to adjust its driving. Kanner says the software first “boxes” a space that appears to contain a defined object type, such as a human, and then runs an analysis on that area to refine the object and test for other qualities, such as the presence of hardhats or safety colors.
Those are only the first safety- and hazard-related qualities Smartvid.io is training VINNIE to recognize, but the potential conditions it could search for are virtually unlimited.
“I would love for it to tell me when it sees a ladder,” says Wynne, explaining that some sites only allow lifts. “I would like to get it to spot a ladder so I could get [the ladder] off the site. I would like it to tell me if people aren’t wearing personal protective devices or if they are not tied off.”
Wynne说违规经常发生，但只有在看到时才能解决。他说：“如果仅通过查看我们的图像并提醒我们就可以看到它，我们会喜欢的。”“这是这些技术之一，我什至无法想象我们可以使用它的所有方式，直到我们拥有它。我想要它，我昨天想要它！”
Taylor Cupp, a project solutions technologist at Mortenson, says, “What’s really interesting to us is the automation and analysis of massive amounts of images and data—something we are already capturing—for the sake of safety. I really like that this first big test was around safety, and I like that they are getting all these companies to contribute images for the training. It’s an interesting approach.”
但是，在Mortenson测试了SmartVid.io图像索引服务之后，它告诉供应商：“这很棒，但是我们不能向我们的项目团队扔更多工具，” Cupp补充说。该功能需要集成为使用系统并已经收集图像的系统的增强。他说：“我们告诉他们，他们需要与[Autodesk] Bim 360 Field，Procore和Kahua集成，其中大约有十几个项目管理系统。”
Integration is happening, Kanner says. Images loaded into Procore now have the image-indexing tool, which includes some safety tags. More safety capabilities will be enabled in early 2017, he adds. Integration with Autodesk’s BIM 360 Field is expected by the end of Q2, as well.
“A couple of years ago, you would have said, ‘That’s way out there.’ But we’ve come a long way,” says Cupp. “We’re close.”
Using ENR’s 2016 Year in Construction Photo Contest submissions,VINNIE processed 1,080 imagesin less than 10 minutes, compared to a Smartvid.io human team tasked with finding people and risks that took 4.5 hours. VINNIE identified 446 images with people—32 images of which were missed by the humans. It flagged 32 images with people missing hardhats and 106 with people not wearing safety colors. But VINNIE’s prediction of 30 images as likely contest winners, based on its comparison with winners of years past, however, needs improvement. Only three were correct.
|
Click Here to View ENR's "2016 Year in Construction" Photo Contest Winners Construction Safety Ranks High in Selecting ENR Photo Contest Winners |
