Climate change demands intelligent buildings.
Here's one way to make them smarter
Intelligent vision sensor commercialized by Sony Semiconductor Solutions Corporation ("Sony") is the world's first image sensor with AI processing functionality. The intelligent vision sensors feature high-speed AI processing on the sensor (edge AI processing), and output only metadata (semantic information) and are expected to expand the usage opportunities for smart cameras, which are used for a variety of use cases such as in retail or industrial environments. Furthermore, Sony started collaboration with Microsoft in May 2020 to create solutions that use AI-powered smart cameras with intelligent vision sensors and video analytics. One of these activities is Co-Innovation Labs, which was jointly launched with the aim of supporting the development, prototyping, and testing of solutions for partners and customers. In this edition, we will introduce via interview the case of
Nomad Go, which efficiently developed its own solution in this lab with the support of Sony and Microsoft.
You rise at 7 a.m. Your schedule shows just one appointment for the day, at noon. Immediately, you head out to your car, start the engine and let it run until it's time to go.
Hours later, you depart for that errand and return home. Now it's 1 p.m. but you decide to keep your car humming until dark. After all, you may need to venture out again. And hey, you're only mimicking the carbon-belching behavior of each office tower, shopping center and apartment complex in town.
"That is what every commercial building does today. It's outrageous and incredibly wasteful," says David Greschler, chief executive officer and co-founder of Nomad Go. The Kirkland, Washington company creates computer vision technology to help physical spaces reduce their carbon footprint, become healthier and save energy.
"It turns out the No. 1 emitter of greenhouse gas emissions is not industry. It's not transportation. It's buildings," Greschler adds.
CEO and co-founder of Nomad Go
According to the United Nations Environment Program, buildings account for 39 percent of the planet's carbon-dioxide (CO2) emissions. That heat-trapping gas, which remains in the atmosphere for hundreds of years, is driving climate change and its brutal weather effects, scientists say.
Why are commercial structures such a problem? Their heating, ventilation and air conditioning (or HVAC) systems are typically controlled by pre-set timers that run 12 hours per day (or longer) － whether people are present or not. They often burn energy to make empty rooms, lobbies and common spaces comfortable just for the promise of occupants.
Nomad Go has created a product to curb that waste: computer vision that can detect whether people are present in a commercial space and, if so, how many for how long. Data from this solution is then used to warm, cool and ventilate a room precisely based on occupants instead of set schedules, significantly reducing energy usage and corresponding greenhouse emissions.
Using computer vision and artificial intelligence (AI), the solution quickly learns and react to each room's nuances, like how much daily sunlight it usually receives or how often a cleaning crew arrives at night. These adjustments further optimize energy consumption in the room.
Nomad Go's engineers were invited to use one of four Co-Innovation Labs recently launched by Sony and Microsoft as part of their 2020 partnership to build a solution using the intelligent vision sensor IMX500 commercialized by Sony and Microsoft Azure AI.
The labs provide customers with tools, expertise and other resources to develop smart camera technologies. As innovative solutions are born in the labs, Sony and Microsoft help their partners and customers validate solutions, cut time to market and help overcome technological challenges.
"What's exciting about working with Microsoft and Sony is this ability to take our technology to scale and reduce the barriers to market entry," Greschler says. "For us, it was a no brainer."
"And this summer, with endless heatwaves and severe droughts, it feels like we've reached a tipping point with climate change. The urgency to launch this product is absolutely there," he adds.
Computer vision emerged in the 1960s among academic researchers exploring the relatively new field of AI. The idea was to build technology to simulate human vision by enabling computers to glean insights from digital images and videos.
As that technology was refined in recent years, users began uploading their photo and video data to the cloud for AI analysis.
But there was a catch to that setup: It traditionally required massive amounts of data to be sent to the cloud. That can cost big money and clog a network.
Nomad Go chose a different path: the edge, where computing is done on the device itself, just feet away from the very source of the data. In this case, the devices are smart sensors.
In the Co-Innovation Lab, Nomad Go engineers built their solution using Sony's Intelligent Vision Sensor IMX500, which is the first image sensor equipped with AI-processing functionality.
"Our solution is substantially less expensive, small, inconspicuous and uses lower power," says Mark Hanson, vice president of technology and business innovation at Sony Semiconductor Solutions America.
"Our solution doesn't continuously send video streaming to the cloud for analysis. It does a lot of those activities on the chip. And it doesn't require a lot of network infrastructure or bandwidth to deliver results since it is not sending a stream of images," Hanson adds.
"What's more, the cameras with our chips address privacy concerns because they output metadata － data that essentially describes all the data being amassed, such as the quantity of people but not their identities," Hanson says.
Vice president of technology and business innovation
Sony Semiconductor Solutions America
The IMX500 has the ability not to send images or video itself from the sensor. While this works very well for facilities it's also relevant for other use cases such as retail.
Nomad Go engineers also devoted lab time to equip their product with Azure Custom Vision, an AI service and end-to-end platform that helps customers apply computer vision. Custom Vision, when accompanied by Sony's toolchain drives a more efficient development process for edge-capable AI models, with fully integrated workflows from training datasets to actual device deployment.
"We were able to help Nomad Go's team walk through the initial stages of training, packaging and deploying a Custom Vision model to the device," says Jun Yamasaki, global lead for Microsoft's AI and IoT Labs. "We also guided their team on key architecture decisions, ensuring that their solution's stability and accuracy will scale into production."
Global lead for Microsoft's AI and IoT Labs
Nomad Go has now deployed its solution within the buildings of a range of companies, including the common areas of residential high-rises, conference rooms at office towers and inside several corporate headquarters.
Following those implementations, the businesses reported lower energy costs, decreased greenhouse gas emissions, improved air quality, and reductions in the wear and maintenance of HVAC equipment.
One of those companies, McKinstry, a national construction and energy services firm, said it has saved more than 38 percent on energy usage and the same amount on CO2 emissions within its Seattle headquarters, including in its main conference room. Previously, that conference room relied on an HVAC system that automatically turned on at 6 a.m. and turned off at 5 p.m. each weekday.
How much cleaner is the environment if just one 12-story, 460,000-square-foot building were to deploy the Nomad Go system? That alone would cut CO2 emissions by more than 1,000 tons each year, according to Nomad Go's math, which is based on a carbon footprint calculator offered by the U.S. Environmental Protection Agency. That represents the emissions from over 230 passenger vehicles driven for a year.
In the U.S. alone, there are nearly 8 million buildings, including 5.6 million commercial structures and 1.7 million apartment complexes.
"We've flipped the paradigm in terms of thinking about buildings," Greschler says. "We've gone from being building centric to people centric."
Nomad Go was invited to participate in the Co-Innovation Lab in part because of its technology's earth-friendly benefits, while both Sony and Microsoft pursue contributing to the development of a sustainable society.
There are four Co-Innovation Labs. One is in Tokyo, Japan and housed at Sony's office. Two more labs － in Redmond, Washington and Munich, Germany － are located on Microsoft campuses. The fourth resides on Shanghai's AI Island. Nomad Go used the Redmond lab, albeit remotely due to the COVID-19 pandemic.
Each lab is fully staffed with Microsoft technical engineers. Sony provides its technical resources for the lab engagements.
To date, the Co-Innovation Labs have hosted four partners. Several more are prepared to participate. Interested companies can apply online. "The primary benefits for participants include faster builds, lower development costs and shorter time to market," Yamasaki says.
"These tools, combined with what partners bring to the table, we believe will shape the future of tiny AI use cases on the edge," Yamasaki says. "We are very excited to see what other projects our future lab partners will be bringing."
The need for Nomad Go's solution will only accelerate if global construction forecasts prove accurate.
In his recently published book, "How to Avoid a Climate Disaster," Microsoft co-founder Bill Gates wrote that the planet's rising population will cause a doubling of the world's stock of buildings through 2060 － the equivalent of "building a New York City every month, for 40 years."
But long before that, the eventual return of millions of workers to hybrid workspaces － perhaps as soon as this fall － will provide yet another obstacle for existing HVAC systems to handle, Greschler says.
"The emergence of the hybrid workspace makes (the focus on) occupancy even higher," Greschler says. "We don't know who's going to be in the room. Will anybody in the room or not? Those kinds of spaces are perfect candidates for this.
"Our team has uncovered something that will make a huge difference."
