Founder and CEO of Facebook, Mark Zuckerberg, revealed in January this year that he aimed to build a fully-functional, artificial intelligence (AI) system to help run his home and help him with his work, and in just under 12 months, he has achieved his goal. After around 100 hours of work over the course of the year, Zuckerberg has spent this week showing off his AI-powered personal assistant inspired by Jarvis, the virtual assistant designed by Tony Stark in Marvel’s Iron Man. “So far this year, I’ve built a simple AI that I can talk to on my phone and computer, that can control my home, including lights, temperature, appliances, music and security, that learns my tastes and patterns, that can learn new words and concepts, and that can even entertain Max,” he wrote in a blog post. “It uses several artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C.” The first issue he encountered was connecting all of the different systems in his house, which he described as being much more complicated than he expected. As the systems are based on different languages and protocols, Zuckerberg had to first write code that would connect all of the disparate systems together. “We use a Crestron system with our lights, thermostat and doors, a Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max, and of course my work is connected to Facebook’s systems,” he explained. “I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.” “Further, most appliances aren’t even connected to the internet yet. It’s possible to control some of these using internet-connected power switches that let you turn the power on and off remotely. But often that isn’t enough.” He continues: “For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other.” Zuckerberg also explains the subtle nuances involved with natural language requests for actions such as playing music, as well as how he incorporated vision and face recognition so the system can identify which room of the house people are in or recognise who someone is when they knock at the front door.
So, what’s next? “In the longer term, I’d like to explore teaching Jarvis how to learn new skills itself rather than me having to teach it how to perform specific tasks. If I spent another year on this challenge, I’d focus more on learning how learning works,” said Facebook’s fonder. Zuckerberg would also like to find ways to make this sort of system “available to the world,” something he was unable to do with this version as it is “too tightly tied” to his home, appliances and network.
