Toyota Research Institute Showcases Latest Robotics Research Aimed at Amplifying Human Ability in the Home

 

• Capability Development to Dramatically Improve Quality of Life
• Robot Fleet Learning Is Enabled by Learning from Humans and Simulation
• Gantry Robot and Soft Bubble Gripper Among Latest Research Prototypes
• New “Mock Home” Research Lab Allows for Robot Development And Testing
• Real World Impact is Achieved Through Understanding User Needs and Collaborating with Others in the Toyota Group

With an eye toward putting their research muscle behind solving broad societal issues, Toyota Research Institute (TRI) opened its laboratory doors to journalists to showcase their latest advances in robotics aimed at assisting humans in their homes (360-degree presentation video here). TRI’s focus is on home-based assistance to address the vexing issue of our aging population. According to the United Nations, over the next three decades the global population over the age of 65 is projected to more than double. That means over 1.5 billion people around the world will be 65 or older by the year 2050. This graying of the population will have profound effects on society, the workforce and the economy. TRI believes their research can provide options to address these challenges with advanced human-assist robots to help people age in place with dignity.

TRI’s robotics research is guided by the Japanese notion of Ikigai, a powerful idea that every person’s life should have meaning and purpose. TRI’s human centered approach turns typical AI philosophy on its head: instead of replacing human beings, TRI’s research uses AI to amplify human ability.  This approach is known as Intelligence Amplification (IA), where machines and humans work in synergy to do something better than neither could do alone. “Studies of Ikigai teach us that we feel most fulfilled when our lives incorporate work that we love and that helps society,” said Gill Pratt, CEO of TRI and Chief Scientist for Toyota Motor Corporation (TMC). “To enable more people to achieve their Ikigai, TRI is pursuing new forms of ‘automation with a human touch’ (known as “Jidoka” in the Toyota Production System) to develop capabilities that amplify, rather than replace, human ability with the goal of bringing deep happiness and fulfilment to all people.”

THE POTENTIALS AND CHALLENGES OF HUMAN AMPLIFICATION ROBOTS

As societies age, there will be huge demand for increased caregiving, systems that enable us to live independently longer, and assistance for an increasingly aging workforce. Robots and automation can play a key role in freeing up people to spend more time with family, assisting people with tasks they enjoy, or helping them perform work for their jobs.

TRI believes that robots are not seen in these roles today because roboticists haven’t yet figured out how to make robots reliably operate in the complex, unstructured environments that people function in every day. Unlike factories, where the environment is structured and programmable, natural human environments, like someone’s home, are unstructured and diverse. For example, every home is unique, with a different combination of objects in distinct configurations that are constantly changing.

“TRI robotics research is focused on the home because it is in that environment that robots can provide the greatest assistance in achieving human fulfillment,” said Max Bajracharya, VP of Robotics at TRI. “It is also one of the most complex environments for robots to master. Our work is focused on two key challenges: teaching robots from human behavior and using simulation to both train and validate robot behaviors. Collectively, we think of this idea as fleet learning, where when one machine learns something, they all learn something. We believe this is going to be the key to making robots in human environments practical.”

TEACHING ROBOTS

To address the diversity a robot faces in a home environment, TRI teaches the robot to perform arbitrary tasks with a variety of objects, rather than program the robot to perform specific predefined tasks with specific objects. In this way, the robot learns to link what it sees with the actions it is taught. When the robot sees a specific object or scenario again, even if the scene has changed slightly, it knows what actions it can take with respect to what it sees.

Leveraging humans as teachers is a key way TRI is enabling robots to learn and acquire real-world skills. Using virtual reality, a human trainer sees what the robot sees in real time then commands the robot to perform a variety of different actions.  The robotics team at TRI is working on generalizing this type of learning, so if one robot learns a wiping task in the mock home kitchen, it could do the same task in any kitchen. TRI’s teaching process and approach to fleet learning is explained in more detail in this story.

USE OF SIMULATION FOR MANIPULATION RESEARCH

TRI has made a significant investment in simulation for both engineering and validating robot behaviors. The mechanics of the way a robot hand interacts with objects is very complicated to simulate, so simulation has traditionally not been used for robotic manipulation research.

In order to teach a robot a new behavior, or refine one that it already knows, TRI’s simulation software provides a way to understand the robot’s performance without having to physically perform all the tasks every time a change is made. The results in simulation are tested on real kitchen sets in the TRI Cambridge lab. TRI researchers’ use of simulation tools can also advance robotic development even during a period of limited access to hardware and testing facilities.

“We’ve used our dish loading robot and clutter-clearing experiments to automatically improve our behaviors in simulation and have that result in improved performance on the real robots,” said Russ Tedrake, TRI Vice President of Robotics Research.

The TRI robotic manipulation team based in Cambridge, Massachusetts, is working on furthering prior research into those challenges. Their progress on taking on hard problems in manipulation was previously shared in this feature.

NEW ROBOTIC HARDWARE AND SOFTWARE

In considering robotic solutions in the home, the TRI team is also looking at some more radical ideas.

One innovative concept is a “gantry robot” that would descend from an overhead framework to perform tasks such as loading the dishwasher, wiping surfaces, and clearing clutter. By traveling on the ceiling, the robot avoids the problems of navigating household floor clutter and navigating cramped spaces. When not in use, the robot would tuck itself up out of the way. To further investigate this idea, the team has built a laboratory prototype robot that can do all the same tasks as a floor-based mobile robot but with the innovative overhead mobility system.

In another innovation, to address the need for soft contact as robots interact with indoor environments, TRI researchers have developed novel soft grippers that have high-density tactile sensing capability.

“The soft grippers are made of a soft bubble that complies when it is in contact with objects or tools, and we can control that compliance by changing the pressure in the bubble,” Tedrake explained. “Inside is a depth camera that senses the shape of the bubble skin, and tracks motion to estimate the shear forces on the surface.”

More details on the soft bubble gripper can be found here.

“MOCK HOME” SPEEDS UP DESIGN AND ITERATION CAPABILITIES

At the TRI headquarters in Los Altos, California, a research team led by Jeremy Ma and Dan Helmick work in a new “mock home” robotics testing facility. This research environment, in which TRI can rearrange floor layouts and move objects, allows TRI to develop fundamental robot capabilities before testing them in a real home. The mock home features a kitchen, dining area, bathroom, and living space similar to those found in an actual home.

“Even though the mock home is a relatively new addition to our office, it truly gives our team the ability to test concepts quickly and efficiently,” said Jeremy Ma, Senior Manager, TRI Robotics.  “We plan to finish the mock home construction by the end of this calendar year and expect the lab to be a vital building block in establishing the future capabilities of robotics at Toyota.”

MEETING REAL USER NEEDS

For robotics to be successful in the home, TRI believes it is important to discover and account for individual human tastes, needs and means of fulfilment. TRI takes a “fail fast” approach to technology development so that its work can more quickly make a positive impact on actual lives and improve social good. Rather than simply developing capabilities that researchers find interesting or think will push the field forward, TRI has a Robotics User Experience and Industrial Design group to uncover and probe real user needs.

“We rely heavily on observational research techniques such as contextual inquiries,” said Steffi Paepcke, a TRI User Experience Team Lead. “Before COVID-19, we went to Japan to work with our research partners to visit the homes of older adults and observe them going about their daily lives, making note of friction points, challenges, and opportunities. We observed that cooking is a beloved activity for many, though it can get more strenuous over time. Sharing meals and feeding loved ones also can serve as a focal point for social connection… so giving elderly people a fully automated cooking robot or pre-cooked meals might be physically beneficial but emotionally detrimental.”

The challenge is to truly understand how to develop human-centered robotics when each individual user represents a unique case. The goal is not to just give people gadgets, but instead find ways to truly enable people to achieve fulfillment and help society, each in their own unique way.

TOYOTA COLLABORATION

TRI’s robotics charter is to develop new robotics capabilities for Toyota that can contribute to solving real world problems and aiding global societies.

To create real-world impact, TRI is working closely with other groups inside of Toyota.

One such group is Toyota AI Ventures, Toyota’s first corporate venture capital firm that strategically invests in early stage startups.

Another group is TRI-AD, soon to be the Woven Planet Holdings Group, which focuses on taking new capabilities from TRI and other parts of Toyota and, in collaboration with them, developing them into product concepts.

Toyota USA Foundation to Address Digital Divide

 Grants Provide Access to Virtual Learning for more than 350,000 Students

As virtual learning continues, many students across the country are still in need of internet access and computers to complete school work. To help meet this need, the Toyota USA Foundation approved grants totaling $3,357,000 to support students in 13 states where the company has operations.

The grants will fund WiFi access points, mobile WiFi devices, laptop computers and software licenses, helping more than 350,000 students access virtual learning.

“All students deserve equal access to education,” said Mike Goss, president, Toyota USA Foundation. “The foundation typically supports STEM, but the pandemic has exposed deeper issues that are a barrier to good education.”

The grants build upon Toyota’s ongoing COVID-19 relief efforts,  prior foundation grants focused on e-learning programs and hunger, and the recent launch of an education hub that includes virtual field trips of the company’s operations.

Mad Apple Recipe Will Help You Feel Less Mad

This spooky and boozy recipe takes your favorite fall fruit to the next level: Mad Apple recipe!

Mad Apples (Yields 6) Ingredients: 4 Cups Granulated Sugar, ¾ Cups Water, ¾ Cups Brandy (Optional), 1 Cup Light Corn Syrup, 6 Granny Smith Apples (Unwaxed), Black or Red Food Coloring

Directions: Grease a piece of baking paper and place on a tray/baking sheet. Insert skewers in all the apples and set aside.In a medium pot, combine the sugar, water, liquor, corn syrup and food coloring and stir over medium heat until the sugar has dissolved and the mixture feels smooth when you rub it between your fingers.

When the sugar has dissolved, turn the heat up and wash the sides of the pot down with a pastry brush dipped into clean water to prevent crystals from forming.

Allow the candy mixture to boil until it reaches the hard crack stage 310°F. Carefully dip the apples into the hot candy mixture and place on the baking paper to set and cool for approximately an hour before serving.

Ways to Trick or Treat Safely This Halloween

 

People have gotten creative all over the country to try to celebrate holidays, birthdays, graduations, and other events safely during the Covid-19 pandemic. From having friends drive by homes on birthdays with signs and well-wishes to Zoom gender reveal parties, we are all doing our best to celebrate safely.

With Halloween approaching, many parents are asking themselves and their friends about how to make it work. Kids (and parents) definitely deserve some fun and joy this Halloween, so here are some creative ideas to have some spooky fun safely:

• Create a treat slide! Instead of trick or treaters walking up to your door, create a fun slide/chute they can put their buckets under and you can slide candy into it from any distance at the top of your driveway.
• Make Halloween more like Easter. Instead of going out for candy, tell the kids that some goblins came to their house this year and hid candy around the house/yard for them to find. Just like how an Easter Egg hunt is traditionally done
•  Coordinate with neighbors for a reverse trick or treat. Instead of kids walking around door to door, have the adults drive around the neighbor and toss candy to the kids standing in their yards all costumed up! Everyone gets to show off their costumes and candy is distributed from a safe distance.

What are you doing this Halloween to make it fun but safe for your community?

Toyota Financial Services Offers Payment Relief to Customers Affected by Hurricanes Laura and Isaias, Midwest Derecho, and California Wildfires

 

Toyota Financial Services (TFS) announced it is offering payment relief options to its customers affected by Hurricanes Laura and Isaias, as well as those impacted by the wildfires in California and the derecho which struck the Midwest earlier this month. This broad outreach includes any Toyota Financial Services (TFS) or Lexus Financial Services (LFS) customer in the designated disaster areas.

 Toyota Financial Services cares about the safety and well-being of its customers, and wants to help those impacted by these natural disasters.  Impacted lease and finance customers residing in the affected areas may be eligible to take advantage of several payment relief options, some of which include:

• extensions and lease deferred payments;
• redirecting billing statements; and
• arranging phone or online payments.

Customers who would like to discuss their account options are encouraged to contact TFS or LFS:

Toyota Financial Services customers may call 800-874-8822 or contact TFS via email using the Mail Center function after logging into ToyotaFinancial.com.

Lexus Financial Services customers may call 800-874-7050 or contact LFS via email using the Mail Center function after logging into LexusFinancial.com.

We extend our heartfelt thoughts to those affected by these devastating disasters.

Postponement Presents Opportunities to Support the Paralympic Movement

August 2020 was supposed to be filled with images and stories from the Paralympic Games Tokyo 2020, a global celebration of human triumph over mobility limitations, many of which seem impossible to overcome.

Like the Olympics, the Paralympics have been postponed a year, and Paralympic athletes, their sports, and sponsors have been adapting to the new timeline. Instead of seeing this as a setback, Toyota is embracing an opportunity to further elevate awareness of the Paralympic Movement and the inspirational stories of those who are a part of it.

Toyota and the Paralympic Movement

Mobility can mean different things to different people. For some, it’s the freedom to move across town; for others, it’s the ability to move across a room, or through life. As a mobility company that considers Respect for People to be its North Star, Toyota is acutely aware of these challenges, and is working to find ways to address them. The company is a firm believer that when a person is free to move, anything is possible, which is why it proudly supports the Paralympic Movement.

In March of 2015, Toyota signed on to become a Worldwide Partner of the International Olympic Committee and International Paralympic Committee in the newly created mobility category from 2017 through 2024. In addition to its global relationship with the IOC and IPC, in the U.S. Toyota is a proud partner of Team USA, various Paralympic National Governing Bodies (NGBs), and 13 Paralympic athletes. Since 2018, Toyota’s “Start Your Impossible” global corporate initiative and campaign has brought attention to Paralympic sport and its athletes, as has the company’s sponsorship of the world’s first wheel park. Furthermore, Toyota was the presenting sponsor of NBC’s coverage of the Paralympic Winter Games PyeongChang 2018.

“While we were heartbroken at not being in Tokyo this summer, we see the postponement as an opportunity to further the work we were already doing to elevate awareness of the Paralympic Movement,“ said Dedra DeLilli, group manager, Olympic and Paralympic marketing, Toyota Motor North America. “The additional time means more opportunities to work with partners such as the USOPC, NBC, our Paralympic NGBs, and our amazing Paralympic athletes.”

To date, that additional support has been seen in a few different ways, including Toyota’s presenting sponsorship of Olympic & Paralympic Day in the U.S., the first time the International Olympic Day (June 23) was renamed to include Paralympic in the U.S., as well as the presenting sponsorship of the re-airing of the Paralympic Games Rio 2016 on NBCSN, which included a number of integrations and Paralympic-themed advertising within the programming.

The additional time leading to Tokyo means not only finding new ways to support the Paralympic Movement, but also being flexible with plans that were already in the works, such as Toyota’s sponsorship of the first-ever Team USA hospitality house during a Paralympic Games, the evolution of the “Start Your Impossible” campaign, and an on-the-ground ambassador program.

“While there is still a lot to be determined, the extra time allows us to flesh out some additional ideas, and look to bridge the summer and winter Games, which will now be just months apart,” says DeLilli. “We’d love to see summer and winter sports as well as athletes supporting each other, working to really amplify the conversation around the Paralympic Movement.”

TRD Lends its Technical Know-how to Advance Paralympic Technology in the U.S.

Looking towards the Paralympic Winter Games Beijing 2022, the conversation will also include technology advances in Paralympic sports. In addition to numerous mobility products that Toyota will have on the ground in Tokyo, its U.S. racing arm, Toyota Racing Development (TRD USA), is working on a sit-ski project with the USOPC and U.S. Paralympics Alpine Skiing.

“TRD is always looking for ways to utilize its engineering know-how to innovate both on and off the track,” said David Wilson, president, TRD USA. ”The Toyota core pillar of continuous improvement is something we practice every day, and we can think of no better way to support our Para partners and athletes than by finding ways to offer our technical expertise to them.”

While details won’t be revealed until a later date, TRD does offer up some information on the project. Says Wilson: “We are currently working with our partners and four-time Paralympic Winter Games medalist Chris Young to use his vast experience for prototype testing and feedback. We’re trying to understand if there are better methods to get sit-skis customized to each athlete and their unique size and shape. We’re also developing and tuning the sit-ski suspensions that allows the athletes to race down some of the most difficult terrain in the world.”

Bringing People Together in an Uncertain Time

Ultimately, the COVID-19 pandemic will chart the course forward for the Paralympic Games Tokyo 2020 and possibly the Paralympic Winter Games Beijing 2022. Says DeLilli, “There’s a lot of uncertainty, but we look forward to the day when we can finally cheer on our Team Toyota Paralympic athletes as they compete on the world stage, celebrating their amazing accomplishments in mobility.”

To learn more about Toyota’s support of the Olympic and Paralympic Games Tokyo 2020, click here.