How Caterpillar is developing virtual and augmented reality to design and service heavy equipment

Updated Nov 24, 2015
Cat’s Galen Faidley demonstrates the company’s virtual reality setup for designing equipment.Cat’s Galen Faidley demonstrates the company’s virtual reality setup for designing equipment.

We are in the midst of a transition between ages.

Data is everywhere. Computers will soon be in nearly every pocket and screens are increasingly being strapped to more and more wrists and heads. Even our homes, a thing that you wouldn’t necessarily have pegged as a data-intensive environment, have become Smart with a capital S, capable of unlocking your front door, adjust the thermostat and turn on the lights with just the wave of a phone.

And though the impact of these technologies is first being felt by consumers, companies are in no way exempt from their potential disruption.

Which is why most smart businesses, especially legacy companies like Caterpillar, are investing in research and development and spending a lot of time in the lab trying to figure out how best to apply new technology to make their businesses stronger, while making their customers more capable.

Recently, Cat gave Equipment World a peek behind the R&D curtain at the DMDII manufacturing research laboratory inside UI Labs in Chicago. While there, we got an up-close look at how the company is already using virtual reality to design its heavy equipment and how it’s researching augmented reality as a way to train service technicians.

“The secret to future success”

While speaking at an event on disruptive technology during Chicago Ideas Week October 15, the president of the Internet of Things Consortium and a developer of IoT technologies at Google, Mark Spates, said we are transitioning from the Information Age into the Intelligence Age. We know how to generate data by the boatload. The Intelligence Age will be defined by how we use it.

At that same event Caterpillar CEO Doug Oberhelman said that is just what his 90-year-old company is trying to do. It’s the reason the company invested in and partnered with analytics firm Uptake—to monitor data created by Cat machines and learn to take meaningful action on that data in order to prevent downtime.

And beyond gaining a bit more insight into the Uptake partnership, Oberhelman’s comments in Chicago were important because they included a clear admission that the company needed help to pivot into the Intelligence Age. “We decided that maybe we better disrupt ourselves,” he said.

Partner Insights
Information to advance your business from industry suppliers

RELATED >> Culture Dash: Inside Caterpillar and Uptake’s data-driven race to solve heavy equipment’s tech dilemma

And while the Uptake partnership will result in a multitude of customer-facing benefits, Cat VP of product development and global technology Gwenne Henricks said the company is also placing a focus on collaboration with outside companies in order to improve its manufacturing process.

“This is a consortium that is really going to allow us to advance manufacturing technologies in a way that none of us could do by ourselves,” Henricks, who also serves as the company’s Chief Technology Officer, said last Friday during a media tour of UI Labs. The facility celebrated its grand opening on Chicago’s Goose Island in May and is a research collaborative comprised of several universities and companies including Caterpillar, John Deere, Rolls Royce and more.

Inside UI Labs is the Digital Manufacturing Design Innovation Institute, aka the DMDII Lab. The lab is federally-funded and had its first give research projects awarded October 15.

“This is going to be the secret to our future success,” Henricks said of the lab.

Virtual reality saves time in design phase

Part of the tour focused on a 3D virtual reality setup that Cat uses while designing its machines. This VR doesn’t involve completely immersive goggles, but does place designers in front of a giant screen with a pair of 3D glasses that allows the system to track their motion. As you can see in the photo above, the system tracks the user with a camera setup that is a more advanced version of the Kinect, which Microsoft used to bundle with the Xbox game console.

Using a remote similar to the Nintendo Wii remote or a PlayStation Move game controller, designers can navigate a 3D model of the machine they’re developing. The model is rendered to scale and allows designers to virtually walk around, climb atop and duck beneath parts of the machine. It even allows them to slice the machine in half to see its insides.

“This allows us to understand customer interactions early to inform the proper design,” said Cat’s Galen Faidley. “We’re able to validate and price out our design before it hits the factory. In the past you’d throw it over the wall to the factory and they would have to figure out the sequence it would be assembled in.

Now, the factory side comes in, examines the machine with the VR setup and makes their recommendations. VR also allows deisgners to make changes to the machine in late stages in its development, in many cases at times when it was too late to incorporate such changes.

Check out a video of the VR system in action below.

Augmented reality in development for machine inspections/servicing

At the end of the UI Labs tour, Cat’s Lonnie Johnson gave us a look at how the company is developing an augmented reality (AR) solution to assist in the inspection and servicing of its equipment.

“There is a critical shortage of experienced maintenance and service personnel in the U.S.,” Johnson said, noting that the system would be helpful in training up inexperienced technicians, allowing them to learn and perform simpler tasks on their own while the more experienced techs can handle more difficult and more pressing issues.

“We’re trying to figure out if we can reduce the service event time and improve the quality of the service event when it happens,” he said.

A pair of augmented reality goggles Cat is using as part of its efforts to develop an app that assists technicians with the servicing and inspections of machines in real time.A pair of augmented reality goggles Cat is using as part of its efforts to develop an app that assists technicians with the servicing and inspections of machines in real time.

In the video below, you can see Johnson give a demonstration of this augmented reality assistant. Though the interface is displayed on a TV for that demo, I took a turn later on with the goggles and can tell you that the system works impressively well for a proof of concept.

The goggles are a bit heavy and ran pretty warm to the touch at a few points, but they do display quite a bit of helpful information for the uninitiated and do so directly into your field of vision. For instance, the system will not only display arrows in order to guide you to the right side of the machine, it also displays an outline of whatever part you’re looking for atop the machine in order to help you find it. And once you’ve located that part, a list of things to check for displays in the upper left hand corner of your vision.

I was able to quickly perform a series of part inspections over the course of 15 minutes, despite a few hiccups with the goggles here and there. You can actually get a feel for the system right now with the Cat Inspect app which is being used as reference point for the development of the AR system.

You can check out how the system works in detail in the video below.