“I believe in horses. The automobile is a transitory phenomenon”, argued the last German monarch, Emperor Wilhelm II.1 Nevertheless, cars started to conquer the world and have by now become the individual mobility device par excellence. At the time of Emperor Wilhelm II, the transition from the horse to the car revolutionized mobility. Today, a new fundamental change seems to be imminent: from the passenger-driven car to the autonomous car. Many passionate drivers still find it hard to imagine that soon their car will drive, steer, brake, and park all on its own. The drivers and their driving skills lose in importance. The large automobile manufacturers have long since started the era of autonomous driving. But companies from sectors other than the automotive industry, too, sense new market opportunities. A search in the Derwent World Patent Index (DWPI) on STN impressively shows that R&D activities in this field have been strongly increasing throughout the last five years.2
What lies behind these innovations? Autonomous cars consist of many individual components. Only the intelligent interaction of these components makes the car autonomous. The published patents cover the individual components such as certain driver assistance and safety systems, sensors, and the digital processing of sensor data. These innovations have changed the lives of motorists amazingly fast. Many expensive cars are already equipped with advanced driver assistance systems (ADAS) and sensors. The first inventions in this field, e.g., anti-lock braking system (ABS) and cruise control, have by now become standard for almost all classes and types of cars. So far, truly autonomous cars have only existed as prototypes, but it is planned to use them on public roads within just a few years’ time.
Fig. 1: Autonomous car patents published from 2000 to 2015 (py.b). Patents published in 2016 were not considered, since the figures for 2016 are incomplete
Toyota ranks first
The ranking list of patent applicants in the technology sector of autonomous cars reads like the Who-is-Who of the automotive industry. Also among the top ten are the three automotive suppliers Bosch (DE), Denso (JP) and Valeo (FR). Obviously, these companies have no intention to leave the promising market of autonomous cars to the automobile manufacturers alone.
Fig.2: The world’s top 10 patent applicants in the field of autonomous car technology, from the year 2000 to August 2016
An all-round assistant
The largest sub-group of the patents we found deals with advanced driver assistance systems (ADAS), the technical precursors of autonomous driving. To develop such a system, all actions performed by the driver of the car are split up into individual steps. Later, each of these steps is carried out by an electronic assistance system. There are, for example, lane departure warning systems, lane keeping assists, night vision systems and traffic jam assistants, to name just a few. The sheer number of different ADAS demonstrates how complex driving a car is. By combining and interconnecting several ADAS, some kind of autopilot can be created. The system keeps the car on the right lane on the motorway, keeps the right distance to the vehicles in front, passes other vehicles properly, respects speed limits, and breaks in time before an obstacle or a traffic jam. This is already working quite well on motorways, whereas it is much more difficult for the system to adapt to and cope with the various situations in city traffic. In May 2016, Tesla suffered a severe drawback when testing its autopilot system. There was a fatal accident because the autopilot had not recognized a truck as an obstacle, while at the same time the driver was blindly relying on the assistance system. In the meantime, Tesla has taken action and redesigned the autopilot. It will now be equipped with a radar supported warning system in addition to the sensors. Besides, Tesla expressly warns against handling the new technology too carelessly: The human operator behind the wheel is still responsible for driving and has to remain in control of the car, even if the autopilot is activated.
Autonomous cars are being tested in several US states, among them Nevada. In these states it is permissible to operate such cars under certain conditions, but the driver must always be present behind the wheel in order to be able to take control of the car in dangerous situations.
And how about Google?
In the press, Google is often mentioned in the same breath with autonomous vehicles. Indeed, our patent analysis has shown that Google, too, is among the very active applicants, albeit not among the top ten. Google follows a different business strategy: It relies on partnerships with leading automobile manufacturers such as Toyota. This means that Google does not have to deal with automobile manufacturing itself and can focus its research on other fields.
Fig. 3: Drawing from Google patent US9097800. On the left, a car with mist of water behind its wheels (526 and 524), on the right, a car with a cloud of sooty exhaust gas (532)
The drawing below is from a Google patent. The invention covers a system of sensors that can clearly distinguish between liquid and gaseous obstacles and send reliable data directly to the autonomous car. This way, a fine mist of rain water stirred up behind the wheels of a car can be unmistakably identified. Exhaust, possibly containing soot particles, can also be clearly identified as gas by the sensor system. All this is important information, as the autonomous car should of course only brake or draw aside when solid obstacles are encountered. It is worth mentioning that Google has extended its patent protection (US9097800) to trucks, buses, motorcycles, boats, airplanes, helicopters, lawn mowers, bulldozers, motor sledges, mobile homes, vehicles at pleasure grounds, agricultural vehicles, construction vehicles, streetcars, golf carts, trains and shopping carts. After all, you never know in which areas autonomous driving may be used in the future.
The word telematics is a combination of “telecommunication” and “informatics”. Inventions in this field deal, for example, with controlling a mobile vehicle via GPS (Global Positioning System). The best known examples for this technology are the navigation systems that are often standard in modern cars. Telematics patents constitute the smallest group of the patents we searched. When analyzing the companies who had filed a patent for a telematics invention in detail we also found other companies besides the large automobile manufacturers.
Fig. 4: The world’s top patent applicants in telematics from 2000 to August 2016
On the ranks below the dominant automobile manufacturers General Motors and Hyundai Motors there are companies from other industry sectors. Marvell Technology Group, for example, produces telecommunication products, storage devices, and semiconductor products. LG Electronics is another company from the telecommunications sector. Of quite a different sort is the logistics company UPS (United Parcel Service). However, when taking a look at the patents filed by UPS, we quickly recognize why a logistics company, too, should be interested in inventions in the field of telematics. Telematic devices allow to continuously monitor the delivery trucks, to locate them, to update the calculated time of travel, and to direct them to vacant parking areas. For this purpose, the trucks are equipped with sensors and guided by GPS. There is a permanent exchange of data with a cloud, see patent drawing below.
Fig. 5: Patent drawing from US20150198450 depicting a networked UPS truck
Other industries, too, benefit from innovations
Innovations in the autonomous car sector also appeal to other industries. Money is always an important factor for inventions, either directly or indirectly. There is also a company from the financial sector that would like to use telematics in cars: Mastercard, who for quite some time had been looking for a method of unambiguously identifying card owners, has now come up with a surprising solution for drivers of autonomous and conventional vehicles. To enable cashless payments of toll, parking fees, and invoices at gas stations and service areas, the car is equipped with a processing device that operates with a certain code. Another numeric or alphanumeric code is assigned and applied to the car’s license plate. Before payment is made, both codes are read and compared with each other using wireless communication, e.g., NFC (US20160247143). If the codes match, payment is released.
The modern groom
A further step ahead is the autonomous car park where cars are no longer parked by their drivers but distributed to, parked in, and removed from the parking spaces by robots. The robots fetch the cars from the transfer station and transport them to their parking space. These parking robots are already in use at the car park of Düsseldorf Airport3 in Germany. We have to use our imagination to read from the sober patent drawing below how efficient the parking robot system really is. A robot lifts the car with forks similar to those of a forklift. Then the car, which has been previously measured by means of cameras and sensors, is accurately parked in a space-saving manner. The cars can be very closely packed because no space for maneuvering or leaving the car is required. In addition, the parking system is linked to the flight database. These data tell the system when the car owner will be back, i.e., when his car has to be made available again. The flight passenger can then directly access his car, ready to start and without any waiting time. Thus, we have come full circle to the times of Wilhelm II. When the German Emperor got off his horse, his groom was already waiting to guide the horse to its stable and to bring the saddled horse back for another ride, if needed.
Even if some people still do not want to believe it: The era of autonomous driving has begun. This technology is progressing fast. The German Federal States of Bavaria and Baden-Württemberg have already established test tracks on their motorways. Now other public sectors, too, will have to catch up on the new developments. Especially insurance companies are supposed to react, and new laws have to be put into place. Liability issues, ethical principles, and sociological aspects of autonomous driving have not yet been conclusively assessed and are still to be clarified. However, the faculty of law of the University of Würzburg has already established a research unit “RobotRecht”4 that deals with the legal issues of man-machine interaction in collaboration with car manufacturers and other industries.
Text and research: Dr. Babett Bolle
Fig. 6: Drawing from patent DE102010052850 by Serva Transport Systems GmbH
Wilhelm II. (1859-1941) was German Emperor from 1888 to the German Revolution of 1918.
Jürgen Rees, Fährt (nicht) alleine, WirtschaftsWoche 28, 2016-07-08.