Open source project: Tesla to make rolling monitoring stations

Automatic License Plate Recognition, Identification of People and Their Habits: An open-source project brings these features to the Tesla Models 3, S and X by evaluating the data streams of the built-in cameras. For the first time, it will be possible to see and feel which privacy problems self-driving cars will produce in the future.

The components of the surveillance system on wheels: A Tesla, developed by Truman Kain Software Scout and Nvidia's small computer-focused Nvidia small computer (Jetson Nano, starting at € 162.90), which is $ 100 more expensive. The software version 0.1 Cain during his presentation at the DEF CON 27 presented, can evaluate the video streams in quasi-real time, the three of the built-in cars cars (left and right side mirrors, front camera) deliver.

Uli Ries

The Scout-Software of Truman Kain (presentation slide)

(Picture: Uli Ries)

Scout recognizes number plates by means of open-source applications such as Tensor Flow or Yolo and stores this information in a database (MongoDB) together with the coordinates at which the respective cars were sighted. The machine learning component (Darknet) of the solution was trained using, among other things, Google's Open Images Dataset. In the US, Scout also compares the license plate with the online service to show the vehicle type.

For example, according to Kain, the software can tell if you're being tracked by the same cars every now and then. Thanks to the Sentry Mode, a kind of alarm system for the parked car, cars would also attract attention, often turning around near their own home or office – which, according to Cain, could be a clue to burglars or car thieves. Depending on the warning thresholds defined by the users, the software then displays the corresponding information on the Tesla owner's smartphone or via the car's own web browser. A video uploaded by the programmer shows the software in action (at 0:15 min.).

Surveillance Detection Scout

But not only number plates captured Scout, but also faces of drivers and passers-by and their coordinates at the time of recording. These data can also be evaluated so that frequent views of the same people can lead to warnings. Penetration testers could use this feature, for example, to find the usual working hours of employees of the attacking company without much effort: Park Tesla in front of the office and then let Scout go to work. A matching of the captured portraits with the photos of the employees at LinkedIn is sufficient to deposit the registered people with names in the evaluation lists of Scout.

In the design of the software Truman Kain has omitted to own information for privacy reasons functions. So Scout runs only locally in the car and not in the cloud. Otherwise, there is a risk that the uploaded mp4 video clips of license plates and passers-by fall into the wrong hands. In addition, there is no feature to let entire vehicle fleets communicate with each other. A correspondingly large, widely distributed fleet could otherwise produce comparatively complete motion profiles.

Cain also points out that the software is still in the experimental stage and Tesla drivers would install it at their own risk. He also addressed possible legal issues. In Germany, for example, in 2008, the Federal Constitutional Court passed laws enacted by Hesse and Schleswig-Holstein that should allow the police to recognize license plates.
(Uli Ries) /


(TagToTranslate) Black Hat and Defcon (t) Def Con (t) Face Recognition (t) Hacker (t) License Plate Scanning (t) Machine Learning (t) Nvidia Jetson Nano (t) Open Source Software (t) Tesla Motors