The Watchmen project was conceived and created in collaboration with DANNIE, which was responsible for the hardware part of the solution, while the Doubletapp team developed the software part of the device.

What does it look like? There is a device with two cameras that is installed on the windshield. It is placed inside the car, with one camera facing the interior and the other facing the exterior; the device operates on Android.

One of the client's requirements was to have on-device functionality, as streaming video from multiple cars would be costly, and processing would require an increase in server capacities. This challenge was solved – all heavy computations are performed directly on the device, following the Edge AI approach.

Watchmen has the following functionality:

• Detection of faces inside the car.
• Decision-making about who the driver is.
• Matching the driver's face with the photo uploaded by the car-sharing service user.
• Using computer vision (without the need for additional sensors other than the camera) to detect smoking inside the car.
• Transferring video from the camera to the server when system operation needs to be checked.
• A web interface for demo mode – to promote the solution to car-sharing services. It provides real-time streaming from the device's camera, and real-time detection of smoking and driver substitution.
• Backend and administrative panel for working with drivers’ data.
• All the described detection and recognition tasks are performed offline on the device.

In terms of development, the most interesting task was the implementation of cigarette detection. For this purpose, we collected a dataset of thousands of photos of people smoking in cars, as well as performing actions similar to smoking, such as scratching the nose or coughing into a fist. Even the company's employees contributed by filming themselves smoking in their cars. Based on this dataset, a neural network was trained and subsequently ported to the device for offline operation.