Integrating facial recognition into your Home Assistant setup can elevate your home automation system to a new level of intelligence and responsiveness. By combining the power of several free tools: Frigate, Double-Take, and CompreFace, you can create smart automations that react to recognized faces, adding a personalized touch to your smart home. This guide will walk you through the process of setting up and using these three components, from installation to training the system to recognize individuals in your home.
The Purpose of Frigate, Double-Take, and CompreFace
Frigate: Frigate is a powerful NVR (Network Video Recorder) designed specifically for use with Home Assistant. It excels at object detection, including recognizing if what it sees in the camera is a face, but it doesn’t perform facial recognition by itself. To identify who the person is, you need additional tools. Beyond facial recognition, Frigate also manages 24/7 video storage for your camera feeds, allowing you to retain recordings for as many days as you need.
Double-Take: Double-Take is a companion service that processes the images captured by Frigate. It handles the facial recognition tasks by communicating with Frigate and sending the relevant frames to CompreFace for analysis. Double-Take is crucial for managing the communication between your video feeds and the facial recognition service, ensuring that the right images are analyzed.
Double-Take is a middleman service that bridges the gap between Frigate and facial recognition tools like CompreFace. While Double-Take doesn’t perform facial recognition on its own, it takes the detected faces from Frigate and sends them to CompreFace for identification. Essentially, it’s a UI that allows you to easily control and manage everything related to facial recognition.
CompreFace: CompreFace is an open-source facial recognition system that performs the actual task of recognizing people from the images provided by Double-Take. It uses machine learning algorithms to match faces with known identities. CompreFace is the engine behind the facial recognition, making it possible for your system to identify and differentiate between individuals.
Installation Overview
Setting up these tools is straightforward, especially if you’re running Unraid or any other system that supports Docker containers. Due to the computational demands of facial recognition, it’s recommended to use a powerful machine with a good CPU, ample RAM, and either a dedicated GPU or Google Coral for hardware acceleration.
Google Coral is a hardware accelerator designed specifically for machine learning applications, offering efficient and powerful processing capabilities. It’s particularly useful for tasks like facial recognition, where high-speed processing of video feeds is essential. Even if you’re using a Raspberry Pi 4b or above, which might not have the raw power of a high-end PC, you can still effectively run these components by using the Google Coral USB. This compact device provides the necessary acceleration to handle facial recognition tasks efficiently. For a PC setup, you can opt for the Google Coral PCI-e, which offers even greater performance. Whether you’re on a Raspberry Pi or a more robust PC, the Google Coral hardware significantly boosts processing, ensuring smoother and faster facial recognition. I consider it a must-have device for this application (if you’re using a PC like me, then I recommend getting the B+M coral version and the PCI-e adapter x1).
If you’re using Unraid, you can easily install the necessary Docker containers from the following repositories:
- Frigate:
ghcr.io/blakeblackshear/frigate:stable - Double-Take:
skrashevich/double-take:latest - CompreFace:
exadel/compreface:1.2.0
For systems equipped with Google Coral, you’ll also need to install the corresponding drivers. In Unraid, you can find these drivers by searching for “Coral Accelerator Module Drivers” in the Apps section. During Frigate installation, if you have a PCI-E Coral, set the “Coral TPU Mapping” setting to /dev/apex_0. For a USB Coral, use /dev/bus/usb.
Setting Up CompreFace
Once installed, start by setting up CompreFace:
- Open CompreFace and create a local account.
- Press
+Create Applicationand thenCreate New Service. - You will receive an API Key—save this for later use.
Configuring Frigate
Next, configure Frigate by editing its configuration file. Below is an example configuration that you can modify to suit your setup:
mqtt:
enabled: true
host: [home assistant IP]
user: [home assistant username]
password: [home assistant password]
detectors:
coral:
type: edgetpu
record:
enabled: true
retain:
days: 14
objects:
track:
- person
filters:
person:
min_area: 30000
threshold: 0.75
snapshots:
enabled: true
bounding_box: false
crop: true
height: 500
retain:
default: 10
objects:
person: 15
quality: 100
ffmpeg:
hwaccel_args: preset-vaapi
detect:
enabled: true
fps: 5
width: 1280
height: 720
cameras:
indoor_camera:
enabled: true
ffmpeg:
inputs:
- path: rtsp://user:[email protected]/live0
roles:
- record
front_gate_camera:
enabled: true
ffmpeg:
inputs:
- path: rtsp://user:[email protected]/live0
roles:
- detect
- record
mqtt:
enabled: true
timestamp: false
bounding_box: false
crop: true
height: 500
quality: 100
version: 0.14
Adjust the IP addresses and other details (Home Assistant username and password) according to your setup. After configuring Frigate, verify that your cameras are visible within the interface.
Configuring Double-Take
With Frigate configured, move on to setting up Double-Take. Below is an example configuration for Double-Take:
mqtt:
host: [home assistant IP]
username: [home assistant username]
password: [home assistant password]
topics:
frigate: frigate/events
homeassistant: homeassistant
matches: double-take/matches
cameras: double-take/cameras
detect:
match:
save: true
base64: false
confidence: 92
purge: 168
min_area: 3200
unknown:
save: true
base64: false
confidence: 40
purge: 96
min_area: 0
frigate:
url: http://192.168.1.69:5055
update_sub_labels: true
cameras:
- front_gate_camera
detectors:
compreface:
url: http://192.168.1.69:8800
key: [your compreface key]
timeout: 15
det_prob_threshold: 0.85
opencv_face_required: false
Replace the placeholders with your actual IP addresses and API key from CompreFace, your Home Assistant username and password, etc.
Setting Up the MQTT Server
MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for low-bandwidth, high-latency networks, making it ideal for smart home applications. In the context of Home Assistant, MQTT acts as a communication bridge between various smart devices and services, allowing them to exchange data in real-time. When setting up facial recognition with tools like Frigate, Double-Take, and CompreFace, MQTT facilitates the seamless transfer of information, such as detection events and recognition results, between these components. By using an MQTT server, you can ensure that all parts of your smart home system communicate efficiently and effectively and that Home Assistant properly receives the information about the detections in real time.
To set up MQTT, install the Mosquitto broker add-on in Home Assistant. No additional configuration is needed.
Training the System to Recognize Specific Individuals
Training the system to recognize specific individuals is a crucial step in setting up facial recognition in your smart home. To do this, you’ll use Double-Take’s “Train” tab, which provides a user-friendly interface for adding and managing faces. Begin by selecting the “Train” tab within Double-Take, then click “Add New” to start the process. You’ll be prompted to enter the name of the person you want the system to recognize. Once you’ve entered the name, you can upload a series of clear images of that individual. It’s important to choose high-quality photos where the person’s face is clearly visible, preferably without too much background noise or distractions. After uploading the images, click “Sync” to send the data to CompreFace, which will analyze the photos and create a recognition profile for that individual. This profile will then be used by the system to identify the person whenever they appear in front of your cameras.
Final Integration in Home Assistant
After setting everything up, you can optionally install the Frigate Proxy Home Assistant integration through HACS, then search for and install it from Settings -> Integration -> Add Integration -> Frigate. This integration will create new devices in Home Assistant for each camera, providing sensors for detected faces. Even if you do not install this integration, you will still get sensors that will automatically be created for each detected person (sensor.double_take_[person name]).
Home Assistant Automation Example
One example for a practical application of facial recognition in my own home is automating the opening of my house’s front gate (not the actual door to the house, that’s the one thing I’m not automating). By using the information gathered from the facial recognition setup, I created an automation that only opens the gate when recognized individuals approach. Below is an example of how you can achieve this using Home Assistant.
alias: Front gate facial recognition
description: ""
trigger:
- platform: state
entity_id:
- sensor.double_take_john
- sensor.double_take_julia
from: unavailable
to: front_gate_camera
condition:
- condition: template
value_template: >-
{{as_timestamp(now()) -
as_timestamp(states.binary_sensor.front_gate_contact_sensor_opening.last_changed)
> 15}}
alias: The front gate hasn't been open in the last 15 seconds
action:
- type: turn_on
device_id: 804dd454a1a855c5bef2865cc50dc37d
entity_id: 5150718670208d9ecbc7d26b1c37d298
domain: switch
This automation is triggered whenever the sensors for double_take_john or double_take_julia report that they see a recognized face through the front_gate_camera. Once a match is detected, the system checks whether the gate has been opened in the last 15 seconds. This check is not required, but it’s super useful, because it prevents the gate from reopening if it was just manually opened when someone left the house. In this setup, a Zigbee door sensor is used to track the last time the gate was opened. If the gate hasn’t been opened recently, the automation proceeds to activate the switch that controls the gate, allowing entry for recognized individuals.
Recommended Cameras
The Frigate documentation recommends cameras that output H.264 video and AAC audio, as these provide the best compatibility with all features of Frigate and Home Assistant. It’s also beneficial if the camera supports multiple substreams, allowing for different resolutions to be used for detection, streaming, and recordings without needing to re-encode the video.
The team behind Frigate suggests Dahua, Hikvision, and Amcrest cameras, in that order. Dahua is slightly preferred over Hikvision due to their availability and ease of purchase, although both brands are known for reliable streams and excellent image quality, especially in low light, thanks to their models with large sensors. It’s noted that larger sensors generally outperform higher resolutions, particularly for night-time image quality. Amcrest, a rebranded version of lower-end Dahua models, is also recommended for its reliability.
On the other hand, Reolink cameras have been reported by many users to experience various issues, so they are generally not recommended. If you already use Reolink cameras, it’s advisable to follow a Reolink-specific configuration. Similarly, the Frigate team does not recommend Wi-Fi cameras due to their less reliable streams, which can lead to connection losses and missing video data.
The cameras recommended by the Frigate team above are wired, requiring an Ethernet cable to connect, which might not be feasible for all setups. I’m continuing to evaluate Wi-Fi cameras to identify reliable models. If you have a Wi-Fi camera that you trust, then I welcome suggestions and feedback.
Summary
By following these steps, you can successfully integrate facial recognition into your Home Assistant setup. With Frigate handling video feeds, Double-Take managing the flow of images, and CompreFace performing the facial recognition, your smart home will be more responsive and personalized. Whether it’s unlocking doors for family members or customizing notifications based on who is at the door, the possibilities for automation are vast.
