Скачать или смотреть Use LLM Vision to describe images in Home Assistant

This automation might appear like a gimmick on the surface, but is actually really useful for telling you whether the person at the door has already gone, what they are holding, and whether their face is concealed.

I've set this up on my doorbell, but would work with any cameras that you can get an image from in Home Assistant.

I'm using a Reolink PoE video doorbell, which I highly recommend: https://amzn.to/3Jv386z

Here's a link to Google AI studio: https://aistudio.google.com/app/apikey
This is the llm vision integration documentation: https://llmvision.gitbook.io/getting-...

This is the prompt that I used to get good results:
```
This is a picture from a video doorbell.

If is nobody there say "they have already gone" and nothing else.
Otherwise, If they are walking away say "they have already gone" and nothing else.
Otherwise, If their face is hidden say "Check the camera, their face is hidden." and nothing else.
Otherwise, If none of the previous are true, say "A {adjective} {man/woman} who looks like {famous actor} holding {what they are holding} is at the door"
```

And here's the complete YAML of the automation:
```yaml
alias: Notifications - Doorbell
description: ""
triggers:
type: turned_on
device_id: 60f0f7dba756a82ed054ec8200829078
entity_id: bd8e8bfd2b39fb7832bbc6cdd5df0fb3
domain: binary_sensor
trigger: device
conditions: []
actions:
delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
action: camera.snapshot
metadata: {}
data:
filename: /config/www/reolink_snapshot/last_snapshot_doorbell.jpg
target:
entity_id: camera.reolink_video_doorbell_poe_fluent
enabled: true
action: llmvision.image_analyzer
data:
include_filename: false
max_tokens: 50
provider: 01JW3EN2J5JB2HVCN7DH6H8269
image_file: /config/www/reolink_snapshot/last_snapshot_doorbell.jpg
model: gemini-2.0-flash
message: This is a picture from a video doorbell.


If is nobody there say "they have already gone" and nothing else.

Otherwise, If they are walking away say "they have already gone" and
nothing else.

Otherwise, If their face is hidden say "Check the camera, their face is
hidden." and nothing else.

Otherwise, If none of the previous are true, say "A {adjective}
{man/woman} who looks like {famous actor} holding {what they are
holding} is at the door"
temperature: 0.2
response_variable: vision_result
action: notify.mobile_app_pixel_9_pro
metadata: {}
data:
title: Doorbell
message: "{{ vision_result.response_text }}"
data:
image: /local/reolink_snapshot/last_snapshot_doorbell.jpg
clickAction: app://com.mcu.reolink
enabled: true
alias: Notify Ben's phone
action: tts.speak
target:
entity_id: tts.home_assistant_cloud
data:
cache: true
media_player_entity_id: media_player.kitchen
message: Ding dong. {{ vision_result.response_text }}
enabled: true
alias: Notify kitchen speaker
mode: single
```

There's a blog version of the video here: https://smarthomeideas.info/posts/sma...

If you want to support my work, remember to like and subscribe - it really helps the channel out.

If you're feeling particularly generous, you could help fund my next cup of coffee!
https://ko-fi.com/smarthomeideas

Any links to products in the description might be affiliate links, which means I get a small kickback, at no extra cost to you.