diff --git a/README.md b/README.md index 7d01bb7..7ebbd39 100644 --- a/README.md +++ b/README.md @@ -346,3 +346,107 @@ sudo chown -R hbrain:www-data /var/www/html/mvlog/.git Do not commit generated media from `in-dir/` or `out-dir/`. `AGENTS.md` is local assistant/project context and should remain ignored by git. + +## Server-side web-push notifications + +The MVLog web UI includes a small server-side push integration so the admin can immediately notify subscribers when a job is made "Show". Keys and sending logic live on the webserver to keep VAPID private keys secret and to avoid exposing them to worker nodes. + +Key files and behavior (deployed under /var/www/html/mvlog): + +- lib/send_push.php — PHP helper that invokes the Node sender and atomically records send state in cache/push_notifications.json. +- send_push.js — Node script (uses the package) to send individual push messages; it reads subscriptions and push config and performs sends. +- cache/push_subscriptions.json — stored subscriber list (array of subscription objects). +- cache/push_notifications.json — recorded sends (shown map) to avoid duplicate notifications. +- api/enable_show.php — authenticated endpoint to toggle Show and trigger a server-side send (requires admin login). +- new.php — admin UI now triggers the server-side helper when Show is enabled. +- /etc/mvlog/push.php or /var/www/html/mvlog/push.json — VAPID key holder (private key must remain secret; set owner to root or www-data and permissions to 600/640). +- log: /var/log/mvlog_notify.log — send attempts and errors are logged here. + +Installation notes: + +1. Install Node and web-push in the webroot: + + npm install web-push --prefix /var/www/html/mvlog + +2. Place VAPID keys where the webserver can read them (recommended: /etc/mvlog/push.php) and set restrictive permissions (600 or 640, owner root:www-data). Do NOT commit private keys to git. + +3. Ensure /var/www/html/mvlog/cache exists and contains push_subscriptions.json and push_notifications.json (create {} if empty), owned by www-data and writable. + +Security notes: + +- api/enable_show.php requires admin login; do not expose it publicly without strong auth. +- Consider adding CSRF protection, rate limiting, and periodic pruning of expired subscriptions (HTTP 410 responses) to reduce send failures. + + +## Gemini-based description generator + +A helper script on the webserver can analyze an input directory (images/videos) and produce a short JSON description using the Google Generative Language (Gemini) API. + +- Script: /var/www/html/mvlog/bin/generate_data.sh +- Purpose: collect representative frames, call Gemini, save the raw response as /gemini_generated.json, and append a Description block to /data.txt only if no Description already exists. + +What it does: + +- Collects up to N frames from the input dir (images -> 1 frame, videos -> up to 3 frames depending on duration). +- Builds a JSON request containing the prompt and inline base64-encoded images using a Python helper to avoid ARG_MAX issues. +- Calls the Gemini API and saves the returned JSON text to /gemini_generated.json for review. +- Appends only a Description block (Description: +## Server-side web-push notifications + +The MVLog web UI includes a small server-side push integration so the admin can immediately notify subscribers when a job is made "Show". Keys and sending logic live on the webserver to keep VAPID private keys secret and to avoid exposing them to worker nodes. + +Key files and behavior (deployed under /var/www/html/mvlog): + +- lib/send_push.php — PHP helper that invokes the Node sender and atomically records send state in cache/push_notifications.json. +- send_push.js — Node script (uses the `web-push` package) to send individual push messages; it reads subscriptions and push config and performs sends. +- cache/push_subscriptions.json — stored subscriber list (array of subscription objects). +- cache/push_notifications.json — recorded sends ("shown" map) to avoid duplicate notifications. +- api/enable_show.php — authenticated endpoint to toggle Show and trigger a server-side send (requires admin login). +- new.php — admin UI now triggers the server-side helper when Show is enabled. +- /etc/mvlog/push.php or /var/www/html/mvlog/push.json — VAPID key holder (private key must remain secret; set owner to root or www-data and permissions to 600/640). +- log: /var/log/mvlog_notify.log — send attempts and errors are logged here. + +Installation notes: + +1. Install Node and web-push in the webroot: + + npm install web-push --prefix /var/www/html/mvlog + +2. Place VAPID keys where the webserver can read them (recommended: /etc/mvlog/push.php) and set restrictive permissions (600 or 640, owner root:www-data). Do NOT commit private keys to git. + +3. Ensure /var/www/html/mvlog/cache exists and contains push_subscriptions.json and push_notifications.json (create {} if empty), owned by www-data and writable. + +Security notes: + +- api/enable_show.php requires admin login; do not expose it publicly without strong auth. +- Consider adding CSRF protection, rate limiting, and periodic pruning of expired subscriptions (HTTP 410 responses) to reduce send failures. + + +## Gemini-based description generator + +A helper script on the webserver can analyze an input directory (images/videos) and produce a short JSON description using the Google Generative Language (Gemini) API. + +- Script: /var/www/html/mvlog/bin/generate_data.sh +- Purpose: collect representative frames, call Gemini, save the raw response as /gemini_generated.json, and append a Description block to /data.txt only if no Description already exists. + +What it does: + +- Collects up to N frames from the input dir (images -> 1 frame, videos -> up to 3 frames depending on duration). +- Builds a JSON request containing the prompt and inline base64-encoded images using a Python helper to avoid ARG_MAX issues. +- Calls the Gemini API and saves the returned JSON text to /gemini_generated.json for review. +- Appends only a Description block ("Description: |\n ...") to /data.txt if data.txt does not already contain a Description key (case-insensitive). It never changes other keys in data.txt. +- Leaves gemini_generated.json unchanged (always written as the raw response). + +Configuration and requirements: + +- GEMINI_API_KEY should be placed in /etc/mvlog/gemini.env and be readable by www-data (recommended owner root:www-data mode 640). +- Required tools on the webserver: jq, python3, ffmpeg, ffprobe, curl. +- Run the script as the webserver user so created files are owned by www-data and permissions are correct: + + sudo -u www-data /var/www/html/mvlog/bin/generate_data.sh /var/www/html/mvlog/in-dir/20230630_napoli "Naples" "Naples, Italy" 8 + +Troubleshooting: + +- If Gemini returns HTTP 429 RESOURCE_EXHAUSTED, ensure the API key is attached to a project with billing and the Generative Language API quota enabled. +- If you see "Argument list too long" errors, ensure generate_data.sh is the deployed version that uses the Python builder (it avoids embedding large base64 arguments directly into jq/curl). +- Run the script as www-data to avoid permission problems reading /etc/mvlog/gemini.env and to create files owned by the webserver.