docs: document Gemini generator (generate_data.sh) behavior and data.txt update rules

README.md

@@ -312,3 +312,49 @@ Security notes
 - Consider adding CSRF protection, rate limiting, and periodic pruning of expired subscriptions (HTTP 410 responses).
 
 If you want deployment automation, alternative flows (worker-based sender or token-protected endpoint), or I should add a short admin checklist, tell me and I will update the docs.
+
+## Automated Gemini description generator (generate_data.sh)
+
+Location: /var/www/html/mvlog/bin/generate_data.sh
+
+Purpose:
+
+- Produce a short human-style description (plus teaser/tags in the raw AI output) for an MVLog input job by sending sampled frames to Google's Gemini model.
+- Save the raw AI output to <inputdir>/gemini_generated.json (preserved for review).
+- If the job's data.txt does not already contain a Description: entry, append a Description block to data.txt and leave all other lines untouched. If data.txt doesn't exist the script will create it with the Description block only.
+- The script never overwrites an existing Description and does not modify gemini_generated.json.
+
+Dependencies & configuration:
+
+- ffmpeg and ffprobe (for extracting frames)
+- jq (for small JSON extraction tasks)
+- python3 (used to build the request payload and avoid argument-list limits)
+- A valid GEMINI API key available in the environment as GEMINI_API_KEY. Recommended placement:
+
+  - Create /etc/mvlog/gemini.env containing: export GEMINI_API_KEY='YOUR_KEY_HERE'
+  - Set owner root:www-data and mode 640 so the webserver (www-data) can source it without exposing the key in git.
+
+How the script works (high level):
+
+1. Collects image/video files from the input directory and extracts scaled JPEG frames (images → 1 frame each; videos → 1–3 frames depending on duration and remaining slots).
+2. Builds a generateContent request that embeds the frames as base64 "inline_data" parts and includes a short prompt describing the expected JSON format.
+3. Calls the Gemini REST API (model defaults to gemini-2.0-flash) and writes the AI response JSON to <inputdir>/gemini_generated.json.
+4. Extracts the "description" field from the AI JSON. If data.txt exists and already contains a Description: key (case-insensitive), the script does nothing to data.txt. Otherwise it appends a YAML-style block scalar to data.txt:
+
+   Description: |
+     <two-space indented description text>
+
+   If data.txt does not exist the script will create it containing only the Description: block.
+
+Notes, running and troubleshooting:
+
+- Run as the webserver user so file ownership is correct:
+
+  sudo -u www-data /var/www/html/mvlog/bin/generate_data.sh /var/www/html/mvlog/in-dir/20230630_napoli "Naples" "Naples, Italy" 8
+
+- If the GEMINI API key has no quota or billing is not enabled, the API will return 429 RESOURCE_EXHAUSTED and no description will be written. The raw gemini_generated.json will contain the error response.
+- Ensure /var/www/html/mvlog/in-dir/<job> is owned/writable by www-data; the script writes gemini_generated.json and may append to data.txt.
+- Install jq if missing: sudo apt install jq
+- The script intentionally preserves the full Gemini output (gemini_generated.json) so you can review and re-run manually if needed. If you prefer the old behavior (write a separate data_gemini.txt instead) or want additional fields appended (teaser/tags), say so and I will update the script and docs.
+
+