Gemma Tuner Multimodal vs TUI-use

Gemma Tuner Multimodal is an open-source fine-tuning toolkit for Google's Gemma 4 and Gemma 3n models that runs entirely on Apple Silicon using PyTorch with Metal Performance Shaders (MPS) backend — no NVIDIA GPU or cloud infrastructure required. It supports LoRA training on multimodal inputs: audio, images, and text simultaneously, using local CSV files or streamed from Google Cloud Storage or BigQuery. The tool targets the growing segment of developers who own M-series Macs but have been locked out of fine-tuning workflows that assume CUDA availability. Gemma 4's architecture is particularly well-suited to this use case: its 4B multimodal variant (designed for on-device deployment) trains efficiently on M3 Max and M4 Pro hardware within the available unified memory constraints. Primary use cases include medical transcription fine-tuning (audio → text with clinical terminology), visual QA systems (image + text → structured response), and private on-device pipelines where cloud API calls are prohibited by compliance requirements. The project fills a specific niche that Google's own fine-tuning documentation doesn't cover well for Apple hardware.

TUI-use is an open-source library that gives AI agents the ability to interact with traditional interactive terminal (TUI) applications — think vim, htop, ssh sessions, database CLIs, and legacy text-based UIs that were never designed for programmatic control. Instead of requiring a GUI or a REST API, TUI-use interprets terminal output as structured state and sends synthetic keystrokes back, enabling agents to "see" and "drive" any TUI application as if they were a human at a keyboard. The project was born from a real pain point: AI coding agents can call bash commands and write files, but they fail badly the moment a tool opens an interactive prompt waiting for user input. TUI-use solves this by building a state machine layer over PTY (pseudo-terminal) interfaces, letting agents read the current screen buffer, detect interactive prompts, and respond intelligently. It ships with adapters for common TUI patterns and a clean API that works with any LLM tool-use framework. The Show HN post attracted genuine interest from the ops and DevOps community — many existing workflows depend on tools that expose only an interactive terminal interface. TUI-use fills a real gap in the "AI agents that control computers" space by handling the long tail of CLI programs that have no API, no GUI, and no intention of ever getting one.