| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat: Maestro E2E functional test suite for Omi app (#6283) * feat: add Maestro E2E functional test suite for Omi app (#3857) - 10 Maestro flow files covering all core app functionality: - Login & Sign-in (Google OAuth) - Onboarding (name, permissions, device skip) - Conversations (list, detail, tabs, CRUD, folder filters) - Memories (view, create, edit, delete) - Chat (send message, AI response) - Apps/Plugins (marketplace browse, detail) - Settings (navigation, key sections) - Device Connection (BLE scan & pair) [device_required] - Recording & Transcription [device_required] - Logout (sign out, verify auth screen) - Tag system: 'core' (simulator-safe) vs 'device_required' (needs Omi HW) - Runner script with tag filtering, parallel-safe output, JUnit XML + Markdown reports - Unified test.sh entry point: --e2e flag for Maestro, default for unit/widget - Clean, DRY flow structure — no code duplication across flows - .gitignore updated to exclude generated reports * fix: address code review — apply global config, isolate screenshots, robust tag parsing - P1: Pass --config global.yaml to maestro test so onFlowStart hooks fire - P1: Use find -newer to only copy screenshots from current flow run - P2: Replace brittle grep -A5 tag extraction with awk block parser * fix: chat flow — meaningful AI response assertion + correct pressKey syntax - P1: Replace text: '.*' (matches anything) with actual response keywords - P1: Fix pressKey optional: true to use block-mapping syntax - Wait for loading indicators to disappear before asserting response * fix: correct pressKey optional syntax in chat flow --------- Co-authored-by: hniane1 <hniane1@users.noreply.github.com> Co-authored-by: Hashim Niane <hahimniane@users.noreply.github.com> | 2 个月前 | |
Update the app > code formater, set line length to 120 (#2675) | 1 年前 | |
perf(android): reduce native BLE audio and battery history overhead (#12942) * perf(android): reduce native BLE audio and battery history overhead Cache stream JSON decoding while synchronously checking current typed prefs, auth, foreground ownership and Custom STT gates. Reuse compatible sockets without per-frame URL/request construction, preserve same-account queued audio across auth/metadata refresh, and reject stale socket callbacks. Sample battery history at the existing iOS 15-minute/5-point/20% threshold policy with a persisted baseline. Combine frame prefix and payload into one immediate reusable-buffer write, retaining the existing fsync and rollback boundary. Run Android native unit tests in the existing mobile CI lane. Verification: app bash test.sh passed 1813 tests (5 skipped); Android Gradle native suite passed, dev ARM64 APK assembled. No attached Android device, so live BLE and physical battery impact remain unmeasured. Failure-Class: none * fix(android): harden batch recovery and inactive streaming gates Propagate rollback failures and hold file publication and ACK barriers until existing parts have a verified complete-frame prefix. Validate and sync stale parts and same-name reopens before publish/append, retaining failed repairs. Keep inactive notification gates outside the streamer monitor, recheck gates at activation and cleanup boundaries, and clear stale-UID transcripts for all stop reasons and inactive sessions. Share battery retention limits. Preserve the unit task's ARM64 Flutter dependency scope and document why it is needed. Verification: 56 native JUnit tests passed, including real batch writer write/rollback/restart/reopen faults and controlled inactive-gate races. Dev ARM64 APK rebuilt. Independent parent review and native rerun passed. Gradle dry-runs prove the unit task includes compileFlutterBuildDevDebug and its platforms expand to three without target-platform. No physical device test. Failure-Class: none * docs(app): keep AGENTS.md within lean size budget after merging main Merging main brought app/AGENTS.md to 11712 bytes against the 11500-byte agents-md-lean ratchet (main itself crossed the budget at #12939). The merge resolution adopts the branch's condensed analyzer-ratchet wording (same facts), and a few verbose sentences are tightened with no factual change so the merged tree fits the budget. | 10 小时前 | |
chore(app): remove 56 unreachable screens, dead bridges, and unused deps Import-graph reachability from lib/main.dart (routing is widget-direct, so the import graph is authoritative) found 56 unreachable files (~11.1k LOC): old email/password auth + onboarding flows, old goals UI, old settings pages, PayPal flow, pages/sdcard/, and singletons. A June 2026 security fix (471b4054b6) landed on one of these while unreachable - the strongest signal they had to go. Also removes: the wifi-permission vertical dead on both ends (Dart service + iOS/Android plugins + registrations incl. all 6 pbxproj entries), the channel-less com.omi/shortcuts service, the deep_links fallback handler in auth_service (native half never existed), bee_audio_capture.py (orphaned 567-line decompiled BLE tool), 9 unused pubspec deps, Education.gif, the commented _handleCalendarCreation block, and stale e2e covers entries. Verification: flutter analyze byte-identical before/after (0 new errors); analyze ratchet replicated per-rule - PASSED with 9 rules improved; rg zero-refs for every deleted basename across lib/test/e2e/ios/android; flutter pub get regenerated pubspec.lock (removals only, lints demoted to transitive). Part of the 2026-09-03 monorepo cleanup (phase 1). | 4 天前 | |
refactor(release): simplify desktop distribution channels | 1 个月前 | |
docs: take operator pages off docs.omi.me Unlisted Mintlify MDX is still a public URL. Move runbooks, flags, invariants, and agent rules next to owning code, add docs/AGENTS.md as the site allow-list, and correct the live kill-switch contract after the JIT authority page leaves the site. Co-authored-by: Cursor <cursoragent@cursor.com> | 8 天前 | |
fix(app): shorten the home chat bar placeholder to "Ask Omi" (#13024) The bar shares its row with the mic and record buttons, so "Ask Omi anything about your life..." was cut off mid-word on a phone. Use the existing localized askOmi string, which every locale already carries. Claude-Session: https://claude.ai/code/session_01BViCmmJpJ3FaExYyxD2TJx Co-authored-by: Nathan <nathan@Nathans-MacBook-Air.local> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> | 1 小时前 | |
evidence(app): P5 WAV splitting 18 chunks transcription complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> | 4 个月前 | |
app: use const constructors in integration_test/animation_performance_test.dart | 1 个月前 | |
Speaker identification: measured threshold + margin, live clip pooling, SpeechBrain retirement; carries #12531 without the onboarding-step removal (#12935) * fix: unblock speech-profile redo and STT pre-flight for already-onboarded accounts Rebased onto origin/main as a single commit. Keep both main's open_provider_selection_circuit and this PR's is_stt_available helpers, then regenerate OpenAPI clients from the rebased backend. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(app): fade transcript words in as they arrive on the speech-profile screens Add FadeInWordsText: a centered word Wrap where only the words appended since the previous render animate from transparent to opaque with a short stagger, existing words stay put, and a rewritten transcript re-reveals from the start. Both the onboarding speech-profile step and the Settings redo page adopt it in the next commit so the live transcript reads the same whether the words come from the server or the on-device fallback. Verification: flutter test test/widgets/fade_in_words_text_test.dart (4 passed); observed on an iPhone 16 Pro via hot reload while dictating. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F1zmRTRh3DV8NsM3QUrrAB * feat(app): fall back to on-device speech recognition when server STT is unavailable The speech-profile question flow (onboarding step and Settings redo) needs a transcript only to drive the questions and progress; the voice print itself is computed server-side from the WAV uploaded at finalize(). So when the backend's streaming STT is down, transcribe on the phone instead of dead-ending: - SpeechProfileProvider gains a local-STT mode. It is entered up front when the stt-availability pre-flight fails, or mid-session after the existing three 1011 closes with no captured speech (previously STT_UNAVAILABLE). The socket becomes the existing CompositeTranscriptionSocket: an on-device polling primary (Apple speech on iOS, downloaded Whisper on Android) forwarding suggested_transcript frames to the backend listen socket in custom_stt mode, which the OnboardingHandler already consumes like server STT output. No backend change; a receiver regression test pins that seam. - iOS on-device recognition hardening (AppDelegate.swift): resolve the app's bare language code to an installed on-device locale (a recognizer built from "en" failed every request with kAFAssistantErrorDomain 1101); reply exactly once per clip on final result, error, or a 20 s timeout, keeping partial results; and expose onDeviceAvailable, which probes a silent clip so a phone with Siri and Dictation disabled (kLSRErrorDomain 201) is reported as "no local STT" instead of entering the fallback blind. - PurePollingSocket bounds each transcribe() with a 30 s timeout. A provider that never answered left the processing flag set forever and silently stopped transcription for the rest of the session; now the audio is requeued and the next tick retries. This also protects the main app's on-device mode. - When neither server nor on-device STT is available, the pre-flight dialog now says to check the connection or turn on Dictation. - Speech-profile UI: subtler mic-level glow, and the live transcript uses the new fade-in words widget. Verification: - flutter test (full suite): 1731 passed, 5 skipped; scripts/analyze_ratchet.sh passed - new tests: speech_profile_provider_test (5 fallback cases), pure_polling_test (hung-provider timeout), fade_in_words_text_test (4), backend test_onboarding_question_start (suggested_transcript reaches the transcript queue only in custom-STT mode) - live on iPhone 16 Pro against the local dev harness with the STT primary forced unavailable: session connects with custom_stt+onboarding flags, Apple on-device recognition returns the spoken answer (~180 ms per clip) and it is forwarded to the backend; with Dictation disabled the probe reports unavailable and the dialog appears. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F1zmRTRh3DV8NsM3QUrrAB * feat(app): use SpeechAnalyzer for on-device speech on iOS 26 SFSpeechRecognizer's on-device mode fails with kLSRErrorDomain 201 whenever Siri and Dictation are turned off in Settings, which is what produced the "turn on Dictation" pre-flight dialog in the speech-profile fallback. iOS 26's SpeechAnalyzer/SpeechTranscriber has no such dependency: the language model is an asset the app installs itself through AssetInventory. - transcribe: on iOS 26 run the clip through SpeechAnalyzer (preset .transcription, analyzeSequence(from:) + finalizeAndFinish), falling back to the SFSpeechRecognizer path only if the analyzer throws. - onDeviceAvailable: report true when a supported locale's model is installed or installs within 8 s; a longer download keeps going in the background and the first transcribe() waits for it. Concurrent callers share one download. - SFSpeechRecognizer remains the path for iOS 15-18. Verified on an iPhone (iOS 26.6.1) with Dictation off: the speech-profile redo enters local-STT mode and transcribes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(app): speech profile talks through three topics and completes on a word target Speech-profile recording (onboarding step and Settings redo) no longer walks one question at a time with a percentage bar. Instead: - A compact white-outlined card headed "Answer with your voice:" lists three topics (where you live, what you do for work, your long-term goal), and a thin bar under it fills as the user speaks. Reaching SpeechProfileProvider.targetWordCount (60 spoken words) finalizes the recording; the backend's onboarding_complete event no longer does, so "bar full" and "done" are the same moment. Omi's own question segments are excluded from the count. - The live transcript is bottom-anchored in a box exactly three lines tall above the card, so whole lines scroll off the top and nothing overlaps. - The Play button on the Settings page plays the saved profile audio in place (just_audio) and turns into Stop, instead of opening the samples page. Redo stops playback first. - Backend ONBOARDING_QUESTIONS is the same three topics, and OnboardingHandler keeps the transcript across questions so one stretch of speech can satisfy several of them. - Removed the unused percentage progress-bar widget and the "Skip this question" button; onboarding keeps "Skip for now". Tests: speech_profile_provider_test (word target fills, finalizes once, ignores Omi segments and the backend completion event); backend test_onboarding_talk_about_flow (one transcript answers every topic; the transcript is kept when it stops answering). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(dev-harness): advertise local-storage links on OMI_DEV_HOST A phone built against OMI_DEV_HOST could reach the backend but not the files it links to: OMI_LOCAL_STORAGE_BASE_URL was always http://127.0.0.1:<port>/_local/storage, so playing the saved speech profile from a device failed. The harness now derives a dev_advertise_host from OMI_DEV_HOST (loopback stays the default) and uses it only for that base URL; every other service address still binds and talks over loopback. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): keep the last three whole transcript lines instead of a clipped scroll The speech-profile transcript was a bottom-scrolled ListView clipped to a three-line box, so a sliver of the line above always showed at the top edge and read as cut-off text. FadeInWordsText now takes visibleLines: it replays the Wrap line breaking with measured word widths and builds only the words on the last N lines, so earlier lines drop off whole, nothing is clipped or scrolled, and words keep their reveal state while on screen. Both screens use visibleLines: 3 inside a fixed three-line, bottom-anchored area, moved a little further above the topics card. Test: fade_in_words_text_test covers short text showing everything, earlier lines dropping once the text exceeds three lines, and the shown words matching the line-break replay. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): measure transcript lines with the effective text style; 40-word target FadeInWordsText replayed the Wrap line breaking with the caller's raw style, but each word's Text inherits the ambient DefaultTextStyle (font family, weight) under that style, so the replay undercounted lines and the real layout could reach four lines and draw over the topics card. Measure with the same merged style, and clip the fixed three-line area on both screens as a safety net so a stray line can never overlap the card. Also lower SpeechProfileProvider.targetWordCount from 60 to 40 so the recording finishes sooner. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): keep the "<Name>'s Speech Profile" title on one line The title wrapped onto two lines for longer names; it now scales down to fit a single line instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): never clip the speech-profile transcript; raise it above the card The three-line transcript area was a fixed-height clipped box, so whenever the rendered lines ran taller than the fontSize*height estimate (text scaling, font metrics) the top line was cut off. FadeInWordsText already guarantees at most three lines, so the area now only has a three-line minimum height (scaled with the text scaler) and grows to its content instead of clipping. Both screens also keep more space between the transcript and the topics card. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(app): finish the speech profile after three sentences; no page-load spinner The recording now completes once the user has spoken three sentences (SpeechProfileProvider.targetSentenceCount, counted on ./!/? boundaries followed by a space or the end of the text, so "3.5" is not one) instead of a word count, and the bar under the topics card fills per sentence. The progress-bar widget is renamed SpeechProgressBar to match. The Settings speech-profile page no longer swaps its Play/Redo or Get Started buttons for a spinner while the page initialises or the STT pre-flight runs; the buttons stay put and startRecording() ignores taps until the check finishes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): play the saved speech profile on the loudspeaker The app's audio session is normally configured for recording, so tapping Play on the Settings speech-profile page routed the WAV to the quiet earpiece. Before playing, configure a playback-category session (default mode, media usage on Android) and play at full volume, so the profile comes out of the main speaker like any other media. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): start every speech-profile recording with an empty transcript Tapping Redo showed the previous recording's words (and counted them toward the sentence target) because nothing cleared the provider's transcript before a new session; only close() did, on leaving the page. initialise() now calls a new resetTranscript() first, which forgets the segments, text, progress, completion and upload flags without touching the audio storage it recreates right after. resetSegments() reuses it. Test: a completed session's transcript is gone after resetTranscript and the fresh session counts sentences from zero and can finalize again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): finish the speech profile after a pause, keeping the last sentence on screen Reaching the third sentence finalized immediately, which stopped the mic mid-utterance (the recognizers punctuate each clip, so a pause can read as a sentence end) and swapped the transcript for a spinner at once. Now: - After the target is reached the provider waits completionGrace (2 s) without new speech before finalizing, restarting the wait on every new segment, and finalizes at completionCap (8 s past the target) at the latest. Once fired it does not re-arm; resetTranscript() clears it. - Both screens keep the last three transcript lines visible through the upload and the All done state, so the final sentence lingers instead of vanishing. Tests: grace/cap timing under fakeAsync, no double finalize, and the reset test now elapses the grace. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(backend): give every speech-profile recording its own conversation Tapping Redo within two minutes of the previous attempt showed last time's words as soon as the user spoke again. The new listen socket attached to the still-open in-progress conversation from the previous attempt (same source, inside conversation_creation_timeout), so combine_segments() merged the first new segment into that conversation's last segment and the merged segment, old text included, was what the client received. LiveConversationController.prepare() now always creates a fresh in-progress conversation for onboarding_mode sessions (the onboarding step and the Settings redo both set it) instead of consulting the in-progress pointer. Ordinary listen sessions are unchanged. Test: test_listen_speech_profile_fresh_conversation.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): cross-fade the speech-profile recording UI into a plain All done button Keeping the transcript on screen through the upload made it pop back in on its own above the spinner and the All done button. The Settings page now cross-fades (450 ms) from the recording UI (transcript, topics card, bar) to nothing while uploading and then to the All done button, which is the same black capsule with a plain white border as the other buttons instead of the gradient box. The onboarding step likewise no longer shows the transcript in its upload and All done states. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * chore(app): remove the speech-samples page and dead progress-state code The Play button now plays the saved profile in place, so the samples page and its provider became unreachable (CI dead-code ratchet). Also drop the scroll controllers and SCROLL_DOWN signal the old clipped transcript used, and the word-count progress-message state (SpeechProfileProgressState, percentageCompleted, questionProgress) nothing reads any more. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): hold the finished speech-profile recording on screen before All done After the third sentence the final words disappeared as soon as the upload began. Both screens now keep the finished recording (last words, topics card, full bar) on screen through the upload and for a further 1.5 s (allDoneHold) after the profile is saved, then cross-fade into the All done button. Onboarding's upload spinner row and its now-unused loading-text helper are gone; Skip for now hides once recording ends. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): fade the finished speech-profile recording out as one block Parts of the finished recording could change on their own before the cross-fade (the transcript and the mic disclaimer are built from live provider state that finalize() and its callbacks touch), so they did not disappear together. Both screens now snapshot the recording view (last words, no-device flag) the moment recording ends and build from that until a new recording starts, and the onboarding step's All done switch is now the same AnimatedSwitcher cross-fade as the Settings page, so the words, the topics card, the bar and the disclaimer fade out at the same time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): keep the speech-profile bar full until the finished recording fades finalize() clears the provider's text once the profile is saved, and the bar derived its value from that text, so it dropped back to zero before the cross-fade. The frozen recording view now pins the bar at full from the moment recording ends. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(app): drop the speech-profile and memory-graph steps from first-run onboarding Onboarding now goes from Permissions straight to the completion screen. The speech profile is recorded from Settings instead, and the memory-graph preview (with its background graph prebuild) is gone. The two step widgets are deleted; their page indices stay as placeholders like the other retired steps so the existing page constants keep working. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): ease the mic glow shut as the finished speech profile fades The white glow behind the device graphic vanished the instant the upload began. It now stays through the upload and hold and eases down to nothing over the same 450 ms in which the recording view fades into All done. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): hold the mic glow at its last size until it eases out The glow followed the live mic level, which drops to zero the instant the microphone stops after the recording ends, so it snapped down to its resting size before the ease-out. The frozen recording view now also captures the last mic level, so the glow holds still and then eases shut with the fade. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * Revert "feat(app): drop the speech-profile and memory-graph steps from first-run onboarding" This reverts commit 4ebcb3d2c773747051dfeb2d519fe88aeb1faea0. * feat(backend): tune speaker verification from measured enrollments and retire the SpeechBrain matcher Speaker identification rejected most of the owner's own audio. The verification threshold (0.45 cosine distance) was copied from a clean-studio VoxCeleb figure; an offline bench over real enrollments in the speech-profiles bucket (229 users with a current profile plus an older one, 16 with extra recordings, 60 taught persons, 400 impostors; wespeaker-voxceleb-resnet34-LM, the diarizer's /v2/embedding model) puts same-user cross-session distance at a median of 0.40-0.53 and other users at 0.93. At 0.45 the owner was rejected 37-71% of the time at a 0.0% false-accept rate; the equal-error threshold is ~0.78. Same-session audio matched at either value, which is why the old constant looked fine in demos. - New utils/stt/speaker_match.py owns the policy (numpy only, shared by the live socket and the sync pipeline): threshold 0.65, plus a 0.10 margin over the runner-up so the owner is not guessed as a taught household member. - Live sessions pool up to three recent clips per diarized speaker and decide on the centroid once 5 s of clip audio has accumulated, instead of letting the first 2 s clip that lands under the threshold stick for the session. - Both surfaces log one structured speaker_id_decision line (best, runner-up, evidence, accepted) so the prod distribution can be checked against the bench from a day of logs. - The bench scripts live in backend/scripts/speaker_id_bench for reruns; user audio never leaves the machine running them. - Retire the dead SpeechBrain speaker-identification path: modal/speech_profile_modal, utils/stt/speech_profile (zero production callers), the /v1/speaker-identification route, HOSTED_SPEECH_PROFILE_API_URL in every chart/env, the speechbrain dependency, the shared-package COPY lines in the modal image, the dev-harness and e2e fakes, and the unused ListenLimits.speaker_id_target_audio field. Drop the now-unused is_same_speaker/find_best_match/bytes helpers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(backend): keep speaker_match real in sync test isolation, allowlist its import cost utils/sync/pipeline.py now imports utils.stt.speaker_match, but the hand-maintained heavy_deps mock list in test_sync_cloud_tasks.py and test_sync_v2.py didn't know about it, so `from utils.stt.speaker_match import select_speaker_match` raised ModuleNotFoundError: 'utils.stt' is not a package once utils.stt was replaced with a MagicMock. Real-import speaker_match (pure, dependency-free, like utils.stt.outcomes) instead of stubbing it, since a MagicMock decision object would also break the %.3f log formatting on decision.best_distance/runner_up_distance. Also allowlist test_speaker_match.py::test_short_clips_are_pooled_before_a_live_decision in the fast-unit duration guard: it's the first test in the file to import routers.listen.speakers, so it amortizes that module's FastAPI router-graph import cost, same structural pattern already documented for other files in the allowlist. * fix(speaker-id): preserve distinct evidence and household ambiguity Serialize live matches per speaker, subtract previously embedded audio, invalidate late session results, and keep all enrolled candidates in sync margin comparisons before enforcing unique assignment. Validation: 396 selected backend tests passed; Python typecheck has zero errors. Five live regression cases and two sync cases failed before the fixes. Changed sync expectations follow PR #12935's measured household-confusion margin. Failure-Class: new * fix(speaker-id): include owners in household benchmark cohorts Include available owner profiles even outside legacy, additional, and impostor cohorts. Distinguish offline benchmark evidence from deployed accuracy. Validation: synthetic manifest regression passes for owners outside other cohorts and people without an owner profile. No private audio or threshold retuning. Failure-Class: new * fix(speech-profile): bound native recognition and discard stale work Use one native completion owner so availability deadlines do not wait for shared model downloads and recognition cleanup precedes timeout completion. Serialize legacy recognition callbacks on the main queue. Propagate native failures to retain audio for retry; remove the polling Future timeout that allowed overlapping work. Scope fallback availability and polling results to their recording session. Validation: full Flutter suite 1836 passed, 5 skipped; analyzer ratchet passed. Native deadline behavioral tests pass and are registered in the existing manifest. Native speech code typechecks for iOS 15 deployment with Flutter boundary stubs; no full iPhone build or live enrollment claim. Preflight passed 53 selected checks. Failure-Class: new * fix(l10n): translate speech-profile flow in every supported locale Translate the eight speech-profile keys across all 48 non-English ARBs and fill two inherited missing keys exposed by generation. Use device-neutral speech recognition guidance and regenerate localization output from source catalogs. Validation: flutter gen-l10n reports zero untranslated messages; owner-name placeholders and complete catalog coverage verified. Full Flutter suite passed. Failure-Class: new * fix(speaker-id): require persisted speech profile before the redo admission bypass cubic P1: the client-supplied speech_profile_redo flag alone proved nothing; any authenticated client could send it to skip the completed-account onboarding-provenance admission gate. The runtime now confirms the redo from durable state (an actually stored speech_profile.wav) before taking the bypass, and an unprovable claim falls through to the provenance admission, failing closed when the check errors. Adds a regression test asserting a redo claim without a persisted profile is judged by the gate. * fix(listen): gate the onboarding fresh-conversation path on server admission cubic P2: onboarding=enabled is a client hint, yet prepare() took the fresh-conversation shortcut on the raw flag even when _bootstrap refused to admit the session — a client could dodge the existing-conversation lookup with a query parameter. The path now requires the runtime's onboarding_admitted (also true for the authorized Settings redo); an unadmitted claim keeps an ordinary session's behavior. Adds a regression test for the unadmitted path. * fix(onboarding): queue segments that arrive during AI answer checks cubic P2: is_checking_answer stayed set across up to three awaited LLM calls in _check_answer, and on_segments_received dropped everything spoken in that window, so answers covering later topics could be lost. Segments received while a check is in flight are now queued and replayed when it finishes, re-entering the normal accumulate-and-timer flow. Adds a regression test. * fix(speaker-id-bench): report the production threshold and true impostor rates cubic P2 x2: score.py evaluated the retired 0.45 operating point while the README and shipped policy (SPEAKER_MATCH_THRESHOLD) sit at 0.65, making its false-reject/false-accept and live-decision numbers misleading; and cohort-C impostor distances included the current user's own owner profile when that user was also sampled as an impostor, folding owner-vs-own-person confusion into the random-impostor sweep. score.py now pins T to the production 0.65 and formats every label from it; sweep.py excludes each cohort-C user's own profile from their impostor pool (the confusion keeps its dedicated diagnostic). Owner profiles for cohort C were already added to the cohort inputs by 8126713612. * fix(speech-profile): close startup, playback, and socket adoption races cubic review follow-ups still present after 1d7a2fd917: - page.dart: _isCheckingAvailability is now held until the entire startup path exits (dialogs, codec lookup, stopDeviceRecording, initialise), not just the availability round-trip, so a second tap cannot race socket and microphone init; context/mounted are rechecked after the language dialog and before initialise. - page.dart: profile playback deactivates the activated audio session on every teardown path (stop, natural completion, failure after activation, disposal) instead of leaving media routing active. - speech_profile_provider: a socket created while the session was closed or reset is discarded instead of adopted, which previously leaked a live backend session stop() never saw. - transcription_service: the speech-profile on-device fallback forwards raw audio per config.sendRawAudioToOmi, matching the conversation composite, instead of hardcoding every frame onto the Omi socket; suggested transcripts still flow and keep the backend session clock alive. Pinned by a factory test. * docs(app): keep AGENTS.md within its lean-budget ratchet after the main merge The merge combined this PR's on-device speech pointer with main's profile-build-mode and batch-contract lines, pushing app/AGENTS.md past its agents-md-lean budget (11747 > 11500 bytes). Tightens wording without dropping any fact: the batch-writer guarantee detail lives in the manifest reason and the ruby test itself; the other compressions are same-fact rewording. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Nathan Cheng <nathanjcx@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Nathan <nathan@Nathans-MacBook-Air.local> | 8 小时前 | |
fix(app): shorten the home chat bar placeholder to "Ask Omi" (#13024) The bar shares its row with the mic and record buttons, so "Ask Omi anything about your life..." was cut off mid-word on a phone. Use the existing localized askOmi string, which every locale already carries. Claude-Session: https://claude.ai/code/session_01BViCmmJpJ3FaExYyxD2TJx Co-authored-by: Nathan <nathan@Nathans-MacBook-Air.local> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> | 1 小时前 | |
Merge remote-tracking branch 'origin/main' into feat/phone-mic-transcribe-later # Conflicts: # app/analysis_baseline.json # app/lib/l10n/app_ar.arb # app/lib/l10n/app_be.arb # app/lib/l10n/app_bg.arb # app/lib/l10n/app_bn.arb # app/lib/l10n/app_bs.arb # app/lib/l10n/app_ca.arb # app/lib/l10n/app_cs.arb # app/lib/l10n/app_da.arb # app/lib/l10n/app_de.arb # app/lib/l10n/app_el.arb # app/lib/l10n/app_en.arb # app/lib/l10n/app_es.arb # app/lib/l10n/app_et.arb # app/lib/l10n/app_fa.arb # app/lib/l10n/app_fi.arb # app/lib/l10n/app_fr.arb # app/lib/l10n/app_he.arb # app/lib/l10n/app_hi.arb # app/lib/l10n/app_hr.arb # app/lib/l10n/app_hu.arb # app/lib/l10n/app_id.arb # app/lib/l10n/app_it.arb # app/lib/l10n/app_ja.arb # app/lib/l10n/app_kn.arb # app/lib/l10n/app_ko.arb # app/lib/l10n/app_localizations.dart # app/lib/l10n/app_localizations_ar.dart # app/lib/l10n/app_localizations_be.dart # app/lib/l10n/app_localizations_bg.dart # app/lib/l10n/app_localizations_bn.dart # app/lib/l10n/app_localizations_bs.dart # app/lib/l10n/app_localizations_ca.dart # app/lib/l10n/app_localizations_cs.dart # app/lib/l10n/app_localizations_da.dart # app/lib/l10n/app_localizations_de.dart # app/lib/l10n/app_localizations_el.dart # app/lib/l10n/app_localizations_en.dart # app/lib/l10n/app_localizations_es.dart # app/lib/l10n/app_localizations_et.dart # app/lib/l10n/app_localizations_fa.dart # app/lib/l10n/app_localizations_fi.dart # app/lib/l10n/app_localizations_fr.dart # app/lib/l10n/app_localizations_he.dart # app/lib/l10n/app_localizations_hi.dart # app/lib/l10n/app_localizations_hr.dart # app/lib/l10n/app_localizations_hu.dart # app/lib/l10n/app_localizations_id.dart # app/lib/l10n/app_localizations_it.dart # app/lib/l10n/app_localizations_ja.dart # app/lib/l10n/app_localizations_kn.dart # app/lib/l10n/app_localizations_ko.dart # app/lib/l10n/app_localizations_lt.dart # app/lib/l10n/app_localizations_lv.dart # app/lib/l10n/app_localizations_mk.dart # app/lib/l10n/app_localizations_mr.dart # app/lib/l10n/app_localizations_ms.dart # app/lib/l10n/app_localizations_nl.dart # app/lib/l10n/app_localizations_no.dart # app/lib/l10n/app_localizations_pl.dart # app/lib/l10n/app_localizations_pt.dart # app/lib/l10n/app_localizations_ro.dart # app/lib/l10n/app_localizations_ru.dart # app/lib/l10n/app_localizations_sk.dart # app/lib/l10n/app_localizations_sl.dart # app/lib/l10n/app_localizations_sr.dart # app/lib/l10n/app_localizations_sv.dart # app/lib/l10n/app_localizations_ta.dart # app/lib/l10n/app_localizations_te.dart # app/lib/l10n/app_localizations_th.dart # app/lib/l10n/app_localizations_tl.dart # app/lib/l10n/app_localizations_tr.dart # app/lib/l10n/app_localizations_uk.dart # app/lib/l10n/app_localizations_ur.dart # app/lib/l10n/app_localizations_vi.dart # app/lib/l10n/app_localizations_zh.dart # app/lib/l10n/app_lt.arb # app/lib/l10n/app_lv.arb # app/lib/l10n/app_mk.arb # app/lib/l10n/app_mr.arb # app/lib/l10n/app_ms.arb # app/lib/l10n/app_nl.arb # app/lib/l10n/app_no.arb # app/lib/l10n/app_pl.arb # app/lib/l10n/app_pt.arb # app/lib/l10n/app_ro.arb # app/lib/l10n/app_ru.arb # app/lib/l10n/app_sk.arb # app/lib/l10n/app_sl.arb # app/lib/l10n/app_sr.arb # app/lib/l10n/app_sv.arb # app/lib/l10n/app_ta.arb # app/lib/l10n/app_te.arb # app/lib/l10n/app_th.arb # app/lib/l10n/app_tl.arb # app/lib/l10n/app_tr.arb # app/lib/l10n/app_uk.arb # app/lib/l10n/app_ur.arb # app/lib/l10n/app_vi.arb # app/lib/l10n/app_zh.arb | 1 个月前 | |
fix(app): address review on the untethered launch fix - Notice on the phone names the canonical `bash setup.sh ios` invocation from app/ instead of a repo-root path. - setup.sh warning describes the shipped behavior (engine-unavailable notice, not a crash) and only fires for the dev flavor, since its remedy builds the dev flavor. - Drop the GOOGLE_REVERSE_CLIENT_ID pin from devProfile/devRelease so Custom.xcconfig keeps owning that value. Failure-Class: none Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YU5BMEAfcFxNSWmLLDNfZ3 | 17 小时前 | |
harden(app): fail fast on mobile build profile drift (#12559) | 6 天前 | |
fix(app): breathing room above conversation markdown headers (#12933) * fix(app): add breathing room above conversation markdown headers flutter_markdown defaults h*Padding to zero, so ##/###/#### sections in conversation summaries sat flush against the previous block (#5622). Failure-Class: none * chore: retrigger CI after flaky local_segment_store test Dart Analyze & Tests failed once on CaptureProvider listener persists when a live session is active (empty digest). Unrelated to markdown header spacing; passes locally and on main. Empty commit to re-run. Failure-Class: none * fix(app): apply markdown header padding to h5 and h6 Match the existing breathing-room const across every header level flutter_markdown defaults to zero padding. Failure-Class: none * chore: drop accidental Flutter ephemeral Package.swift change Failure-Class: none | 1 小时前 | |
rename comments, links, plugins and dcoumentation refs | 1 年前 | |
Add release process guardrails | 2 个月前 | |
Add ARB 4-space indent guard to pre-commit hook (#5273) | 6 个月前 | |
Maps: keyless static previews + native-map handoff; fix Unknown location labels (#12841) * feat(backend): authed static-map proxy route with Redis-cached renders GET /v1/static-map renders dark-styled Google Static Maps images server-side (the only Maps key stays server-restricted) and caches the bytes in Redis keyed by the quantized pin set + size, so repeat renders of the same place across users/sessions cost one upstream call per distinct pin set per week. utils/static_map.py is the single provider seam for app map previews; swapping providers touches that module only. - pins parsed/bounded/de-duplicated/sorted, capped at 50; ~11m quantization makes users at the same place share one cached image - one pin centers at street zoom; several pins use provider auto-fit - failures return 502 (never cached) - the app renders its offline pin-dot canvas; auth prevents an open proxy on the project key - rate policy static_map:get (240/h per uid) stops hot loops; cached hits are one Redis read Tests: tests/routers/test_static_map.py (auth 401, 400 malformed pins, 502 upstream failure, 200 + private cache headers, cache hit/miss/order- insensitivity/no-cache-on-failure/missing key tolerances). Verified: .venv/bin/python -m pytest tests/routers/test_static_map.py -q -> 14 passed; scan_async_blockers clean. * fix(backend): enrich sync-path geolocation at the pipeline coordinator Failure-Class: FC-sync-geolocation-missing-address | new | none Offline-synced conversations were created with raw coordinates and no address: REST create, developer API, integration ingest, and live finalization all run resolve_geolocation, but the sync path shipped raw coords straight through. Recaps built from synced conversations showed 'Unknown' timeline rows. _run_full_pipeline_background_async - the one coordinator both the inline and Cloud Tasks dispatch branches call - now enriches the job's geolocation once, before any segment is processed and before the concurrency gate (no slot held during the geocode call). The resolver keeps the caller's exact coordinates and returns its input unchanged on any geocode miss/error, so a failure never drops the location. Tests: tests/unit/test_sync_geolocation_enrichment.py drives the real coordinator with a fake geocoder - one geocode per job (not per segment), enriched value reaches every segment, raw geolocation survives a geocode failure, None passthrough. Fixture stub lists in test_sync_v2 / test_sync_transcription_prefs gained the utils.conversations.location stub with an identity passthrough (the real resolver's miss behavior). Verified: pytest tests/unit/test_sync_geolocation_enrichment.py test_sync_v2.py test_sync_transcription_prefs.py -> all passed. * fix(backend): fill empty daily-summary pin addresses at read time Failure-Class: FC-sync-geolocation-missing-address | new | none generate_comprehensive_daily_summary copied c.geolocation.address verbatim, so pins from conversations created before write-time enrichment (the sync path) rendered as 'Unknown' recap timeline rows. The pins loop now fills an empty address through the shared ~100m- rounded geocode cache - the same entries write-time enrichment writes, so an already-enriched day costs no extra upstream call. All three callers are sync contexts, so the sync geocoder is used. A geocode miss or error leaves the address empty and keeps the pin (the app labels it 'Unknown'); regenerating a summary retroactively fixes history with no migration. Tests: tests/unit/test_daily_summary_location_address_fill.py (filled from geocoder with exact coords preserved, present address skips the geocoder, miss and exception both keep the pin address-less). Verified: pytest tests/unit/test_daily_summary_location_address_fill.py test_daily_summary_zero_coordinate_locations.py -> all passed. * feat(app): OmiMapPreview - one static-map widget with an offline pin-dot canvas Every map preview now funnels through a single widget backed by the authed backend static-map proxy (GET /v1/static-map): URL built by buildOmiStaticMapUrl (pins quantized to 4 decimals to match the server's cache quantization, deduped, capped at 50), image fetched with the session's Authorization header, and a deterministic dark canvas with one white dot per pin while loading, offline, or on any failure - never an error state. Product direction: previews in-app, tapping opens the native map app via MapsUtil.launchMap. The conversation detail geolocation card migrates from the client-keyed getMapImageUrl URL to this widget; getMapImageUrl is removed (maps_util keeps launchMap + place URL), taking the last direct client-side Google Static Maps call with it. Verified: flutter test test/widgets/omi_map_preview_test.dart test/widgets/daily_summary_card_test.dart (widget+URL builder tests); flutter analyze clean on changed files; analyze_ratchet.sh passed. * feat(app): replace CARTO tile maps with static previews; drop flutter_map CARTO began enforcing API keys on basemaps.cartocdn.com (keyless requests now return watermarked tiles), and Omi has no CARTO key. All three flutter_map surfaces now render OmiMapPreview instead - zero remaining tile traffic, no keyed tile provider in the app: - conversation map page: static preview of every ~100m cluster anchor (tap opens the native map app) above a grouped 'conversations at this place' list; single-conversation places open the conversation directly, multi-conversation places keep the cluster bottom sheet; grouping logic and cluster-row keys unchanged - daily summary card: 96px preview strip of the day's pins in the recap carousel (the highest-frequency map surface) - now one cached server-side render per distinct pin set instead of dozens of tiles per card - daily summary detail 'Your Day's Journey': 200px preview; image tap still opens the day's first stop (Apple Maps cannot take waypoints via map_launcher), per-stop timeline rows unchanged flutter_map is removed from pubspec (latlong2 stays - journey grouping uses it). The client-embedded Env.googleMapsApiKey goes with it: all static maps come from the server-restricted key behind the proxy now. envied outputs regenerated; test EnvFields stubs updated to match. Verified: bash test.sh / flutter test -> 1758 passed, 5 skipped, 0 failed; scripts/analyze_ratchet.sh passed (9 lint counts improved). * test(app): update map-surface tests for static previews - daily summary card: assert the preview strip, proxy URL pins/quantize/ dedupe, offline canvas fallback, and no-map-for-invalid-coordinates (was: tile provider request counting) - daily summary detail page: drop the TileProvider seam (widget no longer takes one) - conversation map groups: group-card key + cluster sheet rows replace marker keys; journey/env stubs gain the EnvFields change Verified: flutter test on all four files -> passed. * test(backend): stub utils.conversations.location in the cloud-tasks loader pipeline.py now imports async_resolve_geolocation at module scope; the cloud-tasks stub loader replaces its parent packages with MagicMocks, so the submodule import fails in file isolation (the earlier green run leaned on a module cached by another test file). Adds the explicit stub with an identity passthrough - the real resolver's miss behavior. Verified: BACKEND_UNIT_TEST_FILE_LIST=<this file> bash test.sh -> 91 passed. * chore(app): drop GOOGLE_MAPS_API_KEY from the env template The client no longer embeds a Maps key - all static-map renders go through the server-restricted key behind GET /v1/static-map. * test(backend): provide redis r on the usage-tracking stub set external_integrations now imports utils.conversations.location (daily- summary address fill), which imports the redis client symbol; the usage-tracking stub environment replaces database.redis_db with an empty module, so the import fails in file isolation. Give the stub an r. Verified: BACKEND_UNIT_TEST_FILE_LIST=<this file> bash test.sh -> 21 passed. * docs(backend): note the static-map provider URL budget with current limits * chore(backend): declare the static-map route policy in the manifest * fix(backend): drop the redundant isinstance on the typed pins parameter * fix(app): hold the preview canvas until the auth header resolves The first build fired the authed proxy request before getAuthHeader() resolved; CachedNetworkImage keys its cache by URL, so the later header-arriving setState never re-fetched - a permanent 401 fallback canvas. The widget now renders the pin-dot canvas (which already doubles as the error/offline path) until the header resolves, and only then mounts the network image with the Authorization header attached. The header resolver is injectable (authHeaderProvider) so tests control when it completes; an explicit imageUrl (test seam) skips the gate. Tests: new regression case asserts no CachedNetworkImage while auth is unresolved and the authed image after the completer fires; the card tests repoint URL assertions at the preview widget's pins (URL building has its own unit tests). Verified: flutter analyze clean; flutter test (full) -> 1759 passed. * fix(app): restore the conversation_map_marker automation key on group rows The per-place tappable key predates the static preview; keep it stable for automation (the PR brief promises preserved keys). Verified: flutter test test/unit/conversation_map_groups_test.dart passed. * fix(backend): normalize static-map dimensions, dedup render stampedes, count canvas fallbacks Review findings on the static-map proxy: - Dimension handling: one proportional scale factor (min(1, 640/w, 640/h)) computed once in fetch_static_map replaces the independent per-axis clamps - aspect is preserved and every request that differs only by scale normalizes onto the SAME cache entry (cache key and provider URL both use the effective size). - Stampede dedup: after a miss, a per-key render lock (r.set nx, 30s TTL) elects one renderer; concurrent misses poll the cache (0.25s interval, 15s budget) for the holder's result. Lock-held-timeout and lock-unavailable (Redis broken) both fail OPEN to an unlocked render - a lost lock never becomes a 502, and a broken Redis never pays the 15s wait budget. - record_fallback (component=static_map, provider_static_map -> client_pin_canvas, outcome=degraded) fires before the 502 so the degrade is counted in the shared telemetry. Tests: proportional normalization + shared cache entry, oversized fetch normalizes (size=640x150), concurrent misses render once, waiter polls a foreign lock holder to its hit, wait-timeout renders unlocked, 422s for out-of-bounds width/height via TestClient (Query contract), fallback telemetry kwargs. Verified: pytest tests/routers/test_static_map.py -> 21 passed; pyright (scripts/typecheck.sh) 0 errors, no findings in changed files. * fix(backend): cap daily-summary geocode attempts; boost-exempt static_map:get - The read-time address fill now bounds geocode ATTEMPTS at 10 per summary generation: cache hits are cheap but attempts are the deterministic wall-clock bound (10 x the geocoder's 10s worst case stays inside the job budget). Pins past the cap keep an empty address and the app's 'Unknown' fallback. Test: 11 empty-address pins -> 10 filled, 11th untouched, 10 geocoder calls. - static_map:get joins the boost-exempt rate policies: under prod's RATE_LIMIT_BOOST the 240/h hot-loop cap would otherwise resolve to 24k/h and stop protecting the billable provider calls. Verified: pytest test_daily_summary_location_address_fill.py -> 5 passed; utils.rate_limit_config import shows static_map:get in BOOST_EXEMPT_POLICIES. * test(backend): self-contained enrichment fakes; model the resolver short-circuit - test_sync_geolocation_enrichment no longer imports the transcription-prefs fixture module: sharing it loaded real heavyweight modules (GCS/Firestore protos, anthropic/jiter) inside the stub window, and stub_modules' teardown evicted them so later files broke on re-import (duplicate proto registration, jiter NameError) when run in one process. The fakes are now minimal and self-contained - every heavy leaf is stubbed, light enum modules (stt.outcomes, sync.lanes, sync.telemetry) stay real. - The None-geolocation test now models the real resolver's short-circuit (falsy input returns immediately, attempts recorded only for truthy input) and asserts: resolver called exactly once with None AND zero geocode attempts. Docstring states what is actually proven. - One-line caveat at the cloud-tasks loader injection site: new submodule imports must be added to heavy_deps explicitly (MagicMock parents do not resolve submodules). Migrating that hand-rolled loader to stub_modules would cascade across its 37 call sites - deliberately left as-is. Verified: pytest on the combined focused group (static_map, summary fill, sync enrichment, cloud tasks, sync v2, transcription prefs, geocode resolve) -> 372 passed in one process. | 1 天前 | |
feat: Maestro E2E functional test suite for Omi app (#6283) * feat: add Maestro E2E functional test suite for Omi app (#3857) - 10 Maestro flow files covering all core app functionality: - Login & Sign-in (Google OAuth) - Onboarding (name, permissions, device skip) - Conversations (list, detail, tabs, CRUD, folder filters) - Memories (view, create, edit, delete) - Chat (send message, AI response) - Apps/Plugins (marketplace browse, detail) - Settings (navigation, key sections) - Device Connection (BLE scan & pair) [device_required] - Recording & Transcription [device_required] - Logout (sign out, verify auth screen) - Tag system: 'core' (simulator-safe) vs 'device_required' (needs Omi HW) - Runner script with tag filtering, parallel-safe output, JUnit XML + Markdown reports - Unified test.sh entry point: --e2e flag for Maestro, default for unit/widget - Clean, DRY flow structure — no code duplication across flows - .gitignore updated to exclude generated reports * fix: address code review — apply global config, isolate screenshots, robust tag parsing - P1: Pass --config global.yaml to maestro test so onFlowStart hooks fire - P1: Use find -newer to only copy screenshots from current flow run - P2: Replace brittle grep -A5 tag extraction with awk block parser * fix: chat flow — meaningful AI response assertion + correct pressKey syntax - P1: Replace text: '.*' (matches anything) with actual response keywords - P1: Fix pressKey optional: true to use block-mapping syntax - Wait for loading indicators to disappear before asserting response * fix: correct pressKey optional syntax in chat flow --------- Co-authored-by: hniane1 <hniane1@users.noreply.github.com> Co-authored-by: Hashim Niane <hahimniane@users.noreply.github.com> | 2 个月前 | |
Speaker identification: measured threshold + margin, live clip pooling, SpeechBrain retirement; carries #12531 without the onboarding-step removal (#12935) * fix: unblock speech-profile redo and STT pre-flight for already-onboarded accounts Rebased onto origin/main as a single commit. Keep both main's open_provider_selection_circuit and this PR's is_stt_available helpers, then regenerate OpenAPI clients from the rebased backend. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(app): fade transcript words in as they arrive on the speech-profile screens Add FadeInWordsText: a centered word Wrap where only the words appended since the previous render animate from transparent to opaque with a short stagger, existing words stay put, and a rewritten transcript re-reveals from the start. Both the onboarding speech-profile step and the Settings redo page adopt it in the next commit so the live transcript reads the same whether the words come from the server or the on-device fallback. Verification: flutter test test/widgets/fade_in_words_text_test.dart (4 passed); observed on an iPhone 16 Pro via hot reload while dictating. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F1zmRTRh3DV8NsM3QUrrAB * feat(app): fall back to on-device speech recognition when server STT is unavailable The speech-profile question flow (onboarding step and Settings redo) needs a transcript only to drive the questions and progress; the voice print itself is computed server-side from the WAV uploaded at finalize(). So when the backend's streaming STT is down, transcribe on the phone instead of dead-ending: - SpeechProfileProvider gains a local-STT mode. It is entered up front when the stt-availability pre-flight fails, or mid-session after the existing three 1011 closes with no captured speech (previously STT_UNAVAILABLE). The socket becomes the existing CompositeTranscriptionSocket: an on-device polling primary (Apple speech on iOS, downloaded Whisper on Android) forwarding suggested_transcript frames to the backend listen socket in custom_stt mode, which the OnboardingHandler already consumes like server STT output. No backend change; a receiver regression test pins that seam. - iOS on-device recognition hardening (AppDelegate.swift): resolve the app's bare language code to an installed on-device locale (a recognizer built from "en" failed every request with kAFAssistantErrorDomain 1101); reply exactly once per clip on final result, error, or a 20 s timeout, keeping partial results; and expose onDeviceAvailable, which probes a silent clip so a phone with Siri and Dictation disabled (kLSRErrorDomain 201) is reported as "no local STT" instead of entering the fallback blind. - PurePollingSocket bounds each transcribe() with a 30 s timeout. A provider that never answered left the processing flag set forever and silently stopped transcription for the rest of the session; now the audio is requeued and the next tick retries. This also protects the main app's on-device mode. - When neither server nor on-device STT is available, the pre-flight dialog now says to check the connection or turn on Dictation. - Speech-profile UI: subtler mic-level glow, and the live transcript uses the new fade-in words widget. Verification: - flutter test (full suite): 1731 passed, 5 skipped; scripts/analyze_ratchet.sh passed - new tests: speech_profile_provider_test (5 fallback cases), pure_polling_test (hung-provider timeout), fade_in_words_text_test (4), backend test_onboarding_question_start (suggested_transcript reaches the transcript queue only in custom-STT mode) - live on iPhone 16 Pro against the local dev harness with the STT primary forced unavailable: session connects with custom_stt+onboarding flags, Apple on-device recognition returns the spoken answer (~180 ms per clip) and it is forwarded to the backend; with Dictation disabled the probe reports unavailable and the dialog appears. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F1zmRTRh3DV8NsM3QUrrAB * feat(app): use SpeechAnalyzer for on-device speech on iOS 26 SFSpeechRecognizer's on-device mode fails with kLSRErrorDomain 201 whenever Siri and Dictation are turned off in Settings, which is what produced the "turn on Dictation" pre-flight dialog in the speech-profile fallback. iOS 26's SpeechAnalyzer/SpeechTranscriber has no such dependency: the language model is an asset the app installs itself through AssetInventory. - transcribe: on iOS 26 run the clip through SpeechAnalyzer (preset .transcription, analyzeSequence(from:) + finalizeAndFinish), falling back to the SFSpeechRecognizer path only if the analyzer throws. - onDeviceAvailable: report true when a supported locale's model is installed or installs within 8 s; a longer download keeps going in the background and the first transcribe() waits for it. Concurrent callers share one download. - SFSpeechRecognizer remains the path for iOS 15-18. Verified on an iPhone (iOS 26.6.1) with Dictation off: the speech-profile redo enters local-STT mode and transcribes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(app): speech profile talks through three topics and completes on a word target Speech-profile recording (onboarding step and Settings redo) no longer walks one question at a time with a percentage bar. Instead: - A compact white-outlined card headed "Answer with your voice:" lists three topics (where you live, what you do for work, your long-term goal), and a thin bar under it fills as the user speaks. Reaching SpeechProfileProvider.targetWordCount (60 spoken words) finalizes the recording; the backend's onboarding_complete event no longer does, so "bar full" and "done" are the same moment. Omi's own question segments are excluded from the count. - The live transcript is bottom-anchored in a box exactly three lines tall above the card, so whole lines scroll off the top and nothing overlaps. - The Play button on the Settings page plays the saved profile audio in place (just_audio) and turns into Stop, instead of opening the samples page. Redo stops playback first. - Backend ONBOARDING_QUESTIONS is the same three topics, and OnboardingHandler keeps the transcript across questions so one stretch of speech can satisfy several of them. - Removed the unused percentage progress-bar widget and the "Skip this question" button; onboarding keeps "Skip for now". Tests: speech_profile_provider_test (word target fills, finalizes once, ignores Omi segments and the backend completion event); backend test_onboarding_talk_about_flow (one transcript answers every topic; the transcript is kept when it stops answering). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(dev-harness): advertise local-storage links on OMI_DEV_HOST A phone built against OMI_DEV_HOST could reach the backend but not the files it links to: OMI_LOCAL_STORAGE_BASE_URL was always http://127.0.0.1:<port>/_local/storage, so playing the saved speech profile from a device failed. The harness now derives a dev_advertise_host from OMI_DEV_HOST (loopback stays the default) and uses it only for that base URL; every other service address still binds and talks over loopback. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): keep the last three whole transcript lines instead of a clipped scroll The speech-profile transcript was a bottom-scrolled ListView clipped to a three-line box, so a sliver of the line above always showed at the top edge and read as cut-off text. FadeInWordsText now takes visibleLines: it replays the Wrap line breaking with measured word widths and builds only the words on the last N lines, so earlier lines drop off whole, nothing is clipped or scrolled, and words keep their reveal state while on screen. Both screens use visibleLines: 3 inside a fixed three-line, bottom-anchored area, moved a little further above the topics card. Test: fade_in_words_text_test covers short text showing everything, earlier lines dropping once the text exceeds three lines, and the shown words matching the line-break replay. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): measure transcript lines with the effective text style; 40-word target FadeInWordsText replayed the Wrap line breaking with the caller's raw style, but each word's Text inherits the ambient DefaultTextStyle (font family, weight) under that style, so the replay undercounted lines and the real layout could reach four lines and draw over the topics card. Measure with the same merged style, and clip the fixed three-line area on both screens as a safety net so a stray line can never overlap the card. Also lower SpeechProfileProvider.targetWordCount from 60 to 40 so the recording finishes sooner. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): keep the "<Name>'s Speech Profile" title on one line The title wrapped onto two lines for longer names; it now scales down to fit a single line instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): never clip the speech-profile transcript; raise it above the card The three-line transcript area was a fixed-height clipped box, so whenever the rendered lines ran taller than the fontSize*height estimate (text scaling, font metrics) the top line was cut off. FadeInWordsText already guarantees at most three lines, so the area now only has a three-line minimum height (scaled with the text scaler) and grows to its content instead of clipping. Both screens also keep more space between the transcript and the topics card. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(app): finish the speech profile after three sentences; no page-load spinner The recording now completes once the user has spoken three sentences (SpeechProfileProvider.targetSentenceCount, counted on ./!/? boundaries followed by a space or the end of the text, so "3.5" is not one) instead of a word count, and the bar under the topics card fills per sentence. The progress-bar widget is renamed SpeechProgressBar to match. The Settings speech-profile page no longer swaps its Play/Redo or Get Started buttons for a spinner while the page initialises or the STT pre-flight runs; the buttons stay put and startRecording() ignores taps until the check finishes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): play the saved speech profile on the loudspeaker The app's audio session is normally configured for recording, so tapping Play on the Settings speech-profile page routed the WAV to the quiet earpiece. Before playing, configure a playback-category session (default mode, media usage on Android) and play at full volume, so the profile comes out of the main speaker like any other media. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): start every speech-profile recording with an empty transcript Tapping Redo showed the previous recording's words (and counted them toward the sentence target) because nothing cleared the provider's transcript before a new session; only close() did, on leaving the page. initialise() now calls a new resetTranscript() first, which forgets the segments, text, progress, completion and upload flags without touching the audio storage it recreates right after. resetSegments() reuses it. Test: a completed session's transcript is gone after resetTranscript and the fresh session counts sentences from zero and can finalize again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): finish the speech profile after a pause, keeping the last sentence on screen Reaching the third sentence finalized immediately, which stopped the mic mid-utterance (the recognizers punctuate each clip, so a pause can read as a sentence end) and swapped the transcript for a spinner at once. Now: - After the target is reached the provider waits completionGrace (2 s) without new speech before finalizing, restarting the wait on every new segment, and finalizes at completionCap (8 s past the target) at the latest. Once fired it does not re-arm; resetTranscript() clears it. - Both screens keep the last three transcript lines visible through the upload and the All done state, so the final sentence lingers instead of vanishing. Tests: grace/cap timing under fakeAsync, no double finalize, and the reset test now elapses the grace. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(backend): give every speech-profile recording its own conversation Tapping Redo within two minutes of the previous attempt showed last time's words as soon as the user spoke again. The new listen socket attached to the still-open in-progress conversation from the previous attempt (same source, inside conversation_creation_timeout), so combine_segments() merged the first new segment into that conversation's last segment and the merged segment, old text included, was what the client received. LiveConversationController.prepare() now always creates a fresh in-progress conversation for onboarding_mode sessions (the onboarding step and the Settings redo both set it) instead of consulting the in-progress pointer. Ordinary listen sessions are unchanged. Test: test_listen_speech_profile_fresh_conversation.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): cross-fade the speech-profile recording UI into a plain All done button Keeping the transcript on screen through the upload made it pop back in on its own above the spinner and the All done button. The Settings page now cross-fades (450 ms) from the recording UI (transcript, topics card, bar) to nothing while uploading and then to the All done button, which is the same black capsule with a plain white border as the other buttons instead of the gradient box. The onboarding step likewise no longer shows the transcript in its upload and All done states. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * chore(app): remove the speech-samples page and dead progress-state code The Play button now plays the saved profile in place, so the samples page and its provider became unreachable (CI dead-code ratchet). Also drop the scroll controllers and SCROLL_DOWN signal the old clipped transcript used, and the word-count progress-message state (SpeechProfileProgressState, percentageCompleted, questionProgress) nothing reads any more. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): hold the finished speech-profile recording on screen before All done After the third sentence the final words disappeared as soon as the upload began. Both screens now keep the finished recording (last words, topics card, full bar) on screen through the upload and for a further 1.5 s (allDoneHold) after the profile is saved, then cross-fade into the All done button. Onboarding's upload spinner row and its now-unused loading-text helper are gone; Skip for now hides once recording ends. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): fade the finished speech-profile recording out as one block Parts of the finished recording could change on their own before the cross-fade (the transcript and the mic disclaimer are built from live provider state that finalize() and its callbacks touch), so they did not disappear together. Both screens now snapshot the recording view (last words, no-device flag) the moment recording ends and build from that until a new recording starts, and the onboarding step's All done switch is now the same AnimatedSwitcher cross-fade as the Settings page, so the words, the topics card, the bar and the disclaimer fade out at the same time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): keep the speech-profile bar full until the finished recording fades finalize() clears the provider's text once the profile is saved, and the bar derived its value from that text, so it dropped back to zero before the cross-fade. The frozen recording view now pins the bar at full from the moment recording ends. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * feat(app): drop the speech-profile and memory-graph steps from first-run onboarding Onboarding now goes from Permissions straight to the completion screen. The speech profile is recorded from Settings instead, and the memory-graph preview (with its background graph prebuild) is gone. The two step widgets are deleted; their page indices stay as placeholders like the other retired steps so the existing page constants keep working. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): ease the mic glow shut as the finished speech profile fades The white glow behind the device graphic vanished the instant the upload began. It now stays through the upload and hold and eases down to nothing over the same 450 ms in which the recording view fades into All done. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * fix(app): hold the mic glow at its last size until it eases out The glow followed the live mic level, which drops to zero the instant the microphone stops after the recording ends, so it snapped down to its resting size before the ease-out. The frozen recording view now also captures the last mic level, so the glow holds still and then eases shut with the fade. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017iFNr6rZDcD1FkCS3dJ3Cm * Revert "feat(app): drop the speech-profile and memory-graph steps from first-run onboarding" This reverts commit 4ebcb3d2c773747051dfeb2d519fe88aeb1faea0. * feat(backend): tune speaker verification from measured enrollments and retire the SpeechBrain matcher Speaker identification rejected most of the owner's own audio. The verification threshold (0.45 cosine distance) was copied from a clean-studio VoxCeleb figure; an offline bench over real enrollments in the speech-profiles bucket (229 users with a current profile plus an older one, 16 with extra recordings, 60 taught persons, 400 impostors; wespeaker-voxceleb-resnet34-LM, the diarizer's /v2/embedding model) puts same-user cross-session distance at a median of 0.40-0.53 and other users at 0.93. At 0.45 the owner was rejected 37-71% of the time at a 0.0% false-accept rate; the equal-error threshold is ~0.78. Same-session audio matched at either value, which is why the old constant looked fine in demos. - New utils/stt/speaker_match.py owns the policy (numpy only, shared by the live socket and the sync pipeline): threshold 0.65, plus a 0.10 margin over the runner-up so the owner is not guessed as a taught household member. - Live sessions pool up to three recent clips per diarized speaker and decide on the centroid once 5 s of clip audio has accumulated, instead of letting the first 2 s clip that lands under the threshold stick for the session. - Both surfaces log one structured speaker_id_decision line (best, runner-up, evidence, accepted) so the prod distribution can be checked against the bench from a day of logs. - The bench scripts live in backend/scripts/speaker_id_bench for reruns; user audio never leaves the machine running them. - Retire the dead SpeechBrain speaker-identification path: modal/speech_profile_modal, utils/stt/speech_profile (zero production callers), the /v1/speaker-identification route, HOSTED_SPEECH_PROFILE_API_URL in every chart/env, the speechbrain dependency, the shared-package COPY lines in the modal image, the dev-harness and e2e fakes, and the unused ListenLimits.speaker_id_target_audio field. Drop the now-unused is_same_speaker/find_best_match/bytes helpers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(backend): keep speaker_match real in sync test isolation, allowlist its import cost utils/sync/pipeline.py now imports utils.stt.speaker_match, but the hand-maintained heavy_deps mock list in test_sync_cloud_tasks.py and test_sync_v2.py didn't know about it, so `from utils.stt.speaker_match import select_speaker_match` raised ModuleNotFoundError: 'utils.stt' is not a package once utils.stt was replaced with a MagicMock. Real-import speaker_match (pure, dependency-free, like utils.stt.outcomes) instead of stubbing it, since a MagicMock decision object would also break the %.3f log formatting on decision.best_distance/runner_up_distance. Also allowlist test_speaker_match.py::test_short_clips_are_pooled_before_a_live_decision in the fast-unit duration guard: it's the first test in the file to import routers.listen.speakers, so it amortizes that module's FastAPI router-graph import cost, same structural pattern already documented for other files in the allowlist. * fix(speaker-id): preserve distinct evidence and household ambiguity Serialize live matches per speaker, subtract previously embedded audio, invalidate late session results, and keep all enrolled candidates in sync margin comparisons before enforcing unique assignment. Validation: 396 selected backend tests passed; Python typecheck has zero errors. Five live regression cases and two sync cases failed before the fixes. Changed sync expectations follow PR #12935's measured household-confusion margin. Failure-Class: new * fix(speaker-id): include owners in household benchmark cohorts Include available owner profiles even outside legacy, additional, and impostor cohorts. Distinguish offline benchmark evidence from deployed accuracy. Validation: synthetic manifest regression passes for owners outside other cohorts and people without an owner profile. No private audio or threshold retuning. Failure-Class: new * fix(speech-profile): bound native recognition and discard stale work Use one native completion owner so availability deadlines do not wait for shared model downloads and recognition cleanup precedes timeout completion. Serialize legacy recognition callbacks on the main queue. Propagate native failures to retain audio for retry; remove the polling Future timeout that allowed overlapping work. Scope fallback availability and polling results to their recording session. Validation: full Flutter suite 1836 passed, 5 skipped; analyzer ratchet passed. Native deadline behavioral tests pass and are registered in the existing manifest. Native speech code typechecks for iOS 15 deployment with Flutter boundary stubs; no full iPhone build or live enrollment claim. Preflight passed 53 selected checks. Failure-Class: new * fix(l10n): translate speech-profile flow in every supported locale Translate the eight speech-profile keys across all 48 non-English ARBs and fill two inherited missing keys exposed by generation. Use device-neutral speech recognition guidance and regenerate localization output from source catalogs. Validation: flutter gen-l10n reports zero untranslated messages; owner-name placeholders and complete catalog coverage verified. Full Flutter suite passed. Failure-Class: new * fix(speaker-id): require persisted speech profile before the redo admission bypass cubic P1: the client-supplied speech_profile_redo flag alone proved nothing; any authenticated client could send it to skip the completed-account onboarding-provenance admission gate. The runtime now confirms the redo from durable state (an actually stored speech_profile.wav) before taking the bypass, and an unprovable claim falls through to the provenance admission, failing closed when the check errors. Adds a regression test asserting a redo claim without a persisted profile is judged by the gate. * fix(listen): gate the onboarding fresh-conversation path on server admission cubic P2: onboarding=enabled is a client hint, yet prepare() took the fresh-conversation shortcut on the raw flag even when _bootstrap refused to admit the session — a client could dodge the existing-conversation lookup with a query parameter. The path now requires the runtime's onboarding_admitted (also true for the authorized Settings redo); an unadmitted claim keeps an ordinary session's behavior. Adds a regression test for the unadmitted path. * fix(onboarding): queue segments that arrive during AI answer checks cubic P2: is_checking_answer stayed set across up to three awaited LLM calls in _check_answer, and on_segments_received dropped everything spoken in that window, so answers covering later topics could be lost. Segments received while a check is in flight are now queued and replayed when it finishes, re-entering the normal accumulate-and-timer flow. Adds a regression test. * fix(speaker-id-bench): report the production threshold and true impostor rates cubic P2 x2: score.py evaluated the retired 0.45 operating point while the README and shipped policy (SPEAKER_MATCH_THRESHOLD) sit at 0.65, making its false-reject/false-accept and live-decision numbers misleading; and cohort-C impostor distances included the current user's own owner profile when that user was also sampled as an impostor, folding owner-vs-own-person confusion into the random-impostor sweep. score.py now pins T to the production 0.65 and formats every label from it; sweep.py excludes each cohort-C user's own profile from their impostor pool (the confusion keeps its dedicated diagnostic). Owner profiles for cohort C were already added to the cohort inputs by 8126713612. * fix(speech-profile): close startup, playback, and socket adoption races cubic review follow-ups still present after 1d7a2fd917: - page.dart: _isCheckingAvailability is now held until the entire startup path exits (dialogs, codec lookup, stopDeviceRecording, initialise), not just the availability round-trip, so a second tap cannot race socket and microphone init; context/mounted are rechecked after the language dialog and before initialise. - page.dart: profile playback deactivates the activated audio session on every teardown path (stop, natural completion, failure after activation, disposal) instead of leaving media routing active. - speech_profile_provider: a socket created while the session was closed or reset is discarded instead of adopted, which previously leaked a live backend session stop() never saw. - transcription_service: the speech-profile on-device fallback forwards raw audio per config.sendRawAudioToOmi, matching the conversation composite, instead of hardcoding every frame onto the Omi socket; suggested transcripts still flow and keep the backend session clock alive. Pinned by a factory test. * docs(app): keep AGENTS.md within its lean-budget ratchet after the main merge The merge combined this PR's on-device speech pointer with main's profile-build-mode and batch-contract lines, pushing app/AGENTS.md past its agents-md-lean budget (11747 > 11500 bytes). Tightens wording without dropping any fact: the batch-writer guarantee detail lives in the manifest reason and the ruby test itself; the other compressions are same-fact rewording. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Nathan Cheng <nathanjcx@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Nathan <nathan@Nathans-MacBook-Air.local> | 8 小时前 | |
Merge remote-tracking branch 'origin/main' into feat/phone-mic-transcribe-later # Conflicts: # app/analysis_baseline.json # app/lib/l10n/app_ar.arb # app/lib/l10n/app_be.arb # app/lib/l10n/app_bg.arb # app/lib/l10n/app_bn.arb # app/lib/l10n/app_bs.arb # app/lib/l10n/app_ca.arb # app/lib/l10n/app_cs.arb # app/lib/l10n/app_da.arb # app/lib/l10n/app_de.arb # app/lib/l10n/app_el.arb # app/lib/l10n/app_en.arb # app/lib/l10n/app_es.arb # app/lib/l10n/app_et.arb # app/lib/l10n/app_fa.arb # app/lib/l10n/app_fi.arb # app/lib/l10n/app_fr.arb # app/lib/l10n/app_he.arb # app/lib/l10n/app_hi.arb # app/lib/l10n/app_hr.arb # app/lib/l10n/app_hu.arb # app/lib/l10n/app_id.arb # app/lib/l10n/app_it.arb # app/lib/l10n/app_ja.arb # app/lib/l10n/app_kn.arb # app/lib/l10n/app_ko.arb # app/lib/l10n/app_localizations.dart # app/lib/l10n/app_localizations_ar.dart # app/lib/l10n/app_localizations_be.dart # app/lib/l10n/app_localizations_bg.dart # app/lib/l10n/app_localizations_bn.dart # app/lib/l10n/app_localizations_bs.dart # app/lib/l10n/app_localizations_ca.dart # app/lib/l10n/app_localizations_cs.dart # app/lib/l10n/app_localizations_da.dart # app/lib/l10n/app_localizations_de.dart # app/lib/l10n/app_localizations_el.dart # app/lib/l10n/app_localizations_en.dart # app/lib/l10n/app_localizations_es.dart # app/lib/l10n/app_localizations_et.dart # app/lib/l10n/app_localizations_fa.dart # app/lib/l10n/app_localizations_fi.dart # app/lib/l10n/app_localizations_fr.dart # app/lib/l10n/app_localizations_he.dart # app/lib/l10n/app_localizations_hi.dart # app/lib/l10n/app_localizations_hr.dart # app/lib/l10n/app_localizations_hu.dart # app/lib/l10n/app_localizations_id.dart # app/lib/l10n/app_localizations_it.dart # app/lib/l10n/app_localizations_ja.dart # app/lib/l10n/app_localizations_kn.dart # app/lib/l10n/app_localizations_ko.dart # app/lib/l10n/app_localizations_lt.dart # app/lib/l10n/app_localizations_lv.dart # app/lib/l10n/app_localizations_mk.dart # app/lib/l10n/app_localizations_mr.dart # app/lib/l10n/app_localizations_ms.dart # app/lib/l10n/app_localizations_nl.dart # app/lib/l10n/app_localizations_no.dart # app/lib/l10n/app_localizations_pl.dart # app/lib/l10n/app_localizations_pt.dart # app/lib/l10n/app_localizations_ro.dart # app/lib/l10n/app_localizations_ru.dart # app/lib/l10n/app_localizations_sk.dart # app/lib/l10n/app_localizations_sl.dart # app/lib/l10n/app_localizations_sr.dart # app/lib/l10n/app_localizations_sv.dart # app/lib/l10n/app_localizations_ta.dart # app/lib/l10n/app_localizations_te.dart # app/lib/l10n/app_localizations_th.dart # app/lib/l10n/app_localizations_tl.dart # app/lib/l10n/app_localizations_tr.dart # app/lib/l10n/app_localizations_uk.dart # app/lib/l10n/app_localizations_ur.dart # app/lib/l10n/app_localizations_vi.dart # app/lib/l10n/app_localizations_zh.dart # app/lib/l10n/app_lt.arb # app/lib/l10n/app_lv.arb # app/lib/l10n/app_mk.arb # app/lib/l10n/app_mr.arb # app/lib/l10n/app_ms.arb # app/lib/l10n/app_nl.arb # app/lib/l10n/app_no.arb # app/lib/l10n/app_pl.arb # app/lib/l10n/app_pt.arb # app/lib/l10n/app_ro.arb # app/lib/l10n/app_ru.arb # app/lib/l10n/app_sk.arb # app/lib/l10n/app_sl.arb # app/lib/l10n/app_sr.arb # app/lib/l10n/app_sv.arb # app/lib/l10n/app_ta.arb # app/lib/l10n/app_te.arb # app/lib/l10n/app_th.arb # app/lib/l10n/app_tl.arb # app/lib/l10n/app_tr.arb # app/lib/l10n/app_uk.arb # app/lib/l10n/app_ur.arb # app/lib/l10n/app_vi.arb # app/lib/l10n/app_zh.arb | 1 个月前 | |
fix(app): keep dev builds from crashing when opened without flutter run A dev-flavor debug build on a physical iPhone works while `flutter run` is attached and crashes the instant it is opened from the Home Screen afterwards (or relaunched by iOS in the background). Every crash report ends in SwiftAwesomeNotificationsPlugin.register(with:) at address 0. iOS only lets Flutter tooling start a JIT Dart VM on a device, so FlutterEngine init returns nil, the storyboard FlutterViewController has no engine, registrar(forPlugin:) returns nil, and the first Swift plugin dereferences it. - setup.sh: OMI_MOBILE_BUILD_MODE=debug|profile|release picks the `flutter run` mode; a debug build headed for a physical iPhone gets a warning naming the untethered crash and the profile alternative. - AppDelegate: skip plugin registration and every binaryMessenger use when the root FlutterViewController has no engine, and show a notice explaining why. Decision and text live in FlutterLaunchEngineGuard.swift with a swiftc-compiled Ruby test registered in the checks manifest. - Docs: AppSetup troubleshooting rewritten around the real symptom, README quick path, app AGENTS build-mode note. Verification: app/scripts/mobile_build_wrapper_test.sh (new build-mode cases), ruby app/ios/test/flutter_launch_engine_guard_test.rb, crash reports pulled from the phone via devicectl systemCrashLogs; the same debug build launched standalone on the simulator does not crash. Failure-Class: none Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YU5BMEAfcFxNSWmLLDNfZ3 | 18 小时前 | |
Add error path test for streamed HTTP responses This adds a test verifying that `_drainStreamedResponse` gracefully handles stream exceptions from aborted network sockets, preventing crashes when the shared HTTP client recovers a 401 response and initiates an authorization token retry. Also updates `analysis_baseline.json` which changed naturally due to the analyzer finding an improved baseline. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> | 27 天前 | |
fix(app): promote use_build_context_synchronously to analyzer error Now that the burndown lands the rule at 0 occurrences (previous commit), promote it to error severity in analysis_options.yaml so the bug class (BuildContext used across an async gap without a mounted check) can never silently re-enter the codebase — a new violation is now a hard analyzer error, not an INFO-level lint the ratchet could tolerate rising and falling. Probe evidence (temporary lib/_bcs_probe.dart with an unguarded Navigator.of(context).pop() after an await, reverted before commit): - `dart analyze --format=machine lib/_bcs_probe.dart` reported it as `ERROR|LINT|USE_BUILD_CONTEXT_SYNCHRONOUSLY|...` (ERROR severity, was INFO before this change). - `app/scripts/analyze_ratchet.sh` exited 1 with "Error-severity diagnostics found (never baselined)" — the script's ERROR path (scripts/analyze_ratchet.sh:103-109) fails before it ever reads or writes analysis_baseline.json. - `app/scripts/analyze_ratchet.sh --update-baseline` also exited 1 on the same ERROR check, confirming the violation cannot be silently locked into the baseline by that flag either. - After deleting the probe file: `app/scripts/analyze_ratchet.sh` passes clean, and `bash app/test.sh` shows 746/746 tests passing. | 1 个月前 | |
add provider extension to devtools | 1 年前 | |
change app name from friend to omi | 1 年前 | |
Revert public-client env hardening that broke the mobile build Builds 1.0.542+969/970 shipped broken: the secret-boundary PRs (#8716, #8717, #8733, #8741) renamed every app env var to a PUBLIC_ prefix and moved Envied's source to .client.env, but the Codemagic dashboard still defines the old names. The PUBLIC_* exports resolved empty, create-public-client-env.sh skipped them, and Envied baked null into prod — so Env.apiBaseUrl! force-unwrapped to a crash on launch. The post-build artifact scanner also false-positived on Firebase/SDK constant strings (ERROR_INVALID_CREDENTIAL, PROJECT_TOKEN) inside the compiled BatteryWidget.appex, failing 'flutter build ipa'. Restores codemagic.yaml and app/lib/env to the last-good (1.0.533) state so the prod build works with the existing dashboard vars, and reverts the coupled secret-scanner apparatus (scanner scripts, client_env_policy, boundary test, backend secret-consumer registry, and the lint.yml/pre-commit/pre-push enforcement) as one unit. Preserves the independent web hardening: the gcp_apps_js/frontend/personas workflows keep their baked-secret removal + env-choice/SHA-tagging, and web/frontend, web/personas-open-source, and plugins/apps-js are untouched. | 2 个月前 | |
fix(l10n): sync on-device STT keys with scoped names Failure-Class: none | 1 个月前 | |
Maps: keyless static previews + native-map handoff; fix Unknown location labels (#12841) * feat(backend): authed static-map proxy route with Redis-cached renders GET /v1/static-map renders dark-styled Google Static Maps images server-side (the only Maps key stays server-restricted) and caches the bytes in Redis keyed by the quantized pin set + size, so repeat renders of the same place across users/sessions cost one upstream call per distinct pin set per week. utils/static_map.py is the single provider seam for app map previews; swapping providers touches that module only. - pins parsed/bounded/de-duplicated/sorted, capped at 50; ~11m quantization makes users at the same place share one cached image - one pin centers at street zoom; several pins use provider auto-fit - failures return 502 (never cached) - the app renders its offline pin-dot canvas; auth prevents an open proxy on the project key - rate policy static_map:get (240/h per uid) stops hot loops; cached hits are one Redis read Tests: tests/routers/test_static_map.py (auth 401, 400 malformed pins, 502 upstream failure, 200 + private cache headers, cache hit/miss/order- insensitivity/no-cache-on-failure/missing key tolerances). Verified: .venv/bin/python -m pytest tests/routers/test_static_map.py -q -> 14 passed; scan_async_blockers clean. * fix(backend): enrich sync-path geolocation at the pipeline coordinator Failure-Class: FC-sync-geolocation-missing-address | new | none Offline-synced conversations were created with raw coordinates and no address: REST create, developer API, integration ingest, and live finalization all run resolve_geolocation, but the sync path shipped raw coords straight through. Recaps built from synced conversations showed 'Unknown' timeline rows. _run_full_pipeline_background_async - the one coordinator both the inline and Cloud Tasks dispatch branches call - now enriches the job's geolocation once, before any segment is processed and before the concurrency gate (no slot held during the geocode call). The resolver keeps the caller's exact coordinates and returns its input unchanged on any geocode miss/error, so a failure never drops the location. Tests: tests/unit/test_sync_geolocation_enrichment.py drives the real coordinator with a fake geocoder - one geocode per job (not per segment), enriched value reaches every segment, raw geolocation survives a geocode failure, None passthrough. Fixture stub lists in test_sync_v2 / test_sync_transcription_prefs gained the utils.conversations.location stub with an identity passthrough (the real resolver's miss behavior). Verified: pytest tests/unit/test_sync_geolocation_enrichment.py test_sync_v2.py test_sync_transcription_prefs.py -> all passed. * fix(backend): fill empty daily-summary pin addresses at read time Failure-Class: FC-sync-geolocation-missing-address | new | none generate_comprehensive_daily_summary copied c.geolocation.address verbatim, so pins from conversations created before write-time enrichment (the sync path) rendered as 'Unknown' recap timeline rows. The pins loop now fills an empty address through the shared ~100m- rounded geocode cache - the same entries write-time enrichment writes, so an already-enriched day costs no extra upstream call. All three callers are sync contexts, so the sync geocoder is used. A geocode miss or error leaves the address empty and keeps the pin (the app labels it 'Unknown'); regenerating a summary retroactively fixes history with no migration. Tests: tests/unit/test_daily_summary_location_address_fill.py (filled from geocoder with exact coords preserved, present address skips the geocoder, miss and exception both keep the pin address-less). Verified: pytest tests/unit/test_daily_summary_location_address_fill.py test_daily_summary_zero_coordinate_locations.py -> all passed. * feat(app): OmiMapPreview - one static-map widget with an offline pin-dot canvas Every map preview now funnels through a single widget backed by the authed backend static-map proxy (GET /v1/static-map): URL built by buildOmiStaticMapUrl (pins quantized to 4 decimals to match the server's cache quantization, deduped, capped at 50), image fetched with the session's Authorization header, and a deterministic dark canvas with one white dot per pin while loading, offline, or on any failure - never an error state. Product direction: previews in-app, tapping opens the native map app via MapsUtil.launchMap. The conversation detail geolocation card migrates from the client-keyed getMapImageUrl URL to this widget; getMapImageUrl is removed (maps_util keeps launchMap + place URL), taking the last direct client-side Google Static Maps call with it. Verified: flutter test test/widgets/omi_map_preview_test.dart test/widgets/daily_summary_card_test.dart (widget+URL builder tests); flutter analyze clean on changed files; analyze_ratchet.sh passed. * feat(app): replace CARTO tile maps with static previews; drop flutter_map CARTO began enforcing API keys on basemaps.cartocdn.com (keyless requests now return watermarked tiles), and Omi has no CARTO key. All three flutter_map surfaces now render OmiMapPreview instead - zero remaining tile traffic, no keyed tile provider in the app: - conversation map page: static preview of every ~100m cluster anchor (tap opens the native map app) above a grouped 'conversations at this place' list; single-conversation places open the conversation directly, multi-conversation places keep the cluster bottom sheet; grouping logic and cluster-row keys unchanged - daily summary card: 96px preview strip of the day's pins in the recap carousel (the highest-frequency map surface) - now one cached server-side render per distinct pin set instead of dozens of tiles per card - daily summary detail 'Your Day's Journey': 200px preview; image tap still opens the day's first stop (Apple Maps cannot take waypoints via map_launcher), per-stop timeline rows unchanged flutter_map is removed from pubspec (latlong2 stays - journey grouping uses it). The client-embedded Env.googleMapsApiKey goes with it: all static maps come from the server-restricted key behind the proxy now. envied outputs regenerated; test EnvFields stubs updated to match. Verified: bash test.sh / flutter test -> 1758 passed, 5 skipped, 0 failed; scripts/analyze_ratchet.sh passed (9 lint counts improved). * test(app): update map-surface tests for static previews - daily summary card: assert the preview strip, proxy URL pins/quantize/ dedupe, offline canvas fallback, and no-map-for-invalid-coordinates (was: tile provider request counting) - daily summary detail page: drop the TileProvider seam (widget no longer takes one) - conversation map groups: group-card key + cluster sheet rows replace marker keys; journey/env stubs gain the EnvFields change Verified: flutter test on all four files -> passed. * test(backend): stub utils.conversations.location in the cloud-tasks loader pipeline.py now imports async_resolve_geolocation at module scope; the cloud-tasks stub loader replaces its parent packages with MagicMocks, so the submodule import fails in file isolation (the earlier green run leaned on a module cached by another test file). Adds the explicit stub with an identity passthrough - the real resolver's miss behavior. Verified: BACKEND_UNIT_TEST_FILE_LIST=<this file> bash test.sh -> 91 passed. * chore(app): drop GOOGLE_MAPS_API_KEY from the env template The client no longer embeds a Maps key - all static-map renders go through the server-restricted key behind GET /v1/static-map. * test(backend): provide redis r on the usage-tracking stub set external_integrations now imports utils.conversations.location (daily- summary address fill), which imports the redis client symbol; the usage-tracking stub environment replaces database.redis_db with an empty module, so the import fails in file isolation. Give the stub an r. Verified: BACKEND_UNIT_TEST_FILE_LIST=<this file> bash test.sh -> 21 passed. * docs(backend): note the static-map provider URL budget with current limits * chore(backend): declare the static-map route policy in the manifest * fix(backend): drop the redundant isinstance on the typed pins parameter * fix(app): hold the preview canvas until the auth header resolves The first build fired the authed proxy request before getAuthHeader() resolved; CachedNetworkImage keys its cache by URL, so the later header-arriving setState never re-fetched - a permanent 401 fallback canvas. The widget now renders the pin-dot canvas (which already doubles as the error/offline path) until the header resolves, and only then mounts the network image with the Authorization header attached. The header resolver is injectable (authHeaderProvider) so tests control when it completes; an explicit imageUrl (test seam) skips the gate. Tests: new regression case asserts no CachedNetworkImage while auth is unresolved and the authed image after the completer fires; the card tests repoint URL assertions at the preview widget's pins (URL building has its own unit tests). Verified: flutter analyze clean; flutter test (full) -> 1759 passed. * fix(app): restore the conversation_map_marker automation key on group rows The per-place tappable key predates the static preview; keep it stable for automation (the PR brief promises preserved keys). Verified: flutter test test/unit/conversation_map_groups_test.dart passed. * fix(backend): normalize static-map dimensions, dedup render stampedes, count canvas fallbacks Review findings on the static-map proxy: - Dimension handling: one proportional scale factor (min(1, 640/w, 640/h)) computed once in fetch_static_map replaces the independent per-axis clamps - aspect is preserved and every request that differs only by scale normalizes onto the SAME cache entry (cache key and provider URL both use the effective size). - Stampede dedup: after a miss, a per-key render lock (r.set nx, 30s TTL) elects one renderer; concurrent misses poll the cache (0.25s interval, 15s budget) for the holder's result. Lock-held-timeout and lock-unavailable (Redis broken) both fail OPEN to an unlocked render - a lost lock never becomes a 502, and a broken Redis never pays the 15s wait budget. - record_fallback (component=static_map, provider_static_map -> client_pin_canvas, outcome=degraded) fires before the 502 so the degrade is counted in the shared telemetry. Tests: proportional normalization + shared cache entry, oversized fetch normalizes (size=640x150), concurrent misses render once, waiter polls a foreign lock holder to its hit, wait-timeout renders unlocked, 422s for out-of-bounds width/height via TestClient (Query contract), fallback telemetry kwargs. Verified: pytest tests/routers/test_static_map.py -> 21 passed; pyright (scripts/typecheck.sh) 0 errors, no findings in changed files. * fix(backend): cap daily-summary geocode attempts; boost-exempt static_map:get - The read-time address fill now bounds geocode ATTEMPTS at 10 per summary generation: cache hits are cheap but attempts are the deterministic wall-clock bound (10 x the geocoder's 10s worst case stays inside the job budget). Pins past the cap keep an empty address and the app's 'Unknown' fallback. Test: 11 empty-address pins -> 10 filled, 11th untouched, 10 geocoder calls. - static_map:get joins the boost-exempt rate policies: under prod's RATE_LIMIT_BOOST the 240/h hot-loop cap would otherwise resolve to 24k/h and stop protecting the billable provider calls. Verified: pytest test_daily_summary_location_address_fill.py -> 5 passed; utils.rate_limit_config import shows static_map:get in BOOST_EXEMPT_POLICIES. * test(backend): self-contained enrichment fakes; model the resolver short-circuit - test_sync_geolocation_enrichment no longer imports the transcription-prefs fixture module: sharing it loaded real heavyweight modules (GCS/Firestore protos, anthropic/jiter) inside the stub window, and stub_modules' teardown evicted them so later files broke on re-import (duplicate proto registration, jiter NameError) when run in one process. The fakes are now minimal and self-contained - every heavy leaf is stubbed, light enum modules (stt.outcomes, sync.lanes, sync.telemetry) stay real. - The None-geolocation test now models the real resolver's short-circuit (falsy input returns immediately, attempts recorded only for truthy input) and asserts: resolver called exactly once with None AND zero geocode attempts. Docstring states what is actually proven. - One-line caveat at the cloud-tasks loader injection site: new submodule imports must be added to heavy_deps explicitly (MagicMock parents do not resolve submodules). Migrating that hand-rolled loader to stub_modules would cascade across its 37 call sites - deliberately left as-is. Verified: pytest on the combined focused group (static_map, summary fill, sync enrichment, cloud tasks, sync v2, transcription prefs, geocode resolve) -> 372 passed in one process. | 1 天前 | |
Maps: keyless static previews + native-map handoff; fix Unknown location labels (#12841) * feat(backend): authed static-map proxy route with Redis-cached renders GET /v1/static-map renders dark-styled Google Static Maps images server-side (the only Maps key stays server-restricted) and caches the bytes in Redis keyed by the quantized pin set + size, so repeat renders of the same place across users/sessions cost one upstream call per distinct pin set per week. utils/static_map.py is the single provider seam for app map previews; swapping providers touches that module only. - pins parsed/bounded/de-duplicated/sorted, capped at 50; ~11m quantization makes users at the same place share one cached image - one pin centers at street zoom; several pins use provider auto-fit - failures return 502 (never cached) - the app renders its offline pin-dot canvas; auth prevents an open proxy on the project key - rate policy static_map:get (240/h per uid) stops hot loops; cached hits are one Redis read Tests: tests/routers/test_static_map.py (auth 401, 400 malformed pins, 502 upstream failure, 200 + private cache headers, cache hit/miss/order- insensitivity/no-cache-on-failure/missing key tolerances). Verified: .venv/bin/python -m pytest tests/routers/test_static_map.py -q -> 14 passed; scan_async_blockers clean. * fix(backend): enrich sync-path geolocation at the pipeline coordinator Failure-Class: FC-sync-geolocation-missing-address | new | none Offline-synced conversations were created with raw coordinates and no address: REST create, developer API, integration ingest, and live finalization all run resolve_geolocation, but the sync path shipped raw coords straight through. Recaps built from synced conversations showed 'Unknown' timeline rows. _run_full_pipeline_background_async - the one coordinator both the inline and Cloud Tasks dispatch branches call - now enriches the job's geolocation once, before any segment is processed and before the concurrency gate (no slot held during the geocode call). The resolver keeps the caller's exact coordinates and returns its input unchanged on any geocode miss/error, so a failure never drops the location. Tests: tests/unit/test_sync_geolocation_enrichment.py drives the real coordinator with a fake geocoder - one geocode per job (not per segment), enriched value reaches every segment, raw geolocation survives a geocode failure, None passthrough. Fixture stub lists in test_sync_v2 / test_sync_transcription_prefs gained the utils.conversations.location stub with an identity passthrough (the real resolver's miss behavior). Verified: pytest tests/unit/test_sync_geolocation_enrichment.py test_sync_v2.py test_sync_transcription_prefs.py -> all passed. * fix(backend): fill empty daily-summary pin addresses at read time Failure-Class: FC-sync-geolocation-missing-address | new | none generate_comprehensive_daily_summary copied c.geolocation.address verbatim, so pins from conversations created before write-time enrichment (the sync path) rendered as 'Unknown' recap timeline rows. The pins loop now fills an empty address through the shared ~100m- rounded geocode cache - the same entries write-time enrichment writes, so an already-enriched day costs no extra upstream call. All three callers are sync contexts, so the sync geocoder is used. A geocode miss or error leaves the address empty and keeps the pin (the app labels it 'Unknown'); regenerating a summary retroactively fixes history with no migration. Tests: tests/unit/test_daily_summary_location_address_fill.py (filled from geocoder with exact coords preserved, present address skips the geocoder, miss and exception both keep the pin address-less). Verified: pytest tests/unit/test_daily_summary_location_address_fill.py test_daily_summary_zero_coordinate_locations.py -> all passed. * feat(app): OmiMapPreview - one static-map widget with an offline pin-dot canvas Every map preview now funnels through a single widget backed by the authed backend static-map proxy (GET /v1/static-map): URL built by buildOmiStaticMapUrl (pins quantized to 4 decimals to match the server's cache quantization, deduped, capped at 50), image fetched with the session's Authorization header, and a deterministic dark canvas with one white dot per pin while loading, offline, or on any failure - never an error state. Product direction: previews in-app, tapping opens the native map app via MapsUtil.launchMap. The conversation detail geolocation card migrates from the client-keyed getMapImageUrl URL to this widget; getMapImageUrl is removed (maps_util keeps launchMap + place URL), taking the last direct client-side Google Static Maps call with it. Verified: flutter test test/widgets/omi_map_preview_test.dart test/widgets/daily_summary_card_test.dart (widget+URL builder tests); flutter analyze clean on changed files; analyze_ratchet.sh passed. * feat(app): replace CARTO tile maps with static previews; drop flutter_map CARTO began enforcing API keys on basemaps.cartocdn.com (keyless requests now return watermarked tiles), and Omi has no CARTO key. All three flutter_map surfaces now render OmiMapPreview instead - zero remaining tile traffic, no keyed tile provider in the app: - conversation map page: static preview of every ~100m cluster anchor (tap opens the native map app) above a grouped 'conversations at this place' list; single-conversation places open the conversation directly, multi-conversation places keep the cluster bottom sheet; grouping logic and cluster-row keys unchanged - daily summary card: 96px preview strip of the day's pins in the recap carousel (the highest-frequency map surface) - now one cached server-side render per distinct pin set instead of dozens of tiles per card - daily summary detail 'Your Day's Journey': 200px preview; image tap still opens the day's first stop (Apple Maps cannot take waypoints via map_launcher), per-stop timeline rows unchanged flutter_map is removed from pubspec (latlong2 stays - journey grouping uses it). The client-embedded Env.googleMapsApiKey goes with it: all static maps come from the server-restricted key behind the proxy now. envied outputs regenerated; test EnvFields stubs updated to match. Verified: bash test.sh / flutter test -> 1758 passed, 5 skipped, 0 failed; scripts/analyze_ratchet.sh passed (9 lint counts improved). * test(app): update map-surface tests for static previews - daily summary card: assert the preview strip, proxy URL pins/quantize/ dedupe, offline canvas fallback, and no-map-for-invalid-coordinates (was: tile provider request counting) - daily summary detail page: drop the TileProvider seam (widget no longer takes one) - conversation map groups: group-card key + cluster sheet rows replace marker keys; journey/env stubs gain the EnvFields change Verified: flutter test on all four files -> passed. * test(backend): stub utils.conversations.location in the cloud-tasks loader pipeline.py now imports async_resolve_geolocation at module scope; the cloud-tasks stub loader replaces its parent packages with MagicMocks, so the submodule import fails in file isolation (the earlier green run leaned on a module cached by another test file). Adds the explicit stub with an identity passthrough - the real resolver's miss behavior. Verified: BACKEND_UNIT_TEST_FILE_LIST=<this file> bash test.sh -> 91 passed. * chore(app): drop GOOGLE_MAPS_API_KEY from the env template The client no longer embeds a Maps key - all static-map renders go through the server-restricted key behind GET /v1/static-map. * test(backend): provide redis r on the usage-tracking stub set external_integrations now imports utils.conversations.location (daily- summary address fill), which imports the redis client symbol; the usage-tracking stub environment replaces database.redis_db with an empty module, so the import fails in file isolation. Give the stub an r. Verified: BACKEND_UNIT_TEST_FILE_LIST=<this file> bash test.sh -> 21 passed. * docs(backend): note the static-map provider URL budget with current limits * chore(backend): declare the static-map route policy in the manifest * fix(backend): drop the redundant isinstance on the typed pins parameter * fix(app): hold the preview canvas until the auth header resolves The first build fired the authed proxy request before getAuthHeader() resolved; CachedNetworkImage keys its cache by URL, so the later header-arriving setState never re-fetched - a permanent 401 fallback canvas. The widget now renders the pin-dot canvas (which already doubles as the error/offline path) until the header resolves, and only then mounts the network image with the Authorization header attached. The header resolver is injectable (authHeaderProvider) so tests control when it completes; an explicit imageUrl (test seam) skips the gate. Tests: new regression case asserts no CachedNetworkImage while auth is unresolved and the authed image after the completer fires; the card tests repoint URL assertions at the preview widget's pins (URL building has its own unit tests). Verified: flutter analyze clean; flutter test (full) -> 1759 passed. * fix(app): restore the conversation_map_marker automation key on group rows The per-place tappable key predates the static preview; keep it stable for automation (the PR brief promises preserved keys). Verified: flutter test test/unit/conversation_map_groups_test.dart passed. * fix(backend): normalize static-map dimensions, dedup render stampedes, count canvas fallbacks Review findings on the static-map proxy: - Dimension handling: one proportional scale factor (min(1, 640/w, 640/h)) computed once in fetch_static_map replaces the independent per-axis clamps - aspect is preserved and every request that differs only by scale normalizes onto the SAME cache entry (cache key and provider URL both use the effective size). - Stampede dedup: after a miss, a per-key render lock (r.set nx, 30s TTL) elects one renderer; concurrent misses poll the cache (0.25s interval, 15s budget) for the holder's result. Lock-held-timeout and lock-unavailable (Redis broken) both fail OPEN to an unlocked render - a lost lock never becomes a 502, and a broken Redis never pays the 15s wait budget. - record_fallback (component=static_map, provider_static_map -> client_pin_canvas, outcome=degraded) fires before the 502 so the degrade is counted in the shared telemetry. Tests: proportional normalization + shared cache entry, oversized fetch normalizes (size=640x150), concurrent misses render once, waiter polls a foreign lock holder to its hit, wait-timeout renders unlocked, 422s for out-of-bounds width/height via TestClient (Query contract), fallback telemetry kwargs. Verified: pytest tests/routers/test_static_map.py -> 21 passed; pyright (scripts/typecheck.sh) 0 errors, no findings in changed files. * fix(backend): cap daily-summary geocode attempts; boost-exempt static_map:get - The read-time address fill now bounds geocode ATTEMPTS at 10 per summary generation: cache hits are cheap but attempts are the deterministic wall-clock bound (10 x the geocoder's 10s worst case stays inside the job budget). Pins past the cap keep an empty address and the app's 'Unknown' fallback. Test: 11 empty-address pins -> 10 filled, 11th untouched, 10 geocoder calls. - static_map:get joins the boost-exempt rate policies: under prod's RATE_LIMIT_BOOST the 240/h hot-loop cap would otherwise resolve to 24k/h and stop protecting the billable provider calls. Verified: pytest test_daily_summary_location_address_fill.py -> 5 passed; utils.rate_limit_config import shows static_map:get in BOOST_EXEMPT_POLICIES. * test(backend): self-contained enrichment fakes; model the resolver short-circuit - test_sync_geolocation_enrichment no longer imports the transcription-prefs fixture module: sharing it loaded real heavyweight modules (GCS/Firestore protos, anthropic/jiter) inside the stub window, and stub_modules' teardown evicted them so later files broke on re-import (duplicate proto registration, jiter NameError) when run in one process. The fakes are now minimal and self-contained - every heavy leaf is stubbed, light enum modules (stt.outcomes, sync.lanes, sync.telemetry) stay real. - The None-geolocation test now models the real resolver's short-circuit (falsy input returns immediately, attempts recorded only for truthy input) and asserts: resolver called exactly once with None AND zero geocode attempts. Docstring states what is actually proven. - One-line caveat at the cloud-tasks loader injection site: new submodule imports must be added to heavy_deps explicitly (MagicMock parents do not resolve submodules). Migrating that hand-rolled loader to stub_modules would cascade across its 37 call sites - deliberately left as-is. Verified: pytest on the combined focused group (static_map, summary fill, sync enrichment, cloud tasks, sync v2, transcription prefs, geocode resolve) -> 372 passed in one process. | 1 天前 | |
harden(app): fail fast on mobile build profile drift (#12559) | 6 天前 | |
chore(dev): add personal build config overlay for local dev builds (#11789) Introduces a .personal_configs/ convention at the repo root for contributors to store machine-local Firebase credentials and dev env config without committing them. Run app/setup-personal.sh after setup.sh to copy them into place. Split out of #7641 at maintainer request (community-build signing fix and this overlay are unrelated concerns and easier to review apart) — carries the same content as that PR's commits 1295dc33a2/b246990fa9, rebased onto current main. Failure-Class: none | 21 天前 | |
fix(app): address review on the untethered launch fix - Notice on the phone names the canonical `bash setup.sh ios` invocation from app/ instead of a repo-root path. - setup.sh warning describes the shipped behavior (engine-unavailable notice, not a crash) and only fires for the dev flavor, since its remedy builds the dev flavor. - Drop the GOOGLE_REVERSE_CLIENT_ID pin from devProfile/devRelease so Custom.xcconfig keeps owning that value. Failure-Class: none Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YU5BMEAfcFxNSWmLLDNfZ3 | 17 小时前 | |
added shorebird.yaml | 2 年前 | |
harden(app): fail fast on mobile build profile drift (#12559) | 6 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 2 个月前 | ||
| 1 年前 | ||
| 10 小时前 | ||
| 4 天前 | ||
| 1 个月前 | ||
| 8 天前 | ||
| 1 小时前 | ||
| 4 个月前 | ||
| 1 个月前 | ||
| 8 小时前 | ||
| 1 小时前 | ||
| 1 个月前 | ||
| 17 小时前 | ||
| 6 天前 | ||
| 1 小时前 | ||
| 1 年前 | ||
| 2 个月前 | ||
| 6 个月前 | ||
| 1 天前 | ||
| 2 个月前 | ||
| 8 小时前 | ||
| 1 个月前 | ||
| 18 小时前 | ||
| 27 天前 | ||
| 1 个月前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 个月前 | ||
| 1 个月前 | ||
| 1 天前 | ||
| 1 天前 | ||
| 6 天前 | ||
| 21 天前 | ||
| 17 小时前 | ||
| 2 年前 | ||
| 6 天前 |