Skip to content

feat(firebase-ai): add transcriptions to Live API sample and reduce abstraction - #2836

Open
thatfiredev wants to merge 4 commits into
masterfrom
rpf/add-live-transcriptions
Open

feat(firebase-ai): add transcriptions to Live API sample and reduce abstraction#2836
thatfiredev wants to merge 4 commits into
masterfrom
rpf/add-live-transcriptions

Conversation

@thatfiredev

Copy link
Copy Markdown
Member

There are 2 main changes in this PR:

  • Add support for showing transcriptions on screen for the Gemini Live API samples.
  • Delete the BidiViewModel class to reduce abstraction. This should make the samples more copy-pastable.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the live streaming feature by replacing the abstract BidiViewModel with two dedicated ViewModels, StreamAudioViewModel and StreamVideoViewModel, and adding support for real-time transcriptions in both audio and video screens. The review feedback highlights several critical improvements: managing the conversation lifecycle with DisposableEffect and a lifecycle-aware CoroutineScope to prevent leaks, avoiding blocking the main thread with runBlocking during ViewModel initialization, and replacing animateScrollToItem with scrollToItem to prevent UI stuttering during rapid transcription updates.

@OptIn(PublicPreviewAPI::class)
class StreamAudioViewModel : BidiViewModel() {
class StreamAudioViewModel : ViewModel() {
private var liveSession: LiveSession

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Declaring liveSession as a non-nullable property initialized via runBlocking in the init block blocks the main thread during ViewModel creation to establish a network connection. This can cause UI freezes or ANR (Application Not Responding) errors. Instead, declare liveSession as lateinit var (or nullable) and connect to the live model asynchronously using viewModelScope.launch.

Suggested change
private var liveSession: LiveSession
private lateinit var liveSession: LiveSession

@OptIn(PublicPreviewAPI::class)
class StreamVideoViewModel : BidiViewModel() {
class StreamVideoViewModel : ViewModel() {
private var liveSession: LiveSession

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Declaring liveSession as a non-nullable property initialized via runBlocking in the init block blocks the main thread during ViewModel creation to establish a network connection. This can cause UI freezes or ANR (Application Not Responding) errors. Instead, declare liveSession as lateinit var (or nullable) and connect to the live model asynchronously using viewModelScope.launch.

Suggested change
private var liveSession: LiveSession
private lateinit var liveSession: LiveSession

Comment on lines +59 to +63
LaunchedEffect(transcriptions.size, transcriptions.lastOrNull()?.text) {
if (transcriptions.isNotEmpty()) {
listState.animateScrollToItem(transcriptions.size - 1)
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using animateScrollToItem for rapidly updating streaming text (like live transcriptions) can cause severe UI stuttering, lag, and high CPU usage because smooth scroll animations are repeatedly interrupted and restarted. Using scrollToItem is much more efficient and ensures smooth rendering during active streaming.

Suggested change
LaunchedEffect(transcriptions.size, transcriptions.lastOrNull()?.text) {
if (transcriptions.isNotEmpty()) {
listState.animateScrollToItem(transcriptions.size - 1)
}
}
LaunchedEffect(transcriptions.size, transcriptions.lastOrNull()?.text) {
if (transcriptions.isNotEmpty()) {
listState.scrollToItem(transcriptions.size - 1)
}
}

Comment on lines +75 to +79
LaunchedEffect(transcriptions.size, transcriptions.lastOrNull()?.text) {
if (transcriptions.isNotEmpty()) {
listState.animateScrollToItem(transcriptions.size - 1)
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using animateScrollToItem for rapidly updating streaming text (like live transcriptions) can cause severe UI stuttering, lag, and high CPU usage because smooth scroll animations are repeatedly interrupted and restarted. Using scrollToItem is much more efficient and ensures smooth rendering during active streaming.

Suggested change
LaunchedEffect(transcriptions.size, transcriptions.lastOrNull()?.text) {
if (transcriptions.isNotEmpty()) {
listState.animateScrollToItem(transcriptions.size - 1)
}
}
LaunchedEffect(transcriptions.size, transcriptions.lastOrNull()?.text) {
if (transcriptions.isNotEmpty()) {
listState.scrollToItem(transcriptions.size - 1)
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant